Runtime¶
Using FleCSI requires proper initialization and configuration of the FleCSI runtime. These examples illustrate some of the basic steps and options that are available.
Example 1: Minimal¶
The core FleCSI runtime has three control points: initialize, start, and finalize. These must be invoked by the user’s application code in that order.
initialize
This control point spins up the FleCSI runtime, and (optionally) any other runtimes on which FleCSI depends.start
This control point begins the actual runtime execution that forms the bulk of any simulation that is performed using FleCSI. The user must pass a top-level action that FleCSI will execute.finalize
This control point shuts down the FleCSI runtime, and any other runtimes that FleCSI itself initialized.
This example demonstrates a minimal use of FleCSI that just executes an action to print out Hello World. Code for this example can be found in tutorial/1-runtime/1-minimal.cc.
/*
@@@@@@@@ @@ @@@@@@ @@@@@@@@ @@
/@@///// /@@ @@////@@ @@////// /@@
/@@ /@@ @@@@@ @@ // /@@ /@@
/@@@@@@@ /@@ @@///@@/@@ /@@@@@@@@@/@@
/@@//// /@@/@@@@@@@/@@ ////////@@/@@
/@@ /@@/@@//// //@@ @@ /@@/@@
/@@ @@@//@@@@@@ //@@@@@@ @@@@@@@@ /@@
// /// ////// ////// //////// //
Copyright (c) 2016, Triad National Security, LLC
All rights reserved.
*/
#include <flecsi/execution.hh>
/*
The top-level action can be any C/C++ function that takes (int, char**) and
returns an int.
In this simple example, we only print a message to indicate that the
top-level action was actually executed by FleCSI. However, in a real
application, the top-level action would execute FleCSI tasks and other
functions to implement the simulation.
*/
int
top_level_action() {
std::cout << "Hello World" << std::endl;
return 0;
} // top_level_action
/*
The main function must invoke initialize, start, and finalize on the FleCSI
runtime. Otherwise, the implementation of main is left to the user.
*/
int
main(int argc, char ** argv) {
auto status = flecsi::initialize(argc, argv);
/*
The status returned by FleCSI's initialize method should be inspected to
see if the end-user specified --help on the command line. FleCSI has
built-in command-line support using Boost Program Options. This is
documented in the next example.
*/
if(status != flecsi::run::status::success) {
return status == flecsi::run::status::help ? 0 : status;
} // if
/*
The top-level action is passed to the start function to tell FleCSI what
it should execute.
*/
status = flecsi::start(top_level_action);
flecsi::finalize();
return status;
} // main
Note
The top-level action can be any C/C++ function that takes no arguments and returns an int. In this simple example, we only print a message to indicate that the top-level action was actually executed by FleCSI. However, in a real application, the top-level action would execute FleCSI tasks and other functions to implement the simulation.
The main function must invoke initialize, start, and finalize on the FleCSI runtime. Otherwise, the implementation of main is left to the user.
The status returned by FleCSI’s initialize method should be inspected to see if the end-user specified –help on the command line. FleCSI has built-in command-line support using Boost Program Options. This is documented in the next example.
Example 2: Program Options¶
FleCSI supports a program options capability based on Boost Program Options to simplify the creation and management of user-defined command-line options. The basic syntax for adding and accessing program options is semantically similar to the Boost interface (You can find documentation using the above link.) However, there are some notable differences:
FleCSI internally manages the boost::program_options::value variables for you, using boost::optional.
Positional options are the mechanism that should be used for required options.
Default, implicit, zero, and multi value attributes are specified in the flecsi::program_option constructor as an std::initializer_list.
This section of the tutorial provides examples of how to use FleCSI’s program option capability.
Example Program¶
In this example, imagine that you have a program that takes information about a taxi service (The options are silly and synthetic. However they demonstrate the basic usage of the flecsi::program_option type.) The command-line options and arguments for the program allow specification of the following: trim level, transmission, child seat, purpose (personal or business), light speed, and a passenger list. The first two options will be in a Car Options section, while the purpose will be under the Ride Options section. The passenger list is a positional argument to the program. The help output for the entire program looks like this:
Usage: program_options <passenger-list>
Positional Options:
passenger-list The list of passengers for this trip [.txt].
Basic Options:
-h [ --help ] Print this message and exit.
Car Options:
-l [ --level ] arg (= 1) Specify the trim level [1-10].
-t [ --transmission ] arg (= manual) Specify the transmission type
["automatic", "manual"].
-c [ --child-seat ] [=arg(= 1)] (= 0) Request a child seat.
Ride Options:
-p [ --purpose ] arg (= 1) Specify the purpose of the trip
(personal=0, business=1).
--lightspeed Travel at the speed of light.
FleCSI Options:
--flog-tags arg (=all) Enable the specified output tags, e.g.,
--flog-tags=tag1,tag2.
--flog-verbose [=arg(=1)] (=0) Enable verbose output. Passing '-1' will strip
any additional decorations added by flog and
will only output the user's message.
--flog-process arg (=-1) Restrict output to the specified process id.
Available FLOG Tags (FleCSI Logging Utility):
execution
task_wrapper
legion_mapper
unbind_accessors
topologies
reduction_wrapper
context
registration
bind_accessors
task_prologue
This shows the program usage, the basic options, e.g., --help, the
command-line and positional options for the example, and some auxiliary
options for controlling the FleCSI logging utility FLOG (described in the
next section of this tutorial). The FLOG options will only appear if
ENABLE_FLOG=ON was set in your FleCSI build.
Declaring Options¶
Note
FleCSI program options must be declared at namespace scope, i.e., outside of any function, class, or enum class. This is not a problem! It is often convenient to declare them in a header file (in which case, they must also be declared inline), or directly before the main function. We use the latter for this example simply for conciseness.
Let’s consider the first Car Options option: --level. To declare
this option, we use the following declaration:
/*
Add an integer-valued command-line option with a default value of '1'. The
option will have a long form --level, and a short form -l. The option will be
added under the "Car Options" section.
*/
flecsi::program_option<int> trim("Car Options",
"level,l",
"Specify the trim level [1-10].",
{{flecsi::option_default, 1}},
[](flecsi::any const & v) {
const int value = flecsi::option_value<int>(v);
return value > 0 && value < 11;
});
First, notice that the flecsi::program_option type is templated on the underlying option type int. In general, this can be any valid C++ type.
This constructor to flecsi::program_option takes the following parameters:
section (“Car Options”):
Identifies the section. Sections are generated automatically, simply by referencing them in a program option.flag (“level,l”):
The long and short forms of the option. If the string contains a comma, it is split into long name,short name. If there is no comma, the string is used as the long name with no short name.help (“Specify…”)
The help description that will be displayed when the usage message is printed.values ({{flecsi::option_default, …}})
This is a std::initializer_list<flecsi::program_option::initializer_value<int>>. The possible values are flecsi::option_default, flecsi::option_implicit, flecsi::option_zero, and flecsi::option_multi. The default value is used if the option is not passed at invocation. The implicit value is used if the option is passed without a value. If zero is specified, the option does not take an argument, and an implicit value must be provided. If multi is specified, the option takes multiple values.check ([](flecsi::any const &…)
An optional, user-defined predicate to validate the value passed by the user.
The next option --transmission is similar, but uses a std::string
value type:
/*
Add a string-valued command-line option with a default value of "manual". The
option will have a long form --transmission, and a short form -t. The option
will be added under the "Car Options" section.
*/
flecsi::program_option<std::string> transmission("Car Options",
"transmission,t",
"Specify the transmission type [\"automatic\", \"manual\"].",
{{flecsi::option_default, "manual"}},
[](flecsi::any const & v) {
const std::string value = flecsi::option_value<std::string>(v);
return value == "manual" || value == "automatic";
});
The only real difference is that (because the underlying type is std::string) the default value is also a string.
The last option in the “Car Options” section --child-seat
demonstrates the use of flecsi::option_implicit:
/*
Add an option that defines an implicit value. If the program is invoked with
--child-seat, the value will be true. If it is invoked without --child-seat,
the value will be false. This style of option should not be used with
positional arguments because Boost appears to have a bug when such options
are invoked directly before a positional option (gets confused about
separation). We break that convention here for the sake of completeness.
*/
flecsi::program_option<bool> child_seat("Car Options",
"child-seat,c",
"Request a child seat.",
{{flecsi::option_default, false}, {flecsi::option_implicit, true}});
Providing an implicit value, defines the behavior for the case that the
user invokes the program with the given flag, but does not assign a
value, e.g., --child-seat vs. --child-seat=1. The value is
implied by the flag itself.
Caution
This style of option should not be used with positional arguments
because Boost appears to have a bug when such options are invoked
directly before a positional option (gets confused about separation).
We break that convention here for the sake of completeness. If you
need an option that simply acts as a switch, i.e., it is either on
or off, consider using the --lightspeed style option below, as
this type of option is safe to use with positional options.
The first option in the Ride Options section --purpose takes an
integer value 0 or 1. This option is declared with the following
code:
/*
Add a an option to a different section, i.e., "Ride Options". The enumeration
type is not enforced by the FleCSI runtime, and is mostly for convenience.
*/
enum purpose_option : size_t { personal, business };
flecsi::program_option<size_t> purpose("Ride Options",
"purpose,p",
"Specify the purpose of the trip (personal=0, business=1).",
{{flecsi::option_default, purpose_option::business}},
[](flecsi::any const & v) {
size_t value = flecsi::option_value<size_t>(v);
return value == personal || value == business;
});
This option demonstrates how an enumeration can be used to define possible values. Although FleCSI does not enforce correctness, the enumeration can be used to check that the user-provided value is valid.
The next option in the Ride Options section --lightspeed defines
an implicit value and zero values (meaning that it takes no values). The
--lightspeed option acts as a switch, taking the implicit value if
the flag is passed. This will be useful to demonstrate how we can check
whether or not an option was passed in the next section:
/*
Add an option with no default. This will allow us to demonstrate testing an
option with has_value().
*/
flecsi::program_option<bool> lightspeed("Ride Options",
"lightspeed",
"Travel at the speed of light.",
{{flecsi::option_implicit, true}, {flecsi::option_zero}});
The final option in this example is a positional option, i.e., it is an argument to the program itself:
/*
Add a positional option. This uses a different constructor from the previous
option declarations. Positional options are a replacement for required
options (in the normal boost::program_options interface).
*/
flecsi::program_option<std::string> passenger_list("passenger-list",
"The list of passengers for this trip [.txt].",
1,
[](flecsi::any const & v) {
const std::string value = flecsi::option_value<std::string>(v);
return value.find(".txt") != std::string::npos;
});
Positional options are required, i.e., the program will error and print the usage message if a value is not passed.
Checking & Using Options¶
FleCSI option variables are implemented using an optional C++ type. The utility of this implementation is that optional already captures the behavior that we want from an option, i.e., it either has a value, or it does not. If the option has a value, the specific value depends on whether or not the user explicitly passed the option on the command line, and its default and implicit values.
Options that have a default value defined do not need to be tested:
/*
Add cost for trim level. This option does not have to be checked with
has_value() because it is defaulted. It is also unnecessary to check the
value because it was declared with a validator function.
*/
price += trim.value() * 100.0;
/*
Add cost for automatic transmission.
*/
price += transmission.value() == "automatic" ? 200.0 : 0.0;
/*
Add cost for child seat.
*/
if(child_seat.value()) {
price += 50.0;
}
/*
Deduction for business.
*/
if(purpose.value() == business) {
price *= 0.8;
}
Here, we simply need to access the value of the option using the value() method.
For options with no default value, we can check whether or not the option has a value using the has_value() method:
/*
Add cost for lightspeed. Since this option does not have a default, we need
to check whether or not the flag was passed.
*/
if(lightspeed.has_value()) {
price += 1000000.0;
}
Our one positional option works like the defaulted options (because it is required), and can be accessed using the value() method:
/*
Do something with the positional argument.
*/
auto read_file = [](std::string const &) {
// Read passengers...
return 5;
};
size_t passengers = read_file(passenger_list.value());
price *= passengers * 1.10 * price;
Here is the full source for this tutorial example:
/*
@@@@@@@@ @@ @@@@@@ @@@@@@@@ @@
/@@///// /@@ @@////@@ @@////// /@@
/@@ /@@ @@@@@ @@ // /@@ /@@
/@@@@@@@ /@@ @@///@@/@@ /@@@@@@@@@/@@
/@@//// /@@/@@@@@@@/@@ ////////@@/@@
/@@ /@@/@@//// //@@ @@ /@@/@@
/@@ @@@//@@@@@@ //@@@@@@ @@@@@@@@ /@@
// /// ////// ////// //////// //
Copyright (c) 2016, Triad National Security, LLC
All rights reserved.
*/
#include <flecsi/execution.hh>
/*
Add an integer-valued command-line option with a default value of '1'. The
option will have a long form --level, and a short form -l. The option will be
added under the "Car Options" section.
*/
flecsi::program_option<int> trim("Car Options",
"level,l",
"Specify the trim level [1-10].",
{{flecsi::option_default, 1}},
[](flecsi::any const & v) {
const int value = flecsi::option_value<int>(v);
return value > 0 && value < 11;
});
/*
Add a string-valued command-line option with a default value of "manual". The
option will have a long form --transmission, and a short form -t. The option
will be added under the "Car Options" section.
*/
flecsi::program_option<std::string> transmission("Car Options",
"transmission,t",
"Specify the transmission type [\"automatic\", \"manual\"].",
{{flecsi::option_default, "manual"}},
[](flecsi::any const & v) {
const std::string value = flecsi::option_value<std::string>(v);
return value == "manual" || value == "automatic";
});
/*
Add an option that defines an implicit value. If the program is invoked with
--child-seat, the value will be true. If it is invoked without --child-seat,
the value will be false. This style of option should not be used with
positional arguments because Boost appears to have a bug when such options
are invoked directly before a positional option (gets confused about
separation). We break that convention here for the sake of completeness.
*/
flecsi::program_option<bool> child_seat("Car Options",
"child-seat,c",
"Request a child seat.",
{{flecsi::option_default, false}, {flecsi::option_implicit, true}});
/*
Add a an option to a different section, i.e., "Ride Options". The enumeration
type is not enforced by the FleCSI runtime, and is mostly for convenience.
*/
enum purpose_option : size_t { personal, business };
flecsi::program_option<size_t> purpose("Ride Options",
"purpose,p",
"Specify the purpose of the trip (personal=0, business=1).",
{{flecsi::option_default, purpose_option::business}},
[](flecsi::any const & v) {
size_t value = flecsi::option_value<size_t>(v);
return value == personal || value == business;
});
/*
Add an option with no default. This will allow us to demonstrate testing an
option with has_value().
*/
flecsi::program_option<bool> lightspeed("Ride Options",
"lightspeed",
"Travel at the speed of light.",
{{flecsi::option_implicit, true}, {flecsi::option_zero}});
/*
Add a positional option. This uses a different constructor from the previous
option declarations. Positional options are a replacement for required
options (in the normal boost::program_options interface).
*/
flecsi::program_option<std::string> passenger_list("passenger-list",
"The list of passengers for this trip [.txt].",
1,
[](flecsi::any const & v) {
const std::string value = flecsi::option_value<std::string>(v);
return value.find(".txt") != std::string::npos;
});
/*
User-defined program options are available after FleCSI initialize has been
invoked.
*/
int
top_level_action() {
double price{0.0};
/*
Add cost for trim level. This option does not have to be checked with
has_value() because it is defaulted. It is also unnecessary to check the
value because it was declared with a validator function.
*/
price += trim.value() * 100.0;
/*
Add cost for automatic transmission.
*/
price += transmission.value() == "automatic" ? 200.0 : 0.0;
/*
Add cost for child seat.
*/
if(child_seat.value()) {
price += 50.0;
}
/*
Deduction for business.
*/
if(purpose.value() == business) {
price *= 0.8;
}
/*
Add cost for lightspeed. Since this option does not have a default, we need
to check whether or not the flag was passed.
*/
if(lightspeed.has_value()) {
price += 1000000.0;
}
/*
Do something with the positional argument.
*/
auto read_file = [](std::string const &) {
// Read passengers...
return 5;
};
size_t passengers = read_file(passenger_list.value());
price *= passengers * 1.10 * price;
std::cout << "Price: $" << price << std::endl;
return 0;
} // top_level_action
int
main(int argc, char ** argv) {
auto status = flecsi::initialize(argc, argv);
if(status != flecsi::run::status::success) {
return status == flecsi::run::status::help ? 0 : status;
} // if
status = flecsi::start(top_level_action);
flecsi::finalize();
return status;
} // main
Example 3: FLOG (FleCSI Logging Utility)¶
FLOG provides users with a mechanism to print logging information to various stream buffers, similar to the C++ objects std::cout, std::cerr, and std::clog. Multiple streams can be used simultaneously, so that information about the running state of a program can be captured and displayed at the same time. In this example, we show how FLOG can be configured to stream output to a file buffer, and the std::clog stream buffer.
Before attempting this example, you should make sure that you have configured and built FleCSI with ENABLE_FLOG=ON. Additional options that are useful are:
FLOG_ENABLE_COLOR_OUTPUT=ON
FLOG_ENABLE_TAGS=ON
FLOG_STRIP_LEVEL=0
Important
One of the challenges of using distributed-memory and tasking runtimes is that output written to the console often gets clobbered because multiple threads of execution are all writing to the same descriptor concurrently. FLOG fixes this by collecting output from different threads and serializing it. This is an important and useful feature of FLOG.
Buffer Configuration¶
By default, FLOG does not produce any output (even when enabled). In order to see or capture output, your application must add at least one output stream. This should be done after flecsi::initialize has been invoked, and before flecsi::start. Consider the main function for this example:
int
main(int argc, char ** argv) {
/*
If FLECSI_ENABLE_FLOG is enabled, FLOG will automatically be initialized
when flecsi::initialize() is invoked.
*/
auto status = flecsi::initialize(argc, argv);
if(status != flecsi::run::status::success) {
return status == flecsi::run::status::help ? 0 : status;
} // if
/*
In order to see or capture any output from FLOG, the user must add at least
one output stream. The function log::add_output_stream provides an
interface for adding output streams to FLOG. The FleCSI runtime must have
been initialized before this function can be invoked.
*/
/*
Add the standard log descriptor to FLOG's buffers.
*/
log::add_output_stream("clog", std::clog, true);
/*
Add an output file to FLOG's buffers.
*/
std::ofstream log_file("output.txt");
log::add_output_stream("log file", log_file);
status = flecsi::start(top_level_action);
flecsi::finalize();
return status;
} // main
The first output stream added is std::clog.
/*
Add the standard log descriptor to FLOG's buffers.
*/
log::add_output_stream("clog", std::clog, true);
The arguments to add_output_stream are:
label (“clog”):
This is an arbitrary label that may be used in future versions to enable or disable output. The label should be unique.stream buffer (std::clog):
A std::ostream object.colorize (true):
A boolean indicating whether or not output to this stream buffer should be colorized. It is useful to turn off colorization for non-interactive output. The default is false.
To add an output stream to a file, we can do the following:
/*
Add an output file to FLOG's buffers.
*/
std::ofstream log_file("output.txt");
log::add_output_stream("log file", log_file);
That’s it! For this example, FLOG is now configured to write output to std::clog, and to output.txt. Next, we will see how to actually write output to these stream buffers.
Writing to Buffers¶
Output with FLOG is similar to std::cout. Consider the FLOG info object:
flog(info) << "The value is " << value << std::endl;
This works just like any of the C++ output objects. FLOG provides four basic output objects: trace, info, warn, and error. These provide different color decorations for easy identification in terminal output, and can be controlled using strip levels (discussed in the next section).
The following code from this example shows some trivial usage of each of the basic output objects:
/*
This output will always be generated because it is not scoped within a tag
guard.
*/
flog(trace) << "Trace level output" << std::endl;
flog(info) << "Info level output" << std::endl;
flog(warn) << "Warn level output" << std::endl;
flog(error) << "Error level output" << std::endl;
Controlling Output - Strip Levels¶
Important
If FleCSI is configured with ENABLE_FLOG=OFF, all FLOG calls are compiled out, i.e., there is no runtime overhead.
The strip level is a preprocessor option FLOG_STRIP_LEVEL that can be specified during FleCSI configuration. Valid strip levels are [0-4]. The default strip level is 0 (most verbose). Depending on the strip level, FLOG limits the type of messages that are output.
trace
Output written to the trace object is enabled for strip levels less than 1.info
Output written to the info object is enabled for strip levels less than 2.warn
Output written to the warn object is enabled for strip levels less than 3.error
Output written to the error object is enabled for strip levels less than 4.
Controlling Output - Tag Groups¶
Tag groups provide a mechanism to control the runtime output generated by FLOG. The main idea is here is that developers can use FLOG to output information that is useful in developing or debugging a program, and leave it in the code. Then, specific groups of messages can be enabled or disabled to only output useful information for the current development focus.
To create a new tag, we use the log::tag type:
/*
Create some tags to control output.
*/
log::tag tag1("tag1");
log::tag tag2("tag2");
Tags take a single std::string argument that is used in the help message to identify available tags.
Important
FLOG tags must be declared at namespace scope.
Once you have declared a tag, it can be used to limit output to one or
more scoped regions. The following code defines a guarded section of
output that will only be generated if tag1 or all is specified to
the --flog-tags option:
/*
This output will only be generated if 'tag1' or 'all' is specified
to '--flog-tags'.
*/
{
log::guard guard(tag1);
flog(trace) << "Trace level output (in tag1 guard)" << std::endl;
flog(info) << "Info level output (in tag1 guard)" << std::endl;
flog(warn) << "Warn level output (in tag1 guard)" << std::endl;
flog(error) << "Error level output (in tag1 guard)" << std::endl;
} // scope
Here is another code example that defines a guarded section for tag2:
/*
This output will only be generated if 'tag2' or 'all' is specified
to '--flog-tags'.
*/
{
log::guard guard(tag2);
flog(trace) << "Trace level output (in tag2 guard)" << std::endl;
flog(info) << "Info level output (in tag2 guard)" << std::endl;
flog(warn) << "Warn level output (in tag2 guard)" << std::endl;
flog(error) << "Error level output (in tag2 guard)" << std::endl;
} // scope
You should experiment with invoking this example:
Invoking this example with the --help flag will show the available
tags:
$ ./flog --help
which should look something like this:
Usage: flog
Basic Options:
-h [ --help ] Print this message and exit.
FleCSI Options:
--flog-tags arg (=all) Enable the specified output tags, e.g.,
--flog-tags=tag1,tag2.
--flog-verbose [=arg(=1)] (=0) Enable verbose output. Passing '-1' will
strip any additional decorations added by
flog and will only output the user's message.
--flog-process arg (=-1) Restrict output to the specified process id.
Available FLOG Tags (FleCSI Logging Utility):
tag2
tag1
Invoking this example with --flog-tags=tag1 will generate output for
unguarded sections, and for output guarded with the tag1 tag:
$ ./flog --flog-tags=tag1
[trace all p0] Trace level output
[info all p0] Info level output
[Warn all p0] Warn level output
[ERROR all p0] Error level output
[trace tag1 p0] Trace level output (in tag1 guard)
[info tag1 p0] Info level output (in tag1 guard)
[Warn tag1 p0] Warn level output (in tag1 guard)
[ERROR tag1 p0] Error level output (in tag1 guard)