.. _library_development: Library development =================== This section provides information on how to develop within the perimeter of the library and it’s documentation. If you are willing to contribute the actual development of the library, please consider reading our coding guidelines and contact us through the website. Install a development version ----------------------------- Install the required dependencies ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ See :ref:`dependencies` for the system requirements. Download openturns ~~~~~~~~~~~~~~~~~~ You can retrieve the development master branch through the git repository by issuing the following commands:: git clone https://github.com/openturns/openturns.git cd openturns Or, you can pick up a stable version:: git clone -b v1.22 https://github.com/openturns/openturns.git cd openturns Build openturns ~~~~~~~~~~~~~~~ CMake presets can be used on Linux, which sets debug build mode, warning flags, build dir, install prefix, etc:: cmake --preset=linux-debug cmake --build build --target install --parallel 4 Run tests ~~~~~~~~~ Python tests can be run once the bindings are finished compiled:: ctest --test-dir build -j4 -R pyinstallcheck C++ tests must be built prior to be launched:: cmake --build build --target tests --parallel 4 ctest --test-dir build -j4 -R cppcheck and all the tests should be successful else check the log file Testing/Temporary/LastTest.log. Adding a single class to an existing directory ---------------------------------------------- This how-to explains the process that must be followed to fully integrate a new class that provides an end-user facility (e.g. a new distribution). We suppose that this class will take place in an existing directory of the sources directories. First, add the class to the C++ library ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #. Create ``openturns/MyClass.hxx`` and ``MyClass.cxx`` in appropriate subdirectories of lib/src. The files must have the standard header comment, with a brief description of the class in Doxygen form and the standard reference to the LGPL license. For the header file ``MyClass.hxx``, the interface must be embraced between the preprocessing clauses: :: #ifndef OPENTURNS_MYCLASS_HXX #define OPENTURNS_MYCLASS_HXX BEGIN_NAMESPACE_OPENTURNS class OT_API MyClass { CLASSNAME; public: // default constructor MyClass(); ... }; END_NAMESPACE_OPENTURNS #endif // OPENTURNS_MYCLASS_HXX to prevent from multiple inclusions. See any pair of .hxx/.cxx files in the current directory and the PGQL document for the  coding rules: case convention for the static methods, the methods and the attributes, trailing underscore for the attribute names for naming a few. #. Modify the ``CMakeLists.txt`` file in the directory containing ``MyClass.hxx`` and ``MyClass.cxx``: - add ``MyClass.hxx`` to the headers using ``ot_install_header_file ( MyClass.hxx )``. - add ``MyClass.cxx`` to the sources using ``ot_add_source_file ( MyClass.cxx )``. #. Add ``MyClass.hxx`` to the file ``OTXXXXXX.hxx``, where ``XXXXXX`` is the name of the current directory. #. Create a test file ``t_MyClass_std.cxx`` in the directory lib/test. This test file must use the standard functionalities of the class MyClass. Keep in mind there are over a thousand unit tests, so they should be reasonably quick to run, lasting preferably less than 10 seconds. #. Create an expected output file ``t_MyClass_std.expout`` that contains a verbatim copy of the expected output (copy-paste the *validated* output of the test in this file). #. Modify the ``CMakeLists.txt`` file in lib/test: add ``ot_check_test ( MyClass_std )`` in this file. #. If the validation of your class involved advanced mathematics, or was a significant work using other tools, you can add this validation in the validation/src directory. - copy all of your files in the validation/src directory. - modify the validation/src/CMakeLists.txt file by appending the list of your files to the list of files to install. #. You may want to update the Changelog file to mention new classes, bug fixes... That’s it! Your class is integrated to the library and will be checked for non-regression in all the subsequent versions of OpenTURNS, assuming that your contribution has been incorporated in the “official”  release. But nobody can use it! Second, add your class to the Python interface ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #. Create MyClass.i in the python/src directory. In most situations, it should be: :: // SWIG file MyClass.i %{ #include "openturns/MyClass.hxx" %} %include MyClass_doc.i %include openturns/MyClass.hxx namespace OT { %extend MyClass { MyClass(const MyClass & other) { return new OT::MyClass(other); } } // MyClass } // OT #. Create MyClass\_doc.i.in docstring documentation in the python/src directory. This will be part of the HTML documentation generated by sphinx. Document every method of your class that’s not inherited. In most situations, it should look like this: :: %feature("docstring") OT::MyClass "MyClass class. Available constructors: MyClass() MyClass(*designPoint, limitStateVariable, isInFailureSpace*) Notes ----- Structure created by the method run() of a :class:`~openturns.Analytical` and obtained thanks to the method *getAnalyticalResult*. Parameters ---------- designPoint : float sequence Design point in the standard space resulting from the optimization algorithm. limitStateVariable : :class:`~openturns.RandomVector` Event of which the probability is calculated. isInFailureSpace : bool Indicates whether the origin of the standard space is in the failure space. Examples -------- >>> import openturns as ot >>> ot.RandomGenerator.SetSeed(0) >>> dp = ot.Normal().getRealization() >>> inst = ot.MyClass(dp, 4.8) >>> print(inst) >>> 4.5677..." // --------------------------------------------------------------------- %feature("docstring") OT::MyClass::foo_method "... " Beware that docstring tests run from the Examples section share the same environment and they can be affected by global settings such as RandomGenerator seed or ResourceMap entries, so for example here we reset the RNG seed prior to sampling to avoid affecting the expected result. Also, these examples must be very quick because they are run as batches per module, more expensive tests can be run in the dedicated unit tests. #. Modify the CMakeLists.txt file in python/src: add MyClass.i, MyClass\_doc.i.in to the relevant ``ot_add_python_module`` clause. #. Locate and modify the file yyyy.i, where yyyy is the name of the python module related to MyClass, to include MyClass.i in the correct set of .i files (see the comments in yyyy.i file). In order to identify the correct python module, remember that the modules map quite closely the source tree organization. #. Create a test file ``t_MyClass_std.py`` in the directory python/test. This test implements the same tests than ``t_MyClass_std.cxx``, but using python. Keep in mind there are over a thousand unit tests, so they should be reasonably quick to run, lasting preferably less than 10 seconds. #. Modify the CMakeLists.txt file in python/test: - add ``t_MyClass_std.py`` to the tests using ``ot_pyinstallcheck_test ( MyClass_std )``. Document your contribution more thoroughly ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If your class introduces important mathematical concepts or impacts the library architecture it may be useful to add some more details in the documentation, see :ref:`sphinx_doc`. That’s all, folks! Some timings from an  Guru: 2 days of work for the most trivial contribution (a copy-paste of a class with 5 methods, no mathematical or algorithmic tricks). For a well-trained  contributor, a user-visible class with a dozen of methods and well-understood algorithms, a new class should not be less than a week of work... Adding a set of classes in a new subdirectory --------------------------------------------- This how-to explains the process that must be followed to fully integrate a set of classes that provides an end-user facility (e.g. a new simulation algorithm) developed in a new subdirectory of the existing sources. The task is very similar to the steps described in the how-to, only the new steps will be described. We suppose that the subdirectory has already been created, as well as the several source files. There are three new steps in addition to those of the how-to: the creation of the cmake infrastructure in the new subdirectory, the modification of the infrastructure in the parent directory and the modification of the infrastructure in the root directory. CMake infrastructure in the parent subdirectory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You have to set up the recursive call of Makefiles from a parent directory to its subdirectories, and to aggregate the libraries related to the subdirectories into the library associated to the parent directory: #. add NewDir subdirectory to the build: :: add_subdirectory (NewDir) CMake infrastructure in the new subdirectory ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You have to create a CMakeLists.txt file. Its general structure is given by the following template: :: # -*- cmake -*- # # CMakeLists.txt # # Copyright 2005-2023 Airbus-EDF-IMACS-ONERA-Phimeca # # This library is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This library is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Lesser General Public License for more details. # # You should have received a copy of the GNU Lesser General Public License # along with this library. If not, see . # # Register current directory files ot_add_current_dir_to_include_dirs () ot_add_source_file (FirstFile.cxx) # ... ot_add_source_file (LastFile.cxx) ot_install_header_file (FirstFile.hxx) # ... ot_install_header_file (LastFile.hxx) # Recurse in subdirectories add_subdirectory (FirstDir) # ... add_subdirectory (LastDir) Version control --------------- The versioning system used for the development of the whole platform is Git. The git repositories are hosted at `Gihub `_ where sources can be browsed. .. figure:: Figures/BrowseSource.png :alt: GitHub interface: the source browser Bug tracking ~~~~~~~~~~~~ GitHub’s tracker is called *Issues*, and has its own section in every repository. The snapshot of the library `bug-tracker `_ shows the list of active tickets: .. figure:: Figures/Tickets1.png :alt: GitHub interface: the ticket browser Each ticket features attributes to help classification, interactive comments and file attachment. This snapshot exposes the details of a ticket: .. figure:: Figures/Tickets2.png :alt: GitHub interface: details of a ticket report Other requirements ------------------ Namespace ~~~~~~~~~ All the classes of the library are accessible within a single namespace named OT and aliased as OpenTURNS. It allows one to insulate these classes from classes from another project that could share the same name. Macros are provided to enclose your code in the namespace as follow: :: BEGIN_NAMESPACE_OPENTURNS // code END_NAMESPACE_OPENTURNS Internationalization ~~~~~~~~~~~~~~~~~~~~ The platform is meant to be widely distributed within the scientific community revolving around probability and statistics, which is essentially an international community. Therefore, the platform should be designed so as to be adjustable to the users, particularly those who do not speak English [1]_. This involves not using any messages directly in the source code of the platform, but rather to create a resource catalogue that can be loaded, according to the locale setting of the user, when the application is launched. Another consequence of internationalization is the need for the Unicode extended character set to be used for all strings. Accessibility ~~~~~~~~~~~~~ The platform shall be accessible to disabled users. This has implications on the ergonomics and the design of the User Interface, particularly the GUI which should offer keyboard shortcuts for any available function as well as keyboard-based (rather than mouse-based) mechanisms to handle and select objects. Profiling ~~~~~~~~~ `Flame Graphs `_ can help visualize where your functions spends the most time. Here are some commands to profile your code paths using the `perf `_ tool and generate the associated graph with `FlameGraph `_. .. figure:: Figures/perf_welch.png :alt: Flame graph of WelchFactory First retrieve the graphing scripts: :: git clone https://github.com/brendangregg/FlameGraph.git /tmp/FlameGraph You will need to build without parallelization and with debug flags: :: cmake -DUSE_TBB=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CXX_FLAGS="-fno-omit-frame-pointer" . You will also want to disable openblas threads or openmp at any other level: :: export OMP_NUM_THREADS=1 Now you are ready to profile your executable: :: perf record --call-graph dwarf -o /tmp/perf.data ./lib/test/t_WelchFactory_std Some Linux distros prevent normal users from collecting stats, in that case: :: # echo "-1" > /proc/sys/kernel/perf_event_paranoid # echo 0 > /proc/sys/kernel/kptr_restrict At this point you should be able to generate the graph from the perf data: :: perf script -i /tmp/perf.data | /tmp/FlameGraph/stackcollapse-perf.pl | /tmp/FlameGraph/flamegraph.pl > /tmp/perf.svg .. [1] English has been chosen as the native language for the platform.