3. Modeller Interface (QGIS Plugin)¶
The 3Di Toolbox is a QGIS plugin for working with 3Di models and netCDF results. The newest plugin (release of March 4th) is only supported by QGIS 3.4.5. An older version of the plugin will remain available for QGIS 2.18. For more information on installing the plugin see 3Di Toolbox plug-in. For more information on viewing and editing 3Di models in QGIS see Adjusting the 3Di model database in QGIS. The section below explains the use of the water balance tool and the raster checker. More subjects will be added soon, as these are only a few of the features of the 3Di Toolbox.
3.2. Installation of plugin¶
The 3Di Toolbox only works for:
- QGIS 2.18 (until february 2019)
- QGIS 3.4.5 (LTR after february 2019)
- 64-bit version of QGIS (see below for more details)
- 3Di v2 results
To install the 3Di-Toolbox plugin follow the steps below:
- Open QGIS and via the menu bar go to ‘Plugins > Manage And Install Plugins’.
- Go to ‘Settings’.
- Check the box for ‘Show also experimental plugins’
- Add a plugin repository
- Fill in a name and copy the URL: https://plugins.lizard.net/plugins.xml into the URL box.
- Go to ‘All’ and choose ‘3Di toolbox’ from the list
- Install the plugin.
3.3. Overview of the plugin¶
After installation of the plugin a toolbar is added to the QGIS interface. The different tools are explained below.
3.4. Load 3Di model and results¶
A model can be loaded by clicking the database icon with the blue plus-sign. A new window will be opened.
- Under ‘Model’ you need to load the Sqlite containing your model
- Under ‘Results’ you can load the NetCDF containing your simulation results (usually named results_3di.nc). It is important to select a result file that is actually belonging to the model you used for your simulation (i.e. your NetCDF must be generated by the sqlite you loaded. Do not use an old or changed Sqlite).
- After the loading finished, click ‘Close’ to return to the QGIS interface
Note: it is not necessary to load results. It is also possible to load a model without results, e.g. for checking and editing the schematization in your Sqlite. To do so, skip step 2.
3.5. Toolbox for working with 3Di models¶
The 3Di toolbox is actived by clicking the toolbox icon in the 3Di-Toolbox bar.
After clicking the toolbox icon, a new window in QGIS is opened. Click the arrow next to the Tools icon to open the toolbox and view the different tools that are available. In time, the descriptions of the various tools will be explained.
The Rasterchecker is launched with the QGIS 3.4.5 version of the Plugin. It is a tool to check the rasters that are used in your 3Di model for consistency. The tool verifies for example
- The correct nodata value
- Consistent projection
- That all rasters are aligned
There are up to 18 checks performed, which are listed below. It is strongly recommended to run this tool before updating the model repository, as it will crash when it encounters any errors in your rasters. It will prevent a failed input generation.
Before the Rasterchecker can be used, you first need to make a connection with the SQlite of your model.
- Open the Data Source Manager under the drop down menu Layer on top of the screen.
- Go to SpatiaLite and click New. Browse to the location of your model Sqlite and open it.
- Now you can close the Data Source Manager window.
- The Rasterchecker can be accessed by opening the Toolbox.
- The Rasterchecker can be found under Step 1 - Check data. By double clicking raster_checker.py the Rasterchecker is opened in a seperate window.
- Under Model schematisation database you can choose the spatialite of your model.
- Click OK to start the rasterchecker. When the tool is finished the following message pops up:
- The log-file of the rasterchecker can be found at the same location as the location of the SQlite. The log-file can be opened with a text editor such as Notepad. The log-file looks similar to:
- The various raster characteristics that are verified are numbered 1 to 18. When we refer to this number, it is called check_id.
- Under subheading ‘Found following raster references’ the rasters used in your model are stated.
Further down in the log-file, the information for each raster is listed.
- The first column (level) shows the importance of the notification (info, warning or error).
- The second column (setting_id) shows the id of the row in the v2_global_settings table of the sqlite, where the raster reference can be found.
- The third column contains the check_id.
- The fourth column (feedback) shows the outcome of the check.
- If one of your rasters is not aligned with the DEM, check_id 18 will give an error. Make sure all your rasters have the same extent and and have nodata pixels at the same location.
The schematization checker analyses your 3Di model database (.sqlite file) for completeness and consistency between tables. With the checker you can make sure most model errors are caught before sending the model to the INP server for model generation.
- Start QGIS
- Add a connection to the model database (Layer -> Data Source Manager, Select SpatiaLite on the left and create a ‘New’ connection or connect to an existing connection)
- Open the schematization checker by opening the Toolbox in the 3Di Plugin, select Step 1: check data, select schematisation_checker.py
- Select the SpatiaLite connection of the model database and the location where to store the output of the schematisation checker. Click run to run the schematisation checker. Click open to open the output.
The output is a comma seperated value file which can be opened in Excel. It contains 6 columns: id, table, column, value, description and check:
- id: identification number of the row where a check encounters an error.
- table: the table in which the error occurs.
- column: the column which contains the error.
- value: the current value in the cell
- description: description of the error
- check: the type of check that found the error, described below
220.127.116.11. What is checked?¶
There are currently different general checks applied on all tables and columns of the model database. These checks are:
Apart from the general checks on the database data and structure there are more 3Di specific checks:
Every cell in every table will be checked if the type of the entered value is correct. A cell is expected to either contain:
- integer (-4, 0,1,2, etc…)
- real (3.6, -5.2)
- varchar (text of limited length)
- geometry (point, linestring or polygon)
- bool (true or false)
- datetime (2019-07-02 14:27+02:00)
Some cells expect specific values. For example, the type of a boundary condition is either 1, 2, 3 or 5 (respectively water level, velocity, discharge or Sommerfeld). Any value other than the enumerated values will result in an EnumCheck error.
If a cell is NULL it does not have a value at all. For some cells this is allowed, but others cells are obliged to contain a value. If this obligation is not met, a NotNullCheck error is given.
n.b. An empty text or varchar does not equal NULL.
Many tables contain foreign key columns which refer to other tables. An example is the column connection_node_start_id in the table v2_channel. This column refers to the column id in the table v2_connection_node. If a channel is entered with connection_node_start_id = 1, there should be an entry in the table v2_connection_nodes with id = 1. If this is not the case a ForeignKeyCheck error will be given.
Some values have to be unique. An example is the name column in v2_global_settings. If multiple rows are entered with the same name, a UniqueCheck error will be given.
If an entered geometry is invalid the GeometryCheck error will be returned. The most occurring reason for invalid geometries is self-intersection of polygons.
This check makes sure the geometry type (point, linestring or polygon) is consistent with the expected geometry type.
Check if the bank_level of v2_cross_section_locations is not NULL when the channel it is connected to is connected or double_connected.
Check if all cross section shapes are valid. Depending on the type of shape the definition has to follow certain requirements:
- Rectangle: A width is required, a height is optional. All supplied dimensions should be positive decimal numbers.
- Circle: Only a width is required, should be a positive decimal number.
- Egg: Only a width is required, should be a positive decimal number.
- Tabulated rectangle or trapezium: A list of widths and heights are required. The lists should contain only positive decimal numbers seperated by spaces and contain the same amount of values. The first value of height should always be 0. The list of heights should be in increasing order.
Check if timeseries are correctly entered and the timesteps are the same as all other timesteps in the same table.
If 0D flow is configured in the global settings, there should be at least 1 (impervious) surface in the model.
3.5.3. Import from SUF-HYD¶
SUF-HYD is a Dutch standardized format for transferring data of sewerage systems for hydraulic analyses.
Before you can use the tool, make sure you have downloaded an empty spatialite to import the data from the SUF-HYD to. Save the Sqlite to a location on your laptop you want to work in.
The tool can be accessed by activating the toolbox and double clicking ‘import_sufhyd.py’ under ‘Step 2 - Convert and import data’
- First, make sure you have a connection with the sqlite you want to import your data to (see the first 3 steps under Rasterchecker).
- After opening the tool, select a sufhyd file and the database (sqlite) to import the data into and click ‘OK’
The data from the sufyd will be loaded into the sqlite. A log file of this process is written to the folder where your sufhyd is located. This file has the name of your sufhyd with a .hyd.log extension. You can open this log file with a text editor such as Notepad. This log-file gives a summary of data errors and warnings.
The following objects are imported:
- Manhole (
- The number of inhabitants will be added as Impervious surface.
- Attention: the shape of the manhole is refered as ‘rnd’ = round, ‘sqr’ = square and ‘rect’ = rectangle
- Manhole (
- Pipe (
- The number of inhabitants will be added as Impervious surface
- Pipe (
- Pumpstation (
- If multiple stages are defined, this will be transformed into seperate pumpstations. Up to 10 stages are supported
- Pumpstation (
- Weir (
- Flow direction (str_rch) is translated into discharge coefficients with a value of 0
- An end node with boundary condition is not automatically added.
- Weir (
- Orifice (
- Flow direction (str_rch) is translated into discharge coefficients with a value of 0
- Orifice (
- Boundary (
- The water level will be the average definition (bws_gem). If not present the summer water level is used and otherwise the winter water level.
- Boundary (
- Extra manhole storage (
- The defined storage area is added to a manhole on the bottomlevel of the manhole. The defined bottom_level of the storage (niv_001) is ignored.
- Only one storage area is supported
- Extra manhole storage (
- Drainage area/ Impervious surface (``*AFV``)
- Linkage nodes (
- The ‘fictive’ linkages (with typ_gkn == 01) are ignored, only real nodes are combined.
- The second node (ide_kn2) is removed. Impervious surfaces and pipes linked to the removed node are redirected to the first node. Extra manhole storage will be lost.
- Linkage nodes (
3.6. Water Balance Tool¶
The water balance tool computes the water balance in a sub-domain of your model. It uses the incoming and outgoing flows in that domain and visualizes the various contributions of the flow in graphs. The development was an initiative of Deltares and jointly developed with Nelen & Schuurmans. The water balance tool is co-funded by the Top Sector Water (Ministry of Economic Affairs)
3.6.1. Settings to use the water balance tool¶
To be able to use the water balance tool, aggregated results are required for a range of variables. This to ensure, that the shown water balance is consistent and complete.
The aggregation settings can be found and configured in the spatialite-table v2_aggregation_settings. For more information on the aggregation settings, see Aggregated output. The default settings for the water balance tool are listed below.
Of course, the time step, cq, the period over which is aggregated, is adjustable. For new models, these settings are included in the empty spatialite database (Empty database). For existing models, these settings must be added to the v2_aggregation_settings -table. These SQL queries will help you in doing so:
Empty v2_aggregation_settings table:
DELETE FROM v2_aggregation_settings;
Add aggregation settings one by one:
INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (1, 9999, 'pump_discharge_cum', 'pump_discharge', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (2, 9999, 'lateral_discharge_cum', 'lateral_discharge', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (3, 9999, 'simple_infiltration_cum', 'simple_infiltration', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (4, 9999, 'rain_cum', 'rain', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (5, 9999, 'leakage_cum', 'leakage', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (6, 9999, 'interception_current', 'interception', 'current', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (7, 9999, 'discharge_cum', 'discharge', 'cum', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (8, 9999, 'discharge_cum_neg', 'discharge', 'cum_negative', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (9, 9999, 'discharge_cum_pos', 'discharge', 'cum_positive', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (10, 9999, 'volume_current', 'volume', 'current', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (11, 9999, 'qsss_cum_pos', 'surface_source_sink_discharge', 'cum_positive', 'FALSE', 300); INSERT INTO v2_aggregation_settings( id, global_settings_id, var_name, flow_variable, aggregation_method, aggregation_in_space, timestep) VALUES (12, 9999, 'qsss_cum_neg', 'surface_source_sink_discharge', 'cum_negative', 'FALSE', 300);
Note, that in both cases, in case of a new model or an existing model, you must update the global settings id to the id of the scenario for which you wish to generate aggregated results. For multiple scenarios, you must add these settings multiple times (and update row id’s). Also, you may choose to change the aggregation time step, but make sure to use the same time step for all aggregation variables in case one wants to use the water balance tool.
3.6.2. Using the water balance tool¶
In a few steps, one can get insight in the water balance of their own system.
- Define a spatialite and the results that are to be analysed by loading your model and results using the ‘Select 3Di results’-button in the toolbox.
- The water balance tool is activated by clicking the balance icon in the 3Di-Toolbox bar.
In case, the aggregated results are missing or incomplete the following error pops up:
- Draw a polygon to define the domain of the model for the area of interest. This can be done by clicking at multiple locations within the model domain. Click Finalize polygon to finish the polygon. The graph shows the water balance over time for the selected area.
- By right-clicking the graph, a menu appears in which the range of the x-axis and y-axis can be adjusted. The visible x-axis determines the period over which the water balance is calculated.
- The button Hide on map the polygon over which the water balance is calculated is hidden.
3.6.3. Display settings¶
- The different colours show the different flow types, explained in the legend on the right.
- By hovering over a flow type in the legend, the corresponding plane lights up in the graph and the corresponding flow lines will be marked with red dotted lines in the map of the model.
- The different flow types can be activated and deactivated in the graph by clicking the box next to the flow type name.
- All flow types can be activated or deactivated using the buttons activate all and deactivate all.
- In the water balance menu different display options can be chosen. In the first drop-down menu (default = ‘1d and 2d’) you can choose to display only 1D-flow (‘1d’) or 2D-flow (‘2d’) or both (‘1d and 2d’).
- In the second drop-down menu (default = ‘everything’) you can choose to display all flows (‘everything’) or only the main flows (‘main flows’).
- In the last drop-down menu (default = ‘m3/s’) you can choose to display flow (‘m3/s’) or cumulative volume (‘m3’).
Note: the different flow types are ‘stacked’ in the graph. This means the flow volumes are added to each other when activating multiple flow types.
Volume change is shown in the graph as well. In this case, the volume change is the result of the total positive and negative flow (inflow and outflow of the area). The volume change is not stacked but shown as a separate line in the graph.
3.6.4. Total balance¶
- By clicking the button Show total balance a new screen will pop-up, showing the total volume balance over the period set on the x-axis of the graph (shown in title).
- To adjust this period, close the screen with the bar diagrams, right click on the water balance graph, open the option x-axis, activate the option manual and set the minimum and maximum time. Then, click again on Show total balance to create the water balance diagrams for the new time range.
The top diagram shows the net water balance from all domains. The bottom diagrams show the water balance per domain.
It is possible to save the graphs as an image or export the water balance data to a CSV-file.
- To save an image of the graphs, right-click on one of the graphs. Choose ‘Export’ in the menu that opens. A new window opens.
- In the first box you can choose the items you want to export. Click ‘Entire Scene’ to export all graphs or choose one of the ‘Plot’-items to export a graph seperately.
- In the second box you can choose the export format. Choose ‘Image file’ for an image and choose ‘CSV from plot data’ to export the actual data.
- Click ‘Export’ to generate your figure.
3.6.5. Explanation of flow types¶
In the overviews the flow is split in several domains. These distinguish themselves based on how the flow is computed. Therefore, you will find the 2D flow, groundwater and the 1D flow domain. Below a more detailed doscription of the various components.
2D Surface water domain
- 2D Boundary flow: Inflow and outflow through 2D boundaries
- 2D Flow: Inflow and outflow in the surface domain crossing the borders of the polygon
- Lateral flow to 2D: Sources or sinks based on 2D laterals
- 2D: 2D flow to 1D: Flow exchange between the 2D surface domain and the 1D network elements within your polygon (for example, surface run-off from rain into a 1D-channel or water that overflows the banks in your channel).
- 2D: 2D flow to 1D (domain exchange): Flow exchange between the 2D surface domain and the 1D network elements crossing the borders of your polygon
- In/exfiltration (domain exchange): Flow exchange between the 2D surface domain and the 2D groundwater domain
- Rain: Incoming water from rain
- Constant infiltration: Flow out of the 2D domain based on simple infiltration
- Interception: Intercepted volume
2D Groundwater domain
- Groundwater flow: Inflow and outflow through the 2D groundwater domain crossing the borders of your polygon
- In/exfiltration (domain exchange): Flow exchange between the 2D surface domain and the 2D groundwater domain (generally inflowing water through infiltration).
- Leakage: sources or sinks based on leakage
1D Network domain
- 0D Rainfall runoff on 1D: Inflow volume from 0D module
- 1D Boundary flow: Inflow and outflow over a 1D boundary
- 1D Flow: Inflow and outflow in 1D network elements crossing the borders of your polygon
- 1D Laterals: Sources and sinks based on 1D laterals
- 1D: 2D flow to 1D: Flow exchange between the 2D surface domain and the 1D network elements (e.g. surface runoff from rain into a 1D-channel) within your polygon
- 1D: 2D flow to 1D (domain exchange) Flow exchange between the 2D surface domain and the 1D network elements crossing the borders of your polygon
- Pump: pumped volume
3.7. Show 3Di results in a graph¶
The graph tool can be used for visualizing model results over time. This can be done for both calculation points and flowlines. In this way you can for example see the variation in water level in a node or the variation in discharge through a flow link (e.g. a channel or pipe) over time
- First, make sure you have loaded a model and the corresponding results (NetCDF) into your QGIS project using Load 3Di model and results.
- Activate the graph tool by clicking the graph button in the 3Di toolbar. A new panel with the title 3Di result plots is launched in your QGIS-project.
- In the layer overview window go to the layer group results: results_3di and activate the ‘flowlines’ layer or the ‘nodes’ layer:
- Activate the Select features tool in QGIS. This can be done by clicking this logo in the Attributes toolbar from QGIS:
- Select the nodes or flowlines from which you want to view the results. You can select multiple nodes or flowlines simultaneously, but for speed purposes it is advised to add a maximum of 20 features to your graph.
- Click the ‘Add’ button in the 3Di results plot panel to add the nodes or flowlines results to the graph. The results for the selected features are loaded from the NetCDF and visualized over time in the graph:
- You can switch between node results and flowline results by activating the tab Q graph for flowlines and H graph for nodes.
- In the drop-down menu on the right side of the panel you can choose the type of results you want to see. The y-axis shows the corresponding range and unit of the results type. The x-axis shows the time. Note: the time is often displayed in kiloseconds (ks). 1 ks = 1000 seconds ≈ 16.7 minutes.
- Below the drop-down menu there is an overview of the nodes/flowlines you selected, with the id of the node/flowline and the type. In this overview you can activate or deactivate the results in the graph by clicking the checkbox next to it. A feature can be deleted by first selecting it in this overview and then click the Delete button below the overview.
- The data from the graph can also be exported to an image or csv-file. Right-click the the graph figure and choose ‘Export’ from the drop-down menu. A new window pops-up in which you can choose the output format and settings.