QGIS 3.20 | LUCCHESE, L. V.

Solved: QGIS 3.20 Python Console shows an error when running Clip Vector by Extent tool

Mon, 22 Nov 2021 12:43:11 -0300

Fellow researchers and open-source GIS enthusiasts,

Welcome to my blog!

I’d like to start with a disclaimer – I may be a researcher of this very area but that doesn’t mean everything I do or write here will work for you, in your own desktop configurations and package versions. I have no responsibility if you lose data or mess up your installation. I also do not authorize any copies of my content.

Today, I am writing about an error I obtained while using QGIS 3.20 Python Console. I was trying to clip a vector layer populated with points I had just created, by an extent of the type $xmin,xmax,ymin,ymax$, using the tool Clip Vector by Extent, from GDAL.

The error

First, I ran the following line, to set the tool parameters:

paramgdal= {'INPUT': points_layer, 'EXTENT':extent_final, 'OUTPUT':output_path}

The variables “extent_final” and “output_path” were previously set. “points_layer” is the vector layer of the type shapefile in which the points I want to clip by the extent are stored. It was loaded previously, using the following snippet:

points_layer = QgsVectorLayer(path_to_file, 'points_loaded', 'ogr')

Well, so far, so good.

But, after setting the parameters, I ran:

processing.run("gdal:clipvectorbyextent", paramgdal)

and then, this error appeared:

ERROR 1: Attempt to write non-multipoint (POINT) geometry to multipoint shapefile.
ERROR 1: Unable to write feature 0 from layer points_layer.
ERROR 1: Terminating translation prematurely after failed translation of layer points_layer (use -skipfailures to skip errors)

And the resulting layer was not correctly generated.

Interpretation of the issue

Re-reading the error message, I noticed that GDAL, for some reason, tries to save the generated layer as a Multipoint shapefile, instead of, simply, a Point shapefile. For this reason, it is unable to write any of the features from the original layer on the new one, because their types do not match.

Solution

Checking on the QGIS documentation we can notice that there are four possible parameters to run gdal:clipvectorbyextent. One of them, “OPTIONS”, is Optional, and refers to GDAL creation options. Because the algorithm Clip Vector by Extent is based on the GDAL utility ogr2ogr, the options available for the original algorithm can be used to build the new layer through the Python Console as well.

Especially, one of the options is useful in this scenario. The “nlt” option of ogr2ogr defines the geometry of the generated file, which is just what we were looking for. In the case here shown, I simply added

'OPTIONS': '-nlt point'

to the $gdalparams$ Python dictionary. This tells GDAL that the layer to be created is a point layer, and it solved my issue completely. The code snippet shown at the start of this post was adapted by me as:

paramgdal= {'INPUT': points_layer, 'EXTENT':extent_final, 'OPTIONS': '-nlt point', 'OUTPUT':output_path}
processing.run("gdal:clipvectorbyextent", paramgdal)

And when I ran the snippet above, the clipped point layer was correctly generated, without any errors or warnings.

A simple fix!

Extras

Why don’t you simply use GDAL from the command line, or click on the tool through the Processing Toolbox and use it directly from the GUI? Why do you want to use the Python Console on QGIS?

Because sometimes the Python Console is the most practical way to perform a calculation, and this code snippet shown here is a small part of a large code. For this motive, it is not worth it to change the whole interface or language in which we are developing code.

I loaded my vector file using the QGIS Python Console, but it did not appear on the layers list on QGIS! What happened?

This is not an issue. The code snippet used to load the vector layer does not list the layer on QGIS Layers panel, nor shows it in the map visualization. To do that, you should run

QgsProject.instance().addMapLayers([points_layer])

I’m not sure what I should import before using the tool “gdal:clipvectorbyextent” on QGIS Python Console.

You should import the QGIS algorithms and the processing tool. See the snippet below:

from qgis.core import *
import processing

Sampling raster values in predetermined points on QGIS

Mon, 04 Oct 2021 10:10:08 -0300

Fellow researchers and open-source GIS enthusiasts,

Welcome to my blog!

Today, I will write about sampling the values of rasters in given points. This may be useful for a range of applications, especially statistical analyses.

In this post, I show how to perform this process using two different tools, one of them is native of QGIS, and the other is a plugin available on the plugin repository. Then, I comment on how to sample the original raster using an interpolation such as cubic, cubic spline, or bilinear for the sampled value.

For the example here shown, I am using the same raster dataset I used for the talk on QGIS Open Day Brazil September/2021, available at https://github.com/luisalucchese/QGISday_092021.

It is based on an “imaginary” Digital Elevation Model (DEM), created by using dem-utils code, from ArMoraer on GitHub. All projections attributed to layers are only for example processing, but the DEM does not portrait a real part of the globe.

I open the rasters on QGIS 3.20 “Odense” and see this:

Then, just for this example, I create a point layer and add three points in it.

Now, there are two main ways to sample the raster in these points.

Using the native tool “Sample raster values”

Find the tool “Sample raster values” or “native:rastersampling” in your Processing Toolbox.

Using this algorithm, you should sample each raster layer individually.

I changed the Output column prefix to “DEM_” because I will have some more rasters to sample later.

Clicking Run, it ran without issues.

Opening the attribute table of my new vector, I find:

The column was given the suffix “1”, probably because it was the first and only band of the raster.

Zooming into the raster cell in which each point is located, I find the values:

The values for the cell in which each point is contained are the same as those extracted by the algorithm. This means that the value is simply extracted from the raster, and no interpolation is performed. This is the same as saying that the tool performed a Nearest Neighbor interpolation.

To extract the values of all four rasters, run the tool three more times. In each run, select the input as the point output from the last run, but select a different raster. When the process was completed, I opened the attribute table of the resulting layer:

This process gets tedious fast if you have many raster layers from which to extract information. And in-built batch processing is not really an option, since you need the output of the last run as an input on the current one. But an option to deal with that would be to build a multiband raster from the rasters you have in separate layers and then run the algorithm.

Using the plugin “Point Sampling Tool”

The plugin “Point Sampling Tool” can be found in the plugin manager of QGIS. I am using the version 0.5.3 of this plugin.

When I open the plugin, I see this:

Select more than one raster by holding Ctrl while clicking the layers’ names.

In this plugin, it is not possible to save a temporary file.

I select the output path on “Browse” and click OK.

The plugin shows this warning “All layers must have the same coordinate refere system. The <layer name> layer seems to have different CRS id (0) than the point layer (32722) […]”

In this case, I am using DEMs created only for example purposes, and I am sure they all “descend” from the same raster, so I clicked “Yes”. However, you should analyze on a case-by-case basis. Usually, it is better to reproject either the raster or the point layer.

Opening the attribute table of the resulting raster:

The values are the same as those acquired by the native tool, only with less significant figures.

They are also acquired with no interpolation a.k.a. Nearest Neighbor.

The names of the columns are based on the names of the layers in your QGIS project.

Extras

- Which one should I use?

Both tools work very well. In my opinion, it depends on the number of raster layers you have to extract information from. If it is only one or two rasters, I would use the native tool, but if you have more than that, I think using the plugin pays off.

- You mentioned that both point samplers use the Nearest Neighbor interpolation. How to use another type of interpolation?

Well, you can’t. But there is a workaround for that. Using “Warp (reproject)” (“gdal:warpreproject”), you can resample your raster layers to a smaller pixel size using a different type of interpolation and then run any of the sampling tools I just showed here.

Example:

Original raster:

Choosing the “Cubic” interpolation and giving the output raster a smaller pixel size, this is the resulting raster:

Now, if I extract the value of this point it will be 61.9293 instead of 61.7967. In this case, in which the original pixels were not too large neither did they differ a lot in value, the difference on the extracted value is in the order of $10^{-1}$, but it can be larger depending on the case. It is better to examine the nature of your raster and analyze it to see if you should resample your raster as a pre-processing step.

Testing some of my favorite changes in QGIS 3.20 Odense

Mon, 21 Jun 2021 16:57:19 -0300

Fellow researchers and open-source GIS enthusiasts,

Welcome to my blog!

Today QGIS 3.20 “Odense” was released!

In today’s post, I will test some of my favorite features of the new version.

To check all the new features and the bugs fixed, check the full Changelog of QGIS 3.20.

Opening an attribute table with just the selected features

Especially useful when dealing with shapefiles with multiple polygons, lines, or points.

I am testing this in a shapefile with the countries of the world.

Select the features

I will select all countries that are on these latitudes.

Resulting on this selection.

Open the atribute table

Click in Open Attribute Table (Selected Features)

Which opens an attribute table with just the selected countries:

If you want to see all features, just click the button “Show Selected Features” and click “Show All Features”.

Stream digitizing

QGIS now supports freehand drawing of lines and polygons.

In the advanced digitizing toolbar, select “Stream Digitizing”:

After clicking the first point, just move your mouse pointer around the screen to draw the line or polygon.

Snapping to endpoints

Like in Computer Aided Design (CAD) applications, now in QGIS you can snap to the endpoints of lines when drawing.

First, I draw one line (in green). Then, I start drawing another line.

Make sure that the Snapping Toolbar is activated in View, Toolbars, Snapping Toolbar. In the snapping toolbar, select the option “Line Endpoints”.

After selecting this option, when your cursor is close to a line endpoint, it should show a special marker.

Just click and the line is drawn.

Variable thickness of lines and painting them with a gradient

~~This feature is listed in QGIS 3.20 Changelog. Unfortunately, I could not make this work in QGIS 3.20.0 Odense.~~

~~I will update this post in case I manage to make it work.~~

Updated! June 22th, 2021

This looks especially useful for hydrology representation. Configure the streams to have thinner lines at one extremity and thicker at the other.

First, open the symbology panel

Click in Simple Line

Choose the type of symbol as Interpolated Line

This panel should open:

Troque o tipo de largura da linha

Switch from Fixed width to Varying width. This should appear:

If you click the selector on the right side of Start value, QGIS will offer you the option “id”. Do not click!

Actually, populate this field with a number. It should be a different number from the one you write on “End value”. If you write the same number the linewidth will not vary.

Write 0.1 in Start Value and 10 in End Value. Click in Apply.

Done! Variable width.

Up next, gradient color

On the dropdown menu, switch from Single color to Varying color. This panel should be opened:

Once more, do not select “id” as Start or End value. Type numbers instead.

I typed 0.1 as Start Value and 10 as End Value. I then clicked in Apply.

At this point, the line looks like this:

However, we can change the color gradient, in “Color ramp”. I chose the one named “Blues” because tones of blue should suit hydrology applications well.

Done!

These are my favorites so far!

If you work with vector labels, many new features on this matter were implemented and I suggest you take a look on the full log.

Is it worth to install QGIS 3.20 if QGIS 3.22 LTS is scheduled to be out in October?

I haven’t had serious bugs on QGIS 3.18 and it has the implementation of continuous raster legends, which I use all the time. I cannot imagine waiting until October to have continuous raster legends in Print Layout, when this feature already exists. I have been installing non-LTS versions of QGIS for over a year so far with little crashes. That said, people have different experiences, and installing a non-LTS version is considered risky by some.