Review of Igalia’s Graphics activities (2018)

This is the first report about Igalia’s activities around Computer Graphics, specifically 3D graphics and, in particular, the Mesa3D Graphics Library (Mesa), focusing on the year 2018.

GL_ARB_gl_spirv and GL_ARB_spirv_extensions

GL_ARB_gl_spirv is an OpenGL extension whose purpose is to enable an OpenGL program to consume SPIR-V shaders. In the case of GL_ARB_spirv_extensions, it provides a mechanism by which an OpenGL implementation would be able to announce which particular SPIR-V extensions it supports, which is a nice complement to GL_ARB_gl_spirv.

As both extensions, GL_ARB_gl_spirv and GL_ARB_spirv_extensions, are core functionality in OpenGL 4.6, the drivers need to provide them in order to be compliant with that version.

Although Igalia picked up the already started implementation of these extensions in Mesa back in 2017, 2018 is a year in which we put a big deal of work to provide the needed push to have all the remaining bits in place. Much of this effort provides general support to all the drivers under the Mesa umbrella but, in particular, Igalia implemented the backend code for Intel‘s i965 driver (gen7+). Assuming that the review process for the remaining patches goes without important bumps, it is expected that the whole implementation will land in Mesa during the beginning of 2019.

Throughout the year, Alejandro Piñeiro gave status updates of the ongoing work through his talks at FOSDEM and XDC 2018. This is a video of the latter:

ETC2/EAC

The ETC and EAC formats are lossy compressed texture formats used mostly in embedded devices. OpenGL implementations of the versions 4.3 and upwards, and OpenGL/ES implementations of the versions 3.0 and upwards must support them in order to be conformant with the standard.

Most modern GPUs are able to work directly with the ETC2/EAC formats. Implementations for older GPUs that don’t have that support but want to be conformant with the latest versions of the specs need to provide that functionality through the software parts of the driver.

During 2018, Igalia implemented the missing bits to support GL_OES_copy_image in Intel’s i965 for gen7+, while gen8+ was already complying through its HW support. As we were writing this entry, the work has finally landed.

VK_KHR_16bit_storage

Igalia finished the work to provide support for the Vulkan extension VK_KHR_16bit_storage into Intel’s Anvil driver.

This extension allows the use of 16-bit types (half floats, 16-bit ints, and 16-bit uints) in push constant blocks, and buffers (shader storage buffer objects).  This feature can help to reduce the memory bandwith for Uniform and Storage Buffer data accessed from the shaders and / or optimize Push Constant space, of which there are only a few bytes available, making it a precious shader resource.

shaderInt16

Igalia added Vulkan’s optional feature shaderInt16 to Intel’s Anvil driver. This new functionality provides the means to operate with 16-bit integers inside a shader which, ideally, would lead to better performance when you don’t need a full 32-bit range. However, not all HW platforms may have native support, still needing to run in 32-bit and, hence, not benefiting from this feature. Such is the case for operations associated with integer division in the case of Intel platforms.

shaderInt16 complements the functionality provided by the VK_KHR_16bit_storage extension.

SPV_KHR_8bit_storage and VK_KHR_8bit_storage

SPV_KHR_8bit_storage is a SPIR-V extension that complements the VK_KHR_8bit_storage Vulkan extension to allow the use of 8-bit types in uniform and storage buffers, and push constant blocks. Similarly to the the VK_KHR_16bit_storage extension, this feature can help to reduce the needed memory bandwith.

Igalia implemented its support into Intel’s Anvil driver.

VK_KHR_shader_float16_int8

Igalia implemented the support for VK_KHR_shader_float16_int8 into Intel’s Anvil driver. This is an extension that enables Vulkan to consume SPIR-V shaders that use Float16 and Int8 types in arithmetic operations. It extends the functionality included with VK_KHR_16bit_storage and VK_KHR_8bit_storage.

In theory, applications that do not need the range and precision of regular 32-bit floating point and integers, can use these new types to improve performance. Additionally, its implementation is mostly API agnostic, so most of the work we did should also help to have a proper mediump implementation for GLSL ES shaders in the future.

The review process for the implementation is still ongoing and is on its way to land in Mesa.

VK_KHR_shader_float_controls

VK_KHR_shader_float_controls is a Vulkan extension which allows applications to query and override the implementation’s default floating point behavior for rounding modes, denormals, signed zero and infinity.

Igalia has coded its support into Intel’s Anvil driver and it is currently under review before being merged into Mesa.

VkRunner

VkRunner is a Vulkan shader tester based on shader_runner in Piglit. Its goal is to make it feasible to test scripts as similar as possible to Piglit’s shader_test format.

Igalia initially created VkRunner as a tool to get more test coverage during the implementation of GL_ARB_gl_spirv. Soon, it was clear that it was useful way beyond the implementation of this specific extension but as a generic way of testing SPIR-V shaders.

Since then, VkRunner has been enabled as an external dependency to run new tests added to the Piglit and VK-GL-CTS suites.

Neil Roberts introduced VkRunner at XDC 2018. This is his talk:

freedreno

During 2018, Igalia has also started contributing to the freedreno Mesa driver for Qualcomm GPUs. Among the work done, we have tackled multiple bugs identified through the usual testing suites used in the graphic drivers development: Piglit and VK-GL-CTS.

Khronos Conformance

The Khronos conformance program is intended to ensure that products that implement Khronos standards (such as OpenGL or Vulkan drivers) do what they are supposed to do and they do it consistently across implementations from the same or different vendors.

This is achieved by producing an extensive test suite, the Conformance Test Suite (VK-GL-CTS or CTS for short), which aims to verify that the semantics of the standard are properly implemented by as many vendors as possible.

In 2018, Igalia has continued its work ensuring that the Intel Mesa drivers for both Vulkan and OpenGL are conformant. This work included reviewing and testing patches submitted for inclusion in VK-GL-CTS and continuously checking that the drivers passed the tests. When failures were encountered we provided patches to correct the problem either in the tests or in the drivers, depending on the outcome of our analysis or, even, brought a discussion forward when the source of the problem was incomplete, ambiguous or incorrect spec language.

The most important result out of this significant dedication has been successfully passing conformance applications.

OpenGL 4.6

Igalia helped making Intel’s i965 driver conformant with OpenGL 4.6 since day zero. This was a significant achievement since, besides Intel Mesa, only nVIDIA managed to do this too.

Igalia specifically contributed to achieve the OpenGL 4.6 milestone providing the GL_ARB_gl_spirv implementation.

Vulkan 1.1

Igalia also helped to make Intel’s Anvil driver conformant with Vulkan 1.1 since day zero, too.

Igalia specifically contributed to achieve the Vulkan 1.1 milestone providing the VK_KHR_16bit_storage implementation.

Mesa Releases

Igalia continued the work that was already carrying on in Mesa’s Release Team throughout 2018. This effort involved a continuous dedication to track the general status of Mesa against the usual test suites and benchmarks but also to react quickly upon detected regressions, specially coordinating with the Mesa developers and the distribution packagers.

The work was obviously visible by releasing multiple bugfix releases as well as doing the branching and creating a feature release.

CI

Continuous Integration is a must in any serious SW project. In the case of API implementations it is even critical since there are many important variables that need to be controlled to avoid regressions and track the progress when including new features: agnostic tests that can be used by different implementations, different OS platforms, CPU architectures and, of course, different GPU architectures and generations.

Igalia has kept a sustained effort to keep Mesa (and Piglit) CI integrations in good health with an eye on the reported regressions to act immediately upon them. This has been a key tool for our work around Mesa releases and the experience allowed us to push the initial proposal for a new CI integration when the FreeDesktop projects decided to start its migration to GitLab.

This work, along with the one done with the Mesa releases, lead to a shared presentation, given by Juan Antonio Suárez during XDC 2018. This is the video of the talk:

XDC 2018

2018 was the year that saw A Coruña hosting the X.Org Developer’s Conference (XDC) and Igalia as Platinum Sponsor.

The conference was organized by GPUL (Galician Linux User and Developer Group) together with University of A Coruña, Igalia and, of course, the X.Org Foundation.

Since A Coruña is the town in which the company originated and where we have our headquarters, Igalia had a key role in the organization, which was greatly benefited by our vast experience running events. Moreover, several Igalians joined the conference crew and, as mentioned above, we delivered talks around GL_ARB_gl_spirv, VkRunner, and Mesa releases and CI testing.

The feedback from the attendees was very rewarding and we believe the conference was a great event. Here you can see the Closing Session speech given by Samuel Iglesias:

Other activities

Conferences

As usual, Igalia was present in many graphics related conferences during the year:

New Igalians in the team

Igalia’s graphics team kept growing. Two new developers joined us in 2018:

  • Hyunjun Ko is an experienced Igalian with a strong background in multimedia. Specifically, GStreamer and Intel’s VAAPI. He is now contributing his impressive expertise into our Graphics team.
  • Arcady Goldmints-Orlov is the latest addition to the team. His previous expertise as a graphics developer around the nVIDIA GPUs fits perfectly for the kind of work we are pushing currently in Igalia.

Conclusion

Thank you for reading this blog post and we look forward to more work on graphics in 2019!

Igalia

Extending the life of your N8x0: Second update for Automatic Skype Launcher

Back again with a new release, the 0.0.3, of Applications Fullscreener and Automatic Skype Launcher.

Just download and install them in your N8x0 for making it become a good 24/7 Skype phone. For more information about the goals and features of this project you can check the previous post Extending the life of your N8x0: Automatic Skype Launcher.

Skype Tango Icon

I took the chance to update also the artwork used in these two applications. First thing, I created a Tango-ized version of the Skype logo.

Then, I used the previous magnificent art works from Andreas Nilsson and Jakub Steiner to create the new icons, and these are the result:

Applications Fullscreener Icon Applications Fullscreener Debian Package Icon Applications Fullscreener Icon Applications Fullscreener Settings Icon

Anyway, going to the actual changes.

The most important change for Applications Fullscreener is:

  • There was a second bash helper to launch a certain application and send the fullscreen key event some time afterwards. It was not fully featured but now it is.

You can, also, check the complete ChangeLog for Applications Fullscreener.

The most important changes for Automatic Skype Launcher are:

  • The Automatic Skype Launcher service was not stopping when uninstalled. Now this is corrected.
  • The init service was not starting correctly in some scenarios due to some uninitialized variables. This has been corrected.
  • In the previous version, I added an alarm triggered by the alarmd daemon to restart Skype (and the service). This was not working always. Now, killing Skype is forced.

You can, also, check the complete ChangeLog for Automatic Skype Launcher.

That’s it. Comments and patches are welcomed!

Extending the life of your N8x0: Update for Automatic Skype Launcher

I’ve just released the 0.0.2 version of Applications Fullscreener and Automatic Skype Launcher.

As I was already commenting in my previous post Extending the life of your N8x0: Automatic Skype Launcher, just download them and install them in your N8x0 for making it become a good 24/7 Skype phone.

The most important change for Applications Fullscreener is:

  • I’ve added a helper to launch a certain application and send the fullscreen key event some time afterwards.
    This is specially useful for launching an application through the DSME watchdog daemon in Maemo4.x. Using this helper as a wrapper, the application will receive the fullscreen event some time after being launched while the watchdog keeps track of the helper. In this case, if either the helper or the application dies, both die and, hence, the DSME will relaunch them both.

You can, also, check the complete ChangeLog for Applications Fullscreener.

The most important changes for Automatic Skype Launcher are:

  • I’ve added a Control Panel plugin for setting several behaviors of the application: starting an init service, using a watchdog for keeping it alive, trying to fullscreen Skype after launching it as a service and the time at which the service will be restarted, if so.

Automatic Skype Launcher Control Panel plugin snapshot

  • I’ve added a Home Desktop applet for making it easier to launch manually Skype, not having to get into the applications menu.

Automatic Skype Launcher Home Desktop applet snapshot

  • The Automatic Skype Launcher service was not stopping when the N8x0 was turned off while charging. Now this is corrected.
  • In the previous version, I added a cron.daily script to restart Skype (and the service). Actually, the N8x0 doesn’t have a cron daemon, by default. This is now corrected through the usage of an alarm triggered by the alarmd daemon.

You can, also, check the complete ChangeLog for Automatic Skype Launcher.

That’s it. Comments and patches are welcomed!

Extending the life of your N8x0: Automatic Skype Launcher

Leading quickly to the “ham”, just install Applications Fullscreener and Automatic Skype Launcher in your N8x0 and turn it into a Skype phone. Now, the long explanation 🙂

After working in Maemo and MeeGo for the last 5 years I’ve become the proud owner of a N810, a N900, a N950 (by now) and a N9. Obviosly, my old and beloved first maemo device, the N810, has been pushed into the background.

Thinking about how to still keep doing a good use of it, came to my mind the idea of turning it into a Skype phone. Nokia’s hardware is superb and the sound quality of the N810 makes it a good choice for this.

Still, I didn’t need another Skype phone myself but my parents, who have a lot of troubles for making use of Skype’s software in the computer, were the perfect candidates. Also, placing the N810 24/7 online side by side to their land line phone would let them receive and check at any moment for the availability of their regular Skype contacts. That’s it, my oldest’s brother family, who lives in Germany, and myself, who have already spent the last year and half in Finland and will be there, at least, for another six months.

Therefore, I copied all the remaining data out of the N810, cleaned the internal MMC and re-flashed the device with the latest available image and the needed flasher. Afterwards, and for having an easy way for maintaining the device, I upgraded to the OS2008 Feature upgrade and installed Skype, openssh and x11vnc packages. Also, I set the “Red Pill” mode in the Application Manager and added the following repositories:

In addition, after being able to ssh into the device, I installed the screen package.

Then, I set the connection manager to be connected to my parents WiFi automatically, without energy saving, with the smallest searching intervals, in case it would lose connection, and with unlimited inactivity time.

I modified the display settings for having the bright level, the bright timeout and the timeout for turning it off to the maximum, unchecked the option for blocking the display and keyboard on turning the display off, and set the display to not being permanently on ever. In the LEDs settings I left everything checked but when the device is on.

Because of the possibility of connecting with a bluetooth headset, I also set the bluetooth to be always on and visible.

Finally, I launched Skype and configured a new account for my parents.

The only thing left was to be able to launch Skype automatically at boot time and in fullscreen mode. As I said before, my parents are not really skilled on technology or gadgets use so the simplest approach would be to have a fully working Skype client just after booting and with no more distractions that the contacts list and the button for calling.

Automatic Skype Launcher Icon

I’ve developed the package skype-autolauncher for this task. This package would launch Skype automatically on boot time through a init script. Yes, I know it is not the cleanest way but it just works™ and other strategies are not much cleaner. The init script makes use of dsmetool to monitor the Skype instance so, if it crashes, or is closed by mistake, the application is relaunched. Also, this package adds a cron.daily task which will restart the service so it will also shut down and re-launch Skype for having fresh memory and a healthier running environment.

Automatic Skype Launcher’s code is kept in gitorious. For downloading the code, just type in a terminal:

Applications Fullscreener Icon

For the matter of automatically turning the launched Skype instance into fullscreen mode, I’ve also developed the package apps-fullscreener. This package sends a fake fullscreen hard key event to a XWindow by its process name.

Applications Fullscreener’s code is also kept in gitorious. For downloading the code, just type in a terminal:

That’s it. Comments and patches are welcomed!

QUrl (mis)usage: appendix, avoid automatic cast from QString

As I was introducing in the former entry QUrl (mis)usage, the direct creation of a QUrl from a QString should be avoided in any software that is not trying to smartly guess what a user input should lead to.

So, going directly to the ham, to avoid mistakes due to automatic conversions from QString to QUrl, I encourage the usage of the QT_NO_URL_CAST_FROM_STRING macro. The only thing you have to do is adding a line to your qmake project file like this:

# Avoid automatic casts from QString to QUrl. Dangerous!!!
DEFINES += QT_NO_URL_CAST_FROM_STRING

Or add it directly to the compilation line, like this

g++ ... -DQT_NO_URL_CAST_FROM_STRING ...

As I was pointing in my previous post, the usage of QUrl::fromLocalFile(QString) and QUrl::fromEncoded(QByteArray, QUrl::StrictMode) is recommended when dealing with QString and QUrl, but committing mistakes is a human condition so it is pretty easy to end passing a QString as a parameter to some API expecting a QUrl, or assigning a QString to a QUrl with the “=” operator through the C++ automatic cast mechanism which is implemented in the QUrl class. That’s why forbidding these automatic casts in our code is of such importantance.

QUrl (mis)usage

Lately, I’ve been developing some software which makes an intensive usage of QUrls as resource locators for local files. Nothing wrong here. QUrl is a powerful way of sharing the locations of those in an universal way. The problem is when you construct those QUrls from QStrings and you actually forget that QUrls are meant for much more than representing local file locations.

Authority chunks on an URL

Authority chunks on an URL

At the moment of writing this, QUrl documentation, although quite complete, could be much more explanatory. For example, it says that the recommended way for creating a QUrl from a QString is:


* When creating an URL QString from a QByteArray or a char*,
always use QString::fromUtf8().
* Favor the use of QUrl::fromEncoded() and QUrl::toEncoded()
instead of QUrl(string) and QUrl::toString() when converting
QUrl to/from string.

But this is explained in the documentation for QUrl::fromUserInput(), instead of in the Detailed Description [ 1 ].

What is important from this explanation is that it is not a matter of favor the use of QUrl::from/toEncoded() over QUrl::(from)toString() but, I would say, a must if you don’t want to end up with bogus corner cases.

Why would this happen? Well, as I was saying, QUrl is meant for much more than universally representing the location of a file so, here go the big tips:

  1. If you want to get the QUrl from a local file represented with a QString, use always QUrl::fromLocalFile ( const QString & localFile ) . Don’t use QUrl::QUrl ( const QString & url ) if you don’t want to end up with some problems. In the same way, get the path to the local file always with QUrl::toLocalFile().
  2. If you want to get a QUrl from a QString representing an URL, be sure that the QString is actually representing a percent encoded URL, as it should to be a valid URL, and always use QUrl::fromEncoded ( const QByteArray & input, ParsingMode parsingMode ), with QUrl::StrictMode as the QUrl::ParsingMode.
  3. If you want to get a QString representation of an URL from a QUrl use always QUrl::toEncoded ().

Bogus examples for each case:

Local file

/mypath/my#file.jpg

Correct:

QUrl myUrl = QUrl::fromLocalFile("/mypath/my#file.jpg")

Incorrect:

QUrl myUrl = QUrl("file:///mypath/my#file.jpg")

The problem here is the way QUrl will treat the “#” character in the second example. It will think, as it actually doesn’t have a way of guessing, that the character is delimiting the fragment part of the URL.

Fragment part on an URL

Fragment part on an URL

As a result, calling to:

myUrl.toLocalFile()

in the first case will result to:

/mypath/my#file.jpg

while in the second will be:

/mypath/my

Parsing mode

/mypath/my#file.jpg

(encoded) url representation:

file:///mypath/my%23file.jpg

Correct:

QUrl myUrl = QUrl::fromEncoded("file:///mypath/my%23file.jpg", QUrl::StrictMode)

Incorrect:

QUrl myUrl = QUrl::fromEncoded("file:///mypath/my%23file.jpg")

The problem here is the way QUrl will treat the “%23” encoding in the second example. Although it is not explicitly explained in the documentation [ 2 ], QUrl will use QUrl::TolerantMode as ParsingMode by default. Therefore, it will think that the input comes from an ignorant user which was actually trying to pass “file:///mypath/my#file.jpg”. Again, it will understand after converting back “%23” to “#”, that the character is delimiting the fragment part of the URL.

As a result, calling to:

myUrl.toLocalFile()

in the first case will result to:

/mypath/my#file.jpg

while in the second will be:

/mypath/my

Encoded usage

/mypath/my#file.jpg

(encoded) url representation:

file:///mypath/my%23file.jpg

(unencoded and wrong) url representation:

file:///mypath/my#file.jpg

Correct:

QUrl myUrl = QUrl::fromEncoded("file:///mypath/my%23file.jpg", QUrl::StrictMode)

Incorrect:

QUrl myUrl = QUrl("file:///mypath/my#file.jpg")

Here, we have another incarnation of the very same problem than the two examples above. QUrl will think, again, as it actually doesn’t have a way of guessing, that the character is delimiting the fragment part of the URL.

As a result, calling to:

myUrl.toLocalFile()

in the first case will result to:

/mypath/my#file.jpg

while in the second will be:

/mypath/my

Corollary:

The default behavior of QUrl is to provide an easy handling of URLs to the user of our programs, the end user, but not the user of QUrl, the developers. I find this quite awkward but, still, it is a decision of Qt people and, as developers, we only have to take this into account when writing our code.

These bogus URLs, which are to be corrected with the usage of the QUrl::TolerantMode ParsingMode, usually come from a text entry box “à là” browser location bar, but this use case is, actually, not so common when talking from the developer’s point of view. When dealing with URLs in our code we have to take into account what an URL is and how it should be formatted/encoded to be valid. Therefore, if I’m receiving a wrongly encoded URL I should go to the source code providing this URL and fix the problem there rather than trying to smartly guess which should be the proper URL. For example, in my software currently in development we use Tracker and I rely on it to feed my code with properly formatted URLs. If for some reason Tracker gives me a wrongly encoded one, the place for solving it is, actually, Tracker, and not my software. I should not and must not interpret what Tracker may have wanted to pass me, but open a bug in its bugzilla and provide as accurate information as I can to help them solve this issue.

Just so my friend Iván Frade doesn’t kill me, make notice that Tracker is, so far, perfectly dealing with URLs 🙂

“maemizing” GUADEMY

This weekend GUAdemy, a mixed conference between GUADEC and Akademy, lands in Coruña. As in most of Gnome related events, Igalia is going to be there.

For me, I’m going to give a talk about maemo on Saturday afternoon, as Miguel, Chema, Susana, Iago and Sergio are going to give other talks about several Gnome related issues.

If you want to learn about the Gtk based environment for embedded devices created by Nokia, come and join us.

See you soon!