A new standard for assets that works in real time, masters PBR, is still open and is supported by all the big names? Yes, it does exist.

According to Wikipedia, “A standard is a comparatively uniform, widely recognised and usually applied (or at least aspired to) way of describing, producing or performing something that has prevailed over other ways or is at least considered a guideline”. With this in mind, the Khronos Group, a consortium of global industry members, came together to create a standard for the real-time visualisation of 3D assets. The requirements for the standard format were a platform-independent, fast and realistic representation of products as interactive 3D objects. Last but not least, the format should also be capable of augmented reality and the 3D assets should look the same on every common viewer.

After much research, discussion and negotiation, glTF (GL Transmission Format) was chosen as the leading licence-free transmission format for the three-dimensional representation of models and scenes. In order to create a legally compliant and high-quality framework, the decision was made in favour of a certification process in accordance with ISO/IEC JTC 1. The process is already underway and confirmation is expected in the course of 2022.

Why do we need a standard for this?

In the 3D format specialist group, leading international experts from Khronos members are specifying standards in the form of glTF extensions, which are aimed at a realistic representation of materials, but also take file size, metadata, etc. into account. We will read more about examples later in the article. Even if the standards are technically correctly taken into account in DCCTools, there can be very large differences in the presentation. The worst case scenario for creatives would be to painstakingly create a 3D asset in the 3D tool of their choice, only to realise that they have to spend several hours adapting it for integration on different platforms.

To avoid this additional work, the 3D Commerce standard and the certification for 3D RealtimeViewer were created. This group includes well-known representatives from all disciplines of e-commerce, such as Amazon, Ikea, Wayfair or Target, but also from the 3D asset creation and editing sector, such as Adobe, Autodesk, Blender, etc. This new demand also provides scope for new players on the market, such as UX3D GmbH from Munich, which has launched the first visual glTF editor called Gestaltor.

How is the file structured and what data is included?

glTF is short for Graphic Language Transmission Format. It contains three-dimensional scenes and models. A glTF file contains either a glTF (JSON/ ASCII) or a GLB (binary) as a possible file extension. A GLB file is completely self-contained. A glTF file can be self-contained or refer to external binary and texture resources.

The glTF standard supports 3D model geometry, appearance, scene graph hierarchy and animation. The aim was to achieve a streamlined and interoperable format for the delivery of 3D assets, while minimising file size and runtime processing. Khronos likes to draw a comparison with the established JPG format from the 2D world and refers to glTF as “JPG for 3D”.

How does glTF differ from FBX, for example?

The main difference between glTF and other established exchange formats such as FBX is the open standard versus the proprietary approach. For example, the FBX format is best used in the environment of Autodesk products. However, if the format is required for an independent 3D real-time display on the web, mobile or as an augmented reality application, there is no way around glTF. Because it is an open standard, it is even explicitly desired to develop extensions for glTF, which are then ideally also ratified as official KHR_extensions.

What is a compliant glTF? Are there non-compliant glTFs?

A valid glTF is characterised, for example, by its official extensions, which are identified by the prefix “KHR_”. They guarantee the same understanding of physically based rendering in terms of transmission, clear coat, etc.

The following official extensions are currently published:

• KHR_draco_mesh_compression
• KHR_lights_punctual
• KHR_materials_clearcoat
• KHR_materials_pbrSpecularGlossiness
• KHR_materials_ior
• KHR_materials_sheen
• KHR_materials_specular
• KHR_materials_transmission
• KHR_materials_unlit
• KHR_materials_variants
• KHR_materials_volume
• KHR_mesh_quantisation
• KHR_texture_basisu
• KHR_texture_transform
• KHR_xmp_json_Id

A complete list for validating a valid glTF can be found on the Khronos GitHub here: github.com/KhronosGroup/ glTF-Validator.

Where are the problems with compliant glTFs in everyday life?

The challenges we often hear from 3D artists are as follows: A 3D artist creates a 3D object in the DCC tool of choice. This can often take several hours. It is all the more annoying when the artist realises that effects or information are not displayed correctly when exporting to glTF or that some proprietary functionality cannot be mapped in glTF at all. Brand owners and product managers are annoyed when the 3D viewer in the DCC tool and the viewer on the target platform (e.g. in the web shop) are displayed differently and materials or colours are suddenly displayed completely differently. Fortunately, there is a solution for both cases: simply use products certified by 3D Commerce.

The following viewers and a glTFEditor are currently certified by Khronos (as of October 2021)

• • UX3D: Gestaltor (glTF editor)
  • Unity Technologies: Unity glTFast –
    4.1.0-preview.1
  • CGTrader: CGTrader Viewer – 1.11.10
  • Babylon.js: Babylon.js Sandbox –
    5.0.0-alpha.24
  • Khronos Group, Inc: 3DC-SampleViewer

And in everyday work?

The digitalisation of objects or products traditionally begins in a CAD or design tool or using scanners. 3D digital creation tools come into play to display the result in 3D and optimise it if necessary. Up to this point, the format of the 3D object is always a proprietary format. There are some free conversion programmes or integrated exporters for converting to the real-time format. The next step is to validate the glTF again for quality control purposes and, if necessary, to optimise the size, polygon count, lighting, etc. Before the 3D object can be loaded onto the respective platform, its requirements must be taken into account. On a smaller scale, 3D artists can certainly map the process from start to finish.

The challenge begins with the generation of several hundred assets per day. Currently, parts of the processes can already be implemented semi-automatically. Many platform operators are working together with tool manufacturers to standardise and automate processes. The biggest challenges currently lie in the various platform requirements with regard to the size of the geometry, referenced material representation, quality assurance, etc. In particular, there is still no consensus on the realistic representation of materials in real time: the furniture and clothing industries demand realistic representation of wood, stone, glass, fabrics, etc. This will be all the more important if e-commerce develops even more strongly and printed catalogues are phased out, as IKEA, for example, announced in a press release at the end of last year. We expect to see a standardised solution for displaying some materials in the coming months. As an open source DCC tool, Blender is naturally widely used and is therefore very often chosen as a reference tool for exchanging formats. The Khronos Group has therefore also commissioned the development of a state of the art importer and exporter for glTF. As long as Blender supports all official glTF extensions, they will be exported correctly. To test whether a glTF is valid and all Khronos extensions work, the official sample viewer from Khronos is available at github. khronos.org/glTF-Sample-Viewer-Release. If you want to go deeper into the analysis, you will do yourself a favour by installing the Gestaltor, because an analysis is started as soon as the glTF is loaded and an error report is output in the logger.

Gestaltor

For those who want to integrate the format deep into their pipeline: Gestaltor is the first visual glTF editor that allows you to work natively with the glTF and GLB file formats in compliance with all official Khronos glTF standards (glTF 2.0 and higher) – which we have developed. In principle, the Gestaltor can be operated like any conventional DCC tool. The special thing about it is that only the functionalities relevant to glTF are deliberately included. So you can be sure: The result is always 100% glTF. No more, but also no less – WYSIWYG (What You See Is What You Get). It offers a powerful interface to massively accelerate workflows in the 3DAsset pipeline. The core functionalities are viewing, checking, editing, optimising and creating scenes. 3D content creation in standard DCC tools remains completely unaffected by the workflow. Gestaltor acts as the final link in the workflow to create a 100% valid glTF.

Gestaltor is officially 3D commerce certified by the Khronos Group. This means that all glTF and GLB files created in Gestaltor will look the same on the upcoming certified viewers from Amazon, Babylon.js, CGTrader, Emersya, Epic Games (Unreal Engine), Facebook (Spark AR), Google ( & Scene Viewer), Samsung (Internet Browser on Android), SketchFab, Unity and many others. In the private sector, the community version of Gestaltor tends to be used to open glTFs and to add or remove assets or slightly change the colour, lighting or materials. In the commercial sector, we know that digital agencies use Gestaltor for customer solutions in the augmented reality sector. In expert forums, we read that there is currently no other alternative on the market that can be used to display transmission effects in real time, for example. We usually work with Gestaltor’s enterprise customers in the form of a strategic project to optimise its use in existing 3D workflows.

Manufacturers are often looking for a way to visualise special materials or effects in real time. To this end, our experts develop a prototype in close consultation with the customer, which is then integrated into Gestaltor as an experimental extension. This is what happened with the KHR_materials_iridescence extension.

What are the next steps for Gestaltor?

One of our main priorities in the further development of Gestaltor is to always be 100% in line with the glTF standard; i.e. as soon as official CHR extensions are ratified, we aim to be the first tool to integrate them. Thanks to the very open exchange with Gestaltor customers and strategic project partners, we have prioritised some features higher or added them to the roadmap. Without giving too much away, by the end of 2021, we will have overhauled the entire UI, added an experimental CHR material iridescence extension, a sophisticated material import (pbr2gltf2) and basic animations. The ambitious plan is that we will continue to establish Gestaltor as the world’s leading DCC tool for glTF.

What is coming in the next versions of glTF?

We are currently seeing a big movement from many industries towards glTF. Automotive manufacturers are already in the process of completely converting their 3D visualisation pipelines for human machine interfaces to glTF and open standards. Representatives of standards for videos and audio files are currently discussing with Khronos sensible options for delivering this data with glTF as well. For the fast-moving consumer goods industry, there are several factors in favour of adding glTF to the pipeline. Firstly, a high-quality 3D real-time asset can replace product photography, and the official extension for metadata khr_xmp_json_ld could be used to supply all information from EAN codes to product descriptions.

What role will Khronos play in the future?

The role of the Khronos Group seems to be constantly strengthening through the development of many useful standards. With the development of the mixed reality market, OpenXR as a standard is here to stay.

Even the otherwise very restrictive Apple Safari browser is opening up to WebGL 2.0 support by the end of 2021. Games built with the Vulkan standard, such as Fortnite or all games on Google’s Stadia, are also proving to be relevant on the market. And it is not yet possible to predict what else will emerge in the direction of WebGPU. Of course, all developments in the direction of augmented reality and metaverse speak in favour of glTF, with Spark AR, Snap AR, TikTok AR, NFT in art and the digital fashion industry etc. opening up promising business models. If you want an overview of all the standards at a glance, you can find them on the official website at www.khronos.org.

What is the future of standards?

The demand for standards from major market players in all industries has become so great that even competing advocates of proprietary formats are sitting down at the same table to reach joint compromises. For us, this is a sign that sooner or later no technology provider will be able to avoid open standards. Just as USD is a valid framework for the internal co-operative exchange of 3D computer graphics, we believe that glTF is the optimal format for the exchange of 3D real-time graphics across platforms to the outside world.

What requirements for interoperable formats such as glTF will we see in 2040?

A democratic desire for non-proprietary solutions could become the norm – business models may change to the extent that users decide what they want to consume with which devices and technologies, and a requirement is established that content and usability are the same regardless of the platform. But we will see.

There is no doubt that the trackball consoles and the Editor Keyboard from Blackmagic Design are solid devices and ideally suited for DaVinci Resolve. The prices are fair, and for specialised workstations in the film and TV industry they are even extremely affordable. But what about lone wolves who are just starting out in the new world of work, or hobby filmmakers who have already had to spend a lot of money on a sufficiently powerful computer? Are there alternatives to pushing a mouse?

Blackmagic Design does not make it easy for third-party providers, because protocols for controlling Resolve are not published. This is part of the business model, as the development of the complex and powerful programme is co-financed by the sale of hardware. This is why, for example, the popular Loupedeck is not yet compatible with Resolve. But a developer from Amsterdam has taken up the challenge and programmed around the handicap. It took Glenn Venghaus three years to persuade Akai’s APC40 controller, a favourite in the music world, to work with Resolve and The APC40 requires fewer switching operations and provides even more feedback, making the functions of large control consoles accessible as far as possible. This device is durable, equipped with very precise controls and, with its numerous operating elements, requires fewer switching operations(tinyurl.com/ssm9wh6).

Addendum for the online version of this article: In the video below, Glenn Venghaus shows an update that allows the HUD to always be enabled via devices with a web browser.

But it’s even cheaper: even with a version for the small Arturia Beatstep, most of the functions can now be controlled(tinyurl.com/sm2f7y3). The small controller/sequencer from France is popular in the music world and can be found for well under 100 euros. You also need the inexpensive ControllerMate software from OrderedBytes (approx. 30 US dollars) and the Beatstep Resolve Edition. This is available in several versions, and this has to do with the type of control. The software benefits from the fact that Resolve’s graphical user interface can hardly be customised, as it has to be able to determine the exact position of the controls.

This is why there are separate versions for different resolutions and for operation with one or two screens. The Base Kit with just one version is available for 100 euros, two versions for 125 euros and the Extreme version for 175 euros includes all resolutions for one or two screens. Upgrades are also available. We tested in the standard resolution of 2,560 x 1,440 for 27-inch monitors. It is important that Resolve runs in the fullscreen window and that only the icons are displayed at the bottom of the pages without labelling so that everything has its correct place. As MIDI unfortunately does not work identically on all localised versions of the Mac OS, the author offers adapted versions of his kit. Otherwise a German system will not find the connections to Fairlight.

Installation

The first installation is time-consuming, because not only the Resolve Edition has to be installed. First comes ControllerMate and its registration. Here, especially with Catalina, you must ensure that you have the necessary rights to control the computer. These permissions must be correct, as the programme itself cannot provide any warnings. This is followed by the one-off installation of a suitably prepared open source toolkit, which is automatically downloaded from the manufacturer’s website. Now the Beatstep Resolve Edition, which is personalised for each user and, if necessary, updated for further downloads in the event of updates. Finally, you need the free MIDI Control Centre from Arturia to load the templates into the controller and configure MIDI in the Mac OS and in Resolve if you also want to control the essential functions in Fairlight. The installation program creates a RAM disc on the desktop, which you should definitely stay away from.

This complex procedure should not deter interested parties, as a detailed video by the developer explains each step in detail and even describes the remedy for operating system or Resolve errors. In addition, each start procedure checks whether everything is working correctly and issues a warning if necessary. Most of these steps are only required once anyway, as long as there are no fundamental changes to the computer or Resolve. We must also emphasise the excellent support from the developer, who responds very quickly and provides individual assistance. This is an advantage of dealing with small companies, where you are not put through endless FAQs and waiting loops only to be left helpless again at the end. Incidentally, his company is called Tachyon Post (tachyon-consulting.com) – tachyons are hypothetical particles that are faster than light.

Hardware

For the price, the small Beatstep controller makes a surprisingly solid impression. It should also be able to withstand a lot in practice, as musicians are usually not exactly gentle with their equipment, depending on their style. The first thing you should do is print out self-adhesive labels for the keys; the template is supplied as a modifiable Illustrator file and as a PDF, otherwise you would be quite overwhelmed. The software uses the key illumination for user guidance, and the respective assignment of the keys or the rotary control is also displayed when the HUD key is held down. You should largely keep your hands off the mouse and only use it for functions that are not yet controlled by the controller – and there aren’t many of those. The computer keyboard, on the other hand, can be assigned to your own preferences, only JKL and the numeric keys for printer lights should remain unchanged.

Operation

Of course, this includes the entire grading, the parameters in the inspector, most of the cut and trim functions and the controls in Fairlight (MIDI also comes into play here). But Glenn has even recently made the custom curves controllable, as well as the parameters of numerous OFX effects and even the new Cut Page in version 16 is supported (switching from Edit via A/B). Even 5 custom OFX assignments can be created with the controller and its visual feedback, but only for parameters that can also be controlled in Resolve with the centre mouse wheel. Of course, the large rotary knob at the top left serves as a jog wheel and proves to be very useful. MIDI must also be configured for Fairlight. precise when cutting and trimming. It goes without saying that the complex possibilities cannot be learnt in a day, even if the assignments reveal a clear, logical structure. The concept makes it clear that the author is not a programmer with little practical experience, but works in post-production himself. His video (vimeo. com/264606144) is helpful and should be watched as a supplement to this report if you are interested. You may want to download it and watch it step by step, as Glenn Venghaus proudly demonstrates how quickly you can work with the device. But in the end, training is essential, similar to learning to write blind.

Comment

Beatstep or its bigger brother, the APC40, are the perfect solution if you’ve upgraded your computer and don’t have enough money for the Blackmagic consoles, but still want to work smoothly without ruining your wrist with the mouse. The smaller controller is even useful for a laptop on the move. There is really only one catch: the whole thing is only available for Mac OS. There are said to be people who have a Hackintosh machine configured for this reason alone ..