Smoothing out the hardware challenge for the metaverse and digital twins

Better file formats and better standards for representing 3D structures like USD and glTF have been crucial in advancing the metaverse and digital twins. However, there has been much less agreement on the representation of materials. Vendors, artists, and businesses mitigate these issues today by sticking to a given ecosystem of design tools and renderers or building multiple versions.

Today, the 3D industry and industrial designers are exploring ways to promote material interoperability between tools. This could allow designers or businesses to create a virtual representation of new fabrics, upholstery, styles, shoes, or paints and render them accurately through various tools and 3D worlds anywhere.

There are actually two complementary hardware interoperability challenges. First, each render engine has a different approach to capturing and representing the physical appearance of materials under various lighting conditions. Second, there are many ways to represent the physical properties of materials, such as how they bend, drape, feel, blow in the wind, or resist fire.

It might take some time for the industry to converge on a single format. Various file formats have sprung up to facilitate the exchange of documents between tools and virtual worlds, including U3M, AXF, MDB, MTL, KMP, and SBS, each with its strengths and weaknesses. It may be that industry-specific formats dominate in their respective fields, while others are used in all areas.

A realistic look

Companies creating 3D assets for games and entertainment are exploring how better material processing techniques like Physical Rendering (PBR) can improve the look of virtual worlds. “People usually think of a material as fabric, but the 3D industry talks about materials as a visual thing,” said Elliott Round, co-founder and CTO of M-XR, a production platform 3D at VentureBeat.

Most people are familiar with how primary paints like red, yellow, and blue are combined to create a variety of colors. Materials take this a step further with additional texture maps representing other properties such as albedo, metallicity, roughness, opacity, and diffusion. This is where it gets complicated. “Different renderers have different amounts of material properties,” Round explained. “Some will have five settings, while others may have ten, so they may all work slightly differently. This is something we hope to work out with other companies to unify 3D a bit better.

The industry has always faced computational and memory constraints for accurate rendering of materials. But now those constraints are beginning to fall with better computers and algorithms. “Hopefully we get to a position where we don’t have to cut corners and hack materials anymore because there are less constraints,” Round said. “It could become unified like in the real world.”

His company develops tools and techniques to quickly capture the visual properties of real-world objects in virtual worlds. They started using tools like photogrammetry and structured light scanning to capture 3D objects. “All of these approaches give you really good 3D geometry, but none of them will give you material information. And that’s arguably the key to photo realism,” Round explained. aspects such as how light reflects off an object and whether it is scattered, absorbed or transmitted.

His team also explored various types of sample scanners often used in the textile industry. These types of scanners from companies like Vizoo and X-Rite can capture the visual properties of materials by scanning fabric samples or pieces of paper. Artists and enterprise applications can then apply them to 3D objects. Round said these scans are really good but don’t work particularly well for scanning an entire object, prompting a search for better whole-object capture techniques. Epic recently invested in M-XR to help scale these tools for 3D creators.

A realistic feel

Companies that manufacture physical materials, such as textiles, home furnishings, and apparel, face additional material challenges. They must also capture the physical feel of things using various tools and approaches. For example, Bru Textiles, a Belgian textile giant, spent four years developing a workflow to capture visually and physically accurate textile digital twins for its new twinbru service. Twinbru Partnership Development Manager Jo De Ridder told VentureBeat: “[The digital twin] is a 100% replica of the physical fabric, both physically and spec-wise.

This helps design firms create realistic prototypes, like a new hotel lobby, and quickly explore variations for clients. In the past, they had to approximate the look through a color chart and create a mockup that didn’t always look like the finished product. “Having digital twins shortens the supply chain, reduces complexity and increases accuracy,” De Ridder said.

However, this is a complex process. It took the Twinbru team years to develop and streamline the workflow to capture visual and physical properties and turn them into digital twins. They used a combination of X-Rite and Vizoo scanners to capture AXF and U3M files representing the visual aspects of the tissues. Additionally, they worked with Labotex to capture the physical properties of the textile into an SAP database which is converted to the appropriate physics engine format. They have created digital twins of the fabric available for Nvidia Omniverse, Chaos Cosmos, ArchiUp and Swatchbook.

Better industry collaboration could help streamline similar workflows for other companies that manufacture and work with textiles, paints, fabrics and other materials. A 2020 Computational Tissue Physics Interoperability A survey by the 3D Retail Coalition concluded that it is now possible to measure five physical attributes of fabric at once and accurately translate them into equivalent physical values ​​for multiple 3D apparel software solutions. These include curvature, stretch/elongation, diagonal shear/stretch, weight and thickness.

Industry leaders are also starting to collaborate on open standards. For example, Browzwear, which makes 3D fashion design software, collaborated with Vizoo to drive adoption of the Unified 3D Material (U3M) standard in the fashion industry. A big advantage over other formats is that it can capture both visual information and physical properties of tissue.

“I truly believe that moving the metaverse to the point of mass adoption requires materials and textures to be accurately represented,” Avihay Feld, CEO of Browzwear, told VentureBeat. “Synthetic visions involving digital twins as frozen snapshots of the physical world are a good start. Digital twins as an evolving image of reality synchronized with reality are even better.

He argues that it’s unclear which direction the metaverse is heading, but it’s easy to imagine two possibilities. One is a metaverse that deviates from reality, where virtual worlds defy the laws of physics. The other is a metaverse that mimics reality so that users have experiences analogous to those possible in the real world.

He believes that a true-to-life representation of visual and physical properties will be essential in this second case. Having realistic things inside the virtual world will make it more immersive and compelling, but it will also allow the Metaverse to support a variety of use cases. One of the main ones is the trade, not of strictly digital items, but of real objects. In this second case, having real digital twins, from the point of view of visualizing textures and simulating the physics of an object, will be essential.

“It’s possible that these two possibilities co-exist, but without the realistic experiences, it’s likely that the metaverse will remain a fantasy world for tech-savvy people instead of the transformative new universe it has the potential to be.” , says Feld.