It was released around 4 years ago, when I worked at a civil engineering tech startup.
We had a lot of Cesium 3D tiles of construction sites, captured by drones. It was quite easy to place them to Unreal engine. Is was fun to place random things in the map and mess around it.
Light Fields camera tech produces dynamic reflections due to capturing and interpolating reality itself, I wonder how well this can be simulated with Cesium Photogrammetry and the Unreal VR pipeline so you don't need an expensive Light Fields camera to produce similar realistic static VR scenes.
itishappy 16 days ago [-]
Lightfields are great, but adding the angular dependence makes them storage expensive, hence the extremely restricted movement from the demos. They're created using photogrametric techniques as well, so in theory I don't see why you can't just take more photos and solve for more angles (but this is very much a load bearing "just"). In practice storing a full BSDF per point is always going to impractical, but I believe that's pretty much exactly the problems that NeRFs have been created to address.
dleeftink 17 days ago [-]
> capturing and interpolating reality itself
That's impressive
Gooblebrai 17 days ago [-]
What does it really mean "interpolating reality itself"?
"An MSI consists of a series of concentric spherical shells, each with an associated RGBA texture map. Like the MPI, the multi-sphere image is a volumetric scene representation. MSI shells exist in three dimensional space, so their content appears at the appropriate positions relative to the viewer, and motion parallax when rendering novel viewpoints works as expected. As with and MPI, the MSI layers should be more closely spaced near the viewer to avoid depth-related aliasing. The familiar inverse depth spacing used for MPI’s
yields almost the correct depth sampling for MSI’s, assuming depth is measured radially from the rig center. The spacing we use is determined by the desired size of the interpolation volume and angular sampling density as described in Appendix A.
3.2.1 MSI Rendering. For efficient rendering, we represent each sphere in the MSI as a texture-mapped triangle mesh and we form the output image by projecting these meshes to the novel viewpoint, and then compositing them in back-to-front order. Specifically, given a ray r corresponding to a pixel in the output view, we first find all ray-mesh intersections along the ray. We denote Cr = {c1, . . . , c }
and Ar = {1, . . . , } as the color and alpha components at each intersection, sorted by decreasing depth. We then compute the output color cr by repeatedly over-compositing these colors. [..] We parameterize the MSI texture maps using equi-angular sampling, although other parameterizations could be used if it were necessary to dedicate more samples to important parts of the scene."
Presumably interpolating from data points recorded IRL to synthesize data points not in the IRL dataset, but which do exist, at least approximately, IRL
itishappy 16 days ago [-]
They use a camera array to capture more details, then solve for depth/angle and reproject.
ksec 16 days ago [-]
On one hand it is great Unreal is being used in many places outside Gaming from VR, VFX, Movie production or anything to do with 3D / Photorealistic Graphics.
On the other hand there are some recent backlash suggesting these direction and default in Unreal is making games look worst. [1] [2] And Nanite may not be the silver bullet we were looking for.
Nanite was never supposed to be a silver bullet but a way to enable high density mesh streaming and high fidelity shadows and lighting when used with Lumen. Both techniques work incredibly well.
The only valid criticism is games getting too expensive (to make) and frame generation can work against gameplay. But both are really up to game designers to use appropriately, not engine providers to dictate.
jayd16 16 days ago [-]
The only real argument I see is "Lumen is bad." It would be pretty hard to work in an alternative GI but Nanite and the upscalers are optional.
HellDunkel 16 days ago [-]
It is not- it is a tremenous relief. If Lumen doesnt fit your needs you can go back to light baking which of course nobody wants to.
kevingadd 16 days ago [-]
1's argument seems to mostly be that shader stuttering is bad in UE5, which doesn't have much to do with UE5... shader stuttering is really bad in lots of UE4-based games and non-Unreal games too, it's a plague that affects most modern high-spec games and driver stacks. If you think stuttering is bad in UE5 games try playing a WebGL or WebGPU based high fidelity game.
I'm not really sure what the alternative here is supposed to be anyway. Is it Unity? Godot isn't a realistic competitor yet for this level of fidelity, though it'll probably get there eventually.
jayd16 16 days ago [-]
There's no alternative. Its just rage/click bait. The shader issue is cross engine and the solution is the same across engine...warm up the shaders before gameplay.
kevingadd 16 days ago [-]
Valve has a fun solution for the Steam Deck, which is shipping pre-compiled shaders for your device across the wire that get installed along with the game. I'm curious to see whether that will become more widespread eventually.
jayd16 16 days ago [-]
That's still pre-warming. Sometimes you can target hardware and do it at dev time but its the essentially the same process.
phatfish 16 days ago [-]
I assume other Steam Deck like hardware which is a fixed target could support it, apparently the Legion Go S is going to ship with Steam OS rather than Windows.
On PC hardware it seems unlikely, more chance of Epic fixing their engine.
kridsdale1 15 days ago [-]
Yeah the fixed hardware is key here. The PS5 doesn’t have to compile shaders at app launch time either.
john_minsk 16 days ago [-]
Even if true, still - unification of efforts across multiple industries will create more value and synergies in the long run compared to narrow optimisations.
bru3s 17 days ago [-]
[dead]
Stevvo 17 days ago [-]
Way too expensive to be useful for anything.
protimewaster 17 days ago [-]
Do they have pricing information easily available? All I'm seeing is that it looks like it's free until you want to use their data, and there's no obvious indication of what that costs.
spookie 17 days ago [-]
Yup, basically that. Furthermore, you are unable to export the meshes themselves.
You are better off using Google Maps API if I'm being honest.
I've been able to do a lot for free with the regular version of Cesium, how is this different?
vouaobrasil 17 days ago [-]
Why do people strive to create such realistic environments when they're a much more realistic world outside? Although I understand the purpose from a fantastical-escapist perspective, I wonder if this sort of increasingly realistic escapism actually is reflective of how we're making the real world far worse than it ever was before and we're preparing an alternative world to escape into due to our destructive tendencies.
spookie 17 days ago [-]
Imagine being able to experience a place that no longer exists. Close to my hometown there was a dam made that completely changed the landscape. But a museum reconstructed as best as they could the area so one can still see it as it was in VR and walk around.
That's one use.
There are many others, such as simulation of human behaviour in a digital twin (you can make sure experiment variables remain the same, difficult to do in the real world), collaboration and ease of access to archeological sites around the world, aiding architectural work, etc...
vouaobrasil 16 days ago [-]
In my opinion, this will just lessen the negative cost of destroying landscapes. So, not a good thing.
drunner 17 days ago [-]
The vast majority of people never leave the area they are raised, much less their country, either due to economics or culture.
There will always be a need for software like this as long as the world has these restrictions.
vouaobrasil 17 days ago [-]
If people in such places had more immediate access to beautiful nature, then such a need wouldn't exist.
jncfhnb 17 days ago [-]
Son, you don’t need to see the desert or the ocean or the rainforest. You have a beautiful Midwest prairie right here at home.
thaumasiotes 16 days ago [-]
Apropos of nothing, it seems weird to me that we observe a vocabulary distinction in identical phenomena based on where those phenomena are located.
A prairie is called a steppe if it's in Asia. A steppe is called a prairie if it's in America.
A hurricane is a typhoon, except it's striking the west coast of the Atlantic instead of the west coast of the Pacific.
None of these make any difference to what the object is like. Why do we care? We don't call mountains something different when they're in Asia. We don't even call them something different when they're underwater, which makes a huge difference.
kedean 16 days ago [-]
Your distinction around hurricane vs typhoon is off. Typhoon is the name in the pacific west of 180 degrees longitude (in the jurisdiction of the JMA), hurricane is the name in the pacific east of 180 degrees, or in the atlantic (jurisdiction of the NHC). In the south pacific and indian ocean, they are called cyclones instead, so there's really three names. The difference in naming is simply because the organization responsible for reporting on them is going to use the name most familiar to it's country of origin.
Also, prairie and steppe are subtly different, though if it weren't for historic reasons they might be named the same. A prairie is more moist and has more vegetation as a result, and can support more trees and general flora/fauna.
thaumasiotes 16 days ago [-]
> In the south pacific and indian ocean, they are called cyclones instead, so there's really three names.
This is not true of American English, where "cyclone" unambiguously refers to a tornado.
kedean 15 days ago [-]
I'm going to disagree with you there. Anecdotally, I've lived in tornado alley my whole life and nobody has ever referred to a tornado as a cyclone in real-life conversation, although I'd know what they're referring to based on context clues.
Outside the heartland, the NHC categorizes many storms below the level of hurricane (64 knot sustained winds) as various kinds of cyclone (tropical cyclone, extratropical cyclone, potential tropical cyclone, post-tropical cyclone, alongside depressions).
In fact, in meteorology terms, a tornado is definitively different from a cyclone (a column of rotating area vs an area of area rotating around a low-pressure system). Hurricanes and typhoons are both kinds cyclones.
snypher 16 days ago [-]
Well, hurricane is a Caribbean word and typhoon is an Arabic word. These two languages named the phenomenon before they could share a word for it.
jncfhnb 16 days ago [-]
Not that weird. The English language emerged from the needs of people from various linguistic backgrounds to describe things. It was not cleanly designed with top down clarity.
vouaobrasil 16 days ago [-]
I agree with the sentiment. My point was not exotic nature, but any nature. Some people do not even have that.
01HNNWZ0MV43FF 16 days ago [-]
I'm balancing my desire to live near nature with my desire to live near a grocery store, a library, and a rock climbing gym
vouaobrasil 16 days ago [-]
If cities were better designed, the balancing would be much easier.
Gooblebrai 17 days ago [-]
Disagree. I can have a beautiful garden and still have curiosity over what's my neighbour's garden looks like.
TeMPOraL 17 days ago [-]
Nature isn't the only, or even the most interesting thing to see out there. Even if you are that much into nature, you'll observe that it too is meaningfully different around the world.
Tadpole9181 17 days ago [-]
You're reading way too much into it. It's just fun.
If I play a mile I'm like Arma, I don't actually want to be a soldier. Me playing Cooking Mama isn't a veiled psychological thing: I like both cooking and the gamified, simulated cooking.
vouaobrasil 17 days ago [-]
I don't mean to say that it's not fun, and fun ins't a valid reason. I only mean to say that the absolute intensity has one component that motivated by real-world destruction. Of course there are other factors.
dagmx 17 days ago [-]
Unreal engine is used for much more than just games.
It’s used for architectural Visualization, film backgrounds, and industrial visualization among other things.
Surely you can imagine why having realistic settings is beneficial in those scenarios.
krapp 17 days ago [-]
>I wonder if this sort of increasingly realistic escapism actually is reflective of how we're making the real world far worse than it ever was before and we're preparing an alternative world to escape into due to our destructive tendencies.
No. Some people just like realism in their video games, and prefer the immersion or just think it looks cool. Not everything is a manifestation of existential dread.
vouaobrasil 17 days ago [-]
True, but there must be a contribution of force towards more realistic environments. I never said it was the ONLY reason.
krapp 17 days ago [-]
Increased quality of processors and graphics hardware allow designers to push the limits, and "realism" has been a competition in games for a long time. Every console generation touts the realism of its graphics versus previous generations and the competition. And for certain games like Cyberpunk or GTA, realism enhances the gameplay experience.
If there were a general trend towards creating realistic enviroments as a means of psychological escape, one would expect those environments to tend to be more pristine and idyllic than reality. But often that realism is used to depict worlds no better, and sometimes far worse, than our own.
I'm sure some people are using realism as a means of escaping the destruction of our real world, walking simulators and the like exist, but I don't think that's a relevant influence overall. Just like with realistic CGI, people mostly do it because they can.
NeutralCrane 16 days ago [-]
Tourism itself is quite destructive both to local communities and to nature. There’s a reason tourist hotspots are increasingly anti-tourist, and national parks are now forced to ration out visits via a lottery. While I agree that a virtual environment is less impressive than the real thing, it’s not a difficult argument to make that it is actually much better for the world to have digital alternatives.
vouaobrasil 16 days ago [-]
I agree. I am not referring to tourism, but nature available for locals.
mwambua 16 days ago [-]
Simulation is a really useful tool for practice! I’m pretty sure this Hang Gliding sim uses Cesium for its environment: https://freeflightexperience.com/
I know people who will practice flying at unfamiliar sites on it first - so that they’re better prepared when they get to the real thing.
EwanG 17 days ago [-]
Have a handicapped daughter who I can't share a hike with since wheelchairs and rugged mountain trails don't go together well. Something like this would make sharing the experience quite a bit better - particularly with AR to allow us both to experience together (VR being a bit restrictive to just the person viewing and not the others there).
pklack 17 days ago [-]
I think it might also have to do with traveling to the places they want to see not being a feasible Thing to do for sind people, for financial reasons for example. Digital recreations (or similar environments) are cheap or even free to "visit", while also bein more accessible (no traveling neeeded).
jayd16 16 days ago [-]
This is like asking why they made film when cartoons exist. It's a means to achieve an artistic vision.
vouaobrasil 16 days ago [-]
I understand that. And I fully support the artistic vision. However, many creations take on secondary functions within a complex system that goes beyond the intention of the inventor or even the people who USE the product.
For example, the INTENTION of antidepressants may have been at one time to treat severe depression, but as society (hypothetically) gets worse and more depressing, they may be used to help people cope with systemic problems and therefore ameliorate people's reaction to them.
Or phones: they initially were created to help people communicate but they are subverted (co-opted) by the system to make people more dependent on technology for the sake of technological growth.
Any number of products or ideas can start as one thing and become co-opted to serve a more insidious function beyond their initial purpose.
scotty79 16 days ago [-]
You'd rather have me set it on fire and exploded in the virtual or in the real world?
They point of having virtual environments is to enable you to see things in them you'd rather not see in real life.
We had a lot of Cesium 3D tiles of construction sites, captured by drones. It was quite easy to place them to Unreal engine. Is was fun to place random things in the map and mess around it.
Light Fields camera tech produces dynamic reflections due to capturing and interpolating reality itself, I wonder how well this can be simulated with Cesium Photogrammetry and the Unreal VR pipeline so you don't need an expensive Light Fields camera to produce similar realistic static VR scenes.
That's impressive
"An MSI consists of a series of concentric spherical shells, each with an associated RGBA texture map. Like the MPI, the multi-sphere image is a volumetric scene representation. MSI shells exist in three dimensional space, so their content appears at the appropriate positions relative to the viewer, and motion parallax when rendering novel viewpoints works as expected. As with and MPI, the MSI layers should be more closely spaced near the viewer to avoid depth-related aliasing. The familiar inverse depth spacing used for MPI’s yields almost the correct depth sampling for MSI’s, assuming depth is measured radially from the rig center. The spacing we use is determined by the desired size of the interpolation volume and angular sampling density as described in Appendix A.
3.2.1 MSI Rendering. For efficient rendering, we represent each sphere in the MSI as a texture-mapped triangle mesh and we form the output image by projecting these meshes to the novel viewpoint, and then compositing them in back-to-front order. Specifically, given a ray r corresponding to a pixel in the output view, we first find all ray-mesh intersections along the ray. We denote Cr = {c1, . . . , c } and Ar = {1, . . . , } as the color and alpha components at each intersection, sorted by decreasing depth. We then compute the output color cr by repeatedly over-compositing these colors. [..] We parameterize the MSI texture maps using equi-angular sampling, although other parameterizations could be used if it were necessary to dedicate more samples to important parts of the scene."
https://storage.googleapis.com/immersive-lf-video-siggraph20...
On the other hand there are some recent backlash suggesting these direction and default in Unreal is making games look worst. [1] [2] And Nanite may not be the silver bullet we were looking for.
[1] https://www.vg247.com/unreal-engine-5-has-been-a-disappointm...
[2] https://www.youtube.com/watch?v=lJu_DgCHfx4
Nanite was never supposed to be a silver bullet but a way to enable high density mesh streaming and high fidelity shadows and lighting when used with Lumen. Both techniques work incredibly well.
The only valid criticism is games getting too expensive (to make) and frame generation can work against gameplay. But both are really up to game designers to use appropriately, not engine providers to dictate.
I'm not really sure what the alternative here is supposed to be anyway. Is it Unity? Godot isn't a realistic competitor yet for this level of fidelity, though it'll probably get there eventually.
On PC hardware it seems unlikely, more chance of Epic fixing their engine.
You are better off using Google Maps API if I'm being honest.
That's one use.
There are many others, such as simulation of human behaviour in a digital twin (you can make sure experiment variables remain the same, difficult to do in the real world), collaboration and ease of access to archeological sites around the world, aiding architectural work, etc...
There will always be a need for software like this as long as the world has these restrictions.
A prairie is called a steppe if it's in Asia. A steppe is called a prairie if it's in America.
A hurricane is a typhoon, except it's striking the west coast of the Atlantic instead of the west coast of the Pacific.
None of these make any difference to what the object is like. Why do we care? We don't call mountains something different when they're in Asia. We don't even call them something different when they're underwater, which makes a huge difference.
Also, prairie and steppe are subtly different, though if it weren't for historic reasons they might be named the same. A prairie is more moist and has more vegetation as a result, and can support more trees and general flora/fauna.
This is not true of American English, where "cyclone" unambiguously refers to a tornado.
Outside the heartland, the NHC categorizes many storms below the level of hurricane (64 knot sustained winds) as various kinds of cyclone (tropical cyclone, extratropical cyclone, potential tropical cyclone, post-tropical cyclone, alongside depressions).
In fact, in meteorology terms, a tornado is definitively different from a cyclone (a column of rotating area vs an area of area rotating around a low-pressure system). Hurricanes and typhoons are both kinds cyclones.
If I play a mile I'm like Arma, I don't actually want to be a soldier. Me playing Cooking Mama isn't a veiled psychological thing: I like both cooking and the gamified, simulated cooking.
It’s used for architectural Visualization, film backgrounds, and industrial visualization among other things.
Surely you can imagine why having realistic settings is beneficial in those scenarios.
No. Some people just like realism in their video games, and prefer the immersion or just think it looks cool. Not everything is a manifestation of existential dread.
If there were a general trend towards creating realistic enviroments as a means of psychological escape, one would expect those environments to tend to be more pristine and idyllic than reality. But often that realism is used to depict worlds no better, and sometimes far worse, than our own.
I'm sure some people are using realism as a means of escaping the destruction of our real world, walking simulators and the like exist, but I don't think that's a relevant influence overall. Just like with realistic CGI, people mostly do it because they can.
I know people who will practice flying at unfamiliar sites on it first - so that they’re better prepared when they get to the real thing.
For example, the INTENTION of antidepressants may have been at one time to treat severe depression, but as society (hypothetically) gets worse and more depressing, they may be used to help people cope with systemic problems and therefore ameliorate people's reaction to them.
Or phones: they initially were created to help people communicate but they are subverted (co-opted) by the system to make people more dependent on technology for the sake of technological growth.
Any number of products or ideas can start as one thing and become co-opted to serve a more insidious function beyond their initial purpose.
They point of having virtual environments is to enable you to see things in them you'd rather not see in real life.