This was also my first thought when I zoomed in into the strawberry. I wonder if you can achieve a microscope like effect with a more suitable setup. E.g. better lighting, zoom, lens, etc.
I have done 2x macro (an ant).. and want to try 5x.. but as you get closer, the depth of field becomes really shallow. You can do focus stacking but you risk that the individual areas in focus are less ideal aligned and the tracking can't make any sense out of the geometry anymore.
From the link: "Shot from 90 perspectives, 88 focus stacked images each. Nikon Z8, full frame, f/7.1, exposure 1/160, ISO 100, Laowa 180mm macro lens, with LED light and bluescreen."
Insane!
i made some decent splats (admittedly much lower quality than this) by taking video with my iphone13 mini and then chopping it up into individual frames via ffmpeg
There is actually still a Lytro type camera for industrial application available: https://raytrix.de/
Too low res, too expensive for my use case unfortunately.
Assuming that the person that did this has not tried that. If you look at the setup photos, the grape is resting on a couple of nails. This suggests that many different things have been tried.
I wonder would a good, sharp needle and thread make for good mounting for a soft object like this? Thread the needle, pass it right through the strawberry and the secure the thread on something above and below. As long as the strawberry doesn't slide down the thread (hopefully a strawberry is light enough friction would hold it in place!)
I glue mounted the strawberry on three nails and used pins to secure it.
I have to think about your idea.. I don't think friction would be enough to hold it in place.. it would probably be hard to knot the top onto something too, as that's where the light is: https://i.imgur.com/vIjw6pc.jpeg
But I'm always experimenting with the mounting, thanks for the inputs.
The splat of the studio has the perfect amount of detail. It looks like you're streaming from the camera direct to the computer usually? How do you check the progress / quality while capturing? Seeing the (great) results makes me more curious about the process of creating now.
Thanks! I use the computer only to setup the shot, adjust lighting make test stacks of different angles. If everything is good, I disconnect and capture to the memory stick. During the shooting I only adjust the beginning or end of the rail a bit, so everything is in focus at some point. Triggering (manually) the rotary disk still happens from the computer.
I'm wondering if the splat community has decided this paper is valuable -- https://github.com/fraunhoferhhi/Self-Organizing-Gaussians -- looking at all the detail in the strawberry splat made me wonder how small one can get the download, and what the current state of the art is for compression.
My intuition is that in theory focus stacking should not be necessary as preprocessing step for 3dgs (or photogrammetry). Does anyone know if there is any recent developments in this regard?
Focus stacking generally is not perfect process and can lead to artifacts/errors and I'd imagine those can then compound when stacked images are used for 3dgs. Also the image focus actually provides some depth data in itself that could be useful?
If you don't focus stack and try to train on partially unfocused images, the optimizer will try to match the rendered view to be also partially unfocused.
You would have to mask out the blurry areas for each image. I guess one could just implement a feature where the optimizer only optimizes gaussians within the sharp distances relative to the camera.
Other way of looking at the question is if you could make focus stacking better by using the full multi-view dataset? Afaik focus stacking essentially does depth estimation so it seems like multiple views would help with that.
Another way would be some kind of 4d GS where one dimension is the focus distance. But I'd guess the renders would inherently have shallow depth then, which is less useful usually.
My take.. at a macro scale, the dof is usually so small, that it's hard to get a reliably track. So you'd need some sort of way to tell that these stacked photos belong into a series, and then you sort of are doing focus stacking :-)
I do think the alignment algorithm could be improved. Maybe the approaches I linked could be used to make a much more robust focus stacking algorithm, that also corrects for 3D geometry. That would be really cool!
Lots of translucent blobs composited to produce the appearance of a strawberry.
There is no mesh or model. The visual surface of the strawberry could be made up of blobs spaced far apart physically and not where the surface appears to be.
This is why they are called radiance fields, they model the light not the geometry.
Practically the blobs positions/rotations can be constrained to better physically match the geometry of a strawberry.
I'm not sure i agree. The blobs are exactly where the surface appear to be because they are constrained by multiple viewing angles.
Otherwise the splat would fall apart as soon as the viewing angle is changed slightly (Which it absolutely does in many examples on supersplat, you cannot really create an out of distribution view with 3GS, it's not magic)
Yes, my statement was loose. The blob doesn’t really have a position since it is theoretically an infinite distribution in 3 space.
It has a mean, and that mean doesn’t have to lie on the surface, consider the case where the mean is deep inside the strawberry but its spike contributes to the surface appearance (e.g a seed could be represented this way, or it could be represented by a small well-oriented blob on the surface, the optimiser doesn’t care)
Oh thanks - I was waiting for a moment where I could turn up sound to watch the other video, but I didn't realise that that would set me back half an hour. This is the perfect amount of background for now!
Others have provided details about how it works. I suggest zooming way in on that image and you'll start 'breaking though' the surface and that'll help you get an idea of how it works. Important thing is there is no defined geometric surface ("mesh"). Also important to know is that it's very, very hard to get a good splat without taking a ton of photos at different angles. It's also really, really easy to create a crappy looking splat. But when it's done right, it's a marvel
When you say its very very hard to create a good splat, what do you mean? And what is good? I would say that strawberry is very detailed and its a good splat. I also kind of like the way some of the 'rougher' splats look. I feel like they'd work well in a car racing simulator.
What I love about gaussian splats is the way they degrade - instead of a hard cutoff or LoD changing spheres into cubes etc., they get increasingly "dreamy" - the basic idea is still there, just less detailed.
I don't know anything about them but it's a cool effect. At least on this strawberry, you're not zooming in but rather traveling closer. I don't see the increasing (made up) detail you'd expect from a zoom, we sort of pop through the skin into an invented interior.
its funny that sky features like clouds and blue patches end up being fit very close to ground level because there isn't a difference in perspective to cue in the algorithm that the skybox should be tall, I wonder if there is any way to incorporate the fitting algorithm with simultaneous lidar data about true distances of things to allow scenes to be viewed from further up
Yeah, it's an incredibly cool effect. Reality breaking down into fog and frosted glass and paint smudges and slivers of northern lights and all the dandelion fluff. Even navigation becomes harder and less predictable as coherence recedes.
This feels so much closer to how minds must store and process spatial information than the usual 3D models do.
Ah the timeless joy of falling through the floor geometry.
Seriously though - it's breathtaking.
The first guy who figures out the bridge between splats and dynamism - animation, editing, responsiveness - is going to be one of the immortals of 3d design.
There are many ways to represent 3D data, but animations really only work properly with polygon meshes (e.g. triangle surface meshes or volumetric tetrahedral meshes).
But the mesh is itself an abstraction, you just need to build that bridge.
We've been leaning away from pure polygons for decades, anyway. Vertex skinning, SDFs, volumetrics, simulation, and a lot more.
The meshes in a From Software game are for exmple hilariously simple, most of the animation is force simulation to make the famous "frizzles" that they like.
I'm not sure that's completely accurate? Vertex skinning isn't (necessarily) tied to polygons . . but to having points (or any parameterized features) that can be transformed by a weighted blend of matrices.
The "vertex" in "vertex skinning" is really just "a thing with a position that gets moved."
p' = Σ wᵢ · Mᵢ · p
It's just a position. Triangles can come along for the ride downstream, but they're not essential, which is one of the reasons it's so efficient for some stuff. Polygons are the optimal surface - but surfaces are often extraneous.
Take all this a few hefty grains of salt, I'm an amateur in the field. My 3d/CAD work is strictly in support of my enterprise stuff. And making wicked battlemaps for gaming VTTs, natch.
But I will stand by the overarching statement that polygons are in fact an abstraction, and bridging that abstraction with whatever is in splats would be wicked awesome.
If you can sample points inside a volume, in theory you could do that with splat geometry. If someone figures out a way to pass in animation time to a sampler, sample along geometry/wireframe or something else, and keep it from overly twinkling it might change everything.
I’m hand waving all the complexity into “if done one figures out”, of course.
I just don’t see why this method can’t evolve in the way diffusion models have evolved (knowing very little of the geberative mechanics of splats).
Imagine if we start designing GPUs around this technology as opposed to vectors. Imagine what voxel engines would look like. Would love a simulated experience or a small scale that theorizes about this.
The filesize of a 3d animated splat is seemingly very small, and the method enables ~arbitrary FPS. But it seems the setup required to record it is still huge and expensive, which limits its usefulness.
Even with that there are some interesting use cases, eg. I'd love to be able to watch concerts this way, and freely move around the stage and crowd from any angle.
This one keeps crashing the browser after it reaches 100%. Safari/iOS, iPhone 13. I was able to navigate and use a few of the other ones linked to from the comments, though. Curious.
There is a faint sensation of translucency, I wonder if that's an artefact of the process, or if it's the actual optics of the surface layer if the strawberry...
Well, at certain angles you actually get very noticeable gaps in the strawberry that are visible when rotating it, but almost invisible when static. I think it's mostly due to that.
It's an artefact unfortunately. Gaussian splats have no concept of refraction and have a hard time dealing with reflection. Highlights, usually so so. Something I always wrangle.
I wonder if that is somehow related to how you seem to have very strong studio light for the subjects? Somehow the bright light penetrating deeper into the material or something like that
The light doesn't need to be completely flat like what you'd do when scanning it more traditionally, for meshing. The problem is when the highlight is sharp and reflects it's surrounding. You could remove it, with cross polarization.. But then it wouldn't look realistic anymore.
Looking at all the outdoor scenes that you can walk around, I wonder how long until we start seeing this in places like Google Maps/Earth, as a replacement for the low-res 3D renderings we have now.
I guess the number of samples required to generate a GS is the constraint now, but maybe that will get solved.
rendering them is also non trivial from what i understand, since the millions of indivudal point positions need to be recalculated all the time while moving
I usually use PostShot.. but the quality was not very good. I want to try LichtFeld but my Graphics Card has too little memory.. so I reached out on twitter, some people ran tests and Mykhailo got some better quality out of it so I took his training. You can d/l the COLMAP dataset for free and try yourself.
(I'm ... still not sure what I'm looking at on TFA, and whether or not my browser configuration fails to fully present the site as intended.... OK, if all you're seeing is a blurred image of a strawberry, yes, you'll want to enable a bunch of JS resources. I'm using uMatrix, several hosts must be enabled.)
I like to think of Gaussian Splatting as a stage full of improv actors who are iteratively/concurrently filling details in regions down to pixels until the full picture is painted.
I know feelings about AI are mixed. But when AI can dream up gaussian splats in real time, from a prompt, and do refinement as you get closer to things... That's going to be pretty bonkers.
That's kinda what NERFs are (neural radience fields). They actually preceeded this Gaussian story, with Gaussians coming in and outperforming them. Maybe they'll merge later for something even better, I don't know enough about them.
Yes they are image generators. You want image generator generators.
A diffusion style process generating gausians instead of pixels. You could possibly do nerfs that way, but it would be effectively generating a trained network. If you managed to do that it would have broad application throughout the field of AI.
Probably? I'm no expert, just a SysAdmin trying to keep up really... but in my head it's would look like a form of MoE that would gen the 'Expert' model on demand instead of having a variety baked in.
That's assuming you could even reasonably train a neural net to output viable weights, of course.
This will be the future of a class of 3d Game. the prompt may not be text however.
An input of a kind of schematic representation of what the designer wants would be better. It may resemble a storyboard or a collection of organised notes that large projects tend to already use.
Fully generative could probably do some cool things, but people will still want to bring their peronal vision to life.
Yeah, when you describe that, I picture Wave Function Collapse to generate a map schematic... And then a text prompt, and some style photos the designers want it to match.
Curious, why wouldn't the future be a full world model like Google's Genie? It just renders every pixel so someone could still make their vision come to life via a prompt too.
It could be done that way but you are spending parameters managing the fact that the output changes completely with a change in view position or orientation. A observer independent model only has to manage changes of things that are actually changing in the world.
Since you can view Gaussian splats from any POV you end up generating an output that is closer to the representation of the world instead of a projection that a single observer sees.
I could see a kind of fun game / design tool / worldbuilding where you get a blurry world and you describe what you are seeing, and it comes into focus. The game world, mechanics, aesthetic, and playstyle build as you form your view. A sort of fog of war meets rorschach game.
This is like a beautiful little miniature. It's cool to see gaussian splatting applied to a detailed small thing versus a big scene.
I have a question about perhaps the most boring aspect of this strawberry: the license. you write, "You can download it under CC BY license, but attribution is appreciated rather than required." IANAL, but I'm pretty sure you can't license your work CC-BY and then waive the BY requirement in the description. Rather, you'd have to license it with something more permissive like CC0 and request attribution if you want attribution to be optional. Is that right?
It's their content. They can do anything they like.
Sort of. There are countries like France where attribution rights are fundamentally inalienable, and the author can always demand attribution, even after forever waiving the right to attribution. But in the US, the party who benefits from a contractual provision can generally choose to ignore or forfeit that provision. Whether it's still called the "CC BY" after that is debatable, since the whole point is attribution, but licensing by the CC BY and immediately waiving that requirement seems legitimate to me.
I want some math Phds to sit down with the Corridor Crew guys and figure out how to make Gaussian Splat reconstruction way better, the way they came up with CorridorKey.
I tried making one, but I couldn't make through the camera position tracking bit, software was super unintuitive. Very interested in gaming applications for this tech, but still waiting for it to be more approachable from a layman's point of view.
One thing you can do with this that works quite well is use it to "decrop" (widen the viewing angle) of a video. This is very useful for stabilization which usually involves cropping. Here's an example: https://x.com/i/status/2051504427287404568
Once they get quicker to train, I expect this to be a popular use of them
Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro.
It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling. But ml-sharp has 2.6 GB weights, a bit too big to run in the browser.
The flesh of the strawberry is very slightly translucent. By taking images from many angles, you can derive some information about the interior. This is not a separate step, it is simply the optimizer finding splats that contribute the right amount of color and light at various viewing angles. Of course those splats aren’t really going to look right to anyone who has ever eaten a strawberry, because they are only derived from the input images and not your past memories of a real strawberry.
You’ll also see the same type of problems in any incomplete part of any scene. Go poke around any of them and you’ll find places where there wasn’t enough information to make a satisfying reconstruction. For example, the back of the moon <https://superspl.at/scene/2ac8f423>, the fridge in the coffee shop <https://superspl.at/scene/6a0c3ccf>, underneath the footbridge in the forest <https://superspl.at/scene/23ebe85c> (or indeed anything in the distance), etc, etc.
A Youtuber that records jookin (a mid-South form of dance) uses Gaussian splats for his videos to bring the dancers out into the forefront and hold center stage, it's a neat trick.
Does anyone know if it's already possible to build gaussian splat of the person that moves/rotates from the single camera? (I.e. to use sequence of frames to reconstruct occluded parts of the body for other frames)
As I understand it, Gussian Splats aren’t a method of constructing missing data in the sense you’re asking here.
But there are other, well established, methods of generating missing data in convincing manner, that are beyond my field of expertise or interest to be able to repeat here.
With the added benefit that if I’m wrong, being wrong can be a great way to spur discussion 8)
As I have learned about Gaussian Splatting just a few weeks ago, I have (perhaps funny/naive/stupid) question: is there any progress or at least theoretical chance to have dynamic lighting?
out of the box, I imagine that surfaces can be lit, but probably not shadowed correctly. (structures aren’t solids, more like particles in 3d space)
it could look like the real-time lighting of an old game engine on rather modern assets. (quake 2-3 era)
or perhaps some "occlusion pre-pass" could be done to create a voxelized sparse volume from the splats that set a "voxel opacity value" for each to absorb light? (not far from how prebaked GI works nowadays)
note: not an expert on rendering, just a nutjob that did stuff in opengl in the old days.
There are some works on doing this directly e.g. https://arxiv.org/abs/2601.23065 but getting accurate materials is a challenge for anything more than diffuse.
AI-based relighting will no doubt start working soon.
Oh, that would be amazing! The photos are also available and some people definitely take it for a test ride. However.. This dataset with the stacking is far from generic.
That new Laowa 180mm macro lens is amazing. So sharp, free of chromatic aberrations, compact, and I like the long working distance and reduced perspective distortion that the 180mm focal length offers.
Thanks for the reminder how fucked up British government is. Can't run VPN here. (imgur is not available in the UK thanks to those authoritarian twats called Labour)
But something looks off: the red area around the "seeds" is pushed towards the center of the strawberry - or that at least the outer most layer is somewhat transparent and some deeper layers are visible.
Not traditional Gaussian splats since they have their lighting baked in but there are alternate forms that have the ability to do so but largely in academia or very specific use cases
carlos-menezes | 10 hours ago
p0w3n3d | 10 hours ago
gobdovan | 10 hours ago
p0w3n3d | 10 hours ago
Edit. TIL Poe's law
dudefeliciano | 10 hours ago
[OP] danybittel | 10 hours ago
josh-wrale | 10 hours ago
Kalendermann | 10 hours ago
[OP] danybittel | 10 hours ago
jcattle | 10 hours ago
For example this bumblebee: https://superspl.at/scene/cf6ac78e
Edit: I completely missed that this was posted by him (:
kridsdale1 | 6 hours ago
galsapir | 10 hours ago
[OP] danybittel | 10 hours ago
dudefeliciano | 8 hours ago
MattCruikshank | 7 hours ago
I was really pulling for OTOY to keep making light field capture and display technology...
[OP] danybittel | 6 hours ago
mgaunard | 10 hours ago
[OP] danybittel | 10 hours ago
gobdovan | 10 hours ago
4gotunameagain | 10 hours ago
CatMustard | 10 hours ago
Anyway, very cool splat, fair play
[OP] danybittel | 8 hours ago
But I'm always experimenting with the mounting, thanks for the inputs.
josh-wrale | 10 hours ago
gobdovan | 10 hours ago
timonoko | 10 hours ago
timonoko | 10 hours ago
Yes. I knoweth what "splats" are: They are splats of fuzzy blobs on the display surface.
brazzy | 10 hours ago
[OP] danybittel | 10 hours ago
a1o | 10 hours ago
(Can we do a Gaussian Splat of the setup of the photograph for the Gaussian Splat of the Strawberry?)
[OP] danybittel | 10 hours ago
a1o | 8 hours ago
burningion | 8 hours ago
[OP] danybittel | 8 hours ago
svetlins | 10 hours ago
voidUpdate | 10 hours ago
bozdemir | 10 hours ago
[OP] danybittel | 10 hours ago
vessenes | 10 hours ago
I'm wondering if the splat community has decided this paper is valuable -- https://github.com/fraunhoferhhi/Self-Organizing-Gaussians -- looking at all the detail in the strawberry splat made me wonder how small one can get the download, and what the current state of the art is for compression.
[OP] danybittel | 10 hours ago
zokier | 10 hours ago
Focus stacking generally is not perfect process and can lead to artifacts/errors and I'd imagine those can then compound when stacked images are used for 3dgs. Also the image focus actually provides some depth data in itself that could be useful?
KeplerBoy | 9 hours ago
You would have to mask out the blurry areas for each image. I guess one could just implement a feature where the optimizer only optimizes gaussians within the sharp distances relative to the camera.
zokier | 9 hours ago
Another way would be some kind of 4d GS where one dimension is the focus distance. But I'd guess the renders would inherently have shallow depth then, which is less useful usually.
zardo | 5 hours ago
That seems like the way to do it, someone has to be working on that.
searching...
https://dof-gaussian.github.io/
[OP] danybittel | 8 hours ago
My take.. at a macro scale, the dof is usually so small, that it's hard to get a reliably track. So you'd need some sort of way to tell that these stacked photos belong into a series, and then you sort of are doing focus stacking :-) I do think the alignment algorithm could be improved. Maybe the approaches I linked could be used to make a much more robust focus stacking algorithm, that also corrects for 3D geometry. That would be really cool!
Vinnl | 10 hours ago
[1] https://en.wikipedia.org/wiki/Gaussian_splatting
marceldegraaf | 10 hours ago
k2xl | 8 hours ago
ddon | 2 hours ago
StevenNunez | 10 hours ago
jaccola | 10 hours ago
There is no mesh or model. The visual surface of the strawberry could be made up of blobs spaced far apart physically and not where the surface appears to be.
This is why they are called radiance fields, they model the light not the geometry.
Practically the blobs positions/rotations can be constrained to better physically match the geometry of a strawberry.
KeplerBoy | 9 hours ago
Otherwise the splat would fall apart as soon as the viewing angle is changed slightly (Which it absolutely does in many examples on supersplat, you cannot really create an out of distribution view with 3GS, it's not magic)
jaccola | 9 hours ago
It has a mean, and that mean doesn’t have to lie on the surface, consider the case where the mean is deep inside the strawberry but its spike contributes to the surface appearance (e.g a seed could be represented this way, or it could be represented by a small well-oriented blob on the surface, the optimiser doesn’t care)
KerrickStaley | 9 hours ago
Vinnl | an hour ago
stevepotter | 7 hours ago
lubesGordi | 5 hours ago
ZeWaka | an hour ago
I took maybe 10 pictures of a model I built and threw it at my 3060 during dinner and it came out quite nice.
Tade0 | 10 hours ago
What I love about gaussian splats is the way they degrade - instead of a hard cutoff or LoD changing spheres into cubes etc., they get increasingly "dreamy" - the basic idea is still there, just less detailed.
Take for example this scene:
https://superspl.at/scene/e721ea7c
If you navigate closer to the trees, things around you become blurry - as if the very fabric of reality unraveled.
scrumper | 9 hours ago
[OP] danybittel | 8 hours ago
xattt | 8 hours ago
jl6 | 6 hours ago
acc_297 | 7 hours ago
amelius | 4 hours ago
0x1ceb00da | 2 hours ago
stevep98 | 4 hours ago
https://www.youtube.com/watch?v=arTIRgdEb1g
ACCount37 | 2 hours ago
This feels so much closer to how minds must store and process spatial information than the usual 3D models do.
squidsoup | 27 minutes ago
https://bayardrandel.com/gaussographs
More recent work on my instagram
https://www.instagram.com/bayardrandel/
ivolimmen | 10 hours ago
data-ottawa | 8 hours ago
That is indeed a very cool scene being about to wander around and still have decent resolution
lopsotronic | 7 hours ago
Seriously though - it's breathtaking.
The first guy who figures out the bridge between splats and dynamism - animation, editing, responsiveness - is going to be one of the immortals of 3d design.
cubefox | 6 hours ago
thfuran | 5 hours ago
lopsotronic | 5 hours ago
We've been leaning away from pure polygons for decades, anyway. Vertex skinning, SDFs, volumetrics, simulation, and a lot more.
The meshes in a From Software game are for exmple hilariously simple, most of the animation is force simulation to make the famous "frizzles" that they like.
cubefox | 2 hours ago
lopsotronic | an hour ago
The "vertex" in "vertex skinning" is really just "a thing with a position that gets moved."
p' = Σ wᵢ · Mᵢ · p
It's just a position. Triangles can come along for the ride downstream, but they're not essential, which is one of the reasons it's so efficient for some stuff. Polygons are the optimal surface - but surfaces are often extraneous.
Take all this a few hefty grains of salt, I'm an amateur in the field. My 3d/CAD work is strictly in support of my enterprise stuff. And making wicked battlemaps for gaming VTTs, natch.
But I will stand by the overarching statement that polygons are in fact an abstraction, and bridging that abstraction with whatever is in splats would be wicked awesome.
data-ottawa | 57 minutes ago
If you can sample points inside a volume, in theory you could do that with splat geometry. If someone figures out a way to pass in animation time to a sampler, sample along geometry/wireframe or something else, and keep it from overly twinkling it might change everything.
I’m hand waving all the complexity into “if done one figures out”, of course.
I just don’t see why this method can’t evolve in the way diffusion models have evolved (knowing very little of the geberative mechanics of splats).
ramon156 | 9 hours ago
ovenchips | 9 hours ago
dormento | 7 hours ago
This is one of the most endearing things about open source IMHO, the way people can find novel uses for it.
ovenchips | 5 hours ago
ovenchips | 3 hours ago
ImJasonH | 9 hours ago
The filesize of a 3d animated splat is seemingly very small, and the method enables ~arbitrary FPS. But it seems the setup required to record it is still huge and expensive, which limits its usefulness.
Even with that there are some interesting use cases, eg. I'd love to be able to watch concerts this way, and freely move around the stage and crowd from any angle.
idoco | 9 hours ago
chimpanzee2 | 9 hours ago
As I scrolled through the website, I was even more impressed with this one though!
https://superspl.at/scene/c67edb74
FigmentEngine | 8 hours ago
MattCruikshank | 7 hours ago
If I'm reading Chrome right, that's 171 MB, for the website, and the data.
If I'm doing the math right, that's 40 seconds worth of the bandwidth from Netflix, at its highest rate.
mkmk | 7 hours ago
ComputerGuru | 5 hours ago
busymom0 | 2 hours ago
bestouff | 9 hours ago
slimbuck | 9 hours ago
probably_wrong | 9 hours ago
corford | 7 hours ago
evrimoztamur | 9 hours ago
vanderZwan | 9 hours ago
[OP] danybittel | 9 hours ago
zokier | 5 hours ago
[OP] danybittel | 4 hours ago
sbarre | 9 hours ago
I guess the number of samples required to generate a GS is the constraint now, but maybe that will get solved.
dudefeliciano | 8 hours ago
kridsdale1 | 6 hours ago
jofzar | 8 hours ago
https://youtu.be/gXug7Kb3p4I
classified | 9 hours ago
arnabdey0503 | 9 hours ago
[OP] danybittel | 8 hours ago
KeplerBoy | 8 hours ago
[OP] danybittel | 8 hours ago
artursapek | 8 hours ago
dredmorbius | 8 hours ago
(I'm ... still not sure what I'm looking at on TFA, and whether or not my browser configuration fails to fully present the site as intended.... OK, if all you're seeing is a blurred image of a strawberry, yes, you'll want to enable a bunch of JS resources. I'm using uMatrix, several hosts must be enabled.)
karmakaze | 3 hours ago
tantalor | 8 hours ago
What's the matter?
jrflo | 7 hours ago
tantalor | 7 hours ago
andybak | 5 hours ago
jrflo | 2 hours ago
duckerduck | 8 hours ago
lubesGordi | 5 hours ago
MattCruikshank | 7 hours ago
perching_aix | 7 hours ago
MattCruikshank | 7 hours ago
Lerc | 7 hours ago
A diffusion style process generating gausians instead of pixels. You could possibly do nerfs that way, but it would be effectively generating a trained network. If you managed to do that it would have broad application throughout the field of AI.
dpoloncsak | 5 hours ago
xigoi | 4 hours ago
dpoloncsak | 3 hours ago
That's assuming you could even reasonably train a neural net to output viable weights, of course.
cubefox | 7 hours ago
thrownthatway | 6 hours ago
https://youtu.be/X8yRlA7jqEQ
notdefio | 7 hours ago
yard2010 | 7 hours ago
Lerc | 7 hours ago
An input of a kind of schematic representation of what the designer wants would be better. It may resemble a storyboard or a collection of organised notes that large projects tend to already use.
Fully generative could probably do some cool things, but people will still want to bring their peronal vision to life.
MattCruikshank | 6 hours ago
satvikpendem | 6 hours ago
Lerc | an hour ago
Since you can view Gaussian splats from any POV you end up generating an output that is closer to the representation of the world instead of a projection that a single observer sees.
basch | 4 hours ago
corysama | 2 hours ago
https://research.nvidia.com/labs/sil/projects/lyra2/
buibuibui | 7 hours ago
Centigonal | 7 hours ago
I have a question about perhaps the most boring aspect of this strawberry: the license. you write, "You can download it under CC BY license, but attribution is appreciated rather than required." IANAL, but I'm pretty sure you can't license your work CC-BY and then waive the BY requirement in the description. Rather, you'd have to license it with something more permissive like CC0 and request attribution if you want attribution to be optional. Is that right?
RobotToaster | 7 hours ago
[OP] danybittel | 6 hours ago
CobrastanJorji | 2 hours ago
Sort of. There are countries like France where attribution rights are fundamentally inalienable, and the author can always demand attribution, even after forever waiving the right to attribution. But in the US, the party who benefits from a contractual provision can generally choose to ignore or forfeit that provision. Whether it's still called the "CC BY" after that is debatable, since the whole point is attribution, but licensing by the CC BY and immediately waiving that requirement seems legitimate to me.
MattCruikshank | 7 hours ago
weystrom | 7 hours ago
ChadNauseam | 4 hours ago
Once they get quicker to train, I expect this to be a popular use of them
sovok | 7 hours ago
Apples model to generate Gaussian splats from a single image. Takes about 30 seconds on an M1 Pro.
It falls apart once you move too much, but for a little side-wiggling or a second-eye view for VR, it's great. And looks a lot better than the old approach of depth map + vertex shaders that I use in https://github.com/combatwombat/tiefling. But ml-sharp has 2.6 GB weights, a bit too big to run in the browser.
RobotToaster | 7 hours ago
sovok | 7 hours ago
kridsdale1 | 6 hours ago
mnorris | 3 hours ago
ben7799 | an hour ago
It seems to work a lot quicker than 30s now on iDevices and Macs.
0xWTF | 7 hours ago
db48x | 6 hours ago
You’ll also see the same type of problems in any incomplete part of any scene. Go poke around any of them and you’ll find places where there wasn’t enough information to make a satisfying reconstruction. For example, the back of the moon <https://superspl.at/scene/2ac8f423>, the fridge in the coffee shop <https://superspl.at/scene/6a0c3ccf>, underneath the footbridge in the forest <https://superspl.at/scene/23ebe85c> (or indeed anything in the distance), etc, etc.
lightedman | 7 hours ago
https://www.youtube.com/watch?v=k0MGoFaIiWs - WARNING - heavily adult lyrics and a loud starting "MOTHERFU*AAAAAAS" right off the top.
hamburglar | 6 hours ago
lightedman | 53 minutes ago
evan_ | 6 hours ago
https://youtu.be/g1-46Nu3HxQ?si=y_u257DzgOg2QV83
divan | 7 hours ago
thrownthatway | 6 hours ago
https://youtu.be/X8yRlA7jqEQ
It kinda splatted my mind.
As I understand it, Gussian Splats aren’t a method of constructing missing data in the sense you’re asking here.
But there are other, well established, methods of generating missing data in convincing manner, that are beyond my field of expertise or interest to be able to repeat here.
With the added benefit that if I’m wrong, being wrong can be a great way to spur discussion 8)
ArekDymalski | 6 hours ago
aruametello | 6 hours ago
it could look like the real-time lighting of an old game engine on rather modern assets. (quake 2-3 era)
or perhaps some "occlusion pre-pass" could be done to create a voxelized sparse volume from the splats that set a "voxel opacity value" for each to absorb light? (not far from how prebaked GI works nowadays)
note: not an expert on rendering, just a nutjob that did stuff in opengl in the old days.
hellohello2 | 6 hours ago
hellohello2 | 6 hours ago
AI-based relighting will no doubt start working soon.
wren6991 | 5 hours ago
[OP] danybittel | 4 hours ago
porphyra | 5 hours ago
stuxnet79 | 4 hours ago
NoSalt | 3 hours ago
ted_dunning | 2 hours ago
varispeed | 2 hours ago
[OP] danybittel | 2 hours ago
ziofill | 2 hours ago
BobbyTables2 | 2 hours ago
painted-now | 2 hours ago
But something looks off: the red area around the "seeds" is pushed towards the center of the strawberry - or that at least the outer most layer is somewhat transparent and some deeper layers are visible.
monkpit | 2 hours ago
dagmx | 2 hours ago
vipshek | an hour ago
lucb1e | 15 minutes ago