It is still behind the screen, instead of video stream what you see is a 3-d rendered model which moves in real time.
wonder how it is different from a video call.
Also, microsoft did this a couple years ago, without fancy volumetric display, just face tracking on their expensive 8k(?) tv for meeting rooms, and it was a complete flop.
I feel like this is the kind of thing I'm going to have to experience in person to appreciate. The video just isn't conveying anything meaningful to me.
Still, the concept is exciting, and if the execution is there, it'll be one of the most important leaps in communications technology in decades.
And I'm looking forward to a company named something like InstaPresence (TM) applying filters and making us all photorealistic cat people.
Looks like tonari[0] will have some heavy competition sooner than expected. This seems to have a lot more attention paid to the sensation of depth than the tonari offering. Could see this being popular at high end senior care facilities.
At Atlassian, they built a Portal inspired portal that connects the Sydney and SF office during one hackathon. Basically a camera behind a 2way mirror (and decorated with blue/orange LEDs in an oval frame). Tonari looks a lot more like that than Starline.
Interesting thing to note is that they don't show the participants touching the glass pane 'separating' them, whereas for that kind of situation it would be very very natural thing to do.
I guess doing so (reaching towards the 'glass pane') would make the imagery distort/degrade real fast as you would start going out of camera's FoV, which that would break the magic.
A bit ironic that this promo video maxxes out in 1080p, despite YouTube supporting 4K and this tech pumping 8K+ res.
There is so much more we can do in terms of quality and immersion that we're not doing simply because bandwidth and connectivity are so low-quality and overpriced at most of our leaf nodes in the USA.
Hmmm. Personally, I would much rather have a lofi hologram that is capable of moving around, as much as I lack technical implementation specificity here :)
Starline seems amazing, but there's a black mirror quality to the ultra pristine resolution.
Is the technology used here for "tricking" the brain to simulate 3D the same as what Nintendo was using in the 3DS? If so, it carries a risk of developmental damage for the eyesight of children under 6, so as much as this might be cool they really shouldn't be showing kids using it, especially not babies.
Am I the only one that thinks one-on-one video is already fine, and what we actually need is to improve the experience so that a group of people feel like they're meeting in the same place?
Why does Google continue to try to produce new products? We all know they will never become reality, or if they do, they'll stop existing soon after they're open to the public. They're now just a regretware company.
Give it a year or 2 and the iPad Pro will have this as a standard feature in Facetime.
> Project Starline is currently available in just a few of our offices and it relies on custom-built hardware and highly specialized equipment. We believe this is where person-to-person communication technology can and should go, and in time, our goal is to make this technology more affordable and accessible, including bringing some of these technical advancements into our suite of communication products.
There’s a funeral scene in Upload (on Prime Video) that does something like this. It felt like a window into another room and I loved that concept. It looks like the tech for actually doing this is closer than I thought. Exciting!
This is getting downvoted but it really is a valid concern. Google’s been losing users on each discontinuation and only the most pro-Google techies I know are still jumping in to something new, especially given the trend towards one cool feature and a bunch of “QA is boring” stuff which will take a year to get fixed. Given their past reputation, it’s really cautionary as a shareholder to hear C-level questions about services like GCP questioning the risks of not going with AWS/Azure and getting stranded.
That’s a big deal for anything which requires hardware you wouldn’t otherwise own. Once you hear “custom-built hardware and highly specialized equipment” that sounds like something you really want a commitment to not just begrudgingly patch but to continue to seriously invest in the product.
It's getting downvoted because it is repetitive, with thousands of effectively identical copies being posted on HN yearly. It is not insightful. It is not clever. It's not reacting to the story, except for lazy pattern matching. It is just a tired meme that's been flogged all the way down to the bone by now.
(It's probably also not true, when compared to Google's peers. Amazon and Microsoft similarly throw a bunch of stuff on the wall, and unceremoniously kill the failures, but neither their launches nor cancellations get this reaction.)
Reminds me of Teliris (https://en.wikipedia.org/wiki/Teliris) which I got to demo 15 years ago. That product ran on dark fiber with special hardware to simulate two sides of a conference table. Really uncanny and unsettling etiquette-wise. One-on-one feels better all around.
If this tech turns out to be too expensive (for normal people) we could still use it on a pay-per-use basis, like with a "video conferencing booth". You'd schedule your call and reserve a local booth for both participants through an app. And most companies should be able to afford having one of these in the office.
That's just reminded me that phone booths were usually filled with advertisements for "escort services". I'm sure it wouldn't take long for this to come back in this brave new world
It would have to be really futuristic to convince people to coordinate timings, reserve a slot, pay and drive to some location for a video call. FaceTime/WhatsApp are still good enough for most.
The office use case is probably more realistic, but some other related products (Surface Hub, Jamboard) haven't become as ubiquitous as originally imagined.
I’d first go with an Internet cafe style booth booking. Book an hour slot and get your coffee included.
Setup cafes in major cities and I think people would use this - could imagine setting a meet with a friend in another country. Parents showing off their new babies. Etc.
I hate to say it but my first impression of this was that it looked like a visiting area for a prison. (Something about the bleak colour palette and minimalist display). But, I think that presents a similar long-distance use-case as the 'phone booth.' I wouldn't be surprised to see this as a pay-per-use option for prisoners/families. It's probably only a matter of time before the 'Echo Shows' capitalize on it.
Well at some point prisoners won't have the choice in practice, because for safety and convenience remote video call will become the norm.
This is the same paradox as social networks; making people supposedly closer, but in reality making humans further and further away.
A fantastic twist would be a reveal showing that the other person (no longer) exists in the real world and is digital construct... or some forbidden knowledge is secretly interdicted and censored by big brother, body-snatching peoples avatars in real time.
Yeah I had the exact same impression. Still vastly preferable over a zoom call, but I expect I'd have to use it extensively before I can shed that association.
This is assuming that this somehow does allow me to interrupt someone's sentence — otherwise might as well do a zoom call.
Totally unrelated but on the radio there was a company talking about making internet pods to be able to communicate without being in the open like in starbucks.
It seems we're seeing a second coming of distributed private comms.
This could bring fax machines back too. Draw something on a piece of paper, put it into the receptacle, and voila - it's instantly printed on the other side.
Or, I mean, live synced drawing tablets. Why paper?
(Unless you like the feeling of writing on paper like me but I bet money they’ll replicate that sooner rather than later).
1. Call for free from home/couch with a regular camera in the privacy of my home.
2. Go to some location and pay a third party to do a less private call, and have a better visual experience.
The most obvious choice would be number 1 in 99.9% of all cases.
In theory, privacy can be increased by creating a sound-proofed environment for your call, but in practice, that would easily become very expensive.
I would like to see this as a franchised service for cafes and bars. It would be nice to be able to treat drinks and the 3d chat experience to a friend who lives in another country. And you know, some conversations are done better at a place other than family home.
The main thing here is the radically improved UX (as advertised). With this kind of tech I might actually hook up with some remote friends with whom I currently just text (I don't really use video calls for anything other than work).
Edit: And even 0.01% of the video call market would be quite large. Naturally this idea wouldn't work if something like 10% of all video calls were replaced with these booths, they would be fully booked for weeks forward. ;)
Edit2: And we already know most people don't care that much about their privacy since they already use services like Google Meet and Zoom and whatnot.
I agree with you, I'm also imagining these in public libraries, retirement homes, small village community centres.
I just hope we don't end up with a bunch of incompatible, proprietary appliances that can't talk to each other.
I get similar vibes to demoing VR the first time. My first thought was, "My tech-illiterate dad would have his mind _blown_ by this."
Unfortunately, even an already-set-up VR experience was too strange/unnatural for him so he never got to experience it. However, this looks easy/natural to use and set up and feels like it'd have a similar mind-blowing effect for many of the older generations, which I think is a good indicator of being revolutionary tech (assuming it can be made available/cheap enough for most people to try it out).
I got my Dad to play SUPERHOT VR for a bit, he liked it, but I suspect the "Shoot yourself in the head to start the demo" bit was possibly a bit on the nose considering he spent quite a few years manning suicide helplines.
The "wow it's really 3D" effect without glasses is what people experience everytime I've shown Looking Glass to someone, glad to see devices with higher resolution are coming and more people in the market, end users will benefit from more concurrence to drive the quality up.
Now make that person in front of you a virtual 3d rendered artificial person. Looking forward to be talking to a virtual Albert Einstein for real, sorta.
I wonder how close you could come to this experience with some "standard" hardware like a standard 100-500mbit connection, a normal 4K TV and a Kinect. That is: given 10% of the cost, how many % of the experience could you deliver?
Same. I suppose they kept the decor and backgrounds stark to showcase what it's doing. But it does have that feel, as-is. At least nobody was wearing orange :)
"The receptionist’s phrase
had prepared her for the only kind of link she knew, a simstim signal routed via Bell
Europa. She’d assumed she’d wear a helmet studded with dermatrodes, that Virek would
use a passive viewer as a human camera.
But Virek’s wealth was on another scale of magnitude entirely.
As her fingers closed around the cool brass knob, it seemed to squirm, sliding along a
touch spectrum of texture and temperature in the first second of contact.
Then it became metal again, green-painted iron, sweeping out and down, along a line of
perspective, an old railing she grasped now in wonder."
A bit off-topic, but I couldn't help thinking about Tantacrul's video, Corporate Music - How to Compose with no Soul [1] when watching the video on this post.
I have very little knowledge of this, but as a product would it not be much cheaper to have something like a wii bar fixed to the top and bottom of a screen, with a small camera at the extremities of each bar (one for each of the four corners)?
You could then use the camera perspectives to create a 3d image of the person you're conversing with and map the colour data correctly to that 3d image (Photogrammetry)
You could also likely use the information from the four cameras to map the orientation of the 3d image of the person you're speaking with to give you that sense of depth as you shift your position. https://www.youtube.com/watch?v=Jd3-eiid-Uw
If you had a speaker and mic in each corner you could also capture / emit subtle differences in audio to further enhance it.
Looks like another Magic Leap but in TV formfactor—technology great in concept, and even MVP but completely unrealistic in terms of consumer grade HW as to affordability and accessibility. It will be another tombstone on a huge grave of Google products. Just like Google Glass and many other. It's too hard to trust Google these days and its initiatives.
It looks like magic, when I saw the video I briefly thought it was a joke (ha-ha, they're actually in the same room!). I wonder how much of the magic is software and how much is hardware?
The video certainly plays up the software, but I've never used zoom or FaceTime in an 8k video call booth before, so I suppose I don't have a point of comparison.
I'm not sure it's the parent's point (I suspect it is), but mine would be that it might be you get 80% of the "it's like they're there in the room" effect by having 8k; good lighting; well-separated, good quality, stereo sound; a purpose made space without distractions; etc.?
If you've only had 480/720 on a small screen, with a mono microphone with poor lighting, lag, dropouts, then just fixing those things and making it 2k+ might be surprisingly good (ie will have an emotional impact due to the truer representation)?
This is cool as hell, but I have to say I feel like we're solving top-level problems when most consumers don't even seem to be getting solutions to the most basic pain-points.
For me, the problem with video-calling isn't the image-quality. It's all the much more mundane technological problems - high latency, lag-spikes caused by bad ISPs, failed noise-cancellation for people who don't use headsets for audio, bad wifi routers cutting out, etc.
First thing I did when I realized we were going to be WFH long-term was buy myself a $100 gaming headset. Next thing I did was get all my home computer stations wired with Cat 6.
That stuff is far more fundamental and far less interesting than 3D telepresence, but it's the real unsexy problem that so many people are suffering through this pandemic.
Even simple things like latency make simple, natural reactions agonizing. Talkcover and crosstalk is incessent and I've developed a filthy habit of just talking over people because otherwise it's a solid 20 seconds of "you go no you go" caused by awful latency. I've had to defuse angry reactions by co-workers who feel they're being interrupted by other co-workers and explain to them that the latency makes interruptions feel worse than they are.
I've tried to push friends to join me on my private Mumble server where the latency is near-nil and the call-quality is excellent, but there's always one person who doesn't have a working headset and wants to just use a laptop or tablet mic with no feedback-cancelling that destroys the conversation through echos (plus Mumble's auth system is needlessly bewildering).
Then with video, problems are similar but less impactful - cheap cameras, poor lighting, compression artifacts, poor sync with the audio, etc. And it's infuriating because every person has a wonderfully powerful camera in their pocket right now - and there's software to connect them but it's just too tricky for most people.
Good on Google for taking an interest in the subject, but I feel like they're decorating the apex of the technological pyramid while most people are pushing stones around at the bottom.
Perhaps one drives the other. Availability of magical tech will drive adoption of best practices in all the areas you have mentioned.
In India, ISPs already advertise low latency and high speeds for "PUBG Gaming". The market evolves to solve the needs of consumers. Advertising for low latency for gaming was unheard of in Indian ISP scene, just a few years ago.
So having this tech, would induce customers to get better hardware, wiring, ISPs etc, and would induce ISPs to provide better service.
I mean to be fair Google has also tried very, very hard to improve home internet access for people, to the point of setting up their own ISP and running municipal fiber networks. That's a pretty big try, and I really wish it had taken off beyond the places where Google Fiber operates.
(NB: I work at Google, but this comment has nothing to do with my work.)
When it really comes down to laying Fiber isn't a job requiring innovation, it require hard man hours to pull cables.
In the end Google tried to innovate around the hard work by burying cables around 5cm deep or so in stead of a meter, which turned out to be short sighted.
Both problems are worth effort and energy. It is worthwhile to push the envelope at the top because that technology, if it really works and can be developed in a more consumer friendly way, will eventually become vastly more accessible.
Solving gigantic scale problems requires a completely different kind of innovation than what you can achieve by pushing the pinnacle of what's possible.
It also in its (albeit) small way could push ISPs to get their act together. Perhaps thats a bit naive to say but if this sort technology is available, this will help push the boat forward in terms of underlying infrastructure.
I mean Google also has Meet for regular videoconferencing that's available to normal humans for free, so it's not like they're ignoring the mainstream.
The issues with connection stability and latency are very real, but I don't know if it's reasonable to expect Google to fix it; the issues there are probably more political than technical.
edit: Also, I think they did mention using AI for noise cancellation while videoconferencing in the keynote today.
Most of that Google can't do anything about, because it's not under their control.
And it's really a question of money. If you want to fix all those mundane problems you describe, literally every one can be addressed by renting a better pipe and buying and setting up better equipment.
This Google product is clearly designed for high-end offices that already have all of those things under control.
Google owns one of the major consumer operating systems (Android) and a lot of videoconferencing goes via Chrome. They could absolutely be doing things like offering better connection quality metrics - look at how everyone currently ends up using a random shady "speed test" site.
Normal video chat at its best is just kind of shitty though, so people don't have a whole lot of motivation to put in a bunch of work to make it still shitty, but slightly less so.
If I could get video chat that felt like real life, that'd be worth running some wires for.
Convenience and necessity. If the apex tech is so wonderful that its convenience becomes necessity, then it will become those proverbial stones on the bottom.
This is a lot like Winscape, from 2012.[1] That's a display hooked to a Kinect head tracking system, so, as you move, the display is adjusted for your point of view. Winscape just used it as a window to the outside world. Same concept as VR headgear, without the headgear.
The illusion only works for one viewer at a time, because the image is POV-dependent.
Google is apparently using this with a 3D TV. Can you still get those?
The main problem with doing this now is getting all the retro technology you need - a 3D TV and the good Kinect. Any good current GPU should be enough engine for this.
I used to say Google has never done anything really great other than their search engine. And generally speaking this opinions tends to get quite a lot of upvote ( as well as some unsuccessful counter arguments ). That is why when another Google project launch or failed. I wont be surprised.
This is the first time, I felt this is going to be great. Something that should have came from Apple, the humane side of technology, for the first time ever came from Google.
Hopefully this isn't a one off thing or an outliner from Google. Apple desperately need some competition.
It’s a pretty silly argument. Maps, android, gmail, translate, docs, mapreduce, kubernetes, tensorflow come to mind off the top of my head. Honestly it’s a stupid contention. Apple’s awesome but the list of “great” things they’ve done is what, a third as long despite existing decades longer?
Feels like the future of this will be using AI approaches to simulate dead loved ones. I don't think that's a dystopian outcome, there are a lot of good aspects to it as well, even if we don't know the full ramifications.
Am I the only one horrified by this? The second photo of the person sitting in a booth looks like a set piece from some cautionary science fiction movie.
No, the whole setup looks dystopian to me. The presentation too knowing Google. Looks like a prison.
People are already depressed as hell from the lack of touch, real family, community and friends in modern society.
That said, of course this tech could have its uses, but mediated by the largest corporations on Earth that collect, sell and mediate everything about you and your friends? No thanks!
I can only imagine how unnerving it would be to hit a spike in latency or something and be shaken from your "suspension of disbelief" while your loved one glitches out and falls into uncanny valley territory.
My dad used to tell a story about my grandfather's first experience with television in the late forties. There was a buxom woman on the screen, and he walked up to it, trying to look down her cleavage. It didn't work.
Does Starline give you a different view when you change your perspective? It looks like it does to some extent. If so, it might work before long, but grandpa died about fifty years too soon for it.
My understanding of the existing tech that's out (3d displays) is that it allows for such on the X axis (different view per-eye) but not on the Y, so a scenario such as you described (looking downwards) might still remain sci-fi for the moment. Perhaps this does something entirely new/novel with display tech but there's nothing to suggest that at the moment.
On the topic of buxom women, my first thought about this tech was, it'll be the next frontier for OnlyFans/porn accounts on Snapchat.
Before you laugh or be prude, porn content was what made VHS and BluRay succeed (or are these urban legends?) and they were pioneers in stuff like online video streaming.
Ah yes, the people whose first thought is not how they can use this to commodify women’s bodies are “prudes”. Yes of course. Women: become literal prostitutes for the only fans corporation. You could be RICH! It’s so fun! They have t-shirts! Remember how I said you could be RICH?!
Not sure my comment will bring anything to change your opinion, you seem to be already a hardcore antiporn person, and I can't defend sex work either because yeah I know it's got a lot of seedy aspects to it; but no, they're not literal prostitutes, cam girls and OnlyFans models have the advantage that they can earn money without having to be in contact with other bodies and potential diseases they bring.
I know some girls who are prostitutes. They seem normal enough. Maybe they're wearing a facade and have hidden PTSDs, I don't know, but if that's the case, I think you might want to extend your crusade against the military.
I don’t know. You should ask some of the abuse victims coming out of the prostitution industry why they’re so upset, and remind them that their job is basically the same as working at google.
The fact that you instantly associate what OP mentioned to women specifically is quite disgusting, not going to lie
FYI More people than just women suffer through that and taking the oxygen of such a discussion via weaponizing linguistics is just nasty and a horrible way to spend the opportunity cost of it
That would really take it to the next level if it had that feature, even with very few degrees of freedom.
The (New) Nintendo 3DS has head tracking, but it doesn't change your perspective into the view port, which gives a very dizzying effect when you deliberately test the feature.
I would imagine its possible to extrapolate perspective if they had an array (N > 2) of cameras.
This is super cool tech, and can't wait to see an array of these installed in the secret undisclosed board rooms for the illuminati.
The biggest barrier to virtual healthcare today is that patients find it hard to connect with and trust doctors over a screen. This seems like a perfect product fit, especially among older people.
Everyone's latching onto the 3D part, but if some version of this just lets us have our Zoom calls without the looking at camera/looking at screen, impossible to make eye contact thing, that'd be a step forward.
I think another good use case for this sort of hardware/service could be for youtubers/streamers/performers who want to provide immersive interaction to the viewers. If I am a fan of a singer, I'd definitely pay for an opportunity to watch the person singing right in front of me, instead of going to the concerts where I watch them singing miles away.
This is actually groundbreaking technology. If this gets widely deployed and then evolves we're looking at telepresence as the next step in ambient computing. When the technology fades into the background that's when you know things are going to be remarkably different. This is quite honestly going to be as close to teleportation as we get.
Asim, are you familiar with how Stephen Baxter portrayed the use of telepresence in his science fiction novels? If not, check it out. But if you are, please explain it to me! It always confused the hell out of me.
I would love to try this - and use it for calls with family, etc.
I’m skeptical on its impact on the in-person vs. remote debate for work. What really distinguishes in-person for me is the potential for informality. Bump into somebody, have a chat, walk over to a colleague’s desk for 5 minutes, decide to go on a walk for a 1x1 instead of sit in a room. This doesn’t do anything for any of that. In fact, it arguably increases formality.
Understood that many folks are not interested in those in-person artifacts - sharing what I see as key distinctions which aren’t in the solution space of Starline.
This conceptually seems pretty similar to what Tonari [7] is doing, but there are some technological differences between the two. Just interesting, that's all.
Wow, will be great to make recordings of people that you can sit and watch long after they're deceased. Maybe with some deep fakery even create AI versions that can respond to conversation.
This, my friends, is how business travel becomes nearly irrelevant.
This is a beautifully executed idea and if the demos live up to expectation the hype may even be warranted. But on a much more fundamental level (i.e. fancy 3D imaging and spatial audio aside), this also possibly suggests people would benefit from dedicated videoconferencing hardware. TVs and telephones do one thing really well (or at least historically they did), which is why even my legally blind grandpa could call his friends or watch^W listen to the news. There's a market for having a plug-and-play videophone now that we have the software to go inside it.
I immediately thought of business travel as well. Project us all in to a virtual conference room and give us a suite of collaboration tools to use while we're all there.
The only thing missing is the after meeting drinks and dinner, but there will inevitably be services to put us all in a restaurant/bar environment, pipe in some bar white noise, have food sent from a local restaurant, etc. for an "in-person" virtual happy hour...
> there will inevitably be services to put us all in a restaurant/bar environment, pipe in some bar white noise, have food sent from a local restaurant, etc. for an "in-person" virtual happy hour...
I honestly can’t believe this still sounds fun to anyone after a year of Zoom dystopia.
The last year has shown pretty clearly that for presented content, video works pretty well. Heck, it maybe even works better along with live chat than in person. (And, to be honest, for big events I would often watch live streamed keynotes rather than crowding into a conference center with 5,000 of my closest friends anyway.)
But everything else about virtual conferences has completely sucked and anyone running events is aggressively trying to get back to in-person. (With a hybrid component for presentations.)
That "the only thing missing" is the main reason to go to many conferences.
A lot of people (those with less social desire/social skills) seem to resent it, but it is true: Networking and casual technical conversations that happen afterhours are the draw for many technical conferences. Talks can be good, and occasionally there are well-constructed lab sandboxes. But mostly, it's going and speed-dating with peers and sales teams to talk about your needs and architecture, and building a good web of contacts.
I also believe fully remote technical/collaboration work, without any periodic physical meetups, will be awful for a lot of people. Sure, those who bought into it pre-pandemic prefer it, and that's fine. But I really think there is concern to be had for the fraying of social bonds and teamwork that can be done, even (or especially) with people you have a tough time working with.
Yeah, the last year has shown us that streaming videos with some side chat is the easy part. Heck, maybe it's even better than in-person a lot of the time. I can re-record stuff when I screw up and do some things I can't do in front of a room of 50-100 people.
And it's even good that people who just go to sessions for the content will be able to do so--for a lot less money and effort. But I'm planning to go back to in-person conferences as soon as possible.
Better videoconferencing hardware is not a solution to people wanting to get together socially and serendipitously at events. And, by the way, how many people are going to turn a room in their house into a work videocall room?
Personally, I find that--for most people--the idea that working remote shoves a lot of cost onto employees vs. commuting probably off-base. (With some exceptions for people living in small city apartments near offices.) But installing a room-sized videoconferencing setup at home even for people with decent-sized houses is pretty silly.
> how many people are going to turn a room in their house into a work videocall room?
Very few while it's $20k+. But I can imagine a lot of people would want one if the price was reasonable. I'm sure you still use the room for other things.
> This, my friends, is how business travel becomes nearly irrelevant
the business trips i've been on either involved the installation of equipment, or were an excuse for somebody with budget to burn it on travel and expensive food/alcohol. or both.
i think plenty of business travel will survive just fine.
Entertainingly, the technology to real-time impersonate someone over zoom photorealistically is also rapidly approaching - the window of irrelevant travel might be small.
> There's a market for having a plug-and-play videophone now that we have the software to go inside it.
> What are Zoom, Facebook or Apple waiting for?
This is already a thing?
Facebook has Portal, Apple has iPad, Amazon has Echo, all of whom support Zoom and other video conferencing apps. The portal even has a moving camera to keep you centered if you're moving in frame, and the iPad does the same thing with an ultra-wide lens and some post-processing.
As far as dedicated videoconferencing hardware is concerned, Starline seems pretty late to the game. Although, I'll admit the fancy 3D imaging features is pretty insane.
Unfortunately, this stage of the Google product cycle is the hardest for me to start getting excited for. I hope for better, but I cannot resist the nagging feeling that this will 1) be very, very cool, 2) be sold to enterprise customers who are OK streaming business calls through Google's cloud, 3) suffer from having no support, 4) be renamed and reclaimed by another team inside Google, 5) sunset.
Is "Starlink" going to be a Gmail or a Wave? Hard to say.
From my experience business travel is as much about sharing an experience as it is the discussion or dissemination of information. That's a hard thing to replicate
Until technology can replicate the experience of staying out late, drunkenly chatting with coworkers in the back of a taxicab as you ride through Manhattan, then grabbing a midnight snack at a hole-in-the-wall diner someone recommended back at the office, business travel will have its place.
My theory is that physical proximity means danger or love to our animal brains .. and knowing it's tech will disengage your brain from feeling close and will change your emotions and engagement to the other person. Now something more natural than a LCD screen might make video calls a bit more lively and efficient.
Maybe for the set of people that don't mind gimmicks and facsimiles, but for those of us that don't want to even turn on video there is no way you're going to get me into an entire videoconferencing box!
HP's Halo conferencing system had several studio quality cameras that would change the video feed based on who was talking. There were also specially placed mics on the conference table to keep audio crisp. The video quality was impressive, however you had the same issues that you have with current web-cam based systems. Eye-contact was non-existent. While the camera placement was above the middle-top of the LCD screen, you'd never feel like the person was making eye-contact with you unless they looked directly at the camera. I had several team meetings in Halo rooms, and for the time, it was the next best thing to having a meeting in person. It also worked well for groups of 4 on each side of the link.
From watching the video, Google's conferencing setup is creating a 3D rep of the people talking and adjusting rendered view based on where the participants are seated. This is blending AR with videoconferencing. It would be interesting to see how their conferencing system works with multiple-people on each side. I know the video had a mother and baby in one segment, however is the 3D rendering based on the eye-level of the main participants?
Why does it appear similar to that tech other than you get a 3d image? As far as I can tell there's no info on the display other than it uses "Light field technology" which would make it different than the parallax barrier display on the 3DS.
The term light field technology is broad enough to cover lenticular arrays in displays. The 3DS had the major limitation that it only rendered two views. If you increase the resolution to be able to display more views for more viewing directions within a wider cone, you already have a light field display - simply because all these views combined form a sampled light field representation by definition. This is exactly the same as glassless 3d displays for multiple viewers of decades past. But advances in display pixel density and computing power apparently make the resulting illusion much more convincing these days.
Isn't it clear glass and not a layer over a traditional display like the 3DS? Glasses would either actively time slice with shutters, or spectrum slice with passive filters on the lens and of course you need the glasses.
How could any of those technologies be what is used here?
E: Looking again, perhaps it could be some layer over a traditional screen. You see through some of the broadcasts but that could just be the digital far plane that shows through.
I'm not sure I'm following. If this is based on a flat panel display, there must be a lens array in front of it. There is no other way to achieve this effect without requiring the users to wear glasses. The lens array can be covered by a protective flat glass pane without issue.
Track both eyes, and then project an image to each eye based on its image in the room. The part I don't really understand is how it's possible to target the image to each eye. Maybe we have displays now which are like the 3DS screen, but with variable focal locations?
Use a demo mode to disable eye tracking and follow your ARcore/Vive puck located camera ?
Or just ask the guy to close his eyes and put googly eyes on your camera...
On a related note, I wanted to share my amazing experience with a similar but lo-fi tech.
My mom and dad for the past 2-3 years have mostly lived in two separate countries (due to work reasons) and I could tell they miss each other quite a bit.
I got both of them an Echo Show 10" device each and set it up for them. I don't think I can explain how our lives have changed for the better, just based on this simple piece of tech.
The Echo Show is now pretty much constantly on video call for 14-16 hours a day in the living room of both houses and it has become an extension of the one another; a window into the other house. The audio and mics are good enough that at times you genuinely forget the other person is not in the same room. This has truly helped them, especially during Covid times.
It's actually good enough that my parents have their morning tea together practically almost the same way they used to when they were physically together. They've told me multiple times "I don't know how we could've lived without this Alexa thing".
If Google can eventually get the prices down to reasonable levels, I really think people would be shocked at how fast this thing becomes part of our daily lives.
I started working fully remote two and half years ago. Company has a couple of meeting rooms fully set up with Zoom, which I and a few other remote people used to connect to with my laptop for standups together with the in-office people. That's not the case everywhere but quite classic.
The company also had a dedicated TV+camera+24/7 Zoom meeting setup in the open space itself, so that remote people could connect to and say hi anytime. It was surprisingly nice.
Then one day on a whim I got a wide-ish USB webcam plugged into my Xbox One†.
The difference in perception is insane: instead of having this wall of people faces, like you're on several phone calls at once, I got a portalesque window between the office and my place. Instead of connecting people, it connected places††. But it only works if both sides are sharing a space instead of a selfie angle, and I think the way sound was picked up by a fixed element in the room played a lot as well.
So I'm a bit partial about this Starline thing, as it seems to be a step forward in "definition" but also feels like a blown up selfie angle.
† There's no Zoom app on Xbox (which is really a shame) so I used Edge, for which Zoom on the web is more limited and barks on webaudio (which works! Zoom just support only Chrome so I had to dial in on a phone). Also, Xbox+Kinect truly had the potential to be something amazing beyond games. I believe the marketing was botched alright but it was also way too early for the audience.
†† Which ends up connecting people on a deeper level. At some point I caught myself intuitively wanting to hand over physical objects through the TV.
>The Echo Show is now pretty much constantly on video call for 14-16 hours a day in the living room of both houses and it has become an extension of the one another; a window into the other house. The audio and mics are good enough that at times you genuinely forget the other person is not in the same room. This has truly helped them, especially during Covid times.
>It's actually good enough that my parents have their morning tea together practically almost the same way they used to when they were physically together.
I don't understand why this sort of persistant 'portal' isn't more popular. But 10 inch screen is so small! A 55 inch TV is what, $300? Put it portrait mode and that's a life size portrait. The feeling of presence when the person's head is the correct size is so strong! I know Facebook sells something for the TV but can it be left on all day like that?
I want it feel like one half of living room is a shared space, at all times. Like the couch is split in half by a portal. Then it just cycles through a list of friends and family with the same setup to find someone else watching Netflix or whatever, and you can casually join them just like a real shared living space.
I don't need 8k or 3D tv. Just a 55 or 65 inch TV on the wall at the end of the couch!
Wouldn't it be really difficult to put a 55 inch TV in portrait mode?
Although I think as I live in a city with small apartments, wall-mounting is much less common. I guess with a decent wall mount perhaps it is trivial to rotate it?
I used the small version of fb portal with my parents for a while and it was great, but I wish it would have been bigger. I’ve been thinking about buying a larger portal, but I might want to wait for this starline thing (although I have a feeling it’ll be very expensive).
Last I checked, Google Nest did not have a "Drop-in" feature like Echo where my parents can just drop-into the living room of the other house (no calling or ringing).
The Echo device just makes a sound that someone is dropping-in and gives a 10 seconds heads-up (of course this only works if you enable the permission for family members. Else it's turned off by default).
I remember when I used to be in a long distance relationship with my now wife. We used to have the laptops on Skype for hours just to have each other's company, even if we were studying or doing something else. It was very unique at the time.
I'm quite jealous cuz back then we didn't have Echo Show, it'd probably be used just like your parents use it.
I can echo (no pun intended) this experience with the Echo Show, but I would love something much bigger and higher-def. My family is separated long-distance, and I would pay big money for a higher-def "window to another home" product.
We tried Portal Plus at work to bring our two remote workers closer to the local office but it never really caught on in the office setting. The quality and tech was quite good though. Fb login required of course which was a minor nuisance.
Side note - why does it feel like so many of Google's products, and especially their most core offerings are in a state of ongoing decay? Search is getting worse. Maps today is worse than maps 10 years ago. Even their Google home assistants, YouTube, Gmail, and other products just seem to be steadily creeping into more annoyances and less usability.
There was a time I felt that Google's search was head and shoulders above anyone else's and when I reluctantly used DDG I felt I was compromising some aspect of search value for privacy value. But now when I use Google search I'm bombarded with ads, find I can't trust search results, and even copying search links results in a mess of Google redirect.
What is happening to the core Google products or am I a curmudgeon for believing that the Internet was at its most functional and user-friendly best circa 2008?
I thing things get bloated and bloated until there’s a rewrite. This is what happened for the facebook app, and google maps is very close to that point. I’ve started using apple maps more and more and I like it. I’m semi joking when I say that I upgraded to the latest iphone just to use gmaps.
To be fair, you would not want to be confronted with the reality of these products 10 years ago, there's been a million bug fixes that have been done since then. But I agree on Search, it's almost like they sabotaged it.
The search too. Wikipedia articles and useful covid graphs. Excellent context inference. The assistant is surprisingly powerful when I ask engineering questions (what's the coefficient for this when that).
I would wager a guess that its due to lack of competition. Apple Maps only works on iOS. DDG is great but the search results aren’t. Does YouTube even have a competitor?
I don’t think you’re a curmudgeon at all. Circa 2008, Google was innovating in a lot of different areas that were very valuable to the average user. The ROI on these services has gone way down.
I thought it was some stereographic encoding on layered displays but the second they mentioned 3d mapped videos I started to see pixar like characters. Very odd.
Much simpler idea but I think I’d get a good sense of being together to play table tennis over the quest with a friend. (Sadly none of my friends will buy a quest.)
I agree I love my Quest, and the sense of presence is amazing. Can't wait until more of my friend social network has them so we can meet up and do things like table tennis.
> What I’m actually looking at is a 65-inch light field display. The Project Starline booths are equipped with more than a dozen different depth sensors and cameras. (Google is cagey when I ask for specifics on the equipment.) These sensors capture photo-realistic, three-dimensional imagery; the system then compresses and transmits the data to each light field display, on both ends of the video conversation, with seemingly little latency.
> All of the data is being transmitted over WebRTC... What Google claims is unique is the compression techniques it has developed that allow it to synchronously stream this 3D video bidirectionally.
> I met with three separate Googlers in Project Starline (all of them men)
Well, at least we know that they weren't all white men; because if they were, this writer would sure have let us know about this very irrelevant information.
Id just be really uncomfortable with google monitoring, recording and analyzing my personal communications to sell me ads or worse later. Everything google does now feels.... invasive for lack of a better word. This is something Id like to see from Apple, and Im not an Apple fan at all.
I do not trust what comes out of Google PR vs what their engineers do behind the scenes. They've been caught in the past doing nefarious things or just a general disconnect/interpretation between what they say and what they do. Because they are an advertising company I always assume that takes priority with direction.
It seems to basically just have a huge number of very dense pixels, and in front it has some tiny lenses that direct each pixel at a different angle.
Presumably there's a maximum distance you can view from because the further you get away from the display, the smaller the angle between your eyes. I would guess it's no coincidence that Google has set it up as a booth with a back wall that would ensure you keep close enough:
Normally, the lenticular lens is glued directly onto a printed sheet of paper, but you can instead glue it onto a _screen_, which lets you change the pixels underneath. There are a lot of challenges here; for example, the pixels are usually located a bit behind the glass, so the standard lenses used for printing won't focus perfectly on a screen.
There's also another, simpler approach called "parallax barrier". This is what the 3DS used. Instead of using lenses to bend light, it _blocks_ the light coming out of certain pixels at certain angles. It's basically just an opaque sheet with periodic slits (the 3DS uses an LCD, so that 3D can be turned off by becoming transparent). Before I found a video of someone making one themselves using an iPhone and transparency sheets, but I can't find it at the moment...
Tangent: There's also been some attempts to do _the reverse_, and use distorted lenses in front of a camera to recover a 3D scene using only a single camera:
It's interesting to see the compression artifacts affecting hair specifically.
I guess we're going to get used to different kinds of compression artifacts in the coming years because we're switching to spatial information being transmitted as opposed to just pixels. Hair is so much harder to get right than a face.
I am sure it is stunning and beyond anything most people have ever experienced, but will it be worth $$ to have a beautifully lit version of visiting someone in prison? It is still not human contact, I don't think we can fool the brain into not missing that.
I wonder what distance they tried this over - there's a hard limit on speed-of-light for latency between continents, plus whatever encoding/decoding has to happen, so seems like we're always going to have that slightly awkward taking-over-each-other issue?
Pretty sure it makes a 3d model of the scene and then if you move around the perspective changes as well, since the screen can show different images based on the angle you look at it. you can see a bit of this in the video when the camera moves around.
Yup, porn is definitely adopt it as soon as it makes economical sense.
Especially, if this is pre-recorded and only sent for viewing (so no need too much bandwidth on upload side). I guess it would come down to the hardware and how much it costs.
Indeed. Business needs screen sharing, screen annotation and audio (Slack covers 99% of this). Home users are relatively happy with what they have already (although no 4K streaming yet and cameras in laptops are kinda tragic). That leaves porn.
The telescreen received and transmitted simultaneously. Any sound that Winston made, above the level of a very low whisper, would be picked up by it, moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live -- did live, from habit that became instinct -- in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized.
It seems like this could be used in the future dystopian economy. Workers sleeping in little capsules next to a screen where they can choose to see their partner in another factory or AI driven escort.
On a serious note likely everyone is going to have at least one at home at some point to use for work meetings, pairing or for home inspections by the government.
I have always wanted to try setting up a low-tech version of this, where instead of using a light-field display, you assume that you have only one viewer and you simply render the correct point of view on the screen. This is the same technique used to shoot The Mandalorian [1], but in this case we're only trying to make a "pane of glass" so an ordinary high-res TV will do. With both participants positioned correctly, you will get correct accommodation, vergence, and parallax depth cues---only stereopsis is missing, and stereopsis is much less important than people think. 1/3 of people don't even have good stereo vision! [2]
As long as you're willing to give up stereopsis, and you have only 1 viewer on each side, I think you could accomplish this type of immersive telepresence with an ordinary TV + software.
This reminds me of the devices described in the short story "The Story of Your Life", which is the basis of the movie Arrival. Excellent, excellent story and very different from the movie, if anyone's into that kind of thing.
Came here to say that.
Asimov was one of the few SF authors that not only imagined advanced technology, but also could see what it would do to society.
Seconded, I’ve really enjoyed all of Ted Chiang’s books and wish he was more prolific as I’ve read them all! An excellent writer and I think his style would appeal to the HN crowd as it’s kind of sci-fi, kind of based in science, and many of his stories are quite thought provoking and unique. Also they’re all short stories so great if you struggle finding time to take on a whole book or whatever!
I recently read Exhalation, and was blown away by how engaging and thought provoking the stories were. Would highly recommend it to anyone with even a passing, high-level interest in sci-fi.
Same here. I’m quite picky when it comes to sci-fi, a lot of it I don’t really like, but he nails a really unique sweet spot where sci-fi is just the framing device used for some beautiful stories. Might reread his books next actually!
Edit: I’d love any recommendations of other authors people think of when discussing his books. Doesn’t necessarily have to be the same style but more just that level of quality and uniqueness.
The first story in Exhalation is written in the style of the Arabian Nights (Thousand and One Nights). Get the unabridged, original translation by Richard Burton - it is some of the most beautiful literature ever written - comes in 16 volumes so there will be no shortage of reading material!
So are they going to use NLP to parse your conversions and build a knowledge graph of all your topics of interest to later run targeted advertising that is shockingly accurate and manipulates you to waste money purchasing random products you really didn’t need to waste money on right now?
They already do that through Google analytics and they already have dark profiles on basically anyone on earth that has touched their products, let alone making accounts
I wonder what the lag is like. I can imagine that's one thing that would break the illusion. I know with something like Zoom I've gotten used to managing the lag over time by taking turns with the other person. However, with the "live" feel of this, there could be an uncanny valley effect if the lag is subtle, but perceptible.
Another thought: this is being presented as ongoing research. I wonder what the corporate thinking is in presenting it now when it's still being tried out. Does Google want capitalize on remote meetings while it's still hot? If the pandemic wanes and we have more in-person meetings, this might not make as big a splash. I remember when I worked at Microsoft, I often noticed research announcements we'd make in public often wouldn't translate to actual product, so I got a bit jaded on any cool new thing that was announced without a product timeline.
It's advantageous to announce when you have a good working prototype so that when a competitor (e.g. Apple) announces something similar, the world is less impressed/amazed.
Zoom lag is an issue with Zoom, not the network generally. If you actually do wired, p2p on the same side of the same continent you can get rid of most of the lag. Current lag comes from services that aren't p2p and bad networks (e.g. wifi).
Yeah, I honestly hate how much Zoom has won because I've found it's the worst for latency. I have a mumble server running on a pi that blows the doors off Zoom for audio-quality and latency but it's unusable for casual groups because nobody wants to wear a headset and feedback destroys it.
Mumble beats nearly all service‘s in latency and audio quality because these two things are the problems it‘s trying to solve.
Did you ever play around with Teamspeak? It‘s a bit worse on the latency side but more mainstream friendly.
Personally I run a mumble server as well because it solves our problems but TS may be worth looking into for you.
The longer the project exists and the more people there are who get to try it, the more likely it is to leak.
To speculate, here are some reasons why keeping it secret longer might be hard:
- They're going to do wider testing within Google.
- They're going to start bringing outside testers in to try it.
- They're going to start working with manufacturers.
- Some newspaper got wind of it and is about to publish a story. (I think this happened with driverless cars?)
Apple is better at keeping secrets, and even they leak.
Also, it's nice for the people working on it when they no longer have to keep what they do a secret.
Edit: although, in this case, the timing mostly has to do with Google I/O starting today.
If they wanted to capitalise on remote meetings, I assume they would have shown one in the demo video. Instead they focused on family members reconnecting.
With the amount of data needed, one on one is probably easiest. We saw teams and zoom struggle to support more people in a call last year. It’s also nice to have them be true to size. Mini people might be somewhat uncomfortable.
If you want to get a sense of what this "feels" like and you have 6DoF VR, try a VR 180 video. I've not experienced the Google Starline project, but I can tell you that when I saw some of the really well-produced VR180 videos it was so realistic I felt I was invading someone's privacy.
Of course this won't be the interactive feeling but it was pretty mind blowing to "feel" how intimate real 3d telepresence could be.
I feel like video calling is not great because you lose that depth which is natural to human communication. That said, I work in field focused on digital audio and sound and so all my colleagues have professional microphones and headphones which makes up 90% of that difference immediately.
In 2059 we probably will invent technology that allows you to talk to people without even having to see them.
Jokes aside, it is cool tech, but I fail to see applications. With my developer colleagues we mostly share a desktop instead of seeing other peoples faces.
Don't know how it is technically solved to simulate depth, but I image it being no different than conventional screens. The difficulty is probably doing a real time stereoscopy of the object displayed.
Would be cool to know if they only use AI supported imaging for that, or if they have some sensors, maybe invisible laser projections or stuff like that. There are probably restrictions how the cameras must be angled too, so a self-made home setup would be difficult to calibrate.
New tech typically debuts at a high price point only wealthy people can afford, then as it get commoditized it becomes more affordable. Wealthy early adopters willing to pay high prices for novelty or business applications are what enable the fast pace of innovation that we have become accustomed to.
And yeah, insane prices at the start funds development for everyone else, I remember when the first 4K screens came out and they where exotic, now they are just normal.
Same thing happened with phones and hell computers, I was the first kid I knew with a computer back in the 80s and we where not wealthy, that think cost more than my dad made in a week, an actual IBM PC was unattainable til I was I was 10.
Now I have hilariously more powerful single board computers shoved in a drawer because I can’t find the time to do anything with them.
Penicillin. But then, he wasn't trying to make money off of it.
And basically anything Nintendo pushes as a console gimmick. It's not that the tech immediately goes from research to broadly accessible, but rather that they tend to take old tech that no one saw as having profitable consumer applications and find one for it. In that way, as far as consumers are concerned, it goes from unknown to widely-used without making a stopover in early-adapter purgatory.
This may be a little unfair, but I do wonder if there isn't a tendency to consider a technology to be widely available when it becomes available to you and the folks farther back in the line don't count or aren't relevant.
To say that a single company's products being used by even a single digit percentage of the world population doesn't meet the requirements to ve considered "widely available" is a stretch.
In any case, you said "important," not "widely available," and yes, Nintendo's products are hugely important. Many of today's technological advancements can be traced back to their proving that a given use case for a primitive version of a given technology was viable.
Whether it's a single company (or product) or multiple is beside the point. If a technology is only available to (say) 1% of the population, I don't think that qualifies as being widely available.
I will also note that my original comment was in response to someone who is "more interested in tech for the other 95%".
I think Minecraft and Linux are more like the content produced on top of the technology that is computers. It's like if a new book is written it's quickly available to everyone in the market who can afford a book. The book isn't really technology, but the printing, publishing, and distribution is and it's been around long enough to be distributed.
Software seems less like technology and more like writing. The distribution cost, once the systems are in place, is marginal. The technology part is creating the systems that enable the software.
IBM in the 90's strongly agrees with you - this software stuff is never going to be profitable and everything people pay for will always end up going through us!
More seriously, I disagree about software being less important because there have been very real innovations for tooling accomplished in software alone. Email is a pretty classic example - but a more modern one might be Google Cardboard which can turn your smart phone into a rather underwhelming VR headset. There are plenty of hardware alternatives but the same basic functionality was accomplished on generalized hardware.
Additionally, all this technology is only really possible due to other technology - we don't discount a new shiny computer just because it's just a dumb oddly shaped box if you can't supply it with electricity - but the costs to develop software are generally lower than hardware so I think it's fair to have a general notion that hardware is more innovative - it's just that you're conflating two different variables - cost and medium.
I think you're conflating technology with profitable or important. That is, you see me saying that software isn't technology and think I'm saying that software isn't profitable or important. That's not at all what I'm saying though. I likened software to writing. Writing can be important and it can be profitable, it's just not technology.
Maybe we could agree on email as a technology. Maybe. I think it's a stretch. I hope we could both agree that the nth email client isn't technology though. It's not adding a new capability to humanity which is how I tend to think about technology. Refrigerator - keep stuff cold. Electricity - power to operate machines and light. Computers - organize, access, modify information. etc. New JavaScript library or new game... Not so much technology.
I think that's fair yea - it might just be a matter of semantics. If you think software is included in technology then I stand by my point but, if your view of technology excludes software then you're quite correct.
Is it not the case that having access to a machine capable of running either Minecraft or Linux in the early days of each (if not now) means you are (or were) fairly affluent?
It depends on what you mean by the early days of each - in the really early days of Linux (1992) computers were probably going to cost north of a few thousand dollars and the type of computer Linux was designed for (a multi-user system for dumb terminals) would cost tens of thousands of dollars. By the time linux became a thing more than a handful of people in any given state knew about you could probably run it on a machine costing somewhere around 300$.
Minecraft has never been demanding resource wise, I'm sure early versions had serious performance issues but running it on a cheapo laptop has always been totally reasonable - it's quite accessible (it was written in java even!)
I disagree - I would agree that Minecraft wasn't a novel technology, just like Linux wasn't a novel technology - it was an alternative version of Minix.
Additionally Zoom isn't a novel technology, it isn't even particularly interesting technically when compared to other video conferencing solutions - but over the past year it's been incredibly important to a number of people.
I think the OC slightly missed the mark in mentioning "important technologies" instead of something closer to "technologically innovative" technologies or, more accurately (but less interesting of a statement) "expensive to develop technologies". Things that are expensive to develop generally aren't cheap to begin with, while things that are cheap to develop need to be cheap to compete with other market entrants and clones. Additionally hardware (a limiting factor on cost for a lot of technology) tends to get cheaper over time and that rate of change is accelerated by a large market of interest (leading to more folks deciding to try and iterate new designs).
I think they meant hardware. For software, yes there are tons of examples (open source software is usually free, games are usually cheap).
But for hardware, it's almost always some expensive thing first. The internet was once very expensive to access, cell phones were initially very expensive, DVD players were initially very expensive, computers in general, etc.
Nah, if Google actually went for it for years it would have went down the wealth requirement. The problem is that Google has demonstrated time and time again that it cannot do reiteration grind.
I was travelling a few years ago. Even Russian bus ladies (they collect your fare) and Mongolians living in traditional huts in the middle of nowhere (and without any signal!) have smartphones now. I found the idea funny that the "To plug in a USB cable, you need 3 tries" experience was maybe universal.
So, 12 years (back then) after the iPhone, it's reached a lot more than 5% of the world.
Things like this make me feel more and more disassociated with science fiction TV shows / films. I'm sure they intentionally don't make communications too good, because then - with tech like this - people would go "lame, that's just the other actor in a box behind a glass pane", instead of "oo that is cool sci fi technology".
I mean we used to marvel at things like Star Trek, but nowadays a smartphone is miles ahead of a lot of "day to day" things they showed in there. Foldable screens are coming too, and now this.
I mean I don't believe this thing will be commonplace at all in the near future, but it's still cool. I think it'll be integrated into smartphones within the decade though. It's already mostly possible with the front-facing camera + light field of iphones, plus AR, plus motion sensor technology, plus the load of processing power they put in there.
A good friend of mine at Google is the technical lead of this project (he has a Phd Princeton, and was a professor before joining Google).
I've tried it in person and it was truly amazing. They used some very fancy tech when I saw the demo, so I'm thrilled it is finally being announced and possibly shared with a larger audience.
Explanation of why it is amazing: It totally fools your perception. No glasses or goggles--but rather an 8k display with special glass that allows your different eyes to see different pixels (a light field).
They also optimize the sound, and the rest, so as all the testimonials point out, you actually feel like the person is in front of you.
It also works for the "cube" around them, so if they hold up some object, it also feels like that is in front of you.
Just very practically, is this a $xxxx.xx improvement over simple video chat with a large screen and simple camera? Will this even work without low-latency high-bandwidth connection up and down?
It seems more like an art project than a tech usable in the coming decade.
So in my company, I've used various avatar's of Cisco TelePresence systems over many years.
The big commercial need it turns out isn't so much realism as it is flexibility to accommodate people dialing in from different systems - phones, laptops, different types of telepresence setups from small single room to big conference rooms or even telephone connections etc.
Many years ago, we were all wowed by the life-size realism and had people come into offices for it . Nowadays these meetings have lots of people crammed in across multiple screens dialing in as they please and all the better for everyone :-)
In other words, you pay market value for it, it doesn't include advertising, and Google is contractually obligated not to snoop on your content.
Which is why even Google competitors use this hardware, because the legal and technological protections are strong enough they don't worry about Google stealing their IP.
It looks like they're doing photogrammetry in real time, which is mind boggling. I'm not familiar with this space, but building a 3D model, texturing it with live video, compressing and sending that over the internet, and doing it with minimal latency for it to be believable/enjoyable? Incredible technical achievement if that's the approach. Using state of the art tech, no doubt, and probably lots of ML magic to smooth the rendering. The 3D display is the icing on the cake, it must look amazing in person.
not that hard to do if you have actual depth sensing cameras, and even without those, something like the oculus quest 2 does that exact task (generate a rough 3d volume based on several 2d video feeds) you can see a neat example when you draw your guardian space, and move objects (and notice how it updates the 3d volume representation)
That's not true -- even with depth sensing cameras, it will still be full of artifacts, and things like curly hair or strands of hair will become disastrous because they're not easily geometrically modeled.
The Oculus Quest 2 doesn't do anything like what you're describing -- it essentially just pipes in stereoscopic video from its stereo cameras and stitches them together in a trivial way. It doesn't attempt to build geometric representations of objects in your environment at all.
(For guardian functionality it does very simple things like using the depth cloud to figure out the height of the floor, and if there are points inside the guardian that shouldn't be there, but that doesn't inferring object geometries.)
if you look at the video you can see there are artifacts around the hair. It is likely applying some matting via AI to make it less obvious, but it is still there.
Good point, I haven't followed the latest VR advancements, that does sound neat. Still, Starline's approach is surely much more sophisticated (the hardware obviously has a lot to do with that, these are prototypes of a desk-sized machine vs a headset). The 3D model looks reasonably detailed, and the final render has very few artifacts. Making it all work over a WAN link with latencies critical for teleconferencing is also impressive.
The difficulty completely depends on the level of quality you're after. They're certainly working on cutting-edge-level quality, so it is likely no easy task. Someone else pointed out that they released a paper on related tech last year:
i got curious (since you said you'd seen this years ago) and found my way to your cppcon 2016 talk. good talk but in particular i want to congratulate you on the weaving pun (first multithreaded tech).
Do you know if it supports multiple people on "screen" at once? Or does it rely on eye tracking of a single person (plus projection of some sort?) to be able to achieve the 3D effect?
I don't know anything more than what's been released, but from my understanding, light field display don't rely on eye tracking, and you can have multiple people looking from different angles (although the range may be limited to near the center). In the video itself, the camera filming the interaction is moving independently of the person sitting, and unless it was faked, it was able to see its own perspective.
Depends on how much bandwidth it needs. I've been video conferencing a lot (who didn't, this past year) and even with people on the other side of the globe I don't recall latency ever being a problem.
yeah, like if your playing keys live for example, you want your audio interface to have around 6ms latency max (the delay time between your finger pressing a key and you hearing the sound from pressing that key) above that and it starts to become detrimental to your playing. above 10 and you can barely play.
No, this sends a different image to each eye. It's not actually a transparent piece of glass with an image on it[0] - I think they just try to make it seem as if it is a transparent pane you're looking through.
The marketing for it is pretty terrible though, because I initially came away with the same impression as you.
I recall there being glasses-less 3D displays about 10 years ago, when the TV industry was trying to make 3D displays the next new thing. I wonder if this is the same technology.
If this was a consumer product I would buy two of these today, one for myself, and one for my parents. It is more compelling that any product I have seen for a long time.
This is a 3D monitor. Not a light field display like Starline.
The traditional glasses free 3d monitors rely on special coatings on the glass to "split" the direction of a pixel. Some coatings are electronically controlled (like Sony's), some are physical (lenticular filters) It still displays a 2D projection of a 3D image, but twice.
By contrast, my understanding is lightfield displays projects the entire volume. So you don't have a pixel, you have a voxel. So each eye gets a different bit of the same voxel
I suspect eventually this will percolate down, somewhat like Facebook's Portal TV (£149) but possibly also integrated with something like a Google Assistant/Siri type smart device to power the software and do other smart-home type things in your home.
Many ordinary folk already have 4k TVs at home and 8k will probably become commonplace in the future. The real bottleneck will be good low-latency broadband both up and down, but fibre to the home should make that easier. I wonder if ISPs in the future will offer QoS guarantees to enable really good videoconferencing?
I mean, Zoom, Meet et al are much better than video conferencing solutions like Webex even from a few years ago, but it's hilarious how much drama there still is around video calls. "oh, sorry, I didn't realise I had muted myself", "can you hear me...?", "I think we just lost Steve", and so on. I'll be glad to see all of that just go away.
I think one of the major aspects of this technology is that the person you are talking with appears in its real size, with the same ratio than in real life. Something you cannot do with a smartphone, except if you are referring to a projector.
I doubt you need gigabit for an 8K stream. The real issue is the abysmal common network infrastructure. Latency will be a bottleneck if most ISPs don't care about treating your packets properly.
Most internet providers here in Finland provides you fiber connections from 100-1000mbs pretty cheap. And 4g/5g connections from 100-300mbs for cheap prices too. Most of them does not have any data caps.
I have 300/100mbs connection which costs 20 euros in month.
Agreed, 10Mbps is likely where it'll be, probably less. If I were to guess, they are going to apply a lot of smarts to texture mapping to avoid needing a lot of bandwidth.
It heavily depends on what encoder and config you use. In my experience, 20Mbps (or less) HEVC by Turing NVEnc realtime encoder is enough for 1080 60p. Also halving frame rate won't halve bitrate because of how video compressed by reference frame. Also video meeting won't move pictures as much as FPS gaming.
Doubtful, IMO. The fact that you need it to be both extremely low latency and with essentially zero compression artifacts (probably lossless, based on their goal). If the numbers listed here[1] are correct, then the most efficient lossy codec at that time was doing 100 minutes of 8k video in ~37GB of data. From that we can intuit that it was using an average of 50 Mbps for that 100 minutes of video. For the most efficient codec, and I'm not even sure from that whether it was using lossy or lossless numbers, because apparently HVEC can do both (but I would assume lossy, since it's about streaming video in that case).
You can't do weird texture mapping or lossy compression and expect people to really seem like they are there. Even if you don't notice that stuff normally watching a video, I think you'll notice it when you're interacting like someone is really in front of you, and that will throw off the immersion.
HVEC is no longer king of the hill when it comes to compression efficiency. AV1 and the upcoming VVC do better.
That said, my intuition is they aren't doing a pure video encoding solution. The fact that they talk about 3d modeling leads me to believe they are doing a combination of model + texture to get the realistic results. That would significantly decrease the amount of bandwidth and computational power needed. Over a low bandwidth situation you'd simply need to send model updates and do some smart interpolation to determine what things should look like.
Similar to the concept that playing a 3d game requires MB of resources but recording the same game at 8k would require a boatload more memory.
My assumption is they are using LIDAR to get a good model, high quality cameras to texture things, and a nice AI to stitch things together and interpolate when data isn't arriving fast enough.
I think this is what will be done in the future. You will interact with the camera for 15 minutes or so and it will create a custom compreasion algorithm for you.
Over a decade ago I used a holographic video conferencing system that used kinect for depth mapping, it was very convincing. It would also reorient the display based on your head position. so you could look "around" the avatar.
There are demos of binocular 3d conferencing done with a lenticular display (lookingglass), although those large displays are extortionately priced (1/4 of the size of this google one is $3000...) - keeping them out of the hands of most devs.
No doubt google are doing the same, but can afford these larger displays.
You can easily find examples (and research papers) by googling for the relevant terms. Google claiming they have invented a "new technology" just shits on all those folk who dont have the publicity/funding of google.
It's neat and I'm waiting to see what creative things people come up with for it. A Japanese developer put together a Wizardry/Megaten-style dungeon-crawler demo:
I have wondered in the past if a similar result could be achieved using a 3d headset with some tracking/cameras and removing the headset from the view through a real-time deep fake that could be achieved through a short scan before the call. Would this even be feasible?
Fascinating. So not only is it feeding it an 8K / 30 (60?) FPS image, it's feeding it numerous incident angle variations and displaying all of them simultaneously?
I only know what I saw from the IO stream, but I think it might send a compressed 3D mesh + texture across the network and render the light field locally.
I think that is where the custom compression algorithm comes in. If you think the fact that human body and face doesn't change much, and the fact that it's a 3d model based, the compression ratio could be very high.
I think what they are transferring is not a video but 3d model and the skin texture applied on the model (all derived from the realtime video / depth recording on the other side). The receiving and then renders it as a 3d model on the screen.
Is existing Looking Glass Factory tech the same though? Not so sure about that. Those displays are typically monitor-sized at the largest and not really aimed at displaying a live feed of a person. This looks to be a more seamless experience on a larger screen.
Do you think it's possible 90% of this is due to the studio quality lighting, large high quality screen, good mic/speakers, and low latency network? It seems like those factors alone would get most of the way there and the 3D aspect is just a bonus. Obviously I haven't used it in person but this was just a thought since most people are used to video calls on their small phone/laptop with poor lighting, mics, etc
It looks like it came out of the high-fidelity Immersive Light Field Video presented at SIGGRAPH 2020. Quite impressive that within a year it's now a consumer product
WebAssembly SIMD is coming to Chrome as well. 2D images and video that only consisted of RGB and Alpha channels may appear downright primitive to future generations as depth camera rigs gain distribution ;)
I am sure that the immersion of the experience is higher. My question (and perhaps that of GP) is: is this greater immersion actually beneficial to communication?
I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical. I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
>I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
>Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical.
>I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
From my experience, 4k or 8k doesn't matter. Sound quality actually matters most, really clear low latency audio alone will give you a surprisingly strong sense of presence.
Video quality is important but 1080p is enough, beyond that the lighting and latency matter more.
Equally important from my personal POV is video size - physical size. Take a cheap 65 inch TV, turn it vertically, and talk to someone on that. When your talking to someone that is actually life size the sense of presence is vastly improved, even at the exact same video quality. And TVs are so cheap this doesn't seem like much of a techical barrier.
If you just screen share from your cell phone to your 65 inch TV and video chat -- holding everything else equal for audio and video quality -- it's SO MUCH BETTER.
I think you might be underestimating the value of viewing a 3D model on a no-glasses 3D display. This is one of the basic aspects of in-person communication we take for granted that current 2D technology can't replicate. You can move your head and actually see a different angle of the person in front of you. This can even be subtle, our brain will still pick up the effect, and it makes the experience beyond what we usually consider as "immersive".
Yes, having low latencies and high definition video is an important aspect of this, but the 3D part is no gimmick. Once the technology improves and gets affordable this is a game changer for how we communicate online. The step after that are holographic displays, and since we'd be used to 3D models and smart displays, it probably won't feel like such a big jump.
I'm _super_ excited about this project. Hopefully Google doesn't axe it. (:
I thought that as well watching the video, but have you used a PSVR?
I've got one and the first minute is always noticing how low res the eye screens are, then as soon as the game starts, I've forgotten and I'm _there_. The 3D part makes up for the low quality
Being able to feel like another person in the room is enough for me to reconsider working from home. As of right now I strongly have a preference for in person, but I do acknowledge most people prefer commute and cost benefits over productivity.
The state of video conferencing today is a poor one and I'm very excited for something that can change the industry like this.
I'm right there with you, and I use a 4k camera and a boom mic and headphones and wired ethernet to videoconference now: I have been regularly complaining about the low resolution and framerates of current videoconferencing systems (10-15fps, 720p, low bitrate - and that's the highest quality setting available!).
If Google wanted to make me believe they care about videoconferencing quality, they'd have a 4k 60fps option that auto-enables in Meet if it detects everyone on the call is on wired gigabit with a 4k camera.
A lot of residential areas in the US have gigabit options, in some cases symmetric. There are lots that have 1000mbps down/40mbps up cable.
Even 100mbps is sufficient for a 1-on-1 4k video call, as high-bitrate 4k is 30-40mbps. Most commercial office buildings in business districts have it available. Even Starlink (20mbps up) should be sufficient for 1-1 30fps 4k videoconferencing with a lower bitrate.
I've spent a decent amount of time and money to make this happen and it helps less than you would hope.
Partially there are just affordance issues of things like eye contact which are physically out of alignment unless you start using two way mirrors [1].
There are technologies that can automatically adjust videos for eye contact today, so I imagine something similar could be implemented for this later on.
Eye tracking is the core feature here--the rendered "hologram" is correct from every possible angle. The things you mention are probably closer to 2% of the final result.
My intuition is that a great lighting+microphone+speaker setup is necessary, but not sufficient, for this demo.
Even from viewing the short demo, the stereo display alone is an entirely new dimension that no amount of studio lighting will recreate. While better lighting and audio setup would certainly improve the average person's videoconferencing experience, this looks to be a genuine step beyond.
That said, we've been seeing holographic-display prototypes for the better part of a decade, and it'll be interesting if this actually pans out or fizzles.
What I think you're trying to describe already exists as a product from Cisco called "telepresence". It is/was insanely expensive, was a permanent installation that only Cisco contracted techs could install, and did what you describe: It is a series of large, curved HD displays with desks at an appropriate distance from the screens/cameras, and copious amounts of indirect lighting from behind the setup to make each party look good.
It seems like the imaging/rendering technology that Google is using is much more advanced.
I’ve used such a Cisco system. Compared to regular video calls the latency and quality was light years ahead, much more natural conversations were possible. By which I mean it was possible to laugh, interject, and generally have a realistic conversation with a colleague in another country without having to compensate for video lag in that very careful way I find necessary on Meet and Zoom.
That said, there was no “emotional connection” like the Google one is described as offering. It was still a video call. There was no forgetting that. I suspect the 3D and the apparent physical closeness to the display add a lot.
Wow I forgot about Telepresence. I used it a decade ago at a Fortune 500 company. With all of the cameras and displays perfectly positions, everyone was life-sized on video, felt like you were sitting around a roundtable. Now I'm imagining that with higher resolution and 3D light field display, wow.
The question for me is how much it matters after the novelty wears off.
I count at least 5 waves of 3D technology starting in 1851 with the Brewster Stereoscope. Each time there's a surge of popularity driven by the legitimately amazing initial experience. And each time people slowly stop caring. People were incredibly excited about Avatar, and many thought it would change the movie industry. But how many people now go out of the way to see something in 3D?
I honestly think it's amazing and I'm sure the novelty would wear off, but it would still be useful. One of the things that stood in the way of 3D viewing was dedicated hardware, which is the same issue here. Although the need to wear glasses is gone, you still need to buy some serious equipment for it. Maybe at some point this will be bundled into a normal TV setup and people will just take it for granted that it's there
If you talk to daily Instagram users, you'll find out what problem it solves. In particular, it was the lucky winner in the crowd of early photo-sharing-on-mobile apps. But the demand for that was proven by early photo sharing successes like Flickr. Realizing people would want to do that on their pocket camera device is not a big leap given what people were already doing to share photos from their mid-aughts cameraphones phones.
There's no such plausible story for 3D video calls. It's not like people are already demanding 3D displays for any of their other 3D stuff. The 3D first-person shooter, for example, has been around for decades. But 3D displays have never been popular despite being available for at least a decade.
>Tell me a problem WhatsApp solves. It's just a fancy SMS.
I gather it mostly solves that SMS is expensive in a lot of contexts. Personally I never use it because most of the people I text with have US phones. And the one person who doesn't, we use Facebook.
Yes? Other than not being hip, how is Instagram better than Myspace? WhatsApp at least added features over SMS, although it sacrificed interoperability and went all-in on a closed system to get there. Newer is not always better.
Replicating the experience of in-person communication much more closely than video and 2D displays will ever do. That's a noble research goal if nothing else, I don't get the skepticism.
There are several reasons 3D content and previous generation displays didn't take off, but there's no reason to believe a revolutionary new approach and product couldn't change this (e.g. electric cars were invented in the 19th century and are only now becoming popular). AFAICT the real time photogrammetry they're using here along with the no-glasses 3D display is a major leap forward. If they can get it cheap and reliable enough to mass market, it would be a game changer.
I certainly know what kind of display and teleconferencing software I want when the next pandemic hits, and it's not what we have now.
> (e.g. electric cars were invented in the 19th century and are only now becoming popular)
I hate this example, and it's like one of the most common ones on HN.
As said by thousands of people and many documentaries before me , the electric car had numerous real conspiracies working against it, some of which were the most powerful financial groups in the world.[0]
The 'electric car' wasn't made popular and possible by recent technological strides -- although it was made better.
The success and popularity of the electric vehicle was made possible by financial shifts away from petroleum exploration, facilitated by dwindling profits and increased scarcity of oil, and encouraged by a movement towards sustainability both from the social culture of the world and the various actions of government from country to country.
Yes, range has improved. Yes, the cars are more intelligent and better to drive -- but these improvements have been seen across the automotive industry since its' inception with ICE based vehicles included.
The real motivating factor behind the electric car is the environment that now exists that allows such endeavors to be profitable -- an environment that not only includes technological improvements like you hint towards, but more importantly it's an environment that fosters development of such things due to the existence of a profit incentive and increased governmental-body cooperation.
All that said, unless Cisco is even more evil than I realized (woah..), I have a hard time presuming that video conferencing has been held back by the same sort of conspiratorial under-handed back-office dealings that slowed the progress of EV adoption.
Why does it matter which circumstances allowed the electric car to become popular only now? Whether that's because of major industry and consumer shifts, or because the technology matured enough for mass market, they have the same effect. It's certainly a combination of both, and we shouldn't downplay the advances in battery technology alone.
I mentioned that example because the previous two comments were dismissing this attempt at 3D teleconferencing on the grounds of it being old technology with past failures. But I think we agree that it takes a certain industry environment along with a technical leap to make a technology truly popular. Even if that never ends up happening in this case and it remains a niche product, we should applaud the technical merits here instead of being dismissive.
For that matter video teleconferencing period has only just really hit critical mass even though there were videophones at the NY World's Fair in the 1960s and camera systems have been around in conference rooms for a few decades.
What really happened was that it became more or less accessible to anyone with a laptop and an even marginal network connection for basically no cost. And, oif course, the last 18 months really pushed it over the finish line if it wasn't already.
> Replicating the experience of in-person communication much more closely
That's not a problem people express much, at least not in ways where "3D" points to a solution. When I want to see people in person, it's not because of a lack of stereoscopy. I want to hug them, to break bread with them.
> there's no reason to believe a revolutionary new approach and product couldn't change this
There is indeed! In specific, the many times we have already had revolutionary new approaches and products that were met with great enthusiasm in the market for a few years.
I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular. (I'm not sure what your work calls are like these days, but quite a lot of people turn off video in mine.) The lesson I take from that is that people mainly self-generate the feeling of interpersonal connection, and they can do it with very little in the way of cues. To me that's another strong indication that no new 3D technology will make much of a difference.
> When I want to see people in person, it's not because of a lack of stereoscopy.
Of course, I'm not saying this will replace physical communication (I should've said "simulating" instead of "replicating"). But it's a clear step forward for traditional teleconferencing solutions. What do you think is the next leap from 2D video and displays? We're at the point of diminishing returns as far as increasing resolution goes, most consumers don't have a need for 8K or higher res displays. VR/AR is chugging along, but we're still a few generations away from mass market adoption.
> I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular.
I don't understand. Video calls were never meant to replace audio calls, they just added a new sensory experience. It's perfectly fine for both technologies to co-exist for different moments and preferences. In a similar way this 3D approach is an extension to traditional 2D video conferencing if people have the equipment and prefer it. Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be, especially if it's polished and cheap enough.
It's not a clear step forward except in technical terms. Those often don't matter. For example, the big revolution in the last 20 years is not faster computers, it's mobile ones.
I don't have much reason to think there is any near term "next leap from 2D video and displays". 2D renderings are more than 40,000 years old. They have improved drastically in resolution and fidelity. Computers added being dynamic and interactive to that. it's really not clear that 3D rendering adds much.
> Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be
I am not, because that kind of novelty-driven excitement has driven every wave of popular 3D rendering technology for 170 years. VR/AR has been close to mass market adoption for 25 years. We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
People have had those excited faces every time. There were people jazzed about the possible impact every time. The Brewster Stereoscope. The ViewMaster (with the US Defense Department purchasing 6 million reels on the theory it would revolutionize training). 3D movies in the 1950s. VR in the 1980s and 1990s. 3D movies again this century. 3D TV for 2 CESes. And then the latest wave of VR, which you agree is still not there despite fantastic investment from companies floating in cash.
Could it be different this time? Maybe! But if we keep measuring it by novelty effects, we're setting ourselves up for the exact same failure that keeps happening.
> the big revolution in the last 20 years is not faster computers, it's mobile ones
Surely the improvements in manufacturing processes, faster hardware and better screens are partly responsible for that. The iPhone as a concept has existed since the 1980s, and revolutionary ideas like what General Magic tried to produce in the 90s were just too early to be successful. When Apple tried it again in the late 00s it was a massive success, but technology finally reached a point when it was commercially feasible.
So it doesn't take much to push a product to mass adoption. Just the right industry circumstances, a manufacturer willing to take the risk and capable hardware and software existing to make it happen.
> We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
Are you dismissing the potential of VR/AR as well? The current innovation wave we're on is much bigger than whatever we had before. Headsets are becoming cheaper, more comfortable and accessible, and the visual tech we have now is leaps and bounds better than previous generations. Once we get to being able to put on sunglasses and experience different worlds, though likely sooner than that, the market adoption will likely go through the roof.
> People have had those excited faces every time.
I think it's different this time. It's not just it being 3D, but the merging of new generations of light field cameras, face/eye tracking, powerful ML algorithms, low latency networks and revolutionary displays is miles ahead of previous attempts. You can't just compare this to the ViewMaster and last century VR. The improvements here are much more substantial, and if they can make it cheap and reliable enough it could be a ground breaking product.
"Hard tech" often only matures after several hype cycles. Sometimes cool tech demos can be produced a century or more beforehand. If you were in the 1980s and people were talking about video calls being the next big thing, you might point out that people have been working on video calls since the 1930s[0], and it hasn't caught on in every one of the hype cycles that have followed, so that's an indication that it won't catch on in the future. Video calls have caught on now though - especially as they've reached mobile devices instead of requiring a literal booth in your house, as with AT&T's initial "Picturephone" tech in the 70s.
I will say though that people tend to assume that each new technology will replace the preceding technology (text->audio->video->VR/light-field->...), but in fact it tends to end up just supplementing the existing tech.
Wow, this is a pretty ableist take. Deaf people or hard of hearing exist.
I've got a hearing problem where I struggle to make out what people are saying on a phone but with a video call I can add lip-reading and visual cues which helps me keep on thread.
I want to try this just to see what happens, if I will have again the feeling something is more real than reality.
Because optometrists are illegal in my country (here only medics can decide what glasses you can use), currently I don't have stereoscopic vision, although my brain CAN do it, if I had the correct images sent to my eyes somehow. (one of my eyes muscles is slightly shorter than the other side thus the images on that eye are shifted unless I had an optometrist design me glasses with a prism).
So when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality, despite it being obvious fantasy with aliens, floating rocks and all that stuff.
EDIT: for those wondering, I am from Brazil, here medical professionals often sue the shit out of anyone offering any service remotely similar to an optometrist, they are quite aggressive about it, some attempted to make even discussion of the subject illegal. And when I was trying to get the prism and asked around my medics about it, one of them went really ballistic, I honestly thought the guy was going to punch me. I believe the reason for that is that for many medics, designing glasses is their only source of income, a guaranteed one, since here is ALSO illegal to buy glasses without a medic desining them for you first, even if the new glasses are supposed to be identical to the old ones!
>my brain CAN do it, if I had the correct images sent to my eyes somehow.
>when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality
Buy an Oculus Quest 2. (Or any VR headset, but that's the best value at the moment.) It sends seperate images to each eye. You should get that same 'more real than reality' feeling. It may even train your eye muscles to see in 3D, I know I've seen some research into that area. (https://www.seevividly.com/ comes up on a Google search, though it seems to be prescription only)
I'm someone with Amblyopia and would love to hear people's experiences using this. Seems like the project made some waves on reddit and kind of died down. I have bought Oculus Quest last year in anticipation of trying this but the pandemic and work pushed it out somewhat.
Watching 3D movies in the theaters just gives me a headache but Oculus actually works fine, I've played a few games and it was amazing...
Brazilian guy here: my girlfriend just bought two pairs of glasses from China, I don't know from which site exactly, but some AliExpress type of site. She paid 5 dollars each, just telling which degrees she needed for each eye. The quality was the same from the 200 dollars pair she bought with a medical note here in Brazil. Unbelievable. Just telling that, if you want it, you can import it through the mail and it will pass through customs without a problem.
If you search for "myopia glasses" on Aliexpress you'll find prescription glasses starting as low as $2. I just added a pair to my cart and the total was $3.31 including shipping. :O
I have glasses from Zenni. Very cheap, and well made. My current glasses cost ~10% of what the previous pair cost me in NZ, and the lenses are better quality. The glasses cost less than the optometrist visit, and I bought a second identical pair as a backup.
Shouldn't an ophthalmologist here in Brazil be able to diagnose you and give you a prescription for corrective lenses, or even perform corrective surgery?
I'm asking in case I might need to look for one in the future.
I'm from Brazil too, and my eye glasses are recommended by a optometrist. The only way an optometrist to work is being an ophthalmologist with specialization in "optometry". It is a sad reality from our bureaucratic government that don't give us freedom to innovate.
The profession of optometry came from physics instead of medicine, many of the early ones with physicists specialized in optics that ALSO understood how the eye worked, optics-wise.
In places where they are legal they spend their time learning more and more about optics, physics, math and eye anatomy, they don't study diseases, infections and so on.
As result you can't rely on one to fix certain stuff where you do need a medic, but if you need fancy lenses, they will calculate them, not just use a number they get from their measurement machine like ophtalmologists do.
They also would be helpful to design proper 3D glasses and whatnot, holograms and so on.
Got it, I was a bit confused as I never had heard about the problem you mentioned. Too bad here in Brazil you can't get the specialized care you need for the condition you have.
This is a major screw up and inefficiency from this country.
Thanks for explaining the concept more in-depth too.
I've seen 3d screens in Japan 20 years ago. TVs, ad displays, mobile phones. It works really well, and it's actually very simple technology. Never understood why that never took off.
This is different. It's a light field display, so you can view things at different angles as you move around, rather than having to sit in a very exact position and only seeing one angle.
My first thought: in 2030 when I’m calling grandma with this, I will first need to accept cookies, then accept the new Google privacy policy, then sign in to my Google account, then watch 2 20-second long pre roll ads.
It's the office of the future! https://www.cs.unc.edu/Research/stc/. The challenge in 2000 was that bandwidth and 3d reconstruction needed several leaps forward in quality.
Super cool technology. It will be interesting to see how this develops, for me google seem to struggle with hardware products, even more with consumer hardware products. I can't see where it fits in to the google ecosystem and how people will access it.
I could imagine Apple selling this in a re-imagined Apple TV with an Apple TV Facetime App. They could probably build something now with the FaceID sensor array / iPhone Camera system / M1 chip plugged into a 4k TV.
sush1612 | 5 years ago
It is still behind the screen, instead of video stream what you see is a 3-d rendered model which moves in real time. wonder how it is different from a video call.
ashekara | 5 years ago
So cool. Anyone have more information about tonari (https://tonari.no/)? They were/are working on solving this “feel like you’re there” problem.
bschwindHN | 5 years ago
I work there currently, we still exist! We actually just published a blog post last week about how we use tonari internally:
https://blog.tonari.no/how-tonari-uses-tonari
fchu | 5 years ago
Meanwhile Google Meet is still horrendous compared to Zoom, despite me and my coworker both having gigabit internet...
Don't get me wrong, this is cool, but a research project becomes really cool when it gets well executed
a_imho | 5 years ago
Even this carefully curated demo looks lackluster, but I'm probably not the target audience.
machello13 | 5 years ago
Reminds me of https://www.youtube.com/watch?v=2prsYbV1TkM from Silicon Valley. I imagine the real-world experience will be pretty similar for a long time.
LightG | 5 years ago
Haha, thanks ... worth hanging on until the second half ...
lurkerasdfh8 | 5 years ago
Reminds? this is literally it!
Also, microsoft did this a couple years ago, without fancy volumetric display, just face tracking on their expensive 8k(?) tv for meeting rooms, and it was a complete flop.
TigeriusKirk | 5 years ago
I feel like this is the kind of thing I'm going to have to experience in person to appreciate. The video just isn't conveying anything meaningful to me.
Still, the concept is exciting, and if the execution is there, it'll be one of the most important leaps in communications technology in decades.
And I'm looking forward to a company named something like InstaPresence (TM) applying filters and making us all photorealistic cat people.
shannifin | 5 years ago
While it looks exciting, I'm also interested in the 3D display's eventual potential for movies and games...
ricopags | 5 years ago
Looks like tonari[0] will have some heavy competition sooner than expected. This seems to have a lot more attention paid to the sensation of depth than the tonari offering. Could see this being popular at high end senior care facilities.
[0]https://tonari.no
chewxy | 5 years ago
At Atlassian, they built a Portal inspired portal that connects the Sydney and SF office during one hackathon. Basically a camera behind a 2way mirror (and decorated with blue/orange LEDs in an oval frame). Tonari looks a lot more like that than Starline.
dmux | 5 years ago
Random thought, but do you "own" a 3d scanned representation of yourself?
achow | 5 years ago
If you watch the video of it:
https://www.youtube.com/watch?v=Q13CishCKXY
Interesting thing to note is that they don't show the participants touching the glass pane 'separating' them, whereas for that kind of situation it would be very very natural thing to do.
I guess doing so (reaching towards the 'glass pane') would make the imagery distort/degrade real fast as you would start going out of camera's FoV, which that would break the magic.
sneak | 5 years ago
A bit ironic that this promo video maxxes out in 1080p, despite YouTube supporting 4K and this tech pumping 8K+ res.
There is so much more we can do in terms of quality and immersion that we're not doing simply because bandwidth and connectivity are so low-quality and overpriced at most of our leaf nodes in the USA.
rajacombinator | 5 years ago
So what’s the even moneyline on Google projects like this? 9 months?
fab1an | 5 years ago
Hmmm. Personally, I would much rather have a lofi hologram that is capable of moving around, as much as I lack technical implementation specificity here :) Starline seems amazing, but there's a black mirror quality to the ultra pristine resolution.
draw_down | 5 years ago
Pretty cool! Seems about as real as a concept car, though.
BlueYoshi | 5 years ago
Is the technology used here for "tricking" the brain to simulate 3D the same as what Nintendo was using in the 3DS? If so, it carries a risk of developmental damage for the eyesight of children under 6, so as much as this might be cool they really shouldn't be showing kids using it, especially not babies.
whymauri | 5 years ago
please don't get google glass'd
nothingreallyma | 5 years ago
Amusingly the video doesn’t load for me... chrome browser, YouTube video, etc... just sits and spins
grogenaut | 5 years ago
youtube is having an outage
domano | 5 years ago
Oh wow, totally reminds me of the Solarians in Asimovs "The Naked Sun"
https://en.wikipedia.org/wiki/The_Naked_Sun#Physical_distanc...
Reading this shortly before the pandemic spread gave me a really strange perspective on the whole remote working boom.
fungiblecog | 5 years ago
Am I the only one that thinks one-on-one video is already fine, and what we actually need is to improve the experience so that a group of people feel like they're meeting in the same place?
piyh | 5 years ago
The inexorable march of technology will continue, curmudgeon or not. There's no reason this approach won't scale to groups or larger areas.
difosfor | 5 years ago
I'm really curious about the 3D video compression codec; besides this, imagine what that could mean for 3D VR streaming!
0xbadcafebee | 5 years ago
Why does Google continue to try to produce new products? We all know they will never become reality, or if they do, they'll stop existing soon after they're open to the public. They're now just a regretware company.
janandonly | 5 years ago
Give it a year or 2 and the iPad Pro will have this as a standard feature in Facetime.
> Project Starline is currently available in just a few of our offices and it relies on custom-built hardware and highly specialized equipment. We believe this is where person-to-person communication technology can and should go, and in time, our goal is to make this technology more affordable and accessible, including bringing some of these technical advancements into our suite of communication products.
sgeorge96 | 5 years ago
Can we reserve these types of names for spaceship projects please.
vbsteven | 5 years ago
There’s a funeral scene in Upload (on Prime Video) that does something like this. It felt like a window into another room and I loved that concept. It looks like the tech for actually doing this is closer than I thought. Exciting!
KETpXDDzR | 5 years ago
Sounds like a solution that a regular consumer will never afford to use. I see the usual bay area behavior: Cool tech for rich people.
crooked-v | 5 years ago
This is a really interesting project, and I wonder how long until Google unceremoniously cancels it and drops support for anyone who's bought into it.
barbazoo | 5 years ago
And have they announced yet how they're planning to invade my privacy with this?
goldenchrome | 5 years ago
Well, they'll have extensive 3D data about your body and your house.
acdha | 5 years ago
This is getting downvoted but it really is a valid concern. Google’s been losing users on each discontinuation and only the most pro-Google techies I know are still jumping in to something new, especially given the trend towards one cool feature and a bunch of “QA is boring” stuff which will take a year to get fixed. Given their past reputation, it’s really cautionary as a shareholder to hear C-level questions about services like GCP questioning the risks of not going with AWS/Azure and getting stranded.
That’s a big deal for anything which requires hardware you wouldn’t otherwise own. Once you hear “custom-built hardware and highly specialized equipment” that sounds like something you really want a commitment to not just begrudgingly patch but to continue to seriously invest in the product.
jsnell | 5 years ago
It's getting downvoted because it is repetitive, with thousands of effectively identical copies being posted on HN yearly. It is not insightful. It is not clever. It's not reacting to the story, except for lazy pattern matching. It is just a tired meme that's been flogged all the way down to the bone by now.
(It's probably also not true, when compared to Google's peers. Amazon and Microsoft similarly throw a bunch of stuff on the wall, and unceremoniously kill the failures, but neither their launches nor cancellations get this reaction.)
annoyingnoob | 5 years ago
Looks like an electronic prison visit.
edderly | 5 years ago
Bingo. Video conferencing meets the uncanny valley.
spurgu | 5 years ago
I.e. ecstatic since you're generally not allowed to see people.
eamon_cas | 5 years ago
Reminds me of Teliris (https://en.wikipedia.org/wiki/Teliris) which I got to demo 15 years ago. That product ran on dark fiber with special hardware to simulate two sides of a conference table. Really uncanny and unsettling etiquette-wise. One-on-one feels better all around.
spurgu | 5 years ago
Thought: This could bring back phone booths.
If this tech turns out to be too expensive (for normal people) we could still use it on a pay-per-use basis, like with a "video conferencing booth". You'd schedule your call and reserve a local booth for both participants through an app. And most companies should be able to afford having one of these in the office.
nickkell | 5 years ago
That's just reminded me that phone booths were usually filled with advertisements for "escort services". I'm sure it wouldn't take long for this to come back in this brave new world
paxys | 5 years ago
It would have to be really futuristic to convince people to coordinate timings, reserve a slot, pay and drive to some location for a video call. FaceTime/WhatsApp are still good enough for most.
The office use case is probably more realistic, but some other related products (Surface Hub, Jamboard) haven't become as ubiquitous as originally imagined.
bitcoinmoney | 5 years ago
Remote job interviews.
wraptile | 5 years ago
No way anyone cares about visual detail for job interviews.
randomsearch | 5 years ago
Great idea.
I’d first go with an Internet cafe style booth booking. Book an hour slot and get your coffee included.
Setup cafes in major cities and I think people would use this - could imagine setting a meet with a friend in another country. Parents showing off their new babies. Etc.
interestica | 5 years ago
I hate to say it but my first impression of this was that it looked like a visiting area for a prison. (Something about the bleak colour palette and minimalist display). But, I think that presents a similar long-distance use-case as the 'phone booth.' I wouldn't be surprised to see this as a pay-per-use option for prisoners/families. It's probably only a matter of time before the 'Echo Shows' capitalize on it.
nmfisher | 5 years ago
That’s exactly the thought I have. I have a feeling it would be truly bizarre to sit down and talk to my mum or dad through this.
baby | 5 years ago
Better than what we have now
rvnx | 5 years ago
Well at some point prisoners won't have the choice in practice, because for safety and convenience remote video call will become the norm. This is the same paradox as social networks; making people supposedly closer, but in reality making humans further and further away.
[Deleted] | 5 years ago
whymauri | 5 years ago
it would certainly sit comfortably in a black mirror episode, lol.
sangnoir | 5 years ago
A fantastic twist would be a reveal showing that the other person (no longer) exists in the real world and is digital construct... or some forbidden knowledge is secretly interdicted and censored by big brother, body-snatching peoples avatars in real time.
hyko | 5 years ago
Plot twist: the person behind the screen is actually sat there, everything else turns out to be a hologram!
Vinnl | 5 years ago
Yeah I had the exact same impression. Still vastly preferable over a zoom call, but I expect I'd have to use it extensively before I can shed that association.
This is assuming that this somehow does allow me to interrupt someone's sentence — otherwise might as well do a zoom call.
[Deleted] | 5 years ago
benhurmarcel | 5 years ago
> And most companies should be able to afford having one of these in the office.
Most companies don't give decent webcams already, I doubt they'd consider paying thousands for such a system.
agumonkey | 5 years ago
Totally unrelated but on the radio there was a company talking about making internet pods to be able to communicate without being in the open like in starbucks.
It seems we're seeing a second coming of distributed private comms.
dgudkov | 5 years ago
This could bring fax machines back too. Draw something on a piece of paper, put it into the receptacle, and voila - it's instantly printed on the other side.
ExtraE | 5 years ago
Or, I mean, live synced drawing tablets. Why paper? (Unless you like the feeling of writing on paper like me but I bet money they’ll replicate that sooner rather than later).
jspaetzel | 5 years ago
I'd pay something for this but I bet there's more of a business market.
ByteWelder | 5 years ago
I doubt that very much. Consider these scenarios:
1. Call for free from home/couch with a regular camera in the privacy of my home.
2. Go to some location and pay a third party to do a less private call, and have a better visual experience.
The most obvious choice would be number 1 in 99.9% of all cases. In theory, privacy can be increased by creating a sound-proofed environment for your call, but in practice, that would easily become very expensive.
evrenesat | 5 years ago
I would like to see this as a franchised service for cafes and bars. It would be nice to be able to treat drinks and the 3d chat experience to a friend who lives in another country. And you know, some conversations are done better at a place other than family home.
spurgu | 5 years ago
The main thing here is the radically improved UX (as advertised). With this kind of tech I might actually hook up with some remote friends with whom I currently just text (I don't really use video calls for anything other than work).
Edit: And even 0.01% of the video call market would be quite large. Naturally this idea wouldn't work if something like 10% of all video calls were replaced with these booths, they would be fully booked for weeks forward. ;)
Edit2: And we already know most people don't care that much about their privacy since they already use services like Google Meet and Zoom and whatnot.
tiagod | 5 years ago
I agree with you, I'm also imagining these in public libraries, retirement homes, small village community centres. I just hope we don't end up with a bunch of incompatible, proprietary appliances that can't talk to each other.
drusepth | 5 years ago
I get similar vibes to demoing VR the first time. My first thought was, "My tech-illiterate dad would have his mind _blown_ by this."
Unfortunately, even an already-set-up VR experience was too strange/unnatural for him so he never got to experience it. However, this looks easy/natural to use and set up and feels like it'd have a similar mind-blowing effect for many of the older generations, which I think is a good indicator of being revolutionary tech (assuming it can be made available/cheap enough for most people to try it out).
mhh__ | 5 years ago
I got my Dad to play SUPERHOT VR for a bit, he liked it, but I suspect the "Shoot yourself in the head to start the demo" bit was possibly a bit on the nose considering he spent quite a few years manning suicide helplines.
wildpeaks | 5 years ago
The "wow it's really 3D" effect without glasses is what people experience everytime I've shown Looking Glass to someone, glad to see devices with higher resolution are coming and more people in the market, end users will benefit from more concurrence to drive the quality up.
Judgmentality | 5 years ago
I can just imagine how crazy this would make my dog. I think the technology is amazing, I'm just imagining how much this will confuse animals.
williesleg | 5 years ago
Why is it the white guys build it but aren't in the video using it? So sick of this bullshit.
LeCow | 5 years ago
Google Glass flop.
Calling it now.
thomasfl | 5 years ago
Now make that person in front of you a virtual 3d rendered artificial person. Looking forward to be talking to a virtual Albert Einstein for real, sorta.
alkonaut | 5 years ago
I wonder how close you could come to this experience with some "standard" hardware like a standard 100-500mbit connection, a normal 4K TV and a Kinect. That is: given 10% of the cost, how many % of the experience could you deliver?
Simulacra | 5 years ago
That is really interesting but I can't shake the feeling of it being like visiting hours at the jail.
tyingq | 5 years ago
Same. I suppose they kept the decor and backgrounds stark to showcase what it's doing. But it does have that feel, as-is. At least nobody was wearing orange :)
[Deleted] | 5 years ago
booleandilemma | 5 years ago
Some fun backgrounds and filters will alleviate that feeling.
adrianmonk | 5 years ago
Yeah, on a certain level it's frustrating if a technology can seem to bring far away to an arm's length away, but not actually any closer.
AppleCandy | 5 years ago
No, just needs some onlyfans optimization. Step aside VR, there's a new medium in town, revolutionary engagement if casters can pony up.
sireat | 5 years ago
"The receptionist’s phrase had prepared her for the only kind of link she knew, a simstim signal routed via Bell Europa. She’d assumed she’d wear a helmet studded with dermatrodes, that Virek would use a passive viewer as a human camera.
But Virek’s wealth was on another scale of magnitude entirely.
As her fingers closed around the cool brass knob, it seemed to squirm, sliding along a touch spectrum of texture and temperature in the first second of contact.
Then it became metal again, green-painted iron, sweeping out and down, along a line of perspective, an old railing she grasped now in wonder."
jraph | 5 years ago
A bit off-topic, but I couldn't help thinking about Tantacrul's video, Corporate Music - How to Compose with no Soul [1] when watching the video on this post.
[1] https://www.youtube.com/watch?v=AIxY_Y9TGWI
illwrks | 5 years ago
I have very little knowledge of this, but as a product would it not be much cheaper to have something like a wii bar fixed to the top and bottom of a screen, with a small camera at the extremities of each bar (one for each of the four corners)?
You could then use the camera perspectives to create a 3d image of the person you're conversing with and map the colour data correctly to that 3d image (Photogrammetry)
You could also likely use the information from the four cameras to map the orientation of the 3d image of the person you're speaking with to give you that sense of depth as you shift your position. https://www.youtube.com/watch?v=Jd3-eiid-Uw
If you had a speaker and mic in each corner you could also capture / emit subtle differences in audio to further enhance it.
Mikho | 5 years ago
Looks like another Magic Leap but in TV formfactor—technology great in concept, and even MVP but completely unrealistic in terms of consumer grade HW as to affordability and accessibility. It will be another tombstone on a huge grave of Google products. Just like Google Glass and many other. It's too hard to trust Google these days and its initiatives.
helen___keller | 5 years ago
It looks like magic, when I saw the video I briefly thought it was a joke (ha-ha, they're actually in the same room!). I wonder how much of the magic is software and how much is hardware?
The video certainly plays up the software, but I've never used zoom or FaceTime in an 8k video call booth before, so I suppose I don't have a point of comparison.
dbbk | 5 years ago
It's not just an 8K video call, it also has 3D depth.
pbhjpbhj | 5 years ago
I'm not sure it's the parent's point (I suspect it is), but mine would be that it might be you get 80% of the "it's like they're there in the room" effect by having 8k; good lighting; well-separated, good quality, stereo sound; a purpose made space without distractions; etc.?
If you've only had 480/720 on a small screen, with a mono microphone with poor lighting, lag, dropouts, then just fixing those things and making it 2k+ might be surprisingly good (ie will have an emotional impact due to the truer representation)?
nynx | 5 years ago
I wonder if this uses a microlens display.
hyko | 5 years ago
This is so cool. I’d buy it.
notyourday | 5 years ago
Will be canceled within two years.
doodpants | 5 years ago
Two years after general release; they need time to develop a dedicated user base to upset by its cancellation.
notyourday | 5 years ago
True. No point in killing it before customers invest time and money into the product that Google was all in!
Pxtl | 5 years ago
This is cool as hell, but I have to say I feel like we're solving top-level problems when most consumers don't even seem to be getting solutions to the most basic pain-points.
For me, the problem with video-calling isn't the image-quality. It's all the much more mundane technological problems - high latency, lag-spikes caused by bad ISPs, failed noise-cancellation for people who don't use headsets for audio, bad wifi routers cutting out, etc.
First thing I did when I realized we were going to be WFH long-term was buy myself a $100 gaming headset. Next thing I did was get all my home computer stations wired with Cat 6.
That stuff is far more fundamental and far less interesting than 3D telepresence, but it's the real unsexy problem that so many people are suffering through this pandemic.
Even simple things like latency make simple, natural reactions agonizing. Talkcover and crosstalk is incessent and I've developed a filthy habit of just talking over people because otherwise it's a solid 20 seconds of "you go no you go" caused by awful latency. I've had to defuse angry reactions by co-workers who feel they're being interrupted by other co-workers and explain to them that the latency makes interruptions feel worse than they are.
I've tried to push friends to join me on my private Mumble server where the latency is near-nil and the call-quality is excellent, but there's always one person who doesn't have a working headset and wants to just use a laptop or tablet mic with no feedback-cancelling that destroys the conversation through echos (plus Mumble's auth system is needlessly bewildering).
Then with video, problems are similar but less impactful - cheap cameras, poor lighting, compression artifacts, poor sync with the audio, etc. And it's infuriating because every person has a wonderfully powerful camera in their pocket right now - and there's software to connect them but it's just too tricky for most people.
Good on Google for taking an interest in the subject, but I feel like they're decorating the apex of the technological pyramid while most people are pushing stones around at the bottom.
kumarvvr | 5 years ago
Perhaps one drives the other. Availability of magical tech will drive adoption of best practices in all the areas you have mentioned.
In India, ISPs already advertise low latency and high speeds for "PUBG Gaming". The market evolves to solve the needs of consumers. Advertising for low latency for gaming was unheard of in Indian ISP scene, just a few years ago.
So having this tech, would induce customers to get better hardware, wiring, ISPs etc, and would induce ISPs to provide better service.
noahl | 5 years ago
I mean to be fair Google has also tried very, very hard to improve home internet access for people, to the point of setting up their own ISP and running municipal fiber networks. That's a pretty big try, and I really wish it had taken off beyond the places where Google Fiber operates.
(NB: I work at Google, but this comment has nothing to do with my work.)
apexalpha | 5 years ago
When it really comes down to laying Fiber isn't a job requiring innovation, it require hard man hours to pull cables.
In the end Google tried to innovate around the hard work by burying cables around 5cm deep or so in stead of a meter, which turned out to be short sighted.
hyko | 5 years ago
The poor you will always have with you
julienb_sea | 5 years ago
Both problems are worth effort and energy. It is worthwhile to push the envelope at the top because that technology, if it really works and can be developed in a more consumer friendly way, will eventually become vastly more accessible.
Solving gigantic scale problems requires a completely different kind of innovation than what you can achieve by pushing the pinnacle of what's possible.
miralize | 5 years ago
It also in its (albeit) small way could push ISPs to get their act together. Perhaps thats a bit naive to say but if this sort technology is available, this will help push the boat forward in terms of underlying infrastructure.
TulliusCicero | 5 years ago
I mean Google also has Meet for regular videoconferencing that's available to normal humans for free, so it's not like they're ignoring the mainstream.
The issues with connection stability and latency are very real, but I don't know if it's reasonable to expect Google to fix it; the issues there are probably more political than technical.
edit: Also, I think they did mention using AI for noise cancellation while videoconferencing in the keynote today.
topkeks | 5 years ago
I think Meet has had noise cancellation for a while now.
mpalmer | 5 years ago
For what it's worth, Google used Meet internally for years before making it publicly available.
crazygringo | 5 years ago
Most of that Google can't do anything about, because it's not under their control.
And it's really a question of money. If you want to fix all those mundane problems you describe, literally every one can be addressed by renting a better pipe and buying and setting up better equipment.
This Google product is clearly designed for high-end offices that already have all of those things under control.
lmm | 5 years ago
Google owns one of the major consumer operating systems (Android) and a lot of videoconferencing goes via Chrome. They could absolutely be doing things like offering better connection quality metrics - look at how everyone currently ends up using a random shady "speed test" site.
blululu | 5 years ago
That exists now. If you Google search ‘speed test’ (on mobile at least) the first result is now a simple speed test provided by Google.
lmm | 5 years ago
Not for me, either on android or desktop chrome.
notatoad | 5 years ago
Normal video chat at its best is just kind of shitty though, so people don't have a whole lot of motivation to put in a bunch of work to make it still shitty, but slightly less so.
If I could get video chat that felt like real life, that'd be worth running some wires for.
rubatuga | 5 years ago
Anybody here want to fork Mumble for a better UI and authentication experience?
russdpale | 5 years ago
Convenience and necessity. If the apex tech is so wonderful that its convenience becomes necessity, then it will become those proverbial stones on the bottom.
jaza | 5 years ago
There I was, reading the intro blurb, thinking yay, we've finally built a holodeck.
But alas, reading further down, just the same old flat screen, with slightly better spatial trickery than what we've already had for a while now.
Computer, end program.
jarek83 | 5 years ago
Great tech, shame it's google - whatever they do, they do it to sell you, not to you.
busymom0 | 5 years ago
Is it just me or does this pic used in the header article look quite depressing? It makes me feel like talking to someone in prison:
https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...
Animats | 5 years ago
This is a lot like Winscape, from 2012.[1] That's a display hooked to a Kinect head tracking system, so, as you move, the display is adjusted for your point of view. Winscape just used it as a window to the outside world. Same concept as VR headgear, without the headgear.
The illusion only works for one viewer at a time, because the image is POV-dependent.
Google is apparently using this with a 3D TV. Can you still get those? The main problem with doing this now is getting all the retro technology you need - a 3D TV and the good Kinect. Any good current GPU should be enough engine for this.
[1] https://youtu.be/V2hxaijuZ6w
ksec | 5 years ago
I used to say Google has never done anything really great other than their search engine. And generally speaking this opinions tends to get quite a lot of upvote ( as well as some unsuccessful counter arguments ). That is why when another Google project launch or failed. I wont be surprised.
This is the first time, I felt this is going to be great. Something that should have came from Apple, the humane side of technology, for the first time ever came from Google.
Hopefully this isn't a one off thing or an outliner from Google. Apple desperately need some competition.
abrahamepton | 5 years ago
It’s a pretty silly argument. Maps, android, gmail, translate, docs, mapreduce, kubernetes, tensorflow come to mind off the top of my head. Honestly it’s a stupid contention. Apple’s awesome but the list of “great” things they’ve done is what, a third as long despite existing decades longer?
mritchie712 | 5 years ago
It could also completely disappear due to lack of a compelling enough business model. We'll see!
sammalloy | 5 years ago
The biggest selling point of this technology is its ability to reduce business travel and cut down on carbon emissions.
bscphil | 5 years ago
Feels like the future of this will be using AI approaches to simulate dead loved ones. I don't think that's a dystopian outcome, there are a lot of good aspects to it as well, even if we don't know the full ramifications.
thepasswordis | 5 years ago
Am I the only one horrified by this? The second photo of the person sitting in a booth looks like a set piece from some cautionary science fiction movie.
kossTKR | 5 years ago
No, the whole setup looks dystopian to me. The presentation too knowing Google. Looks like a prison.
People are already depressed as hell from the lack of touch, real family, community and friends in modern society.
That said, of course this tech could have its uses, but mediated by the largest corporations on Earth that collect, sell and mediate everything about you and your friends? No thanks!
dmux | 5 years ago
I can only imagine how unnerving it would be to hit a spike in latency or something and be shaken from your "suspension of disbelief" while your loved one glitches out and falls into uncanny valley territory.
rdtwo | 5 years ago
Google is bringing back the phone both! In 3D pretty cool
nprateem | 5 years ago
I wish they'd pump their money and brainpower into trying to reverse climate change instead of an incremental improvement to chat like this
hirundo | 5 years ago
My dad used to tell a story about my grandfather's first experience with television in the late forties. There was a buxom woman on the screen, and he walked up to it, trying to look down her cleavage. It didn't work.
Does Starline give you a different view when you change your perspective? It looks like it does to some extent. If so, it might work before long, but grandpa died about fifty years too soon for it.
spoonjim | 5 years ago
Does your dad think your grandfather's problematic behavior was funny?
sneak | 5 years ago
My understanding of the existing tech that's out (3d displays) is that it allows for such on the X axis (different view per-eye) but not on the Y, so a scenario such as you described (looking downwards) might still remain sci-fi for the moment. Perhaps this does something entirely new/novel with display tech but there's nothing to suggest that at the moment.
blastro | 5 years ago
Great story re: your grandfather. Thanks for sharing.
bellyfullofbac | 5 years ago
On the topic of buxom women, my first thought about this tech was, it'll be the next frontier for OnlyFans/porn accounts on Snapchat.
Before you laugh or be prude, porn content was what made VHS and BluRay succeed (or are these urban legends?) and they were pioneers in stuff like online video streaming.
adhoc32 | 5 years ago
Yes, like VR this will be used extensively by the porn industry.
thepasswordis | 5 years ago
Ah yes, the people whose first thought is not how they can use this to commodify women’s bodies are “prudes”. Yes of course. Women: become literal prostitutes for the only fans corporation. You could be RICH! It’s so fun! They have t-shirts! Remember how I said you could be RICH?!
bellyfullofbac | 5 years ago
Not sure my comment will bring anything to change your opinion, you seem to be already a hardcore antiporn person, and I can't defend sex work either because yeah I know it's got a lot of seedy aspects to it; but no, they're not literal prostitutes, cam girls and OnlyFans models have the advantage that they can earn money without having to be in contact with other bodies and potential diseases they bring.
I know some girls who are prostitutes. They seem normal enough. Maybe they're wearing a facade and have hidden PTSDs, I don't know, but if that's the case, I think you might want to extend your crusade against the military.
goldenchrome | 5 years ago
How is this any different from me wasting my life working for FAANG? Why is commodifying my brain any better than commodifying my body?
thepasswordis | 5 years ago
I don’t know. You should ask some of the abuse victims coming out of the prostitution industry why they’re so upset, and remind them that their job is basically the same as working at google.
[Deleted] | 5 years ago
CyanBird | 5 years ago
> commodify women’s bodies
The fact that you instantly associate what OP mentioned to women specifically is quite disgusting, not going to lie
FYI More people than just women suffer through that and taking the oxygen of such a discussion via weaponizing linguistics is just nasty and a horrible way to spend the opportunity cost of it
[Deleted] | 5 years ago
flowerlad | 5 years ago
Does Starline include haptic gloves?
agumonkey | 5 years ago
maternal grandfather had trouble adjusting to tv too, he never stopped greeting female tv hosts on screen
yakkityyak | 5 years ago
That would really take it to the next level if it had that feature, even with very few degrees of freedom.
The (New) Nintendo 3DS has head tracking, but it doesn't change your perspective into the view port, which gives a very dizzying effect when you deliberately test the feature.
I would imagine its possible to extrapolate perspective if they had an array (N > 2) of cameras.
This is super cool tech, and can't wait to see an array of these installed in the secret undisclosed board rooms for the illuminati.
ibrahimsow1 | 5 years ago
If this uses fully fledged lightfield[0] it might be able to.
[0] https://www.google.com/amp/s/techcrunch.com/2020/06/23/googl...
alkonaut | 5 years ago
Why would you need lightfields to make a TV that shows a different projection if you change vantagepoint?
The device knows where the viewer's eyes are in 3D, so can display an image of the other end, as viewed from that point (within some constraint).
ExtraE | 5 years ago
Non-amp link: https://techcrunch.com/2020/06/23/googles-46-camera-light-fi...
paxys | 5 years ago
The biggest barrier to virtual healthcare today is that patients find it hard to connect with and trust doctors over a screen. This seems like a perfect product fit, especially among older people.
homedepotdave | 5 years ago
Is there evidence of this? This is not what I would guess the greatest barrier is to virtual healthcare
mNovak | 5 years ago
Everyone's latching onto the 3D part, but if some version of this just lets us have our Zoom calls without the looking at camera/looking at screen, impossible to make eye contact thing, that'd be a step forward.
nemonemo | 5 years ago
I think another good use case for this sort of hardware/service could be for youtubers/streamers/performers who want to provide immersive interaction to the viewers. If I am a fan of a singer, I'd definitely pay for an opportunity to watch the person singing right in front of me, instead of going to the concerts where I watch them singing miles away.
asim | 5 years ago
This is actually groundbreaking technology. If this gets widely deployed and then evolves we're looking at telepresence as the next step in ambient computing. When the technology fades into the background that's when you know things are going to be remarkably different. This is quite honestly going to be as close to teleportation as we get.
Pxtl | 5 years ago
If it got widely deployed onto infrastructure not owned by Google, real-world internet conditions would ruin it with latency and lag-spikes.
sammalloy | 5 years ago
Asim, are you familiar with how Stephen Baxter portrayed the use of telepresence in his science fiction novels? If not, check it out. But if you are, please explain it to me! It always confused the hell out of me.
kaonwarb | 5 years ago
I would love to try this - and use it for calls with family, etc.
I’m skeptical on its impact on the in-person vs. remote debate for work. What really distinguishes in-person for me is the potential for informality. Bump into somebody, have a chat, walk over to a colleague’s desk for 5 minutes, decide to go on a walk for a 1x1 instead of sit in a room. This doesn’t do anything for any of that. In fact, it arguably increases formality.
Understood that many folks are not interested in those in-person artifacts - sharing what I see as key distinctions which aren’t in the solution space of Starline.
the_gipsy | 5 years ago
Can't wait to see ads in 3d
isaacimagine | 5 years ago
This conceptually seems pretty similar to what Tonari [7] is doing, but there are some technological differences between the two. Just interesting, that's all.
7: https://tonari.no
xwdv | 5 years ago
Wow, will be great to make recordings of people that you can sit and watch long after they're deceased. Maybe with some deep fakery even create AI versions that can respond to conversation.
joelkesler | 5 years ago
This looks really cool. Exciting to see how this plays out in the future.
Does anyone know if the display is similar to the display in the Looking Glass 8K holographic light field display?
https://lookingglassfactory.com/8k
airstrike | 5 years ago
This, my friends, is how business travel becomes nearly irrelevant.
This is a beautifully executed idea and if the demos live up to expectation the hype may even be warranted. But on a much more fundamental level (i.e. fancy 3D imaging and spatial audio aside), this also possibly suggests people would benefit from dedicated videoconferencing hardware. TVs and telephones do one thing really well (or at least historically they did), which is why even my legally blind grandpa could call his friends or watch^W listen to the news. There's a market for having a plug-and-play videophone now that we have the software to go inside it.
What are Zoom, Facebook or Apple waiting for?
adam | 5 years ago
I immediately thought of business travel as well. Project us all in to a virtual conference room and give us a suite of collaboration tools to use while we're all there.
The only thing missing is the after meeting drinks and dinner, but there will inevitably be services to put us all in a restaurant/bar environment, pipe in some bar white noise, have food sent from a local restaurant, etc. for an "in-person" virtual happy hour...
umanwizard | 5 years ago
> there will inevitably be services to put us all in a restaurant/bar environment, pipe in some bar white noise, have food sent from a local restaurant, etc. for an "in-person" virtual happy hour...
I honestly can’t believe this still sounds fun to anyone after a year of Zoom dystopia.
ghaff | 5 years ago
The last year has shown pretty clearly that for presented content, video works pretty well. Heck, it maybe even works better along with live chat than in person. (And, to be honest, for big events I would often watch live streamed keynotes rather than crowding into a conference center with 5,000 of my closest friends anyway.)
But everything else about virtual conferences has completely sucked and anyone running events is aggressively trying to get back to in-person. (With a hybrid component for presentations.)
unethical_ban | 5 years ago
That "the only thing missing" is the main reason to go to many conferences.
A lot of people (those with less social desire/social skills) seem to resent it, but it is true: Networking and casual technical conversations that happen afterhours are the draw for many technical conferences. Talks can be good, and occasionally there are well-constructed lab sandboxes. But mostly, it's going and speed-dating with peers and sales teams to talk about your needs and architecture, and building a good web of contacts.
I also believe fully remote technical/collaboration work, without any periodic physical meetups, will be awful for a lot of people. Sure, those who bought into it pre-pandemic prefer it, and that's fine. But I really think there is concern to be had for the fraying of social bonds and teamwork that can be done, even (or especially) with people you have a tough time working with.
ghaff | 5 years ago
Yeah, the last year has shown us that streaming videos with some side chat is the easy part. Heck, maybe it's even better than in-person a lot of the time. I can re-record stuff when I screw up and do some things I can't do in front of a room of 50-100 people.
And it's even good that people who just go to sessions for the content will be able to do so--for a lot less money and effort. But I'm planning to go back to in-person conferences as soon as possible.
ghaff | 5 years ago
Better videoconferencing hardware is not a solution to people wanting to get together socially and serendipitously at events. And, by the way, how many people are going to turn a room in their house into a work videocall room?
Personally, I find that--for most people--the idea that working remote shoves a lot of cost onto employees vs. commuting probably off-base. (With some exceptions for people living in small city apartments near offices.) But installing a room-sized videoconferencing setup at home even for people with decent-sized houses is pretty silly.
nicoburns | 5 years ago
> how many people are going to turn a room in their house into a work videocall room?
Very few while it's $20k+. But I can imagine a lot of people would want one if the price was reasonable. I'm sure you still use the room for other things.
IshKebab | 5 years ago
Who said anything about a room in your house? This will be for offices first.
ghaff | 5 years ago
Sure. But given that this sort of thing has been discussed many times before it's hard to ignore the context of remote work.
sigstoat | 5 years ago
> This, my friends, is how business travel becomes nearly irrelevant
the business trips i've been on either involved the installation of equipment, or were an excuse for somebody with budget to burn it on travel and expensive food/alcohol. or both.
i think plenty of business travel will survive just fine.
nmca | 5 years ago
Entertainingly, the technology to real-time impersonate someone over zoom photorealistically is also rapidly approaching - the window of irrelevant travel might be small.
Sargos | 5 years ago
I don't think this will actually be a real problem as you can just sign the call with your identity which renders deep fakes useless.
ajliu | 5 years ago
> There's a market for having a plug-and-play videophone now that we have the software to go inside it.
> What are Zoom, Facebook or Apple waiting for?
This is already a thing?
Facebook has Portal, Apple has iPad, Amazon has Echo, all of whom support Zoom and other video conferencing apps. The portal even has a moving camera to keep you centered if you're moving in frame, and the iPad does the same thing with an ultra-wide lens and some post-processing.
As far as dedicated videoconferencing hardware is concerned, Starline seems pretty late to the game. Although, I'll admit the fancy 3D imaging features is pretty insane.
hbosch | 5 years ago
Unfortunately, this stage of the Google product cycle is the hardest for me to start getting excited for. I hope for better, but I cannot resist the nagging feeling that this will 1) be very, very cool, 2) be sold to enterprise customers who are OK streaming business calls through Google's cloud, 3) suffer from having no support, 4) be renamed and reclaimed by another team inside Google, 5) sunset.
Is "Starlink" going to be a Gmail or a Wave? Hard to say.
umanwizard | 5 years ago
> how business travel becomes nearly irrelevant
I hope not. Video chat can really never be the same thing as meeting people in person.
dontwannabe | 5 years ago
True. Covid Pandemic has proven 2 things.
1. Work From Home works. 2. Business Travel is totally unnecessary and saves a lot of money.
End of an era for Travel whores.
alwayshumans | 5 years ago
From my experience business travel is as much about sharing an experience as it is the discussion or dissemination of information. That's a hard thing to replicate
Bognar | 5 years ago
The real value of business travel is what you do with people outside of work hours.
baud147258 | 5 years ago
and business kickbacks and shared drinks at the hotel bar
mortenjorck | 5 years ago
Until technology can replicate the experience of staying out late, drunkenly chatting with coworkers in the back of a taxicab as you ride through Manhattan, then grabbing a midnight snack at a hole-in-the-wall diner someone recommended back at the office, business travel will have its place.
agumonkey | 5 years ago
My theory is that physical proximity means danger or love to our animal brains .. and knowing it's tech will disengage your brain from feeling close and will change your emotions and engagement to the other person. Now something more natural than a LCD screen might make video calls a bit more lively and efficient.
tomc1985 | 5 years ago
Maybe for the set of people that don't mind gimmicks and facsimiles, but for those of us that don't want to even turn on video there is no way you're going to get me into an entire videoconferencing box!
Nothing beats in-person. Nothing!
ilaksh | 5 years ago
https://en.wikipedia.org/wiki/Autostereoscopy
Which type of autostereoscopy is it?
riffic | 5 years ago
HP did this at least 15 years ago with their "Halo" conferencing systems:
https://www.networkworld.com/article/2258553/inside-the-halo...
https://www.youtube.com/watch?v=TC4sNztp8dk
LogicCowboy | 5 years ago
HP's Halo conferencing system had several studio quality cameras that would change the video feed based on who was talking. There were also specially placed mics on the conference table to keep audio crisp. The video quality was impressive, however you had the same issues that you have with current web-cam based systems. Eye-contact was non-existent. While the camera placement was above the middle-top of the LCD screen, you'd never feel like the person was making eye-contact with you unless they looked directly at the camera. I had several team meetings in Halo rooms, and for the time, it was the next best thing to having a meeting in person. It also worked well for groups of 4 on each side of the link.
From watching the video, Google's conferencing setup is creating a 3D rep of the people talking and adjusting rendered view based on where the participants are seated. This is blending AR with videoconferencing. It would be interesting to see how their conferencing system works with multiple-people on each side. I know the video had a mother and baby in one segment, however is the 3D rendering based on the eye-level of the main participants?
com2kid | 5 years ago
> Eye-contact was non-existent.
Potentially a solved problem, just fix it in post. :)
https://www.zdnet.com/article/windows-10-microsoft-finally-l...
I wonder how well it works, and how much latency (if any) it adds to the feed.
I'm also sad it isn't rolled out more generally, a very strange feature to lock behind a small-ish volume hardware product.
percentcer | 5 years ago
Halo was just a big video screen. I worked at DW for six years, it was nice but it didn't feel like you were there in person.
mrfusion | 5 years ago
I’m confused what the 3d display screen is? I thought we didn’t have technology like that without glasses?
AdamTReineke | 5 years ago
Appears similar to the tech in the Nintendo 3DS: lenses over the screen so each eye sees a different picture. See https://en.wikipedia.org/wiki/Autostereoscopy
jayd16 | 5 years ago
Why does it appear similar to that tech other than you get a 3d image? As far as I can tell there's no info on the display other than it uses "Light field technology" which would make it different than the parallax barrier display on the 3DS.
gmueckl | 5 years ago
The term light field technology is broad enough to cover lenticular arrays in displays. The 3DS had the major limitation that it only rendered two views. If you increase the resolution to be able to display more views for more viewing directions within a wider cone, you already have a light field display - simply because all these views combined form a sampled light field representation by definition. This is exactly the same as glassless 3d displays for multiple viewers of decades past. But advances in display pixel density and computing power apparently make the resulting illusion much more convincing these days.
jayd16 | 5 years ago
Isn't it clear glass and not a layer over a traditional display like the 3DS? Glasses would either actively time slice with shutters, or spectrum slice with passive filters on the lens and of course you need the glasses.
How could any of those technologies be what is used here?
E: Looking again, perhaps it could be some layer over a traditional screen. You see through some of the broadcasts but that could just be the digital far plane that shows through.
gmueckl | 5 years ago
I'm not sure I'm following. If this is based on a flat panel display, there must be a lens array in front of it. There is no other way to achieve this effect without requiring the users to wear glasses. The lens array can be covered by a protective flat glass pane without issue.
chis | 5 years ago
I think it's something like https://www.youtube.com/watch?v=pI__qNx8Gdk
Track both eyes, and then project an image to each eye based on its image in the room. The part I don't really understand is how it's possible to target the image to each eye. Maybe we have displays now which are like the 3DS screen, but with variable focal locations?
TulliusCicero | 5 years ago
If it's that, why does the camera see a gradually different image as it pans around?
See: https://storage.googleapis.com/gweb-uniblog-publish-prod/ori...
Notice how the angle of her face changes as the camera moves: first you only see her left ear, at the end of the animation you only see her right ear.
dieze | 5 years ago
Use a demo mode to disable eye tracking and follow your ARcore/Vive puck located camera ? Or just ask the guy to close his eyes and put googly eyes on your camera...
Hakeemmidan | 5 years ago
This would be incredible for immigrants that haven't seen their families in a long time
Can't wait for them (Google) to get there (large-scale production)
beefman | 5 years ago
Why does the copy on this page read like something out of dystopian novel?
drumhead | 5 years ago
Nice promotion project, I wonder how long it'll last.
pc2g4d | 5 years ago
Make an entire wall out of this, now you are virtually colocated.
rmccue | 5 years ago
The capture and compression part might be related to this Google Research:
https://augmentedperception.github.io/deepviewvideo/
Interview about the SIGGRAPH paper here: https://blog.siggraph.org/2020/08/how-google-is-making-strea...
helios_invictus | 5 years ago
Doesn't look like these people are in a prison visit center at all.
andyxor | 5 years ago
that.. looks a bit like prison visit
I hope they don't start using this for "whiteboard" interviews, would be super stressful to be interrogated from behind a mirror with nowhere to go
asenna | 5 years ago
On a related note, I wanted to share my amazing experience with a similar but lo-fi tech.
My mom and dad for the past 2-3 years have mostly lived in two separate countries (due to work reasons) and I could tell they miss each other quite a bit.
I got both of them an Echo Show 10" device each and set it up for them. I don't think I can explain how our lives have changed for the better, just based on this simple piece of tech.
The Echo Show is now pretty much constantly on video call for 14-16 hours a day in the living room of both houses and it has become an extension of the one another; a window into the other house. The audio and mics are good enough that at times you genuinely forget the other person is not in the same room. This has truly helped them, especially during Covid times.
It's actually good enough that my parents have their morning tea together practically almost the same way they used to when they were physically together. They've told me multiple times "I don't know how we could've lived without this Alexa thing".
If Google can eventually get the prices down to reasonable levels, I really think people would be shocked at how fast this thing becomes part of our daily lives.
lloeki | 5 years ago
I share the feeling.
I started working fully remote two and half years ago. Company has a couple of meeting rooms fully set up with Zoom, which I and a few other remote people used to connect to with my laptop for standups together with the in-office people. That's not the case everywhere but quite classic.
The company also had a dedicated TV+camera+24/7 Zoom meeting setup in the open space itself, so that remote people could connect to and say hi anytime. It was surprisingly nice.
Then one day on a whim I got a wide-ish USB webcam plugged into my Xbox One†.
The difference in perception is insane: instead of having this wall of people faces, like you're on several phone calls at once, I got a portalesque window between the office and my place. Instead of connecting people, it connected places††. But it only works if both sides are sharing a space instead of a selfie angle, and I think the way sound was picked up by a fixed element in the room played a lot as well.
So I'm a bit partial about this Starline thing, as it seems to be a step forward in "definition" but also feels like a blown up selfie angle.
† There's no Zoom app on Xbox (which is really a shame) so I used Edge, for which Zoom on the web is more limited and barks on webaudio (which works! Zoom just support only Chrome so I had to dial in on a phone). Also, Xbox+Kinect truly had the potential to be something amazing beyond games. I believe the marketing was botched alright but it was also way too early for the audience.
†† Which ends up connecting people on a deeper level. At some point I caught myself intuitively wanting to hand over physical objects through the TV.
JonathanFly | 5 years ago
>The Echo Show is now pretty much constantly on video call for 14-16 hours a day in the living room of both houses and it has become an extension of the one another; a window into the other house. The audio and mics are good enough that at times you genuinely forget the other person is not in the same room. This has truly helped them, especially during Covid times.
>It's actually good enough that my parents have their morning tea together practically almost the same way they used to when they were physically together.
I don't understand why this sort of persistant 'portal' isn't more popular. But 10 inch screen is so small! A 55 inch TV is what, $300? Put it portrait mode and that's a life size portrait. The feeling of presence when the person's head is the correct size is so strong! I know Facebook sells something for the TV but can it be left on all day like that?
I want it feel like one half of living room is a shared space, at all times. Like the couch is split in half by a portal. Then it just cycles through a list of friends and family with the same setup to find someone else watching Netflix or whatever, and you can casually join them just like a real shared living space.
I don't need 8k or 3D tv. Just a 55 or 65 inch TV on the wall at the end of the couch!
alexgmcm | 5 years ago
Wouldn't it be really difficult to put a 55 inch TV in portrait mode?
Although I think as I live in a city with small apartments, wall-mounting is much less common. I guess with a decent wall mount perhaps it is trivial to rotate it?
baby | 5 years ago
I used the small version of fb portal with my parents for a while and it was great, but I wish it would have been bigger. I’ve been thinking about buying a larger portal, but I might want to wait for this starline thing (although I have a feeling it’ll be very expensive).
chaostheory | 5 years ago
If you don't mind Facebook invading your house, I find their Portal TV peripheral to be good since the screen size is much larger.
https://portal.facebook.com/products/portal-tv/
There's another portal model that's as big as a small mirror.
faitswulff | 5 years ago
Did you consider Google Nest Hub for this use case? I’ve been thinking of doing the same thing but I’m not sure which to get.
asenna | 5 years ago
Last I checked, Google Nest did not have a "Drop-in" feature like Echo where my parents can just drop-into the living room of the other house (no calling or ringing).
The Echo device just makes a sound that someone is dropping-in and gives a 10 seconds heads-up (of course this only works if you enable the permission for family members. Else it's turned off by default).
faitswulff | 5 years ago
Neat! I’m sold on that feature. Thanks.
thegginthesky | 5 years ago
This is very very cool.
I remember when I used to be in a long distance relationship with my now wife. We used to have the laptops on Skype for hours just to have each other's company, even if we were studying or doing something else. It was very unique at the time.
I'm quite jealous cuz back then we didn't have Echo Show, it'd probably be used just like your parents use it.
[Deleted] | 5 years ago
pinko | 5 years ago
I can echo (no pun intended) this experience with the Echo Show, but I would love something much bigger and higher-def. My family is separated long-distance, and I would pay big money for a higher-def "window to another home" product.
fataliss | 5 years ago
Can't one facetime on an appleTV/macMini with a webcam? Or use an ipad Pro?
asenna | 5 years ago
"Alexa, drop-in to Kenya" works amazingly well.
[Deleted] | 5 years ago
deadfire55 | 5 years ago
Facebook has a 15.6" version of this called Portal Plus. I can't speak to the quality/value of it.
retreatguru | 5 years ago
We tried Portal Plus at work to bring our two remote workers closer to the local office but it never really caught on in the office setting. The quality and tech was quite good though. Fb login required of course which was a minor nuisance.
russdpale | 5 years ago
That is really, really awesome!! I'm so glad it worked out for them! Its nice when a comment restores some faith in humanity.
[Deleted] | 5 years ago
[Deleted] | 5 years ago
rexreed | 5 years ago
Side note - why does it feel like so many of Google's products, and especially their most core offerings are in a state of ongoing decay? Search is getting worse. Maps today is worse than maps 10 years ago. Even their Google home assistants, YouTube, Gmail, and other products just seem to be steadily creeping into more annoyances and less usability.
There was a time I felt that Google's search was head and shoulders above anyone else's and when I reluctantly used DDG I felt I was compromising some aspect of search value for privacy value. But now when I use Google search I'm bombarded with ads, find I can't trust search results, and even copying search links results in a mess of Google redirect.
What is happening to the core Google products or am I a curmudgeon for believing that the Internet was at its most functional and user-friendly best circa 2008?
baby | 5 years ago
I thing things get bloated and bloated until there’s a rewrite. This is what happened for the facebook app, and google maps is very close to that point. I’ve started using apple maps more and more and I like it. I’m semi joking when I say that I upgraded to the latest iphone just to use gmaps.
Mangalor | 5 years ago
To be fair, you would not want to be confronted with the reality of these products 10 years ago, there's been a million bug fixes that have been done since then. But I agree on Search, it's almost like they sabotaged it.
a1371 | 5 years ago
The search too. Wikipedia articles and useful covid graphs. Excellent context inference. The assistant is surprisingly powerful when I ask engineering questions (what's the coefficient for this when that).
mcbutterbunz | 5 years ago
I would wager a guess that its due to lack of competition. Apple Maps only works on iOS. DDG is great but the search results aren’t. Does YouTube even have a competitor?
I don’t think you’re a curmudgeon at all. Circa 2008, Google was innovating in a lot of different areas that were very valuable to the average user. The ROI on these services has gone way down.
kjakm | 5 years ago
NB: Apple Maps also has a dedicated Mac app.
tempodox | 5 years ago
While that's true, GP's point still stands. You can use G-Maps on all platforms that have a browser, but you can't use A-Maps that way.
[Deleted] | 5 years ago
netik | 5 years ago
This technology is unreal, there is a deep-dive here:
https://creativity.arch.tamu.edu/_common/FoVI3D_DeepDrive.pd...
agumonkey | 5 years ago
google reinvents the uncanny valley ?
I thought it was some stereographic encoding on layered displays but the second they mentioned 3d mapped videos I started to see pixar like characters. Very odd.
mrfusion | 5 years ago
Much simpler idea but I think I’d get a good sense of being together to play table tennis over the quest with a friend. (Sadly none of my friends will buy a quest.)
bradneuberg | 5 years ago
I agree I love my Quest, and the sense of presence is amazing. Can't wait until more of my friend social network has them so we can meet up and do things like table tennis.
jonas21 | 5 years ago
There's a bit more detail in this Wired article (though it's still quite vague):
https://www.wired.com/story/google-project-starline/
> What I’m actually looking at is a 65-inch light field display. The Project Starline booths are equipped with more than a dozen different depth sensors and cameras. (Google is cagey when I ask for specifics on the equipment.) These sensors capture photo-realistic, three-dimensional imagery; the system then compresses and transmits the data to each light field display, on both ends of the video conversation, with seemingly little latency.
> All of the data is being transmitted over WebRTC... What Google claims is unique is the compression techniques it has developed that allow it to synchronously stream this 3D video bidirectionally.
randompwd | 5 years ago
> I met with three separate Googlers in Project Starline (all of them men)
Well, at least we know that they weren't all white men; because if they were, this writer would sure have let us know about this very irrelevant information.
[Deleted] | 5 years ago
ProAm | 5 years ago
Id just be really uncomfortable with google monitoring, recording and analyzing my personal communications to sell me ads or worse later. Everything google does now feels.... invasive for lack of a better word. This is something Id like to see from Apple, and Im not an Apple fan at all.
silentsea90 | 5 years ago
Do you pay for youtube?
ProAm | 5 years ago
I don't publish content on YouTube, but YouTube is also not intimate communication with friends and family.
hu3 | 5 years ago
I was surprised today when I opened Gmail and it displayed this: https://i.imgur.com/SHSOSJ7.png
I was ok with Gmail scanning my emails to finetune targeted advertisement. But they don't do that, which is a nice surprise.
throwaway3699 | 5 years ago
> Google Cloud (which offers Meet) doesn't use customer data for advertising. Google Cloud doesn't sell customer data to third parties.
https://support.google.com/meet/answer/9852160?hl=en-GB#zipp...
Why would this be any different?
ProAm | 5 years ago
I do not trust what comes out of Google PR vs what their engineers do behind the scenes. They've been caught in the past doing nefarious things or just a general disconnect/interpretation between what they say and what they do. Because they are an advertising company I always assume that takes priority with direction.
intricatedetail | 5 years ago
How can you prove they don't do that?
crazygringo | 5 years ago
Does anyone have a clue how the light field display works?
Someone suggested this is the tech:
https://lookingglassfactory.com/tech
But it still has no explanation of what the "proprietary light field technology" is.
How do you get each pixel to show up as a different color with 45-100 separate angles?
I had no idea this actually existed in production!
adrianmonk | 5 years ago
I was curious too and found this slide deck:
http://creativity.arch.tamu.edu/_common/FoVI3D_DeepDrive.pdf
The explanation of how it works begins on p. 13.
It seems to basically just have a huge number of very dense pixels, and in front it has some tiny lenses that direct each pixel at a different angle.
Presumably there's a maximum distance you can view from because the further you get away from the display, the smaller the angle between your eyes. I would guess it's no coincidence that Google has set it up as a booth with a back wall that would ensure you keep close enough:
https://storage.googleapis.com/gweb-uniblog-publish-prod/ima...
curtisf | 5 years ago
I've been very curious about this too.
I believe there are two basic approaches.
The first is "lenticular lenses" -- the same technology used to make those shiny postcards that change depending on what angle you tilt them at.
Here's a site with an example GIF and kinda explains how it works: https://setosa.io/blog/2014/07/07/lenticular/
Normally, the lenticular lens is glued directly onto a printed sheet of paper, but you can instead glue it onto a _screen_, which lets you change the pixels underneath. There are a lot of challenges here; for example, the pixels are usually located a bit behind the glass, so the standard lenses used for printing won't focus perfectly on a screen.
There's also another, simpler approach called "parallax barrier". This is what the 3DS used. Instead of using lenses to bend light, it _blocks_ the light coming out of certain pixels at certain angles. It's basically just an opaque sheet with periodic slits (the 3DS uses an LCD, so that 3D can be turned off by becoming transparent). Before I found a video of someone making one themselves using an iPhone and transparency sheets, but I can't find it at the moment...
3DS parallax barrier: https://www.youtube.com/watch?v=D-LzRT7Bvc0
Diagram from Wikipedia article: https://en.wikipedia.org/wiki/Parallax_barrier#/media/File:P...
A big disadvantage of parallax-barrier is that you end up blocking a lot of the screen, making the image darker.
Supposedly you can combine the two approaches to reduce the darkness.
https://www.youtube.com/watch?v=yetX2C3AdCI
There are probably even more tricks involved, and this is all a lot easier-said-than-done.
Here's a project that uses an LCD to make a dynamic parallax-barrier, that increases brightness by skipping the barriers wherever they don't matter:
https://www.youtube.com/watch?v=YrRnURU1W3E
Tangent: There's also been some attempts to do _the reverse_, and use distorted lenses in front of a camera to recover a 3D scene using only a single camera:
https://dl.acm.org/doi/10.1145/3072959.3073589 (link has a SIGGRAPH talk recorded which shows example results)
breakingcups | 5 years ago
It's interesting to see the compression artifacts affecting hair specifically.
I guess we're going to get used to different kinds of compression artifacts in the coming years because we're switching to spatial information being transmitted as opposed to just pixels. Hair is so much harder to get right than a face.
d3ntb3ev1l | 5 years ago
The ads will be amazing
billiam | 5 years ago
I am sure it is stunning and beyond anything most people have ever experienced, but will it be worth $$ to have a beautifully lit version of visiting someone in prison? It is still not human contact, I don't think we can fool the brain into not missing that.
jonplackett | 5 years ago
I wonder what distance they tried this over - there's a hard limit on speed-of-light for latency between continents, plus whatever encoding/decoding has to happen, so seems like we're always going to have that slightly awkward taking-over-each-other issue?
dontwannabe | 5 years ago
How is this different from a high resolution screen?
presentation | 5 years ago
Pretty sure it makes a 3d model of the scene and then if you move around the perspective changes as well, since the screen can show different images based on the angle you look at it. you can see a bit of this in the video when the camera moves around.
dontwannabe | 5 years ago
Does it track eyes to change the perspective? Since the image is in 2d, i can not think of any other way?
homedepotdave | 5 years ago
The porn industry is about to be reinvented
llsf | 5 years ago
Yup, porn is definitely adopt it as soon as it makes economical sense.
Especially, if this is pre-recorded and only sent for viewing (so no need too much bandwidth on upload side). I guess it would come down to the hardware and how much it costs.
oars | 5 years ago
This is a game changer. Much better than VR/AR.
megous | 5 years ago
More like cam industry...
dzhiurgis | 5 years ago
Indeed. Business needs screen sharing, screen annotation and audio (Slack covers 99% of this). Home users are relatively happy with what they have already (although no 4K streaming yet and cameras in laptops are kinda tragic). That leaves porn.
LightG | 5 years ago
When can I try this on my phone?
Imagine gaming on this.
I assume the porn industry will be early adopters (sorry, it's probably true).
Jach | 5 years ago
Inferior to VR for gaming or pron.
craigc | 5 years ago
The telescreen received and transmitted simultaneously. Any sound that Winston made, above the level of a very low whisper, would be picked up by it, moreover, so long as he remained within the field of vision which the metal plaque commanded, he could be seen as well as heard. There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live -- did live, from habit that became instinct -- in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized.
varispeed | 5 years ago
It seems like this could be used in the future dystopian economy. Workers sleeping in little capsules next to a screen where they can choose to see their partner in another factory or AI driven escort. On a serious note likely everyone is going to have at least one at home at some point to use for work meetings, pairing or for home inspections by the government.
georgewfraser | 5 years ago
I have always wanted to try setting up a low-tech version of this, where instead of using a light-field display, you assume that you have only one viewer and you simply render the correct point of view on the screen. This is the same technique used to shoot The Mandalorian [1], but in this case we're only trying to make a "pane of glass" so an ordinary high-res TV will do. With both participants positioned correctly, you will get correct accommodation, vergence, and parallax depth cues---only stereopsis is missing, and stereopsis is much less important than people think. 1/3 of people don't even have good stereo vision! [2]
As long as you're willing to give up stereopsis, and you have only 1 viewer on each side, I think you could accomplish this type of immersive telepresence with an ordinary TV + software.
[1] https://www.youtube.com/watch?v=Ufp8weYYDE8 [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4934608/
aerovistae | 5 years ago
This reminds me of the devices described in the short story "The Story of Your Life", which is the basis of the movie Arrival. Excellent, excellent story and very different from the movie, if anyone's into that kind of thing.
Jommi | 5 years ago
Literally some of the best modern short stories there is.
fellowniusmonk | 5 years ago
I definitely got vibes of the "viewing" tech from Asimov's "The Naked Sun", always exciting to see sci-fi tech reach the real world.
karlkloss | 5 years ago
Came here to say that. Asimov was one of the few SF authors that not only imagined advanced technology, but also could see what it would do to society.
tomduncalf | 5 years ago
Seconded, I’ve really enjoyed all of Ted Chiang’s books and wish he was more prolific as I’ve read them all! An excellent writer and I think his style would appeal to the HN crowd as it’s kind of sci-fi, kind of based in science, and many of his stories are quite thought provoking and unique. Also they’re all short stories so great if you struggle finding time to take on a whole book or whatever!
paxys | 5 years ago
I recently read Exhalation, and was blown away by how engaging and thought provoking the stories were. Would highly recommend it to anyone with even a passing, high-level interest in sci-fi.
sangnoir | 5 years ago
To those who want to read a sample of the anthology, the complete titular short story ("Exhalation") can be read online for free: https://www.lightspeedmagazine.com/fiction/exhalation/
It is 6552 words long.
[Deleted] | 5 years ago
tomduncalf | 5 years ago
Same here. I’m quite picky when it comes to sci-fi, a lot of it I don’t really like, but he nails a really unique sweet spot where sci-fi is just the framing device used for some beautiful stories. Might reread his books next actually!
Edit: I’d love any recommendations of other authors people think of when discussing his books. Doesn’t necessarily have to be the same style but more just that level of quality and uniqueness.
Jommi | 5 years ago
I'd love to do a Clubhouse room just chatting around his stories.
shriphani | 5 years ago
The first story in Exhalation is written in the style of the Arabian Nights (Thousand and One Nights). Get the unabridged, original translation by Richard Burton - it is some of the most beautiful literature ever written - comes in 16 volumes so there will be no shortage of reading material!
tomduncalf | 5 years ago
Awesome! Thanks very much.
shriphani | 5 years ago
And of course I feel I have to mention Richard Burton himself who led possibly the most interesting life in his time: https://en.wikipedia.org/wiki/Richard_Francis_Burton
sizzle | 5 years ago
So are they going to use NLP to parse your conversions and build a knowledge graph of all your topics of interest to later run targeted advertising that is shockingly accurate and manipulates you to waste money purchasing random products you really didn’t need to waste money on right now?
sizzle | 5 years ago
Conversations*
CyanBird | 5 years ago
They already do that through Google analytics and they already have dark profiles on basically anyone on earth that has touched their products, let alone making accounts
So yeah...
allenu | 5 years ago
Really cool execution.
I wonder what the lag is like. I can imagine that's one thing that would break the illusion. I know with something like Zoom I've gotten used to managing the lag over time by taking turns with the other person. However, with the "live" feel of this, there could be an uncanny valley effect if the lag is subtle, but perceptible.
Another thought: this is being presented as ongoing research. I wonder what the corporate thinking is in presenting it now when it's still being tried out. Does Google want capitalize on remote meetings while it's still hot? If the pandemic wanes and we have more in-person meetings, this might not make as big a splash. I remember when I worked at Microsoft, I often noticed research announcements we'd make in public often wouldn't translate to actual product, so I got a bit jaded on any cool new thing that was announced without a product timeline.
xnx | 5 years ago
It's advantageous to announce when you have a good working prototype so that when a competitor (e.g. Apple) announces something similar, the world is less impressed/amazed.
MetalGuru | 5 years ago
Advantageous even if you don't have a working prototype. See Microsoft vaporware
sneak | 5 years ago
Zoom lag is an issue with Zoom, not the network generally. If you actually do wired, p2p on the same side of the same continent you can get rid of most of the lag. Current lag comes from services that aren't p2p and bad networks (e.g. wifi).
Pxtl | 5 years ago
Yeah, I honestly hate how much Zoom has won because I've found it's the worst for latency. I have a mumble server running on a pi that blows the doors off Zoom for audio-quality and latency but it's unusable for casual groups because nobody wants to wear a headset and feedback destroys it.
sneak | 5 years ago
> Yeah, I honestly hate how much Zoom has won
Zoom has won in the sense that MySpace had won, or perhaps in the sense that Facebook has won.
This too shall pass.
Trollmann | 5 years ago
Mumble beats nearly all service‘s in latency and audio quality because these two things are the problems it‘s trying to solve. Did you ever play around with Teamspeak? It‘s a bit worse on the latency side but more mainstream friendly. Personally I run a mumble server as well because it solves our problems but TS may be worth looking into for you.
jayd16 | 5 years ago
Google has plenty of dark fiber. I'm sure they could get something close to land line levels of lag for an office to office connection.
skybrian | 5 years ago
The longer the project exists and the more people there are who get to try it, the more likely it is to leak.
To speculate, here are some reasons why keeping it secret longer might be hard:
Apple is better at keeping secrets, and even they leak.Also, it's nice for the people working on it when they no longer have to keep what they do a secret.
Edit: although, in this case, the timing mostly has to do with Google I/O starting today.
dbbk | 5 years ago
If they wanted to capitalise on remote meetings, I assume they would have shown one in the demo video. Instead they focused on family members reconnecting.
twobitshifter | 5 years ago
With the amount of data needed, one on one is probably easiest. We saw teams and zoom struggle to support more people in a call last year. It’s also nice to have them be true to size. Mini people might be somewhat uncomfortable.
apinstein | 5 years ago
If you want to get a sense of what this "feels" like and you have 6DoF VR, try a VR 180 video. I've not experienced the Google Starline project, but I can tell you that when I saw some of the really well-produced VR180 videos it was so realistic I felt I was invading someone's privacy.
Of course this won't be the interactive feeling but it was pretty mind blowing to "feel" how intimate real 3d telepresence could be.
They have some demo's here:
https://arvr.google.com/vr180/
russdpale | 5 years ago
This is really cool, and I'm sure they didn't mean it, but it looks like they are in and ultra low security prison during visitations .
qmmmur | 5 years ago
I feel like video calling is not great because you lose that depth which is natural to human communication. That said, I work in field focused on digital audio and sound and so all my colleagues have professional microphones and headphones which makes up 90% of that difference immediately.
raxxorrax | 5 years ago
In 2059 we probably will invent technology that allows you to talk to people without even having to see them.
Jokes aside, it is cool tech, but I fail to see applications. With my developer colleagues we mostly share a desktop instead of seeing other peoples faces.
Don't know how it is technically solved to simulate depth, but I image it being no different than conventional screens. The difficulty is probably doing a real time stereoscopy of the object displayed.
Would be cool to know if they only use AI supported imaging for that, or if they have some sensors, maybe invisible laser projections or stuff like that. There are probably restrictions how the cameras must be angled too, so a self-made home setup would be difficult to calibrate.
czechdeveloper | 5 years ago
This is for family and friend meetings, not for work calls.
This is all made to evoke sense of closeness.
encryptluks2 | 5 years ago
While cool for the top 5% wealthiest people, I'd be more interested in tech for the other 95% that will actually be affordable.
weird-eye-issue | 5 years ago
Such an unnecessarily pessimistic comment. How do you think tech becomes affordable? It has to start somewhere and then we go from there...
OminousWeapons | 5 years ago
New tech typically debuts at a high price point only wealthy people can afford, then as it get commoditized it becomes more affordable. Wealthy early adopters willing to pay high prices for novelty or business applications are what enable the fast pace of innovation that we have become accustomed to.
microtherion | 5 years ago
Think of it as "The future is already here — it's just not very evenly distributed."
Quite a bit of the technology used there seems destined to get more affordable as it's getting more widely adopted.
noir_lord | 5 years ago
William Gibson quote I think.
And yeah, insane prices at the start funds development for everyone else, I remember when the first 4K screens came out and they where exotic, now they are just normal.
Same thing happened with phones and hell computers, I was the first kid I knew with a computer back in the 80s and we where not wealthy, that think cost more than my dad made in a week, an actual IBM PC was unattainable til I was I was 10.
Now I have hilariously more powerful single board computers shoved in a drawer because I can’t find the time to do anything with them.
dpratt71 | 5 years ago
What is an example of important technology that did not start as something only available to the affluent/connected?
bsanr2 | 5 years ago
Penicillin. But then, he wasn't trying to make money off of it.
And basically anything Nintendo pushes as a console gimmick. It's not that the tech immediately goes from research to broadly accessible, but rather that they tend to take old tech that no one saw as having profitable consumer applications and find one for it. In that way, as far as consumers are concerned, it goes from unknown to widely-used without making a stopover in early-adapter purgatory.
dpratt71 | 5 years ago
Penicillin, maybe? But I wonder how quickly it became readily available outside of the Western world.
And I see that Nintendo has apparently sold an extremely impressive number of consoles (https://www.nintendolife.com/news/2019/11/nintendo_has_now_s...). But even if everyone only bought one console each, that's only about 10% of world population.
This may be a little unfair, but I do wonder if there isn't a tendency to consider a technology to be widely available when it becomes available to you and the folks farther back in the line don't count or aren't relevant.
bsanr2 | 5 years ago
To say that a single company's products being used by even a single digit percentage of the world population doesn't meet the requirements to ve considered "widely available" is a stretch.
In any case, you said "important," not "widely available," and yes, Nintendo's products are hugely important. Many of today's technological advancements can be traced back to their proving that a given use case for a primitive version of a given technology was viable.
dpratt71 | 5 years ago
Whether it's a single company (or product) or multiple is beside the point. If a technology is only available to (say) 1% of the population, I don't think that qualifies as being widely available.
I will also note that my original comment was in response to someone who is "more interested in tech for the other 95%".
encryptluks2 | 5 years ago
Is this technology important though?
varispeed | 5 years ago
A wheel
wizzwizz4 | 5 years ago
Minecraft. Linux. (Really want to write GNU, but that's not true.)
ALittleLight | 5 years ago
I think Minecraft and Linux are more like the content produced on top of the technology that is computers. It's like if a new book is written it's quickly available to everyone in the market who can afford a book. The book isn't really technology, but the printing, publishing, and distribution is and it's been around long enough to be distributed.
Software seems less like technology and more like writing. The distribution cost, once the systems are in place, is marginal. The technology part is creating the systems that enable the software.
munk-a | 5 years ago
IBM in the 90's strongly agrees with you - this software stuff is never going to be profitable and everything people pay for will always end up going through us!
More seriously, I disagree about software being less important because there have been very real innovations for tooling accomplished in software alone. Email is a pretty classic example - but a more modern one might be Google Cardboard which can turn your smart phone into a rather underwhelming VR headset. There are plenty of hardware alternatives but the same basic functionality was accomplished on generalized hardware.
Additionally, all this technology is only really possible due to other technology - we don't discount a new shiny computer just because it's just a dumb oddly shaped box if you can't supply it with electricity - but the costs to develop software are generally lower than hardware so I think it's fair to have a general notion that hardware is more innovative - it's just that you're conflating two different variables - cost and medium.
ALittleLight | 5 years ago
I think you're conflating technology with profitable or important. That is, you see me saying that software isn't technology and think I'm saying that software isn't profitable or important. That's not at all what I'm saying though. I likened software to writing. Writing can be important and it can be profitable, it's just not technology.
Maybe we could agree on email as a technology. Maybe. I think it's a stretch. I hope we could both agree that the nth email client isn't technology though. It's not adding a new capability to humanity which is how I tend to think about technology. Refrigerator - keep stuff cold. Electricity - power to operate machines and light. Computers - organize, access, modify information. etc. New JavaScript library or new game... Not so much technology.
munk-a | 5 years ago
I think that's fair yea - it might just be a matter of semantics. If you think software is included in technology then I stand by my point but, if your view of technology excludes software then you're quite correct.
dpratt71 | 5 years ago
Is it not the case that having access to a machine capable of running either Minecraft or Linux in the early days of each (if not now) means you are (or were) fairly affluent?
munk-a | 5 years ago
It depends on what you mean by the early days of each - in the really early days of Linux (1992) computers were probably going to cost north of a few thousand dollars and the type of computer Linux was designed for (a multi-user system for dumb terminals) would cost tens of thousands of dollars. By the time linux became a thing more than a handful of people in any given state knew about you could probably run it on a machine costing somewhere around 300$.
Minecraft has never been demanding resource wise, I'm sure early versions had serious performance issues but running it on a cheapo laptop has always been totally reasonable - it's quite accessible (it was written in java even!)
robertlagrant | 5 years ago
Minecraft wasn't an important technology. It's a game built with important technologies.
wizzwizz4 | 5 years ago
Tell that to the mathematicians.
munk-a | 5 years ago
I disagree - I would agree that Minecraft wasn't a novel technology, just like Linux wasn't a novel technology - it was an alternative version of Minix.
Additionally Zoom isn't a novel technology, it isn't even particularly interesting technically when compared to other video conferencing solutions - but over the past year it's been incredibly important to a number of people.
I think the OC slightly missed the mark in mentioning "important technologies" instead of something closer to "technologically innovative" technologies or, more accurately (but less interesting of a statement) "expensive to develop technologies". Things that are expensive to develop generally aren't cheap to begin with, while things that are cheap to develop need to be cheap to compete with other market entrants and clones. Additionally hardware (a limiting factor on cost for a lot of technology) tends to get cheaper over time and that rate of change is accelerated by a large market of interest (leading to more folks deciding to try and iterate new designs).
TulliusCicero | 5 years ago
I think they meant hardware. For software, yes there are tons of examples (open source software is usually free, games are usually cheap).
But for hardware, it's almost always some expensive thing first. The internet was once very expensive to access, cell phones were initially very expensive, DVD players were initially very expensive, computers in general, etc.
TameAntelope | 5 years ago
Install these in neighborhood libraries, or as phone booths!
notyourday | 5 years ago
Nah, if Google actually went for it for years it would have went down the wealth requirement. The problem is that Google has demonstrated time and time again that it cannot do reiteration grind.
bellyfullofbac | 5 years ago
I was travelling a few years ago. Even Russian bus ladies (they collect your fare) and Mongolians living in traditional huts in the middle of nowhere (and without any signal!) have smartphones now. I found the idea funny that the "To plug in a USB cable, you need 3 tries" experience was maybe universal.
So, 12 years (back then) after the iPhone, it's reached a lot more than 5% of the world.
frakkingcylons | 5 years ago
I would pay a lot of money if this was commercially available. Very impressive
Cthulhu_ | 5 years ago
Things like this make me feel more and more disassociated with science fiction TV shows / films. I'm sure they intentionally don't make communications too good, because then - with tech like this - people would go "lame, that's just the other actor in a box behind a glass pane", instead of "oo that is cool sci fi technology".
I mean we used to marvel at things like Star Trek, but nowadays a smartphone is miles ahead of a lot of "day to day" things they showed in there. Foldable screens are coming too, and now this.
I mean I don't believe this thing will be commonplace at all in the near future, but it's still cool. I think it'll be integrated into smartphones within the decade though. It's already mostly possible with the front-facing camera + light field of iphones, plus AR, plus motion sensor technology, plus the load of processing power they put in there.
meowface | 5 years ago
Makes you wonder in what ways our contemporary sci-fi is going to look totally off-base in 50 years.
abhv | 5 years ago
A good friend of mine at Google is the technical lead of this project (he has a Phd Princeton, and was a professor before joining Google).
I've tried it in person and it was truly amazing. They used some very fancy tech when I saw the demo, so I'm thrilled it is finally being announced and possibly shared with a larger audience.
Explanation of why it is amazing: It totally fools your perception. No glasses or goggles--but rather an 8k display with special glass that allows your different eyes to see different pixels (a light field).
They also optimize the sound, and the rest, so as all the testimonials point out, you actually feel like the person is in front of you.
It also works for the "cube" around them, so if they hold up some object, it also feels like that is in front of you.
Amazing...
estaseuropano | 5 years ago
Just very practically, is this a $xxxx.xx improvement over simple video chat with a large screen and simple camera? Will this even work without low-latency high-bandwidth connection up and down?
It seems more like an art project than a tech usable in the coming decade.
patall | 5 years ago
Interesting. Does the light field work only for one person or multiple (they show mother and baby in the video)?
sfifs | 5 years ago
So in my company, I've used various avatar's of Cisco TelePresence systems over many years.
The big commercial need it turns out isn't so much realism as it is flexibility to accommodate people dialing in from different systems - phones, laptops, different types of telepresence setups from small single room to big conference rooms or even telephone connections etc.
Many years ago, we were all wowed by the life-size realism and had people come into offices for it . Nowadays these meetings have lots of people crammed in across multiple screens dialing in as they please and all the better for everyone :-)
rejectedandsad | 5 years ago
That's very cool, but out of curiosity
> project (he has a Phd Princeton
Why is this part relevant? Was the PhD in light field technology?
1vuio0pswjnm7 | 5 years ago
Questions:
How does this relate to advertising and the necessary surveillance to support it.
Will this "product" be set up to phone home and "update" by default. Will the price be "free".
Commercial viability. How much would someone pay for this.
crazygringo | 5 years ago
Sheesh. Why the FUD?
It would presumably work exactly the same way as Google's existing videoconferencing hardware:
https://workspace.google.com/products/meet-hardware/
In other words, you pay market value for it, it doesn't include advertising, and Google is contractually obligated not to snoop on your content.
Which is why even Google competitors use this hardware, because the legal and technological protections are strong enough they don't worry about Google stealing their IP.
draw_down | 5 years ago
It does seem really cool but I don't see any reason to think they are sharing it with a larger audience.
imiric | 5 years ago
It looks like they're doing photogrammetry in real time, which is mind boggling. I'm not familiar with this space, but building a 3D model, texturing it with live video, compressing and sending that over the internet, and doing it with minimal latency for it to be believable/enjoyable? Incredible technical achievement if that's the approach. Using state of the art tech, no doubt, and probably lots of ML magic to smooth the rendering. The 3D display is the icing on the cake, it must look amazing in person.
shahar2k | 5 years ago
not that hard to do if you have actual depth sensing cameras, and even without those, something like the oculus quest 2 does that exact task (generate a rough 3d volume based on several 2d video feeds) you can see a neat example when you draw your guardian space, and move objects (and notice how it updates the 3d volume representation)
Aeolun | 5 years ago
Using Oculus Quest 2 had me just walking around my room in wonder about seeing straight through the headset for a bit.
crazygringo | 5 years ago
That's not true -- even with depth sensing cameras, it will still be full of artifacts, and things like curly hair or strands of hair will become disastrous because they're not easily geometrically modeled.
The Oculus Quest 2 doesn't do anything like what you're describing -- it essentially just pipes in stereoscopic video from its stereo cameras and stitches them together in a trivial way. It doesn't attempt to build geometric representations of objects in your environment at all.
(For guardian functionality it does very simple things like using the depth cloud to figure out the height of the floor, and if there are points inside the guardian that shouldn't be there, but that doesn't inferring object geometries.)
supermatt | 5 years ago
if you look at the video you can see there are artifacts around the hair. It is likely applying some matting via AI to make it less obvious, but it is still there.
panarky | 5 years ago
> not that hard to do
Then it would be done already.
imiric | 5 years ago
Good point, I haven't followed the latest VR advancements, that does sound neat. Still, Starline's approach is surely much more sophisticated (the hardware obviously has a lot to do with that, these are prototypes of a desk-sized machine vs a headset). The 3D model looks reasonably detailed, and the final render has very few artifacts. Making it all work over a WAN link with latencies critical for teleconferencing is also impressive.
rewq4321 | 5 years ago
The difficulty completely depends on the level of quality you're after. They're certainly working on cutting-edge-level quality, so it is likely no easy task. Someone else pointed out that they released a paper on related tech last year:
https://augmentedperception.github.io/deepviewvideo/
jlebar | 5 years ago
My mind was also blown when I got to demo this a few years ago.
You sit down and you forget that there's technology happening. The person is there, in front of you. I don't know how else to describe it.
The testimonials in the video aren't exaggerating compared to my experience.
anon_tor_12345 | 5 years ago
i got curious (since you said you'd seen this years ago) and found my way to your cppcon 2016 talk. good talk but in particular i want to congratulate you on the weaving pun (first multithreaded tech).
rewq4321 | 5 years ago
Do you know if it supports multiple people on "screen" at once? Or does it rely on eye tracking of a single person (plus projection of some sort?) to be able to achieve the 3D effect?
ehsankia | 5 years ago
I don't know anything more than what's been released, but from my understanding, light field display don't rely on eye tracking, and you can have multiple people looking from different angles (although the range may be limited to near the center). In the video itself, the camera filming the interaction is moving independently of the person sitting, and unless it was faked, it was able to see its own perspective.
nexuist | 5 years ago
It seems like it does, given in the demo video there's a lady and a baby in the same shot.
ForHackernews | 5 years ago
We don't know if the baby is seeing the full 3d effect, but my understanding of how light-field displays work is that it isn't based on eye-tracking.
lultimouomo | 5 years ago
I suspect this worked because the person was, indeed, there.
I mean, not in the same room, but down the hall or so.
You can have the most perfect rendering in the world and 100ms of latency will be enough to make the experience miserable.
closeparen | 5 years ago
You could imagine commercial/institutional sites hundreds to thousands of miles apart with <10ms latency on dedicated fiber.
tomklein | 5 years ago
Wouldn’t that be way faster than the speed of light?
gearhart | 5 years ago
Depends how many thousands ;) light travels about 3000km in 10ms
https://www.google.com/search?q=speed+of+light+*+10ms
soVeryTired | 5 years ago
That’s in a vacuum, in a straight line. In fibre optic cable, light travels around two thirds of that speed.
akie | 5 years ago
Depends on how much bandwidth it needs. I've been video conferencing a lot (who didn't, this past year) and even with people on the other side of the globe I don't recall latency ever being a problem.
dash2 | 5 years ago
Also depends what you're doing. Singing together is impossible over standard video conferencing, for example.
everyone | 5 years ago
yeah, like if your playing keys live for example, you want your audio interface to have around 6ms latency max (the delay time between your finger pressing a key and you hearing the sound from pressing that key) above that and it starts to become detrimental to your playing. above 10 and you can barely play.
nipponese | 5 years ago
Isn't it just the "Pepper's Ghost" technique?
https://en.wikipedia.org/wiki/Pepper%27s_ghost
nimchimpsky | 5 years ago
huh ? how is it related to that in any way ?
rewq4321 | 5 years ago
No, this sends a different image to each eye. It's not actually a transparent piece of glass with an image on it[0] - I think they just try to make it seem as if it is a transparent pane you're looking through.
The marketing for it is pretty terrible though, because I initially came away with the same impression as you.
[0] See image here: https://www.engadget.com/google-project-starline-191228699.h...
soylentgraham | 5 years ago
Is it lenticular, or something else?
jayd16 | 5 years ago
Does it have perceptible regions where the view angles are ideal?
If it's as seamless as it looks in the video that would be truly novel and exciting.
amelius | 5 years ago
Was this display developed by Google, or did they buy it from elsewhere? Is it still on the market?
shard | 5 years ago
I recall there being glasses-less 3D displays about 10 years ago, when the TV industry was trying to make 3D displays the next new thing. I wonder if this is the same technology.
bsanr2 | 5 years ago
IIRC those were displays that used sterescopy, while these simulate a light field instead.
bullfightonmars | 5 years ago
If this was a consumer product I would buy two of these today, one for myself, and one for my parents. It is more compelling that any product I have seen for a long time.
codeecan | 5 years ago
https://electronics.sony.com/spatial-reality-display/p/elfsr...
chewxy | 5 years ago
This is a 3D monitor. Not a light field display like Starline.
The traditional glasses free 3d monitors rely on special coatings on the glass to "split" the direction of a pixel. Some coatings are electronically controlled (like Sony's), some are physical (lenticular filters) It still displays a 2D projection of a 3D image, but twice.
By contrast, my understanding is lightfield displays projects the entire volume. So you don't have a pixel, you have a voxel. So each eye gets a different bit of the same voxel
[Deleted] | 5 years ago
ipsum2 | 5 years ago
How much would you pay? I expect the lower end to be around $20,000 from hardware alone, plus at least gigabit down/up connectivity.
signal11 | 5 years ago
I suspect eventually this will percolate down, somewhat like Facebook's Portal TV (£149) but possibly also integrated with something like a Google Assistant/Siri type smart device to power the software and do other smart-home type things in your home.
Many ordinary folk already have 4k TVs at home and 8k will probably become commonplace in the future. The real bottleneck will be good low-latency broadband both up and down, but fibre to the home should make that easier. I wonder if ISPs in the future will offer QoS guarantees to enable really good videoconferencing?
I mean, Zoom, Meet et al are much better than video conferencing solutions like Webex even from a few years ago, but it's hilarious how much drama there still is around video calls. "oh, sorry, I didn't realise I had muted myself", "can you hear me...?", "I think we just lost Steve", and so on. I'll be glad to see all of that just go away.
eps | 5 years ago
Facebook Telescreen™
kbenson | 5 years ago
To be followed up by Facebook Telepresence™ and then Facebook Omnipresence™? ;)
dane-pgp | 5 years ago
Mark Zuckerberg might announce that, but I think he'll go straight from Facebook Telescreen™ to Facebook Panopticon™.
true_religion | 5 years ago
Max I'd pay would be $6,000 because I'd need to buy two: one for me, and one for my parents. If it were $3000, I'd have already bought it.
$20,000 is a bit out of the reach of most people, and reserves this for business use or desperate need.
I could see a bunch of execs getting this installed in their home offices as a company perk, and then using it for personal reasons too.
ghaff | 5 years ago
A lot of execs don’t have optimal setups with current tech. A lot of people just don’t care.
benhurmarcel | 5 years ago
> reserves this for business use
Even for business use a price with 5 digits would make it confined to a few executive offices.
fnord77 | 5 years ago
probably built into smartphones in 10 years.
cocoggu | 5 years ago
I think one of the major aspects of this technology is that the person you are talking with appears in its real size, with the same ratio than in real life. Something you cannot do with a smartphone, except if you are referring to a projector.
bullfightonmars | 5 years ago
Well that wouldn't be a consumer product then. But I would pay as much as a high end tv. 2500? Maybe more? The experience looks to be transformative.
throwaway3699 | 5 years ago
I doubt you need gigabit for an 8K stream. The real issue is the abysmal common network infrastructure. Latency will be a bottleneck if most ISPs don't care about treating your packets properly.
perttir | 5 years ago
Most internet providers here in Finland provides you fiber connections from 100-1000mbs pretty cheap. And 4g/5g connections from 100-300mbs for cheap prices too. Most of them does not have any data caps.
I have 300/100mbs connection which costs 20 euros in month.
cogman10 | 5 years ago
Agreed, 10Mbps is likely where it'll be, probably less. If I were to guess, they are going to apply a lot of smarts to texture mapping to avoid needing a lot of bandwidth.
Aeolun | 5 years ago
That seems unlikely. My game stream in 1080p at 60fps already takes 40Mbps. So 4K at 30fps would need more like 80Mbps, and 8K 320Mbps
fomine3 | 5 years ago
It heavily depends on what encoder and config you use. In my experience, 20Mbps (or less) HEVC by Turing NVEnc realtime encoder is enough for 1080 60p. Also halving frame rate won't halve bitrate because of how video compressed by reference frame. Also video meeting won't move pictures as much as FPS gaming.
IshKebab | 5 years ago
You can't even do basic video calling on a 10 Mbps connection. You definitely need the reliability and low latency of fibre for this.
[Deleted] | 5 years ago
kbenson | 5 years ago
Doubtful, IMO. The fact that you need it to be both extremely low latency and with essentially zero compression artifacts (probably lossless, based on their goal). If the numbers listed here[1] are correct, then the most efficient lossy codec at that time was doing 100 minutes of 8k video in ~37GB of data. From that we can intuit that it was using an average of 50 Mbps for that 100 minutes of video. For the most efficient codec, and I'm not even sure from that whether it was using lossy or lossless numbers, because apparently HVEC can do both (but I would assume lossy, since it's about streaming video in that case).
You can't do weird texture mapping or lossy compression and expect people to really seem like they are there. Even if you don't notice that stuff normally watching a video, I think you'll notice it when you're interacting like someone is really in front of you, and that will throw off the immersion.
1: https://www.quora.com/In-regards-to-filesize-how-big-is-1-mi...
cogman10 | 5 years ago
HVEC is no longer king of the hill when it comes to compression efficiency. AV1 and the upcoming VVC do better.
That said, my intuition is they aren't doing a pure video encoding solution. The fact that they talk about 3d modeling leads me to believe they are doing a combination of model + texture to get the realistic results. That would significantly decrease the amount of bandwidth and computational power needed. Over a low bandwidth situation you'd simply need to send model updates and do some smart interpolation to determine what things should look like.
Similar to the concept that playing a 3d game requires MB of resources but recording the same game at 8k would require a boatload more memory.
My assumption is they are using LIDAR to get a good model, high quality cameras to texture things, and a nice AI to stitch things together and interpolate when data isn't arriving fast enough.
scoopertrooper | 5 years ago
There have been great strides made in ai compression in recent years. I wonder if they are going to leverage that.
https://blogs.nvidia.com/blog/2020/10/05/gan-video-conferenc...
https://ai.googleblog.com/2021/02/lyra-new-very-low-bitrate-...
https://aibots.my/officialBlog/deepminds-ai-agent-muzero-cou...
mensetmanusman | 5 years ago
I think this is what will be done in the future. You will interact with the camera for 15 minutes or so and it will create a custom compreasion algorithm for you.
paxys | 5 years ago
I imagine it isn't (yet) because of the price tag.
bostonvaulter2 | 5 years ago
And it would probably be very difficult to install at the moment as well.
supermatt | 5 years ago
Over a decade ago I used a holographic video conferencing system that used kinect for depth mapping, it was very convincing. It would also reorient the display based on your head position. so you could look "around" the avatar.
There are demos of binocular 3d conferencing done with a lenticular display (lookingglass), although those large displays are extortionately priced (1/4 of the size of this google one is $3000...) - keeping them out of the hands of most devs.
No doubt google are doing the same, but can afford these larger displays.
You can easily find examples (and research papers) by googling for the relevant terms. Google claiming they have invented a "new technology" just shits on all those folk who dont have the publicity/funding of google.
Pxtl | 5 years ago
I'm still not understanding how the 3D works... is it like the 3DS? Because that required you position your head in a very specific place.
Philip-J-Fry | 5 years ago
The "New 3DS" introduced head tracking and no longer needed your head in a fixed position by the way.
somebodythere | 5 years ago
Sounds super cool. I can't wait for LFD tech to become more accessible to consumers.
ortusdux | 5 years ago
I've been thinking about pre-ordering the Looking glass portrait. $250 for the unit, and I already have a leap motion laying around.
https://lookingglassfactory.com/product/portrait
bsanr2 | 5 years ago
It's neat and I'm waiting to see what creative things people come up with for it. A Japanese developer put together a Wizardry/Megaten-style dungeon-crawler demo:
https://twitter.com/mizzmayo/status/1394171128491487234
mywacaday | 5 years ago
I have wondered in the past if a similar result could be achieved using a 3d headset with some tracking/cameras and removing the headset from the view through a real-time deep fake that could be achieved through a short scan before the call. Would this even be feasible?
purplecats | 5 years ago
probably would have an uncanny valley effect
Valgrim | 5 years ago
It sounds like they are using this technology or something similar:
https://lookingglassfactory.com/8k
[Deleted] | 5 years ago
[Deleted] | 5 years ago
hathawsh | 5 years ago
Does anyone know whether they're using that tech, something similar, or something else? I've always been interested in light field technology.
tengbretson | 5 years ago
Wouldn't there also have to be some kind of head tracking involved? Otherwise you're still limited to just showing a fixed perspective in 3d.
wesleyy | 5 years ago
No, that's what the lightfield does. You see different physical images depending on your angle to the screen
defaultname | 5 years ago
Fascinating. So not only is it feeding it an 8K / 30 (60?) FPS image, it's feeding it numerous incident angle variations and displaying all of them simultaneously?
Sounds like a monster data rate.
zaptrem | 5 years ago
I only know what I saw from the IO stream, but I think it might send a compressed 3D mesh + texture across the network and render the light field locally.
dialogbox | 5 years ago
I think that is where the custom compression algorithm comes in. If you think the fact that human body and face doesn't change much, and the fact that it's a 3d model based, the compression ratio could be very high.
hiharryhere | 5 years ago
Good point. Also the very neutral background would contribute to that.
true_religion | 5 years ago
I bet they'd also just fix focus on the person and whatever they're holding, then blur out the background in most cases.
dkarras | 5 years ago
I think what they are transferring is not a video but 3d model and the skin texture applied on the model (all derived from the realtime video / depth recording on the other side). The receiving and then renders it as a 3d model on the screen.
eps | 5 years ago
Sounds like eye tracking could still be useful to not bother with images for angles that are 100% not visible at the moment.
ithkuil | 5 years ago
https://news.ycombinator.com/item?id=23316225
> the 8K is their Input Resolution. > That resolution is then divided into the 45 viewing directions:
datameta | 5 years ago
Is existing Looking Glass Factory tech the same though? Not so sure about that. Those displays are typically monitor-sized at the largest and not really aimed at displaying a live feed of a person. This looks to be a more seamless experience on a larger screen.
datameta | 5 years ago
Downvoted, with no response, for posing an open question. Shame on you, honestly.
[Deleted] | 5 years ago
[Deleted] | 5 years ago
weird-eye-issue | 5 years ago
Do you think it's possible 90% of this is due to the studio quality lighting, large high quality screen, good mic/speakers, and low latency network? It seems like those factors alone would get most of the way there and the 3D aspect is just a bonus. Obviously I haven't used it in person but this was just a thought since most people are used to video calls on their small phone/laptop with poor lighting, mics, etc
abhv | 5 years ago
NO WAY.
It is impossible for me to explain how/why it works so well.
shahar2k | 5 years ago
I imagine they are using lightfield type displays like the ones made at this company - https://lookingglassfactory.com/
ArtWomb | 5 years ago
It looks like it came out of the high-fidelity Immersive Light Field Video presented at SIGGRAPH 2020. Quite impressive that within a year it's now a consumer product
https://augmentedperception.github.io/deepviewvideo/
WebAssembly SIMD is coming to Chrome as well. 2D images and video that only consisted of RGB and Alpha channels may appear downright primitive to future generations as depth camera rigs gain distribution ;)
https://www.chromestatus.com/feature/6533147810332672
sneak | 5 years ago
I am sure that the immersion of the experience is higher. My question (and perhaps that of GP) is: is this greater immersion actually beneficial to communication?
I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical. I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
JonathanFly | 5 years ago
>I think this is cool tech, and valuable. I'm just not sure that it offers a communication benefit over well-lit, well-miced, wired, low latency, 8K videoconferencing.
>Maybe there's some 3D emotional perception face processing stuff that we have deep in our brains that can immensely benefit from this, but I'm skeptical.
>I think simply doing 4k or 8k low latency high quality videoconferencing might be a 90 or 95% solution without needing special cameras/displays.
From my experience, 4k or 8k doesn't matter. Sound quality actually matters most, really clear low latency audio alone will give you a surprisingly strong sense of presence.
Video quality is important but 1080p is enough, beyond that the lighting and latency matter more.
Equally important from my personal POV is video size - physical size. Take a cheap 65 inch TV, turn it vertically, and talk to someone on that. When your talking to someone that is actually life size the sense of presence is vastly improved, even at the exact same video quality. And TVs are so cheap this doesn't seem like much of a techical barrier.
If you just screen share from your cell phone to your 65 inch TV and video chat -- holding everything else equal for audio and video quality -- it's SO MUCH BETTER.
imiric | 5 years ago
I think you might be underestimating the value of viewing a 3D model on a no-glasses 3D display. This is one of the basic aspects of in-person communication we take for granted that current 2D technology can't replicate. You can move your head and actually see a different angle of the person in front of you. This can even be subtle, our brain will still pick up the effect, and it makes the experience beyond what we usually consider as "immersive".
Yes, having low latencies and high definition video is an important aspect of this, but the 3D part is no gimmick. Once the technology improves and gets affordable this is a game changer for how we communicate online. The step after that are holographic displays, and since we'd be used to 3D models and smart displays, it probably won't feel like such a big jump.
I'm _super_ excited about this project. Hopefully Google doesn't axe it. (:
birdman3131 | 5 years ago
Any benefit from the 3d seems like it would be vastly overshadowed by the massive artifacting in the hair though.
adamauckland | 5 years ago
I thought that as well watching the video, but have you used a PSVR?
I've got one and the first minute is always noticing how low res the eye screens are, then as soon as the game starts, I've forgotten and I'm _there_. The 3D part makes up for the low quality
throwaway3699 | 5 years ago
Being able to feel like another person in the room is enough for me to reconsider working from home. As of right now I strongly have a preference for in person, but I do acknowledge most people prefer commute and cost benefits over productivity.
The state of video conferencing today is a poor one and I'm very excited for something that can change the industry like this.
sneak | 5 years ago
I'm right there with you, and I use a 4k camera and a boom mic and headphones and wired ethernet to videoconference now: I have been regularly complaining about the low resolution and framerates of current videoconferencing systems (10-15fps, 720p, low bitrate - and that's the highest quality setting available!).
If Google wanted to make me believe they care about videoconferencing quality, they'd have a 4k 60fps option that auto-enables in Meet if it detects everyone on the call is on wired gigabit with a 4k camera.
ghaff | 5 years ago
And most people won't have wired gigabit and can't,
throwaway3699 | 5 years ago
They will eventually. Hopefully the ISP monopolies are broken again.
sneak | 5 years ago
A lot of residential areas in the US have gigabit options, in some cases symmetric. There are lots that have 1000mbps down/40mbps up cable.
Even 100mbps is sufficient for a 1-on-1 4k video call, as high-bitrate 4k is 30-40mbps. Most commercial office buildings in business districts have it available. Even Starlink (20mbps up) should be sufficient for 1-1 30fps 4k videoconferencing with a lower bitrate.
dboreham | 5 years ago
40mbit != gigabit
vbezhenar | 5 years ago
50 megabits is enough for most scenes in 4K Blu-ray. 10 megabits should be enough for 4K stream with good enough quality.
michaelbuckbee | 5 years ago
I've spent a decent amount of time and money to make this happen and it helps less than you would hope.
Partially there are just affordance issues of things like eye contact which are physically out of alignment unless you start using two way mirrors [1].
https://hackaday.com/2020/05/29/two-way-mirror-improves-vide...
peterthehacker | 5 years ago
Essentially eye-contact is the missing ingredient. I believe it! Eye contact is key to a conversation feeling authentic.
isaacimagine | 5 years ago
There are technologies that can automatically adjust videos for eye contact today, so I imagine something similar could be implemented for this later on.
porcc | 5 years ago
Eye tracking is the core feature here--the rendered "hologram" is correct from every possible angle. The things you mention are probably closer to 2% of the final result.
plokiju | 5 years ago
Source on the eye tracking? The light field displays I've seen (https://lookingglassfactory.com/) don't need eye tracking to work
akersten | 5 years ago
My intuition is that a great lighting+microphone+speaker setup is necessary, but not sufficient, for this demo.
Even from viewing the short demo, the stereo display alone is an entirely new dimension that no amount of studio lighting will recreate. While better lighting and audio setup would certainly improve the average person's videoconferencing experience, this looks to be a genuine step beyond.
That said, we've been seeing holographic-display prototypes for the better part of a decade, and it'll be interesting if this actually pans out or fizzles.
unethical_ban | 5 years ago
What I think you're trying to describe already exists as a product from Cisco called "telepresence". It is/was insanely expensive, was a permanent installation that only Cisco contracted techs could install, and did what you describe: It is a series of large, curved HD displays with desks at an appropriate distance from the screens/cameras, and copious amounts of indirect lighting from behind the setup to make each party look good.
It seems like the imaging/rendering technology that Google is using is much more advanced.
ElliotH | 5 years ago
I’ve used such a Cisco system. Compared to regular video calls the latency and quality was light years ahead, much more natural conversations were possible. By which I mean it was possible to laugh, interject, and generally have a realistic conversation with a colleague in another country without having to compensate for video lag in that very careful way I find necessary on Meet and Zoom.
That said, there was no “emotional connection” like the Google one is described as offering. It was still a video call. There was no forgetting that. I suspect the 3D and the apparent physical closeness to the display add a lot.
noveltyaccount | 5 years ago
Wow I forgot about Telepresence. I used it a decade ago at a Fortune 500 company. With all of the cameras and displays perfectly positions, everyone was life-sized on video, felt like you were sitting around a roundtable. Now I'm imagining that with higher resolution and 3D light field display, wow.
wpietri | 5 years ago
The question for me is how much it matters after the novelty wears off.
I count at least 5 waves of 3D technology starting in 1851 with the Brewster Stereoscope. Each time there's a surge of popularity driven by the legitimately amazing initial experience. And each time people slowly stop caring. People were incredibly excited about Avatar, and many thought it would change the movie industry. But how many people now go out of the way to see something in 3D?
swsieber | 5 years ago
I do. Ha ha. I really loved Dr. Strange in 3D. I'm sad 3D movies aren't really a thing anymore.
nickkell | 5 years ago
I honestly think it's amazing and I'm sure the novelty would wear off, but it would still be useful. One of the things that stood in the way of 3D viewing was dedicated hardware, which is the same issue here. Although the need to wear glasses is gone, you still need to buy some serious equipment for it. Maybe at some point this will be bundled into a normal TV setup and people will just take it for granted that it's there
suyash | 5 years ago
Exactly, it's just a fancy FaceTime technology, I would be bored after few days. Tell me a problem that it solves.
taejavu | 5 years ago
Eye contact
golfer | 5 years ago
Tell me a problem Instagram solves. It's just a fancy Myspace.
Tell me a problem WhatsApp solves. It's just a fancy SMS.
Tell me a problem X solves. It's just a fancy Y.
Etc etc
wpietri | 5 years ago
If you talk to daily Instagram users, you'll find out what problem it solves. In particular, it was the lucky winner in the crowd of early photo-sharing-on-mobile apps. But the demand for that was proven by early photo sharing successes like Flickr. Realizing people would want to do that on their pocket camera device is not a big leap given what people were already doing to share photos from their mid-aughts cameraphones phones.
There's no such plausible story for 3D video calls. It's not like people are already demanding 3D displays for any of their other 3D stuff. The 3D first-person shooter, for example, has been around for decades. But 3D displays have never been popular despite being available for at least a decade.
varjag | 5 years ago
I (and probably everyone in immigrant communities) will get such a thing to my family and our parents as soon as we can afford it.
ghaff | 5 years ago
>Tell me a problem WhatsApp solves. It's just a fancy SMS.
I gather it mostly solves that SMS is expensive in a lot of contexts. Personally I never use it because most of the people I text with have US phones. And the one person who doesn't, we use Facebook.
yjftsjthsd-h | 5 years ago
Yes? Other than not being hip, how is Instagram better than Myspace? WhatsApp at least added features over SMS, although it sacrificed interoperability and went all-in on a closed system to get there. Newer is not always better.
imiric | 5 years ago
> Tell me a problem that it solves.
Replicating the experience of in-person communication much more closely than video and 2D displays will ever do. That's a noble research goal if nothing else, I don't get the skepticism.
There are several reasons 3D content and previous generation displays didn't take off, but there's no reason to believe a revolutionary new approach and product couldn't change this (e.g. electric cars were invented in the 19th century and are only now becoming popular). AFAICT the real time photogrammetry they're using here along with the no-glasses 3D display is a major leap forward. If they can get it cheap and reliable enough to mass market, it would be a game changer.
I certainly know what kind of display and teleconferencing software I want when the next pandemic hits, and it's not what we have now.
serf | 5 years ago
> (e.g. electric cars were invented in the 19th century and are only now becoming popular)
I hate this example, and it's like one of the most common ones on HN.
As said by thousands of people and many documentaries before me , the electric car had numerous real conspiracies working against it, some of which were the most powerful financial groups in the world.[0]
The 'electric car' wasn't made popular and possible by recent technological strides -- although it was made better.
The success and popularity of the electric vehicle was made possible by financial shifts away from petroleum exploration, facilitated by dwindling profits and increased scarcity of oil, and encouraged by a movement towards sustainability both from the social culture of the world and the various actions of government from country to country.
Yes, range has improved. Yes, the cars are more intelligent and better to drive -- but these improvements have been seen across the automotive industry since its' inception with ICE based vehicles included.
The real motivating factor behind the electric car is the environment that now exists that allows such endeavors to be profitable -- an environment that not only includes technological improvements like you hint towards, but more importantly it's an environment that fosters development of such things due to the existence of a profit incentive and increased governmental-body cooperation.
All that said, unless Cisco is even more evil than I realized (woah..), I have a hard time presuming that video conferencing has been held back by the same sort of conspiratorial under-handed back-office dealings that slowed the progress of EV adoption.
[0]: https://en.wikipedia.org/wiki/Who_Killed_the_Electric_Car%3F
imiric | 5 years ago
Why does it matter which circumstances allowed the electric car to become popular only now? Whether that's because of major industry and consumer shifts, or because the technology matured enough for mass market, they have the same effect. It's certainly a combination of both, and we shouldn't downplay the advances in battery technology alone.
I mentioned that example because the previous two comments were dismissing this attempt at 3D teleconferencing on the grounds of it being old technology with past failures. But I think we agree that it takes a certain industry environment along with a technical leap to make a technology truly popular. Even if that never ends up happening in this case and it remains a niche product, we should applaud the technical merits here instead of being dismissive.
ghaff | 5 years ago
>this attempt at 3D teleconferencing
For that matter video teleconferencing period has only just really hit critical mass even though there were videophones at the NY World's Fair in the 1960s and camera systems have been around in conference rooms for a few decades.
What really happened was that it became more or less accessible to anyone with a laptop and an even marginal network connection for basically no cost. And, oif course, the last 18 months really pushed it over the finish line if it wasn't already.
wpietri | 5 years ago
> Replicating the experience of in-person communication much more closely
That's not a problem people express much, at least not in ways where "3D" points to a solution. When I want to see people in person, it's not because of a lack of stereoscopy. I want to hug them, to break bread with them.
> there's no reason to believe a revolutionary new approach and product couldn't change this
There is indeed! In specific, the many times we have already had revolutionary new approaches and products that were met with great enthusiasm in the market for a few years.
I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular. (I'm not sure what your work calls are like these days, but quite a lot of people turn off video in mine.) The lesson I take from that is that people mainly self-generate the feeling of interpersonal connection, and they can do it with very little in the way of cues. To me that's another strong indication that no new 3D technology will make much of a difference.
imiric | 5 years ago
> When I want to see people in person, it's not because of a lack of stereoscopy.
Of course, I'm not saying this will replace physical communication (I should've said "simulating" instead of "replicating"). But it's a clear step forward for traditional teleconferencing solutions. What do you think is the next leap from 2D video and displays? We're at the point of diminishing returns as far as increasing resolution goes, most consumers don't have a need for 8K or higher res displays. VR/AR is chugging along, but we're still a few generations away from mass market adoption.
> I'd add that the telephone was not only a very successful technology for a century, audio calls still remain very popular.
I don't understand. Video calls were never meant to replace audio calls, they just added a new sensory experience. It's perfectly fine for both technologies to co-exist for different moments and preferences. In a similar way this 3D approach is an extension to traditional 2D video conferencing if people have the equipment and prefer it. Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be, especially if it's polished and cheap enough.
wpietri | 5 years ago
It's not a clear step forward except in technical terms. Those often don't matter. For example, the big revolution in the last 20 years is not faster computers, it's mobile ones.
I don't have much reason to think there is any near term "next leap from 2D video and displays". 2D renderings are more than 40,000 years old. They have improved drastically in resolution and fidelity. Computers added being dynamic and interactive to that. it's really not clear that 3D rendering adds much.
> Judging by the expressions of the people in the demo and some of the comments here, you're underestimating how impactful this could be
I am not, because that kind of novelty-driven excitement has driven every wave of popular 3D rendering technology for 170 years. VR/AR has been close to mass market adoption for 25 years. We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
People have had those excited faces every time. There were people jazzed about the possible impact every time. The Brewster Stereoscope. The ViewMaster (with the US Defense Department purchasing 6 million reels on the theory it would revolutionize training). 3D movies in the 1950s. VR in the 1980s and 1990s. 3D movies again this century. 3D TV for 2 CESes. And then the latest wave of VR, which you agree is still not there despite fantastic investment from companies floating in cash.
Could it be different this time? Maybe! But if we keep measuring it by novelty effects, we're setting ourselves up for the exact same failure that keeps happening.
imiric | 5 years ago
> the big revolution in the last 20 years is not faster computers, it's mobile ones
Surely the improvements in manufacturing processes, faster hardware and better screens are partly responsible for that. The iPhone as a concept has existed since the 1980s, and revolutionary ideas like what General Magic tried to produce in the 90s were just too early to be successful. When Apple tried it again in the late 00s it was a massive success, but technology finally reached a point when it was commercially feasible.
So it doesn't take much to push a product to mass adoption. Just the right industry circumstances, a manufacturer willing to take the risk and capable hardware and software existing to make it happen.
> We've just been through an unprecedented period of demand for at-home entertainment, and the hardware that many said was finally, finally the thing turns out once again not to matter.
Are you dismissing the potential of VR/AR as well? The current innovation wave we're on is much bigger than whatever we had before. Headsets are becoming cheaper, more comfortable and accessible, and the visual tech we have now is leaps and bounds better than previous generations. Once we get to being able to put on sunglasses and experience different worlds, though likely sooner than that, the market adoption will likely go through the roof.
> People have had those excited faces every time.
I think it's different this time. It's not just it being 3D, but the merging of new generations of light field cameras, face/eye tracking, powerful ML algorithms, low latency networks and revolutionary displays is miles ahead of previous attempts. You can't just compare this to the ViewMaster and last century VR. The improvements here are much more substantial, and if they can make it cheap and reliable enough it could be a ground breaking product.
rewq4321 | 5 years ago
"Hard tech" often only matures after several hype cycles. Sometimes cool tech demos can be produced a century or more beforehand. If you were in the 1980s and people were talking about video calls being the next big thing, you might point out that people have been working on video calls since the 1930s[0], and it hasn't caught on in every one of the hype cycles that have followed, so that's an indication that it won't catch on in the future. Video calls have caught on now though - especially as they've reached mobile devices instead of requiring a literal booth in your house, as with AT&T's initial "Picturephone" tech in the 70s.
I will say though that people tend to assume that each new technology will replace the preceding technology (text->audio->video->VR/light-field->...), but in fact it tends to end up just supplementing the existing tech.
[0] https://en.wikipedia.org/wiki/History_of_videotelephony
adamauckland | 5 years ago
Wow, this is a pretty ableist take. Deaf people or hard of hearing exist.
I've got a hearing problem where I struggle to make out what people are saying on a phone but with a video call I can add lip-reading and visual cues which helps me keep on thread.
Aeolun | 5 years ago
How does it do the light field if multiple people are behind the glass? They can’t really optimize it for every person individually right?
speeder | 5 years ago
I want to try this just to see what happens, if I will have again the feeling something is more real than reality.
Because optometrists are illegal in my country (here only medics can decide what glasses you can use), currently I don't have stereoscopic vision, although my brain CAN do it, if I had the correct images sent to my eyes somehow. (one of my eyes muscles is slightly shorter than the other side thus the images on that eye are shifted unless I had an optometrist design me glasses with a prism).
So when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality, despite it being obvious fantasy with aliens, floating rocks and all that stuff.
EDIT: for those wondering, I am from Brazil, here medical professionals often sue the shit out of anyone offering any service remotely similar to an optometrist, they are quite aggressive about it, some attempted to make even discussion of the subject illegal. And when I was trying to get the prism and asked around my medics about it, one of them went really ballistic, I honestly thought the guy was going to punch me. I believe the reason for that is that for many medics, designing glasses is their only source of income, a guaranteed one, since here is ALSO illegal to buy glasses without a medic desining them for you first, even if the new glasses are supposed to be identical to the old ones!
graup | 5 years ago
Wait – I also don't have stereoscopic vision, do you have any details or sources for those magical glasses with prisms?
speeder | 5 years ago
I first learned about this prism business when I read this article: https://www.newyorker.com/magazine/2006/06/19/stereo-sue
Back then people even were saying it was pseudoscience or a scam, but seemly there is ongoing research that shows that the article wasn't lying.
djrogers | 5 years ago
Sounds like a great reason to do a little medical tourism once the present situation is under control.
JonathanFly | 5 years ago
>currently I don't have stereoscopic vision
>my brain CAN do it, if I had the correct images sent to my eyes somehow.
>when I watched Avatar, an actually well made 3D movie, it literally felt more real than reality
Buy an Oculus Quest 2. (Or any VR headset, but that's the best value at the moment.) It sends seperate images to each eye. You should get that same 'more real than reality' feeling. It may even train your eye muscles to see in 3D, I know I've seen some research into that area. (https://www.seevividly.com/ comes up on a Google search, though it seems to be prescription only)
karolist | 5 years ago
I'm someone with Amblyopia and would love to hear people's experiences using this. Seems like the project made some waves on reddit and kind of died down. I have bought Oculus Quest last year in anticipation of trying this but the pandemic and work pushed it out somewhat.
Watching 3D movies in the theaters just gives me a headache but Oculus actually works fine, I've played a few games and it was amazing...
ako | 5 years ago
Online eye exam: https://www.easee.online/en/
ethbr0 | 5 years ago
Can you import glasses? Or, say, receive a box that someone accidentally dropped a pair of glasses into, along with the item you ordered?
kikokikokiko | 5 years ago
Brazilian guy here: my girlfriend just bought two pairs of glasses from China, I don't know from which site exactly, but some AliExpress type of site. She paid 5 dollars each, just telling which degrees she needed for each eye. The quality was the same from the 200 dollars pair she bought with a medical note here in Brazil. Unbelievable. Just telling that, if you want it, you can import it through the mail and it will pass through customs without a problem.
The_rationalist | 5 years ago
would be nice to share the site
WillPostForFood | 5 years ago
If you search for "myopia glasses" on Aliexpress you'll find prescription glasses starting as low as $2. I just added a pair to my cart and the total was $3.31 including shipping. :O
numpad0 | 5 years ago
Could be Zenni Optical. Prescription glasses, right as ordered, no questions asked.
mkl | 5 years ago
I have glasses from Zenni. Very cheap, and well made. My current glasses cost ~10% of what the previous pair cost me in NZ, and the lenses are better quality. The glasses cost less than the optometrist visit, and I bought a second identical pair as a backup.
thegginthesky | 5 years ago
Shouldn't an ophthalmologist here in Brazil be able to diagnose you and give you a prescription for corrective lenses, or even perform corrective surgery?
I'm asking in case I might need to look for one in the future.
ZeroClickOk | 5 years ago
I'm from Brazil too, and my eye glasses are recommended by a optometrist. The only way an optometrist to work is being an ophthalmologist with specialization in "optometry". It is a sad reality from our bureaucratic government that don't give us freedom to innovate.
thegginthesky | 5 years ago
Cool, thanks for explaining this for me!
speeder | 5 years ago
The profession of optometry came from physics instead of medicine, many of the early ones with physicists specialized in optics that ALSO understood how the eye worked, optics-wise.
In places where they are legal they spend their time learning more and more about optics, physics, math and eye anatomy, they don't study diseases, infections and so on.
As result you can't rely on one to fix certain stuff where you do need a medic, but if you need fancy lenses, they will calculate them, not just use a number they get from their measurement machine like ophtalmologists do.
They also would be helpful to design proper 3D glasses and whatnot, holograms and so on.
thegginthesky | 5 years ago
Got it, I was a bit confused as I never had heard about the problem you mentioned. Too bad here in Brazil you can't get the specialized care you need for the condition you have.
This is a major screw up and inefficiency from this country.
Thanks for explaining the concept more in-depth too.
sleepybrett | 5 years ago
Similar tech in the display of the nintendo 3ds?
jbverschoor | 5 years ago
I've seen 3d screens in Japan 20 years ago. TVs, ad displays, mobile phones. It works really well, and it's actually very simple technology. Never understood why that never took off.
colordrops | 5 years ago
This is different. It's a light field display, so you can view things at different angles as you move around, rather than having to sit in a very exact position and only seeing one angle.
XnoiVeX | 5 years ago
Sony has it's own version of the light field display. https://youtu.be/KrLMnQM0_Ps
leokennis | 5 years ago
My first thought: in 2030 when I’m calling grandma with this, I will first need to accept cookies, then accept the new Google privacy policy, then sign in to my Google account, then watch 2 20-second long pre roll ads.
booleandilemma | 5 years ago
Google will have definitely discontinued this project by then of course :)
silentsea90 | 5 years ago
We're witnessing magical tech and innovation, and here we are back to making fun of signing in, ads and cynicism about Google shutting down projects.
meibo | 5 years ago
Especially because most of Google's productivity apps don't even have ads. I think it's just Gmail on mobile at the moment.
hu3 | 5 years ago
I thought Gmail "read" my emails to finetune ads but today it told me they don't: https://i.imgur.com/SHSOSJ7.png
gregcoombe | 5 years ago
It's the office of the future! https://www.cs.unc.edu/Research/stc/. The challenge in 2000 was that bandwidth and 3d reconstruction needed several leaps forward in quality.
wbsun | 5 years ago
Reminded me of this paper on SigGraph 2007: https://ict.usc.edu/pubs/An%20Interactive%20360%20Light%20Fi..., video: https://youtu.be/8gvPS1m40gw
vicary | 5 years ago
That looks amazing
crossroadsguy | 5 years ago
The two things that stayed with me after reading this and feeling excited for a very brief period were:
1. Google shutting it down after sometime out of the blue. Just like that.
2. Google needing nothing less than a Gmail account always logged-in and no choice about it.
gehsty | 5 years ago
Super cool technology. It will be interesting to see how this develops, for me google seem to struggle with hardware products, even more with consumer hardware products. I can't see where it fits in to the google ecosystem and how people will access it.
I could imagine Apple selling this in a re-imagined Apple TV with an Apple TV Facetime App. They could probably build something now with the FaceID sensor array / iPhone Camera system / M1 chip plugged into a 4k TV.