back to article Amount of pixels needed to make VR less crap may set your PC on fire

Put on a virtual reality headset and it's hard to believe that your visual system is being stretched beyond its limit. Individual pixels are still visible and the narrow field of view makes it feel like you're wearing ski googles. Yet even now VR bombards our visual system with more information than it can process. Engineers …

  1. Jason Bloomberg Silver badge

    Make it sew

    "don't expect to do any VR sewing in the next decade or two"

    Sewing is actually a case where 'tunnel vision' isn't a problem and even current VR appears to be suited to that

    It's the effort and strain of dealing only with 'what's in the tunnel' for prolonged periods which is the real problem, whether sewing, performing surgery, or similar.

    1. Sampler

      Re: Make it sew

      Isn't this what Fove were supposed to be doing to distinguish themselves in the VR space, I recall reading it a while back and being the only VR headset I was interested in, before I lost interest in having a VR headset..

      Kickstarter snakeoil? www.getfove.com

  2. LDS Silver badge
    Joke

    That's what Magic Leaip is working on.

    They achieved it boiling frogs and flies eyes into a cauldron with bats ears, to use their ultrasonic capabilities to track eye movements.

    1. Sir Runcible Spoon Silver badge

      Re: That's what Magic Leaip is working on.

      How are they addressing the response time for eye tracking?

      If I move my eyes 20 degrees to the left my personal optics seem to take a moment to focus - is that enough time for the tracked eye movement to hit the CPU and re-render that area in high resolution in a VR headset?

      1. TRT Silver badge

        Re: That's what Magic Leaip is working on.

        Micro-saccades are way faster than that.

      2. katrinab Silver badge

        Re: That's what Magic Leaip is working on.

        Probably not, because most of that is the delay between the light hitting your eyes, and your brain doing all the processing to figure out what it is. That takes about 0.1 seconds.

  3. karlkarl Bronze badge

    Interesting read

    Nice to see some research into actual VR going on rather than on their monetisation platforms and "VR stores" ;)

    1. Prst. V.Jeltz Silver badge

      Re: Interesting read

      I cant help feeling a kind a "mechanical sympathy" for the amount of computing horsepower we put into "making things look pretty".

      Computers are great things ideal for repetitve tasks and calculations. theyve changed the world with their databases and accounting and engineering modelling, and ecoding of music and video..

      But why is it necessary to animate every blade of grass waving around in the background while I play Farcry ? The video card in my pc sits there whirring , throwing heat out , using extra psu sockets and costing more than the rest put together - like some kind of over developed muscle.

      1. BlueTemplar

        Re: Interesting read

        The issue is that current VR has such a low resolution that you even have trouble reading any not-oversized text...

        1. Prst. V.Jeltz Silver badge

          Re: Interesting read

          oh , I didnt notice that when I had a bash on a mate's Playstation setup. i was too busy bouncing balls around a room and shooting spaceships :)

          pchoo! pchoo! bzzzzt! ping! Whizz! boom!

      2. Luiz Abdala
        Pint

        Re: Interesting read

        Q: [But why is it necessary to animate every blade of grass waving around in the background while I play Farcry ?]

        A: https://en.wikipedia.org/wiki/Escapism

        People want to forget about their dull daily lives and immerse in something pleasant to the brain. Somewhere on the webs it is said that the visual cortex takes 70% of our brains, audio takes another hefty chunk of the remaining... so fooling them both will immerse you pretty handily.

        And yes, my GPU also has more computing power than 2x my CPU, and guzzles just as much 'leccy as the whole rest of the system.

        To almost complete the immersion of brain into pleasant things, you need a beer while playing.

      3. Paul Westerman

        Re: Interesting read

        The grass is to distract you while the eagles are swooping down at you. Moving grass in FC3 meant honey badgers incoming!

  4. Voland's right hand Silver badge

    Direct to-brain interface

    I suspect that the "direct to brain" or direct to eye nerve interface will emerge before we figure out a viable tech to be able to do all the crazy stuff needed to track eye motion and produce variable resolution in the different parts of the headset.

    1. Dave 126 Silver badge

      Re: Direct to-brain interface

      Hehe, made me think of the eyePhone from Futurama!

      Episode 'Attack of the Killer App'

      https://en.m.wikipedia.org/wiki/Attack_of_the_Killer_App

    2. Anonymous Coward
      Anonymous Coward

      Re: I did not upvote/downvote

      Because you are both right and wrong for another reason. We already have such technology, and have for thousands of years. It's called "hard rugs". It at least allows such suggesting "trips" anywhere you wish.

      As with all technology though, bug solving and customer support can be a real *nightmare*.

    3. Prst. V.Jeltz Silver badge

      Re: Direct to-brain interface

      I suspect that the "direct to brain" or direct to eye nerve interface will emerge before we figure out a viable tech to be able to do all the crazy stuff needed to track eye motion and produce variable resolution in the different parts of the headset.

      Yeah - current VR is the "Accoustic coupler" of video.

  5. Steve K Silver badge

    Fovea or FauxVR..?

    Fovea or FauxVR..?

    I wonder if we are hitting the same VR tech wall as in the early 1990s, just a bit further along (due to the better tech available now)?

    In other words the 80/20 rule, but the 80% we have is not enough for VR to take off, and the 20% is really rather hard to sort out (hence the 5+ year timeframe for the rendering taking account of fovea and periphery differences)

    1. Dave 126 Silver badge

      Re: Fovea or FauxVR..?

      The inability to render scenes indistinguishable from real life was no barrier to video games of the 80s and 90s.

      If it can be made fun without nausea and other negatives, it could still achieve some success even without solving the problems described in the article.

      1. Sir Runcible Spoon Silver badge

        Re: Fovea or FauxVR..?

        I'm having a lot of fun with my PSVR.

        Is it perfect? Of course, not, that doesn't stop me enjoying it.

        I've played Skyrim on the xb360 and ps4 (flat) and now on PSVR - it's now a totally different game imho.

        Once upon a time I'd click through the NPC dialogue to get the quest markers (for example) but now I find I am actually engaging in the world and taking an interest in the people that are in it. They are somehow more real now and I find I am caring. There are people playing it on PSVR who will actually take a seat in an inn and just chill out watching the fire and listening to the bard and the general conversations. Never heard of that happening in the 2d version.

        Still, this does raise some interesting ethical questions about what you get up to in VR. It's so much more engaging that repeated anti-social activities that I really can see it having a negative impact on some people's behavior. I'm old and gnarly enough to sense it happening and be aware of the risks - others might just soak it up and let it influence how they act in real life.

        Extremely powerful stuff, even at this graphical level. I don't want to see the old 'videogames made my son a killer' thing resurrected to be honest.

        1. disgustedoftunbridgewells Silver badge

          Re: Fovea or FauxVR..?

          I find I have to be a bit more careful driving in real life just after playing on a driving game. It's a bit like coming off a motorway and having to watch your speed more.

          1. cdrcat

            Re: Fovea or FauxVR..?

            You need to also take more care after driving go-carts. My old boss crashed his M3 after racing around a track with work mates!

            1. Glenturret Single Malt

              Re: Fovea or FauxVR..?

              Any big change in the type of vehicle requires care initially. I used to find problems with the relative responsiveness of my car immediately after driving a fully loaded minibus for a couple of hours. I am sure that the same applies to drivers of all heavy commercial vehicles.

      2. Prst. V.Jeltz Silver badge

        Re: Fovea or FauxVR..?

        The inability to render scenes indistinguishable from real life was no barrier to video games of the 80s and 90s.

        Spot on. I have just written similar comment.

        They'd be better focusing on the other aspects of vr "immersion" like how to stop the user running around the china shop like a crazed bull.

        Or getting force feedback when manipulating objects.

        Or a smell genarator

    2. spold Bronze badge

      Re: Fovea or FauxVR..?

      Pretty much, a lot of the limit back then was the graphics quality. Graphics processing was offboarded to the "host" - in the example I worked with, to 4 RISC processors on a board in the PC. So firstly the same strategy is key, don't make the crappy hardware in the visor do the processing. For augmented reality you are going to need to push the processing even further out - probably cloud - to deal with the large database to process application things and give you "headsup" data such as "This is Jenny, you met her at a conference last year, she likes dogs and gin and tonics, looks like she is interested in you" - you can't do that in native hardware.

      Lastly they should buffer the display, then all you need to render/update is the bits that have changed.

      p.s. I was product manager for immersive virtual reality systems for IBM in 1995.

    3. David 18

      Re: Fovea or FauxVR..?

      Have an upvote for the pun sir!

    4. Anonymous Coward
      Anonymous Coward

      Re: Fovea or FauxVR..?

      Indeed. I saw PlaystationVR demo a few weeks back, and it blew my mind how good it was, and it's actually a very reasonable entry price (if you have a PS4). You can get the full kit for £250 if you shop around.

      Is the FacebookVR or Vive worth the thousands of pounds more (when you look at the supporting hardware investment), Absolutely not, especially given the huge userbase and game title advantage that PSVR has.

      1. BlueTemplar

        Re: Fovea or FauxVR..?

        You don't need an overly expensive PC for VR :

        I'm having good enough performance for ~90% of the use cases with a

        AMD (Bulldozer) FX-8320E (£100) (the 8 cores seem to help a lot)

        and a

        Radeon RX 470 (£175)

        (and I blame most of the issues with the remaining 10% at the developers not bothering to test their software with Bulldozer-line CPUs)

        http://steamcommunity.com/sharedfiles/filedetails/?id=1116639906

  6. allthecoolshortnamesweretaken Silver badge

    Isn't it obvious? VR only works in combination with a hipster beard. Sorry, ladies.

    1. onefang Silver badge

      "Isn't it obvious? VR only works in combination with a hipster beard."

      I have a proper Unix beard, it works well with that to.

      1. Destroy All Monsters Silver badge
        Trollface

        A Unix beard meets a hipster beard:

        Hipster: "I recognize you. You are one of those annoying Unix beards!"

        Unixer: "Here is a nickel kid. Grow yourself a real beard!"

        "And stop using Node.js".

  7. Andy E

    Glasses anyone?

    What if you normally wear glasses? Can the VR headsets be adjusted for this or do you have to wear contact lenses?

    Just wondering....

    1. Valeyard

      Re: Glasses anyone?

      not sure if you're long-sighted, but if you're short sighted you're fine since the objects only look far away but aren't actually. it's like being in a magical world where i can see

      1. Sir Runcible Spoon Silver badge

        Re: Glasses anyone?

        not sure if you're long-sighted, but if you're short sighted you're fine since the objects only look far away but aren't actually. it's like being in a magical world where i can see

        Really? I'm short sighted and I still need to use my glasses. Which system do you use?

        1. TRT Silver badge

          Re: Glasses anyone?

          There's a really weird effect I found with the HP Z system. The screen image is in focus, some 50-60cm in front of you, but the optical disparity L-R eye puts the object almost on your nose. So you have a perfectly in focus virtual Angel fish sat on the end of your nose when if it was real it would be blurred beyond recognition.

        2. Brenda McViking

          Re: Glasses anyone?

          I'm on windows mixed reality with the HP head mounted display and need my glasses (which do fit with little room to spare) as there is no inbuilt adjustment. I'm short sighted +2.00 and +3.75

      2. BlueTemplar

        Re: Glasses anyone?

        That's weird because I'm short-sighted (slightly, -0.5 diopters), and my (Oculus Rift, CV1) experience is noticeably degraded without glasses -

        (which are also annoying, start to bother me over 30 minutes in VR, and risk damaging the (non-removable, extremely easy to scratch) lenses when removing the headset - I'll get lens inserts when I get around to it)

        I've assumed that's because the lenses are actually to angle the light rays to make like the picture is coming from far away...

        (but I also have light astigmatism, maybe that's a factor too?)

    2. Excellentsword

      Re: Glasses anyone?

      I have wondered, but I'm assuming headsets can fit over them. I've started wearing contacts so it's less of an issue. Still, I'm a big "gamer" who has yet to try VR. Why? Because I knew from the beginning that it's a big fat gimmick unable to truly rival experiences with a monitor, and in part due to the issues outlines in this article. Sure, it's a different experience, but it's not really useful, it doesn't give an "edge".

      1. Bernard M. Orwell Silver badge

        Re: Glasses anyone?

        I've worn vari-focals, single vision and contact lenses in both Rift and Vive and had little to no problems.The Vive performed slightly better with contact lenses, but the vari-focals are the business.

        As for VR not giving an "edge", it may not be what you were driving at, but in Elite Dangerous the Vive certainly delivers a competitive edge. Yeah, its true, that its not as gloriously pretty as a 4k monitor, but the sense of "presence" counters that very well. The ability to "g-lock" your view to an enemy like a fighter pilot is something else, and in CQC its saved my hide many times.

        1. onefang Silver badge

          Re: Glasses anyone?

          I'm shortsighted, I have distance glasses, reading glasses, and often go without glasses to peer closely at something to see the fine detail. Actually my reading glasses are purely for my own computer monitor, I don't need them for anything else, not even other peoples monitors, unless they use crazy small fonts like I do.

          I have three varieties of VR headset, an Oculus DK2, a plastic Google Cardboard 1 compatible I bought from Aldi, and a Google Daydream View 2017. Oddly enough, the Oculus is best in my reading glasses, the Cardboard best with none, and the Daydream best with my distance glasses. The Oculus came with a few different lens sets for dealing with glasses, and I found the combination of the C lens with my reading glasses gave the best view. The Cardboard has enough focus adjustment that I could go without glasses to get the best view. The Daydream has fixed focus that assumes your vision is perfect, or perfectly corrected, so distance glasses it is.

          The Cardboard and the Daydream I have been using to demonstrate VR to seniors over the last few months. Coz I only need my phone with these headsets, don't have to drag along an entire computer system. The Cardboards focus is both fiddly and hard to use if you don't have a lot of strength in your fingers, so I've generally been recommending they use their normal glasses with the Daydream. No one has complained about the Daydream and their glasses, a few complained about how hard it is to focus that particular model of Cardboard. The seniors have a variety of eye problems and suitable corrective glasses or contacts.

          1. kernelpickle

            Re: Glasses anyone?

            Correction--you my friend have presbyopia! Which is what happens to everyone as they age, because our crystalline lens loses it's ability to change shape, and adjust our focal depth--hence the reason for progressive lenses that correct for distance, intermediate, and near vision.

            https://en.wikipedia.org/wiki/Presbyopia

            I've looked into some headsets with lens inserts, and they're pretty straightforward for folks that rely on a spherical correction (either - for myopia or + for hyperopia) to their vision, because you can get close enough with the OC height, and PD and they'll work.

            However, for folks with more complicated prescriptions, it's not that easy, because there are tighter tolerances required. If you have any cylindrical correction (for astigmatism) you have the added variable of Axis to worry about, and minor changes to the OC height, PD, or angle of the lenses will produce a fishbowl effect--which can induce nausea in someone with a high enough prescription. A lens with only 0.25 correction to cylinder can be rotated by as much as 5-10 degrees without the wearer having any issues--but an RX with higher amounts of cylinder (mine I have over -3.00 in mine) can be greatly affected by changes of less than 1<5 degrees! That said, you can flip cylindrical lenses by 180 degrees and be perfectly fine.

            https://en.wikipedia.org/wiki/Prism_correction#Prentice's_rule

            Then there's the most complicated prescriptions, that have multi-focal correction (for presbyopia), which still requires lenses remain perfectly in position, because the seg height measurements need to line up correctly, so that the intermediate segment doesn't intrude on the distance vision and the wearer doesn't have tilt their head up drastically to see something at arms length--so a mm difference can be huge!

            Basically--I'm all sorts of fucked now (as far as getting fancy lens inserts goes) and I'm only going to be more fucked as I age! So, I'm REALLY hoping that in my lifetime, that they'll be able to figure out how to interface with the optic nerve directly.

            In the much nearer future, there may be some hope, because they've come a long way with implanted IOLs (used to repair cataracts) and now they're not only able to correct any refractive errors (including astigmatism) but the latest technology are accommodating IOLs that are able correct presbyopia! They're obviously not quite perfect yet, so you wouldn't want to get them if you didn't already have cataracts--but it seems like we're closer to making the technological advances needed to create a mechanical (or even a bio-mechanical) solution to the problem.

      2. bondyboy

        Re: Glasses anyone?

        For driving sims it absolutely does give you an "edge" and is certainly not a gimmick; speed sensation, immersion/sense of presence, depth perception and "feel" of what the car is doing under you are all greatly increased, allows you to more easily position the car exactly where you want (or be able to tell by just how much you missed that apex).

        Before the Oculus arrived I was using high end triple screens in my sim rig, they haven't been used for almost two years now

        1. 68K

          Re: Glasses anyone?

          > For driving sims it absolutely does give you an "edge" and is certainly not a gimmick; speed sensation, immersion/sense of presence, depth perception and "feel" of what the car is doing under you are all greatly increased, allows you to more easily position the car exactly where you want (or be able to tell by just how much you missed that apex).

          I tried out PSVR with DriveClubVR and while the freedom of vision was astounding (loved being able to know where the other cars were with a quick flick of my head), the lack of any G-forces really didn't agree with me. Mashing the brakes hard or turning corners didn't agree with what my brain knew I should be experiencing and after a few minutes I felt quite spewy.

          Maybe it's something you could get used to. Unfortunately I haven't been able to spend enough time with a VR HMD to work that out.

          1. bondyboy

            Re: Glasses anyone?

            I've never had any motion sickness from the Oculus while in the driving sim but I know of others who had to "build up their VR legs" although all who have tried mine didn't have any issues.

            Did you try it on a proper rig? fixed seat, wheels and pedals? maybe that helps? plus I tend to lean into corners while looking through the apex - could that be helping to trick my inner ear?

            Within my sim (Assetto Corsa) there are various settings that can relate to "Comfort/sickness inducing" , one which has divided opinion is the Lock to horizon - some love it with it on others can't drive with it. Not sure if any of those types of options are available on PSVR and driveclub.

            1. Simon Harris Silver badge

              Re: Glasses anyone?

              "I've never had any motion sickness from the Oculus while in the driving sim"

              We got an Oculus at work and one day I spent way too much time playing with one of the flying demos - no motion sickness in the sim itself, but I felt pretty nauseous all the way home on the bus.

              Normally I never get motion sick on real-world transport.

              1. bondyboy

                Re: Glasses anyone?

                Yes I've found the couple of times I've fired up Valkrie (space combat) I've experienced similar, but never in the driving sim

      3. anyoldfool

        Re: Glasses anyone?

        Depends what kind of 'edge' you're after from your gaming, if you're the type to invest in over-priced mice and keyboards to get that millisecond edge in whatever MOBA type games you like to play then no VR isn't going to be your thing. VR is for gamers who actually like to enjoy gaming, those who want to feel the wonder of a new world and marvel at the experience of just simply gaming again. Gimmicks are only gimmicks until they're ubiquitous and personally I would never go back to majority pancake gaming, it's just a big step backwards from actually experiencing a game world, no matter how high res your monitor. For something like sim racing it's undeniably the future of that genre, flat gaming had nothing on VR+wheel there and never will again I'm afraid.

    3. Dave 126 Silver badge

      Re: Glasses anyone?

      Yes, headsets can be adjusted for glasses, just as the virtual view finders on mirrorless digital cameras can be.

      1. Sir Runcible Spoon Silver badge

        Re: Glasses anyone?

        I wear glasses (short sighted) and I use PSVR quite effectively. It's not ideal obviously, but do-able.

    4. blarpman

      Re: Glasses anyone?

      I had the same thoughts. Maybe something like entering some details about your prescription or something, and it adjusts to something you can see clearly.

      1. myhandler

        Re: Glasses anyone?

        Um - not going to work until they can project the image directly onto the retina.

      2. This post has been deleted by its author

    5. x 7 Silver badge

      Re: Glasses anyone?

      On an Oculus Rift, the glasses fit under the headset without problems.

      What did cause an issue was the enforced 3D perspective - I've got a lazy eye and have very little in the way of 3D vision. The Oculus Rift effectively rams 3D down your eyeballs and I found it bloody discoordinating - enough to make me feel decidedly queasy

      1. This post has been deleted by its author

  8. tiggity Silver badge

    Saccades

    Bear in mind that eye movement (and thus fovea "focus" is not necessarily a nice smooth predictable scenario, the eye position often rapidly "jumps" with "focus" now on a different area.

    Any "Focus tracking" (if using that approach to try and give acceptable visual image in "focus" area on a system without the pixels / hardware grunt to render all data at high res) will need to handle that as will pixel rendering (with smooth focus change lots of time to prep the hi res pixels at the expected new focus point, with position "jump", need to "instantly" render a new area as the hi res pixel zone.

    1. Mike Moyle Silver badge

      Re: Saccades

      "Any "Focus tracking" (if using that approach to try and give acceptable visual image in "focus" area on a system without the pixels / hardware grunt to render all data at high res) will need to handle that as will pixel rendering (with smooth focus change lots of time to prep the hi res pixels at the expected new focus point, with position "jump", need to "instantly" render a new area as the hi res pixel zone."

      Heh.

      I have a pretty nasty nystagmus. My eyes "jiggle" constantly, to the degree that, if there is an emergency vehicle with lights flashing by the side of the road, from a distance the first flash may be directly on top of the vehicle but the next may appear to be 10 - 15 feet up and to the right, say, depending on the distance away it is. The closer we get, the closer the flashes are to their actual locations. So, we're talking a couple of degrees of twitch happening several times a second. It's fast enough, and I'm sufficiently accustomed to it, that unless something like flashing lights brings it to my attention I never notice it. On the other hand, something directly in front of my eyes trying to keep up with that and constantly shifting focus to whatever it thinks I'm looking at could end up being a strain on both its hardware and my software!

      I wonder if I could rent myself out to VR developers as a test subject...

  9. Tom 7 Silver badge

    The best way to make VR less crap

    would be to take it away from the idiots who think it's a good idea.

    1. This post has been deleted by its author

  10. Randolf McKinley

    Number! It's the flipping number!!!

    Pixels are countable, they come in "Number of pixels", not "Amount of pixels"! Water comes in amounts, so does sand, unless you're talking about atoms of water or grains of sand, then it's number again. I suppose you could have said "Amount of pixel needed ..."

    Anal? Moi?

    1. TRT Silver badge

      Re: Number! It's the flipping number!!!

      Pixel is short for Pixie Dust, as ane fule kno.

      1. x 7 Silver badge

        Re: Number! It's the flipping number!!!

        And if you extrapolate from all those fairy take books, pixel dust are the spores of fly agaric mushrooms (Amanita muscaria). Eating those really helps expand visual reality remarkably. Even helps you see things that would be otherwise invisible.

    2. Steve K Silver badge

      Re: Number! It's the flipping number!!!

      "Every time you correct me on my grammar I love you a little bit fewer'.” - George Lewis.

    3. phuzz Silver badge

      Re: Number! It's the flipping number!!!

      If we actually get displays with 120 pixels per degree, I think referring to an 'amount' of pixels might be justified, it'll be close to the equivalent of a glass of sand.

      (A VR headset would need of the order of 10^7 pixels per eye, a very small sand pit might have 10^8 grains of sand, so a standard 250ml glass would have roughly the same *amount*)

    4. Anonymous Coward
      Headmaster

      Amount

      A quantity of something, especially the total of a thing or things in number, size, value, or extent.

      - https://en.oxforddictionaries.com/definition/amount

      1. Randolf McKinley

        Re: Amount

        Ah, yes, but we're not talking the definitions of the words, we're talking grammar. Those two do not always intersect. In grammar you have a "number of things you can count" but an "amount or quantity of stuff you can't count". However a number can be both an amount or a quantity (34 pounds is a large amount of cash but a large number of pound coins). That's English, one of the least logical grammars in existence.

    5. Mike Moyle Silver badge
      Devil

      Re: Number! It's the flipping number!!!

      Inappropriate use of "amount of..." instead of "number of..." still grates on my nerves, but I've pretty much given up on fighting that battle in order to save my resources for the truly IMPORTANT ones.

      ...like the Oxford Comma.

      1. Randolf McKinley
        Coat

        Re: Number! It's the flipping number!!!

        I know what you mean, Mike. I get upset about grammar, punctuation, and the Oxford comma.

        I'll get my coat ...

        1. John H Woods Silver badge

          Re: Number! It's the flipping number!!!

          Fabulous.... Nearly on a par with the fauxVR pun

        2. DropBear Silver badge
          Trollface

          Re: Number! It's the flipping number!!!

          "I get upset about grammar, punctuation, and the Oxford comma."

          I now what your talking about, and there are indeed few thing's more annoying (except maybe this...

  11. ukgnome Silver badge

    Surely you want VR to look a bit off

    Other wise eXistenZ becomes real.

  12. John Smith 19 Gold badge
    Unhappy

    "Michael Abrash, chief scientist at Oculus,"

    Wonder if he's the guy who did the video column in Dr Dobbs Journal back in the day.

    Which would suggest he's been around this block a few times.

    I also wonder if people how much not like a TV camera the human eye really is?

    Starting with the fact the rods and cones are not laid out in a square pattern, but an exponential spiral, starting at the centre of the FoV.

    And then it gets complicated...

    1. Simon Harris Silver badge

      Re: "Michael Abrash, chief scientist at Oculus,"

      It does appear he is the very same Michael Abrash.

  13. Nimby Bronze badge
    Terminator

    It's still the 90s and VR must be En Vogue, because you're never gonna get it.

    Numbers numbers numbers. These are all just generalizations. Even if you managed to achieve them, you would still have a considerable gap between "everyone" and "the average person". Not every eye exactly the same, nor the brain that process the data. So aim higher yet.

    And meanwhile, I'll continue to ignore the technology because it is just not ready. Better than where it was is still not where it needs to be.

    Here's looking forward to one day not needing it at all as I jack in to the Direct Neural Interface of my deck.

  14. John Mangan

    If I read that correctly..

    it sounds like one way around it would be to have a wrap-around background scene which is essentially static and hi-res and then use AR to put the moving stuff 'on top' of that. Most of your AR view would be transparent reducing the processing overload and should help with head tracking and eliminate the need for eye-tracking.

    Not many people with room for a spherical video wall at home unfortunately :-(

    1. TechnicalBen Silver badge

      Re: If I read that correctly..

      Ooooh. Yeah. Something like that could work to some degree.

      You would need transparent displays. With 3d active glass flicker free. Take a panoramic (or 3 monitor setup) or full wall projection of a scene in low res. Then the high res at the centre or so of the VR headset.

      Downside, you only get the "room" feel on the screen/wall the projection is on.

  15. David Nash Silver badge

    Obligatory Apple reference

    " To match this sensitivity on a computer monitor would require a pixel density that "beggars belief" "

    So much for so-called "Retina display" then!

    1. Dave 126 Silver badge

      Re: Obligatory Apple reference

      Regina Display is a marketing term, true, but most of the time its claim holds up when used at its intended distance of around 12" (it's a phone) as opposed to 1" (on VR headsets).

      As the article notes, there are scenarios which show up resolution more than others; it's easier to spot a low Res monitor if it's displaying a CAD diagram than a photograph of a tree.

      1. Teiwaz Silver badge
        Coat

        Re: Obligatory Apple reference

        Who is this Regina Display?

        Sounds like a Victorian Strippers show name...

  16. tony72

    Do we really need much more pixels?

    The biggest problem at the moment is not so much resolution as fill factor; the so-called "screen door effect". You can see individual pixels not so much because the resolution is too low, but because you can see the gaps around the individual pixels. Things don't look blocky as when the resolution is too low, but it looks as if you are looking through a fine mesh. Even just using my Pixel and Daydream headset, not the highest resolution rig, I rarely really feel like resolution is what's lacking.

    Now you could solve this simply by throwing pixels at it - the higher the resolution, the finer the "mesh", and if you go fine enough, then it will seem to disappear. This will, as noted, require a great deal of processing power to render all those pixels, however, and seems like a pretty brute force way of dealing with it.

    Foveated rendering doesn't sound to me like much better of a solution. The complexity of getting the eye tracking to work well enough and fast enough, the variable-resolution rendering, and the other issues mentioned, sounds like an absolute nightmare.

    Now I don't know displays, but I do know a bit about image sensors. Image sensors also suffer from fill factor issues - light falling between the pixels is wasted, thus the sensitivity of the sensor is less than optimal. One common solution to that is micro-lenses - as it sounds, an array of tiny lenses is bonded over the pixels, and the lenses capture light that would have fallen into the dead area around the pixel and focus it onto the active area. I don't see any reason why something similar wouldn't work for displays, and a quick bit of Googling says I'm not the first to think that way. No idea of the practicality or cost of manufacturing that kind of thing, but it may well be that we don't need so many pixels to make VR less crap after all, a 4k display plus micro-diffusers (or whatever those things end up being called) to fix the fill factor may be more than good enough.

    1. imanidiot Silver badge

      Re: Do we really need much more pixels?

      This is what I was thinking. Much of the Screen-door effect comes from the hard-edged nature of the individual pixels. So maybe what we need is slightly blurry pixels instead of hard edged ones.

      1. Sir Runcible Spoon Silver badge

        Re: Do we really need much more pixels?

        Even if you remove the screen door, you still have the challenge of resolution.

        I.e. higher resolution will allow you to display more detail, but also requires more rendering power. It isn't really about screen door from what I read.

        1. onefang Silver badge

          Re: Do we really need much more pixels?

          There's screen door, resolution, and "mura". Current generation consumer headsets don't have enough resolution to be monitor replacements, especially not for crazy small font users like myself. Makes it hard to be a VR developer, I have to rip the headset off to read and write source code, then put it back on again to see the results. Fuzzy pixels might help with screen door, but the resolution is still low. "Mura" is caused by inconstant brightness levels of individual pixels, so that there is what looks like a cloudy texture that is fixed in your view, like wearing dirty glasses. All three get in the way of faking reality, unless you happen to be used to dirty glasses and looking through fly screens.

          BTW, a Motorolo Moto Z with a Google Daydream is actually higher resolution than an Oculus Rift CV1. I dunno what the resolution of the above mentioned Pixel in a Daydream is off the top of my head. I only know about the Moto Z coz that's what I use in my Daydream. As mentioned in my previous post, I'm shortsighted, I can see tiny details if I take my glasses off, but I can't see individual pixels on my Moto Z (like I could on my Samsung GS3) unless I put it in the Daydream or Cardboard.

          1. tony72

            Re: Do we really need much more pixels?

            @onefang - sure, I didn't say the we don't need any more resolution, just that we might not need resolutions so high that our PCs will melt. I mentioned 4k displays at the end of my post, and 8k displays are being demo'd right now with current, albeit high-end PC hardware, e.g. Pimax 8k (some guy's video review), so that doesn't seem too crazy.

            The Pixel in a VR headset is a bit lower resolution than the Rift, it's 1080p, so 960x1080 per eye, versus 1080x1200 for the Rift, but it's not too bad.

  17. Anonymous Coward
    Anonymous Coward

    Thanks

    Well, this is one of the best articles I've read on this site in a good while. Lucid, intelligent and informative. Please, more technical articles of this caliber, and fewer political "Op Ed" pieces.

  18. Lord_Beavis
    Paris Hilton

    Really, did no one think to ask...

    about pr0n?

    1. Anonymous Coward
      Anonymous Coward

      Re: Really, did no one think to ask...

      We were all thinking it. The great technological driver throughout the ages.

      1. x 7 Silver badge

        Re: Really, did no one think to ask...

        Porn........the idea of a 3D popup in the browser is something to consider with trepidation (or hope depending on your orientation)

        1. onefang Silver badge

          Re: Really, did no one think to ask...

          VR porn works quite well actually, even with the low resolution, and even with most of them being just 180 degree movies, and only 3 DOF. The immersion you get is great, it's like really being there, so intimate. And due to it working great with the low resolution, there's no need for foveated rendering. Any old phone in a Cardboard should work fine. There's even a few porn movies which include data for driving robotic sex toys to match the VR action, though I don't have any of those toys, so can't comment.

          The next stage could add VR cameras and long distance lovers.

          1. Anonymous Coward
            Anonymous Coward

            Re: Really, did no one think to ask...

            I've tried a few free videosphere ones, and I disagree : I didn't get any "intimacy" impression, and the tiny bit of extra immersion you get is ruined by the inconvenience of the headset and the weird camera placement.

            Of course YMMV, I've yet to see some high-quality, "real" 3D ones...

  19. Clive Galway

    Foveated rendering is not a magic bullet

    It's not gonna be popular with the live streamers if all their viewers only get a decent view of where the player's eyes are

    1. TechnicalBen Silver badge

      Re: Foveated rendering is not a magic bullet

      The stream is in VR? No. So they can output to 1080p fine for the whole scene, then add the extra 4k/8k render for just the one foveated render spot on the VR headset *only* (streaming software and VR headsets already support such features!).

      1. Clive Galway

        Re: Foveated rendering is not a magic bullet

        "So they can output to 1080p fine for the whole scene, then add the extra 4k/8k render for just the one foveated render spot"

        A Vive is 1920x1200, so the 1080p render could not be used for the non-foveated parts.

        The foveated part is not suitable for focusing on, it's only designed for peripheral vision.

        Besides, the whole point of foveated rendering is to not have to render all parts of the screen to a decent quality, so by rendering that part of the screen at full quality for the streamers, you negate the whole point of foveated rendering.

        1. Hargrove

          Re: Foveated rendering is not a magic bullet

          @Clive Galway

          Did you mean non-foveated in the third paragraph?

    2. BlueTemplar

      Re: Foveated rendering is not a magic bullet

      IMHO you're overstating the importance of live streamers...

  20. Cynic_999 Silver badge

    Offload to wetware

    Our brain is great at adapting to different situations. If you were to wear glasses that inverts the image, or swaps right and left eye, after a while of continuous use your brain adapts and the World looks completely normal again - you stop noticing. We also know that the brain fills in the gaps when there is visual information missing - we do not perceive the big dark spot that is always present due to the optic nerve's entry point. Various optical illusions show that a huge percentage of what we "see" is really created by our brain rather than seen by our eyes.

    I should think therefore that anyone using a VR headset regularly will experience the same effect - the brain will adapt and compensate for the shortcomings in the display and the virtual World will appear realistic.

    1. BlueTemplar

      Re: Offload to wetware

      Won't the need to switch between real wold view and VR view prevent this kind of adaptation from happening?

  21. Christian Berger Silver badge

    One solution for it is obvious

    Just track the eyes, then have 2 screens for your eyes, one that's low res and large for the background, and one higher resolution one which is optically placed at the point where you see the sharpest and at a focal distance that's close to the object you're looking at.

    This would dramatically cut back the amount of data you need to generate.

    1. TechnicalBen Silver badge

      Re: One solution for it is obvious

      I'm not sure what you are suggesting. If you have a high res screen, then why also add a low res one?

      If the screen is static, then it does not avoid the eye movement problem. In a VR headset, you can look straight ahead, but you can also move your eyes. So you need a screen that covers all the eye movement.

      A small screen in the middle, even if high res, would give you blurred tunnel vision in VR, and that would either make you feel sick, or look worse than a high res screen with the "best res" you can get.

    2. onefang Silver badge

      Re: One solution for it is obvious

      So obvious at least a couple of VR companies are trying that.

  22. TheGreatCabbage

    I really don't think that 90Hz is necessary, after using an Oculus Rift DK2 - with a 75Hz refresh rate - for extended periods. 75Hz is definitely good enough for me, but perhaps other people are more sensitive?

    Personally, I think VR headsets will be awesome when they get to 4K per eye. Maybe by the time 4K displays with a 90Hz refresh rate are available for VR headsets, the graphics technology will be almost good enough to handle it...

    1. onefang Silver badge

      "I really don't think that 90Hz is necessary, after using an Oculus Rift DK2 - with a 75Hz refresh rate - for extended periods. 75Hz is definitely good enough for me, but perhaps other people are more sensitive?"

      I think it largely depends on how likely you are to get ill. For people like me, who has only ever thrown up three times in my entire life, VR isn't any problem. And indeed, developing on my DK2, even when the code was in an initial technicolour yawn inducing state that made the client feel ill quickly, never had a problem. Oculus I think set the bar at 90 Hz for being best for everyone, but I think they where covering their arses a bit.

      1. Sir Runcible Spoon Silver badge

        Even if, like me, you are extremely* motion sensitive, you can earn your VR legs.

        As long as you stop when you start to feel queasy or you start to feel hot under the collar you can retry the next day and it will be marginally better. After a couple of weeks it's no problem.

        I didn't use my psvr for about a month, and then I got Skyrim. I started to get those hot under the collar feelings again (although not quite as bad as I did when I first got the headset) only took me 2-3 days and it's all good again.

        *if I get motion sick irl it can take about four hours for my stomach/head to settle enough to walk around.

  23. Deltics

    Humans may have foveated vision but we use it to observe a fully resolved world.

    This "problem" doesn't seem to affect 2D VR (i.e. TV/monitor output) from PC's/consoles - human vision is still foveated when viewing VR on a 2D plane, so the "problem" here is - I suspect - less about trying to make stereo-scopic VR "work" but rather to grasp desperately for an excuse as to why the technology has failed to take off as predicted despite being hailed as The Next Big Thing.

    If they ever do produce eyeball tracking foveated rendering I suspect that the result will be even worse - foveated vision perceiving a pre-foveated render.... I suspect the results will be worse, not better.

    1. Dave 126 Silver badge

      The article wasn't talking about what is needed to make VR fun, but rather what is needed to make VR indistinguishable from reality, so for that reason comparison to TVs and monitors is only of limited use.

      If we ignore that most people have two eyes, we can think about how we could try to render a real looking scene in the entirety of a users field of view using a huge bank of monitors (if they are low DPI then no problem - just use more of them and place them further away from the user!). The issue would still be processing power.

      We aren't conscious of only being able to focus sharply on a small area because our eyes move around a lot and our brain builds up an image that it presents to us. There's lots of optical illusions that illustrate this.

  24. HildyJ

    Why pixels? Why not color palette?

    Instead of reducing the resolution outside of the fovea area, why not reduce the number of color bits from full in the center down to 8 (4?, 2?) at the periphery?

    1. Dave 126 Silver badge

      Re: Why pixels? Why not color palette?

      And here was me thinking I'd never play a game displayed in EGA or CGA again! :)

  25. Destroy All Monsters Silver badge
    Pint

    That was a good read!

    +1 El Reg

    Have some Greg Egan in return (Permutation City, 1994):

    Paul uncovered his eyes, and looked around the room. Away from a few dazzling patches of direct sunshine, everything glowed softly in the diffuse light: the matte white brick walls, the imitation (imitation) mahogany furniture; even the posters -- Bosch, Dali, Ernst, and Giger -- looked harmless, domesticated. Wherever he turned his gaze (if nowhere else), the simulation was utterly convincing; the spotlight of his attention made it so. Hypothetical light rays were being traced backward from individual rod and cone cells on his simulated retinas, and projected out into the virtual environment to determine exactly what needed to be computed: a lot of detail near the center of his vision, much less toward the periphery. Objects out of sight didn't 'vanish' entirely, if they influenced the ambient light, but Paul knew that the calculations would rarely be pursued beyond the crudest first-order approximations: Bosch's Garden of Earthly Delights reduced to an average reflectance value, a single gray rectangle -- because once his back was turned, any more detail would have been wasted. Everything in the room was as finely resolved, at any given moment, as it needed to be to fool him -- no more, no less. He had been aware of the technique for decades. It was something else to experience it. He resisted the urge to wheel around suddenly, in a futile attempt to catch the process out -- but for a moment it was almost unbearable, just knowing what was happening at the edge of his vision. The fact that his view of the room remained flawless only made it worse, an irrefutable paranoid fixation: No matter how fast you turn your head, you'll never even catch a glimpse of what's going on all around you ...

  26. This post has been deleted by its author

  27. Ubermik

    Arent they overcomplicating this a bit?

    From how it read you need a certain amount of pixels for the eye and brain to not struggle, but then they try to drive every pixel independently which is where the bottleneck comes from

    Surely the solution would be to go "anadigilogue" a melding of analogue and digital

    make the screen with the required pixel density, but then feed into it a lower resolution image and have silicon that "smears" the picture across the extra pixels in either a gradient or hard transition as required

    This gives the eye the density it needs up close, but would allow the hardware to only feed in a monitors level of resolution which the hardware is already capable of doing

  28. FlamingDeath Bronze badge

    I know kungfoo

  29. Milton Silver badge

    Touching a nerve

    I feel some sympathy with the post about direct-to-brain interfacing. Having to jump through so many hoops to satisfy the annoyingly choosy Mk#1 Eyeball would just go away if we could plug into the Visual Data Bus (used to be called the Optic Nerve) and feed whatever the hell we like, properly formatted pix, to the brain. Right?

    But no. The problem is, as I understand it (i.e. no expert) that the eye effectively does a lot of the filtering, call it pre-processing work, before it reveals to the brain whatever it thinks the latter should be allowed to worry about. I believe this includes "fixing" the image, by for example inferring that areas of colour are sharply bounded if lines are perceptible between them—even if, in reality, there is bleed between the colours. Also there is selective blindness: simply ignoring quite large and otherwise noticeable features because something more interesting is the focus of concentration. There's also the question of relative sizes, whereby the eye reports erroneously on dimensions because it is taking cues from other parts of a scene.

    We've all seen these. The first is one reason early colour TV was able to use so little broadcast bandwidth: it relied upon the human eye putting in features that just weren't there, in the picture, ever. The second you'll have noted if you were ever fooled by the gorilla-behind-the-basketball-players vignette: your eye was following the frantic motion of the players and you never so much as noticed that a gorilla had walked across the court. The third is a favourite of trompe l'oeil and other illusions you can find all over the web. (And: how many of the decisions about the *distances* of objects are already made by the time the consciousness "sees" that heavily filtered, processed, image? Read up on why good 2D movies are already kinda 3D movies, for some answers.) My wild-assed guess is that the eye is making dozens if not hundreds of modifications to pre-process stuff for the consciousness, and we're just barely scratching the surface with current understanding.

    The eye isn't doing this in isolation from the brain, but the brain isn't doing it in isolation from the eye, either: in fact, arguably, the eye is a highly specialised part of the brain. So, tempting as it may be, we can't simply decode the Visual Data Bus and then shovel our own bits onto it.

    That said, there are some awesomely clever people working in the field of biological vision, and I don't doubt that we will eventually find a way to get images into the brain directly—though perhaps it were more accurate to say, "to get a detailed subjective impression of an optical environment accepted by the brain as valid".

    My guess, FWIW, is that when this happens VR will *still* not be good enough to fool the brain, or convince anyone that they are looking at reality.

  30. Thomas Gray

    Monochrome?

    Assuming the problem of fovea focus could be dealt with, would it save processing power to place the surroundings in monochrome, since rods only detect light intensity rather than actual frequency?

  31. BlueTemplar
    Thumb Up

    Great Article ! Some extra numbers :

    1.) Remember that putting our visual bandwidth in terms of bits/second involves making not only assumptions about resolution, but also about colors, brightness, and framerate too !

    (I'm assuming that the quoted "74 gigabytes of visual data are available to us each second" are assuming the sRGB color space and 90 Hz ? Source ?)

    2.) Japan's Broadcasting Corporation (NHK)'s research "claimed the tests showed 310 pixels/degree are needed for an image to reach the limit for human resolution" (which is a lot more than the quoted 120 pixels /degree)

    https://www.homecinemaguru.com/can-we-see-4kuhd-on-a-normal-sized-screen-you-betcha/

    (Though you can indeed see from the graph that you start indeed to get into diminishing returns over 80 pixels / degree - which also corresponds to the average 20/15 vision - but then it *also* looks like that the Nyquist sampling requirement doubles that number, and you end up with a minimum of the double : 160 pixels / degree.)

    That Samsung 850ppi display, put in a Oculus CV1 would result in ~23 pixels per degree :

    https://forums.oculusvr.com/community/discussion/comment/521849/#Comment_521849

    The US Air Force hypothetical 10,300ppi display would therefore result in 282 pixels per degree ? (notice it's close to NHK's research result!)

    1. BlueTemplar

      Re: Great Article ! Some extra numbers :

      On second thought, it's weird that a maximum angular resolution with (fixed) televisions in mind would be similar to the maximum angular resolution of (head-following) Head-Mounted Displays !

  32. ProgrammerForHire

    I might be wrong but the 90 Hz figure is computed considering you need 2 frames each time, for each eye

    So it's 45Hz overall

    1. BlueTemplar

      I'm pretty sure that you're wrong :

      https://thevrbase.com/why-does-the-90-hz-refresh-rate-matter-for-virtual-reality/

      You might confuse it with the "Asynchronous SpaceWarp" visual trickery that the Rift does that allows it to cut the framerate pushed by the graphic card in half to 45 Hz with hardly any noticeable difference?

      https://developer.oculus.com/blog/asynchronous-spacewarp/

      1. ProgrammerForHire

        This is confusing. 90Hz and 60 fps ? I thought the numbers should be equal ?

        1. BlueTemplar

          If I'm not mistaken that's 60 fps when talking about the video if it was shown on a regular screen, and 90 Hz for the actual refresh rate of the HMD's screens.

          Hence some trickery is involved : see my second link (I probably should have said 45 fps (and 90 Hz) in my previous comment ?)

  33. Mark Simon

    Correction

    Number of pixels.

  34. Indolent Wretch

    Maybe returning to the concept of a vector display (like asteroids and the old vectrex gaming system) would be a good idea. That didn't have pixels at all.

    Admittedly the world would like your in Battlezone but that's not necessarily a bad thing.

  35. Anonymous Coward
    Anonymous Coward

    I question the numbers

    I question th numbers in this article.

    Field of view 180 degrees - more like 80 i.m.h.o.

    All of our fine resolution takes place in the central one degree, shouldn't that be more like 10 degrees?

    But I agree we will need a high screen refresh rates to prevent nausea when using headsets.

    1. BlueTemplar

      Re: I question the numbers

      No :

      https://en.wikipedia.org/wiki/Human_eye#Field_of_view

      "For both eyes combined (binocular) visual field is 135° vertical and 200° horizontal."

      the high resolution area of the eye doesn't seem to have have a clear cut-off... (until the blind spot at ~15° ?) :

      https://en.wikipedia.org/wiki/Fovea_centralis

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019