My New Project & Thanksgiving Day Stream

Friday, November 21, 2014

This new project started out as practice project for later trying to get emulators displaying in VR scenes. And when I finally got working on it, it was with being for VRChat in mind. Things change, and well, here we are!

This will be somewhat familiar of a concept, but with some differences. Namely, you would think of ggodins awesome VRDesktop program. There will be some parallels, but I think their should be value in both.

What I've been working on is a Unity plugin/script to display your desktop in a Unity scene. My original intention was to just release this as a plugin for Unity devs to include in their own projects. I figured that in VR it might be nice to be able to press a button while you're playing a game or socializing, and bring up a image of your desktop. Keep track of Skype windows. Check a online map. See the time. Check a stream of your webcam and see nobody is sneaking up on you, or check a security cam to see if the pizza guy is here. Who knows. You couldn't interact with the windows on the virtual desktop screen like in VRDesktop, but there is at least some value in seeing.

I soon realized you could have some measure of interaction with the windows on your desktop. Like with many media players supporting global media keys on many keyboards. You could load up a movie, pause it, then load up a program with my plugin, press play and watch that movie inside VR. But similar to my original intention, you could play a emulator like this too. Just the emulator needs to support continuing to play when not the foreground window, and accepting gamepad input as a background window (RetroArch does this!).

That's nice, but your hands feel too tied still. People will naturally want to interact with their desktop if they see it. But seeing as my project is in Unity, it has some limitations compared to VRDesktop. If I were to allow you to click a virtual mouse cursor on a window to bring it to the foreground, doing so would cause the Unity app itself to go to the background, stop accepting input, updating, and maybe even minimize.

But that was just theory, and I didn't want to give up without at least trying. In the end I've come up with a way of supporting sending input to the windows. It's limited, but its something. You just position a cursor over a window/control, and the input you type/click will be sent to that directly. Without moving that window to the foreground. I've been able to chat in Skype with it, and even scroll the chat history down. I can navigate folders too. I haven't explored everything it can do yet. But atm some things don't work. And if a window is partly hidden behind another, it will remain as such.

I'm not giving up yet. My window recognition code could be better. The way I handle input could maybe see improvement. Maybe I can change the Z sort order of windows, without bringing windows to the foreground. Who knows.

Atm the goal is to release it as a demo itself. Currently I'm using the Theater scene from the Oculus Moblie SDK as its a good fit. But I may end up using something else. Later I might still release the plugin for other devs to use in their own projects. Maybe on the Unity asset store, maybe not. I just need to see I've anyone's interested in this before I make decisions. If you're a dev that would be interested in including it in your project, just let me know somehow. If you're a VR user that would just like to see most games/programs with something like this in them, please let me know too!

Live Stream!

Thanksgiving and Christmas day I sort of have a tradition. Often, for some reason or another I end up playing a Zelda game all day. So I was thinking this might be a time to show my demo in action! So Thanksgiving day, if you don't celebrate it or have any plans, come tune into my Twitch channel to see the demo in action. I'll likely be playing Zelda A Link to the Past. Maybe I can even get some people on Skype calls just to chat about whatever, while I play. I think I'll try to start around 12PM PST. And go until I'm done with the game. Hope to see you there! =)