Dev Diary (2/21/2014) – v0.3.3, Updated Networking, Persistent Worlds and a New Level

The last two weeks have been pretty busy. Last week I was bunkered down rewriting VRChat’s networking code, in an attempt to make the interpolation smoother and more accurate between clients. Once I was happy with the result, I moved onto something I was really excited about, the World Hub aka a persistent world that started looking more and more MMOesque. I put a good deal of time into building this world but had to take a break this passed week because of school work (midterms week). While I was studying for tests, one of my collaborators, Calen, finished up his first 3D environment for VRChat!

I’ve added the new environment and made a few small fixes to v0.3.3 which is available now.

Updated Networking

Initially, I was using a very basic form of interpolation. I was sending everybody’s player state to everybody else 10 times a second over TCP and interpolating between the last two states I received. The result was satisfactory, but it still opened itself to multiple issues including the “ice skating effect” where the animation and locomotion did not match up, jitter and lag. I started looking into more robust solutions and came across a TNet forum thread with some very helpful concepts and code to work with. It used the concept of synced times across all clients and using that, along with the ping time of the client to calculate the rate at which a networked player is interpolated. After a bit of tweaking, it ended up working really well!

Persistent Worlds

One feature I was really excited about was creating a persistent world consisting of a single instance of each chat room level. Therefore, the world would consist of the CoffeeShop, the Palace and our new Gallery level. Players could move from one to another by just walking across the street. The main problem to tackle here was that I wanted each building to be a different channel, so people inside of the CoffeeShop wouldn’t know about people in the Gallery. I spent a few days tinkering around with different solutions to the problem and got about half way done before I was interrupted by school work. During my break from VRChat, I started to realize that this persistent world starting looking more and more like an MMO, something I would not be able to build on my own. Therefore, I’m deciding to put this on the back burner for now.

New Level

Probably the most exciting news I’ve had this week has been that one of my collaborators, Calen, has finished his first VRChat 3D environment…the Gallery. The Gallery is an art gallery consisting of two floors, a balcony and lots of art! Here are some screen shots. 

Dev Diary (2/12/14) – v0.3.1 Bug Hunting

At the Sunday VR meetup, I was able to see whether the new update would hold it’s own when the server load increased…it didn’t do as well as I had hoped. The issues stemmed from an increased load on the server and users doing things I didn’t expect (even though I should have). The top issues I found were 1) players were sometimes not able to connect to the world hub channel 2) sometimes players in the world hub could not each other although others could 3) alt-tabbing from full screen mode on a Windows machine wreaked havoc on everything. Unfortunately I haven’t gotten the chance to bring in a large number of users to test again, but I will probably do so soon. 

Bug 1 – Players were sometimes not able to connect to the world hub

I haven’t pinpointed the exact cause of this bug, but I have some ideas. Until I get another chance to reproduce the issue, it’s hard to figure out exactly what is going on.  By the time players are trying to join the world hub channel, the player is already connected to the server and in their own “Pad” channel. Thus, switching channels shouldn’t really be that hard…unless there was a cap on the world hub channel that we hit….I’ll have to go check that.

Bug 2 – Sometimes players in the world hub could not see each other

This bug perplexes me. I’m not doing anything THAT different from v0.2.0 networking wise and I’ve brought up the issue with the developer of my networking solution, TNet. Until I see this issue again and am able to see what the heck is going on, there’s not much I can do now.

Bug 3 – Alt tabbing from full screen mode in Windows breaks everything

One of my dedicated testers, Khailz loves to break things. He discovered that when you alt tabbed out of full screen mode and come back a few seconds later, all sorts of nonsense occurs, including, but not limited to, random textures on your face, going into 3rd person mode, not being able to move, being able to move but being pushed back to your original place and many others. After some digging, it turns out that all of this was occurring because he was no longer connected to the server. First, I should have handled what happens when a user loses connection to the server and second, why was he losing connection to the server? Turns out it’s a Unity bug. Apparently full screen Windows builds have an issue with alt tabbing, aka the program stops (instead of running the background like it should), therefore not sending data to the server and timing out. For now I can only handle this on my end (send the user back to the starting logo so they reconnect to the server).

Development has been slow the last few days, but I wanted to fix all the issues before adding more stuff!

VRC Dev Diary (2/2/2014) – Mirrors, In-Game Console and Switching Avatars

The thing about software development is that if you don’t have an exact direction to go in, you’re probably going to spend a lot of time on some feature and then scrap it. That’s what has happened since my last post. Previously, I decided to take the GUI approach for joining servers and channels and abstracting that into walking through virtual doors that do the exact same thing. The result was cool. It worked well and allowed the user to stay in VR throughout the entire experience.

Centralized Player Hub

After showing off my new prototype to a fellow VR enthusiast and 3D environment designer, he mentioned how it’d be cool if instead of the player having to choose a server and channel from the offline setup room, there was only one door from this room which automatically had everybody join the same persistent channel on the same server. That way, at least in the beginning, everybody is congregated together with zero hassle. From this central networked hub, there would be other doors leading to other persistent channels. Eventually I think it’d be really cool for users to be able to upload their own environments to become one of these other channels.

Mirrors

So I guess I should say my previous work wasn’t completely scrapped, as I just tweaked the old implementation to work for the new one. Next on my list was making the offline setup room have something more than a single door leading to the central hub channel. I wanted players to do all of their setup in the setup room (:P), including things like selecting avatars and setting names. The first issue was how the user was going to see their avatar as they changed it. One option would be to have the user have an “out of body experience”  that would allow them to move around the avatar and see it from any direction. Another option would be to add mirrors, allowing the user to see what they looked like in real time. I liked this idea a lot. The main issue here was getting a mirror script/shader/material, which I found after searching the inter-webs for a little bit. The wiki gives the shader code for a mirror but I think the newest version of Unity has a built in mirror shader under FX/Mirror Reflection. I went ahead and used the built in shader but still used the wiki’s mirror script. This worked well enough…for if I wanted more than one mirror in the scene I had to create multiple mirror materials (otherwise I get some really weird reflections). There is room for improvement in the future, but I quite pleased with the results.

Mirror

In Game Console

One of the issues I’ve started to face as a lone developer is that I don’t have enough time to add everything I want to add. Adding features take time and what takes even more time is designing a way for the user to easily use those new features. In a sense there are two main parts to developing a feature for a program like VRChat – first you have to write the code to actually do whatever it is you want to do (ie. change avatars) and then you have to write code that allows the user to access that that feature (ie. button input or in-game menus). For instance, let’s say I’ve implemented a way to switch avatars. All you have to do now is call SwitchAvatar(string avatarName). Great! Except…how do I let the user do this? There is no correct solution, but no matter the design choice you make, it will take time. A lot of time. And not fun time. I dislike GUI coding…a lot. Therefore, I decided that the easiest way for me to pound out some new features is to completely ignore fancy VR user input and just use a good ol’ fashioned in game console!

The idea struck me when I saw a post to the Unity3D subreddit regarding an open-source Quake style in-game console. User mikelovesrobots had built an in-game console for Unity and man is it cool. I downloaded the package, dragged in the folder and bam, after adding a single prefab I now could bring up the GUI console on screen with the click of the ` key. Adding commands to the console is super easy and I knew this would work perfectly with VRChat. Only one, small, itty bitty issue stood in my way….this console was a full-screen GUI, and that wouldn’t work for VR.

The obvious thing to do was look at how Oculus took a normal Unity GUI menu (OVRMainMenu) and transformed it into an in-game menu that looked really good in the Rift. Unfortunately the OVRMainMenu script is one of my least favorites to work with because I feel that there’s a ton of mumbo jumbo I don’t quite understand (also did I mention I really dislike GUI coding?). I dedicated my entire Sunday to figuring out what witchcraft allowed Oculus to do what they did and after a few hours of tinkering, commenting out code, head-to-desk contact and hacking together code chunks, I finally got things working. I actually don’t understand fully why it works but from what I can tell Oculus is manipulating the GUI.matrix and RenderTexture and applying it to the OVRCameraController to get the GUI to render in both eyes. At this point my console works but as much as I hate to say it, one day I will have to revisit this code and clean it up.

In-Game Console

Switching Avatars

In VRC v0.2.0 I used a drop down menu to allow users to pick and choose which avatar they wanted to use. Based on the chosen avatar, I would instantiate an prefab unique to that avatar. That means I have a saved prefab per avatar. Not a good design. Users also could not see what the avatars looked like in v0.2.0…all they had to go on were names. With the Setup Room, users should be able to easily switch between avatars and see how that avatar looks before joining the online world. My new solution to avatar switching is a single prefab with multiple sets of graphics attached to it. Still not the best solution as I have as many avatar graphics on that single prefab as I do avatar choices, but it’s a step in the right direction. This way, when the user is looking the mirror and wants to change avatars, the switch happens instantly because all we’re doing is turning off the old avatar graphics and turning on the new one. There are few other things that must be updated as well that have to do with the avatar rig and the animator, but it wasn’t too difficult or complicated

.VRCPlayer Graphic Hierarchy

VRC Dev Diary (1/31/2014) – Redesigning the Server/Channel Lobby (aka The Hub)

The first thing I wanted to do post v0.2.0 was redesign the Server/Channel Lobby GUI. The logical option that came to mind was to make the existing GUI more VR friendly. As I started designing my implementation, it dawned on me that the coolest way of implementing a lobby system would be the actual VR way, where there were virtual doors that lead to a server or a channel. And that’s where I am now.

Virtual Portals to Servers and Lobbys

All I’m doing here is abstracting the “Join” and “Create” buttons on the old GUI and turning those into doors that correspond to servers and rooms. When you walk into a door (I’ve been calling the portals), a menu pops up confirming that you want to enter. This is where I needed to make a design decision on how to get user input. I could do mouse (ew), keyboard arrows, or have items be selected based on head gaze. I went with the head gaze option and implemented it pretty easily using RayCasts. Things are coming along pretty well and now I’m just waiting on an cool HUB environment to use!

VRC v0.2.0 Released

Hey everybody! After lots of feedback and development, I’m finally ready to release the next version of VRChat. You can download it from my site[1] (I’m using bitly to monitor download stats and reddit auto-removes posts with bitly links). Also, this will be the last VRChat build you’ll have to download because I’ve added a patching service!

IMPORTANT: v0.1.0 No longer works with the server infrastructure. Please be sure to upgrade!

If you get a chance to try it out, leave me some feedback![2]

Creating/Connecting to a server

  1. Server Address – Where you can start your own server or connect directly to the external IP of somebody who is. NOTE: Only users with a router that supports uPnP or manually forwards ports 5127-5129 will be able to host a server.
  2. Server List – Where a list of created servers will appear (if there are any). You can join any of these servers and join/create a room (or channel) on that server.
  3. I’m currently hosting a dedicated server at 129.59.30.56

Creating/Joining a Room (Channel)

  1. You can create your own channel or join one that’s already been created
  2. If you try to join a channel and nothing happens, it’s because there’s a password and you’re inputting it incorrectly.
  3. Note that some levels (including Palace) take a bit of time to load. Once you Create/Join a map, give it a few seconds to load. Timeout is set at 60 seconds.

Mouse/Keyboard

  • Hold “V” – Voice Chat
  • Press “M” – Return to channel menu
  • WASD for movement
  • Press “B” – Reset Rift orientation

Xbox Controller (Note: Must be connected before VRChat launches)

  • Hold “left bumper” – voice chat
  • FPS style movement scheme
  • D-Pad Down – Reset Rift orientation

Change Log

v0.1

  • NEW: Setup lobby server so users can find servers
  • NEW: Basic networked avatars (ie. position and rotation are tracked)
  • NEW: Nametags float above players
  • NEW: 3D Voice Chat that lights up nametags when player talks
  • NEW: Allow for both rift and non-rift compatabilty
  • NEW: Users can host servers (if they have a uPnP router)
  • NEW: Users can create rooms(channels) on a server w/ passwords

v0.2

  • NEW: Server list now shows number of active players on server
  • NEW: Separate head and body movement
  • NEW: Avatar animations
  • NEW: Xbox Controller support (left bumper to talk)
  • NEW: Removed player-player collisions
  • NEW: Added patch downloader
  • NEW: Added two more avatars
  • NEW: Added “Palace” level

VRC Dev Diary (1/23/2014) – Plans for the Future

As of yesterday I felt like I was at the point where I was ready to release VRChat v0.2.0, but then I got thinking. Do I really want to do weekly updates, that don’t contain much new content, and make users download a new version of the app every single time? I didn’t think so. Therefore, I decided to prolong the next release of VRC until there’s a significant update, and don’t worry…after the next update you won’t ever have to download another version of VRC again (that is unless I royally screw up). I plan using Crafty, an incredible patching system I came across in the Unity Asset Store. All future updates (after the next one) will be downloaded inside the app.

Anyways, I’m up in Seattle for the weekend and figured this would give me some time to think about the future of VRChat. In it’s current state, it’s nothing more than a simple networked 3D game that almost anybody could have put together. I just happened to be the first to publicize it. There is no doubt in my mind that other individuals (or companies) are working on the “Killer App for VR”, which in my opinion would be an ever expanding MMO style world that acts as a hub for all other VR content…similar to the Oasis from Ready Player One. However, I still plan on working on my own solution for said application. I’ve talked with other developers who plan on using the Torque3D game engine to work on an open source solution and I think we’ll end up working very closely together. Depending on which seems to be the better option after more development, we may join forces to continue developing.

As I continue development on VRChat, I’ve started to realize that I can’t do it all on my own and need to form a team who work very closely together on this. I know I wanted to make this an “open-source” project, but it seems like a hard task for such a networked application. For now I need to find the right group of people to take this on with me…I’ll figure it out soon.

VRC Dev Diary (1/19/2014)

My plan for yesterday was to get animations working but I got distracted with the shiny new toys that Unity delivered to us (the Sample Assets Package). Therefore, avatar animations were pushed to today. I also chatted with a couple of other VR developers who were interested in building a similar application in Torque3d, an open source game engine.

Avatar Animations

I wanted to get very basic animations for the Carl (floating Jesus) avatar, including standing idle and walking. There’s an animation library called Mixamo that is supposed to have any animation you can think of and is supposed to work with most rigged avatars. I looked into it and was prepared to spend a little bit of money to get professional grade animations, but the Unity Mixamo store plugin wouldn’t work. I gave up pretty quickly and decided to take a peek at Unity’s tutorial on their animation solution, Mecanim. The tutorial (like most of Unity’s tutorials) included a demo project that came with animations for standing idle, walking, running, turning, jumping and waving. Bingo. I downloaded the demo package, watched the tutorial on how to use Mecanim (it’s really cool) and added standing idle, walking forward, and walking backward animations to our carl avatar. The only annoying part was that I had to tinker a bit to get the walking animation to match the speed the avatar was moving as well as the head bob and footsteps.

The next step was to get animations working over the network (or appearing to work over the network). A non-networked animation knew which animation to play based on the input from the keyboard. However, a networked player doesn’t have keyboard input to work with, only positions. So my solution was to play the idle animation unless the networked player’s position was being updated. Simple enough. The next issue is whether we should play the walk forwards or backwards clip. We can figure out which one needs to be played based on the orientation of avatar and which direction the new, updated position is.

Torque3D

Torque3D is an open-source game engine that focuses on the first person shooter genre. It has Rift support and gives users complete control over the engine since you have access to the source code (something you don’t get with the regular Unity Pro). Unity3D is super powerful and has more than enough tools to take VRC to where it needs to be, but depending on how far VRC goes, access to source code can be a very powerful thing. I’m not dropping everything for this new engine but I am going to take a look and see if it’s something to think about.

VRC Dev Diary (1/18/2014)

I was able to test the separate head and body tracking last night with a few redditors and it seems to work pretty well. It adds a level to the immersion you wouldn’t otherwise get. The next step I think is to add animations for standing idle and walking around. But before I do that I wanted to look at Unity’s new Sample Asset package they released recently.

Unity Sample Assets

I took a look at Unity’s beta sample assets package they just launched on the Asset Store and I’ve got to say they are really impressive. Tons of new stuff including crazy particle effect examples, 3rd person car and plane demos and my favorite, a completely redesigned First Person Controller WITH A RIGIDBODY!! No more dealing with the CharacterController as your form of interaction with the environment. Ugh. Oh and it gets even better. The new First Person Controller has built in head bob and walking sounds….which is perfect for VR. Thus, I spent the day integrating the new First Person Controller into the current VRC avatar.

Head Bob

The way head bob works is that the camera moves up and down as the player walks around and it works great…unless you have graphics on your First Person Controller (FPC). With the addition of graphics, the camera will sometimes bob inside your body, which means you can see the inside of your head and mouth and stuff…not what we want. So I basically spent a TON of time tweaking the offsets to get it just perfect. My guess is that I’ll eventually change the way I’ve implemented avatars, but for now I really like what we have.

VRC Dev Diary (1/17/2014)

There has been a ton of great feedback surrounding VRChat v0.1 and it’s shown me how much the community cares about VR and having a solid VRChat program. That being said, in an attempt to be as involved as possible with that community, I’m going to start doing a daily blog about the bugs and features I accomplish each day in VRC. That way, everybody knows what I’m up to and can help me get back on course if I start slacking. And in the future, we can start having polls on what features people want to see!

Last night, I jumped back into development. I use Trello as my project management system and have a ton of cards under “ToDo”. Towards the top of these, however, are showing the number of active users in a server and separating head rotation from body rotation.

Showing the number of active users in a server

Being able to see the number of users in a server before you choose one will not only help y’all decide which servers are worth joining but also helps me when I’m debugging or testing. I honestly thought this was going to be a challenge because of the way my networking solution, TNet, is set up. My first thoughts on a solution were hey, I can “join” the server, loop over every channel in the server, grab the number of players in each channel and add them all up! This was a poor solution for a number of reasons. When I finally decided to actually spend a little time deciding how getting the number of players on a server should actually work, I realized that when the client requests the server list, I should also send the number of players in the server. Easy. Now I have to dive into TNet’s server code and see where the number of players is stored (if at all) and the best way to return it to the client (since we’re networking at a pretty low level using packets). After five minutes of reading through the server docs, I realize the server already returns the number of players in it! I just wasn’t using that variable on the client side. Score. Now I just have to find a pretty way of presenting it to the user…I really don’t enjoy GUI stuff.

active_players

Separating head rotation from body rotation

Currently in VRC, when a Rift Player turns their head in the real world, it translates to a full body turn in the virtual world. This was okay for v0.1 but what we really want is real world head orientation to match up with virtual world head orientation. I didn’t really know how to approach this problem. I don’t have much experience with 3D models and don’t know how they work or how you manipulate them. Therefore, I reached out to the VR community for help. I figured I’d need somebody to build an avatar from the ground up with animations and a special rig that would allow for Rift control.  Calen, a VR enthusiast and redditor, reached out to me and said he could be of assistance. After investigating the floating Jesus model I had downloaded from the Unity Asset store, Calen realized that I could use that same model to do everything I needed. I quickly started to poke around the asset and found out he was right! It’s always a relief when a feature you think is going to be super difficult turns out to be super simple :) I still need to get networked head rotations working though.

head_rotation

Stay tuned for more updates!

VRChat v0.1 Released

Hey guys! I’m the developer of VRChat, the virtual reality meeting space, and I have some great news! VRChat v0.1 is now available for download! The application is still in its VERY early stages and bugs may be found. If you do stumble upon some issue, or want to leave a suggestion or want to help with development, feel free to fill out a feedback form.

Features:

* Virtual coffee shop environment
* Anybody can host a server/create a “room” up to 16 people
* Private rooms, secured by a password
* Networked Avatars (no animations yet)
* 3D Voice Chat
* Name Tags
* Both Rift and Non-Rift Compatable

Windows
Mac