Using VRTK toolkit with AC

DADA_universe

I've looked up as much as I've been able to find of forum posts here on using VRTK toolkit with AC. I noticed @Alverik worked on a project using VRTK. Could you please kindly help with suggested best practices for getting the integration to work from your experience? The last post I saw was from 2017 so I don't know if some things have changed in AC which might mean a different workflow. Please anyone else with this experience, do share. The main reason for using VRTK is to be able to develop and test the VR integrations as I do not have a HMD at hand.

I built the game originally with AC, now I want to use VRTK to adapt it to VR, final destination would be Occulus Go. I want the player to be able to trigger pick ups and open access doors using controllers to trigger the hotspots I already set up in AC. I'm also wondering about changing AC menus (there are 42 of them currently) to world space. Is this necessary / practical for all the AC menus in VR? Or should I choose only those ones that make sense when interactable in 3d space? Should menus that just give you updates on gameplay remain in screen space or will that jar the experience? Please help with some tips while I keep tinkering with the toolkit. Thanks!

ChrisIceBox

AC doesn't offer official support for VR platforms, but the Manual does provide a "Working with VR" chapter for those who wish to attempt it on their own.

DADA_universe

Thanks Chris, I've looked up past threads where this was referred to and I have tried it out. I'm however hoping to get some insight from those in the community who have tried using AC with the VRTK toolkit

DADA_universe

Archiving some posts by @Alverik on the subject, here for future reference:

Working with AC and VRTK

Resolving Main Camera VR conflict with VRTK

Video play test of adventure game he worked on using AC and VRTK

ChrisIceBox

I'm afraid links don't seem to get saved when editing a post - you're best off pasting urls in directly.

DADA_universe

Thanks, I didn't realise that.
Reposted below:

Archiving some posts by @Alverik on the subject, here for future reference:

Working with AC and VRTK: https://adventurecreator.org/forum/discussion/5661/pyramid-2016-demo-now-out

Resolving Main Camera VR conflict with VRTK: https://adventurecreator.org/forum/discussion/5791/maincameravr-vrtk-conflict

Video play test of adventure game he worked on using AC and VRTK: https://www.youtube.com/watch?v=8EWqpCOswxI&feature=youtu.be

DADA_universe

Got in touch with Alverik and he was quite helpful with some tips, which I'll copy out here below, first general advise about VR:

"When using VR with AC you have to setup AC as if you were using a custom character controller. I was also using a custom way to run hotspots and actionlists so I had also configured it to Custom

You're going to want to find a friend or someone (if you don't have a headset) to help you with Player setup, because setting up a real VR player takes a little tweaking. Seeing the simulator work doesn't mean the actual player's are going to work.....

...3rd person VR requires a little more finesse. Moving the player in VR (AKA the camera), has to be done carefully or else you'll get the Player nauseated...

...a lot of starting players get nauseous from just Touchpad walking, and having the player follow the Character around continuously could be an issue if you aren't careful (if your code to get the Player to follow the Character isn't smooth it could be a problem).....

.....Remember the first, most important rule in VR is to never take control of the camera away from the player. You can move the player around, but never the camera itself.

well, you better leave the camera in it's default position. Rotating or moving the camera in the prefab is moot in VR. The camera's pos/rot will be overriden by the player's real head position and rotation.

3rd person VR is basically a first person Player that "follows" a character....

...(Well, position might be not be overriden in headsets that don't have RoomScale, not sure what would happen with Oculus Go, since I don't have one of those...).(edited)

Have you at least tried VR before? I highlight recommend at least trying it if you want to develop for VR (maybe at a kiosk or a friend's). I made a lot of wrong choices when I made my first AC-VR integration because I didn't know what VR was like..."

DADA_universe

Specific to VRTK and AC: 

"...Well, depends on how your own codebase works. To avoid having to set scripts on all your players (I had sim, Rift and Vive) you'll want to add your custom scripts (if any) at runtime (by placing them in the same objects you find VRTKs's scripts). This makes it easy to access the hands. You will basically be accessing the ControllerEvents scripts to Access input.

If you have UI you'll have two options: convert them to Worldspace then 1). manage the canvas location in 3D space, or 2). create an object to anchor them to (like a PDA, laptop, book, etc).

Unfortunately Overlay UI and general OnGUI UI isn't visible in VR. It's worldspace UI or nothing. You can fake overlay UI by parenting the canvas to the camera, but I'd avoid that. Overlay like UI can make people sick in VR.

You can now natively access some generic VR input by creating axis in the Input manager and calling those in code, but the setup is rather long and annoying. It's still easier to tap into VRTK or the SDKs themselves to get VR input.....

https://www.dropbox.com/s/ck6t77o2zqe2zmu/ACVR_ActionlistAssetTrigger.cs?dl=0

That'll teach you what you need to access VR input in scripts using VRTK (the link above). Anyway, be careful about input, some inputs only exist for some devices. For example, Vive controller's have less buttons than the Rift's (which makes Button Two always return negative). I think the Go also has less buttons than the Rift, so you'll have to play around with it."

DADA_universe

Thanks to Alverik for taking the time to help with tips. 

The script opens up slots in the inspector you could plug action lists into, to react to the different VRTK inputs, I'm still trying to think through how to take advantage of that, ditto sorting out how to set up AC to use the VRTK character controller rather than its own.

DADA_universe

Comments seem to be queued for approval.....is this a bug or a feature?

Edit: It seems to happen when a link is pasted in the body of the comment rather than embedding it in text.

DADA_universe

Another useful find:

"Ummm, just letting everyone know (for those who don't want to do any extra scripting), that the idea to use the hotspot detector to make only hotspots that "touch" the controller activate, really does work. But the result of my testing were a little inconsistent. I attached Hotspot scripts on some walls (which are just stretched cubes), but for some reason the hotsposts only activate when the hotspot detector is contacting/intersecting right at the center of the wall. Somehow the Hotspots don't activate if the detector is entering/intersecting only the edges (even though the hotspot should encompass the whole wall). This happens even if the Hotspot detector is not attached to the controls so I'm guessing something else is causing it. It's like the detector needs to be just in the right place inside the hotspot to activate it (by the way, the hotspot detector is like the size of a football, not the usual player size)...

Still using the detector does work and gives the illusion that the controller is activating the hotspot.... We've also decided we want only the right hand to activate hotspots so I'll keep this option in mind. But for now I'll just make a custom script instead, cause sadly I can't afford the small inconsistencies in using the Player Vicinity/Hotspot Detector method. It might not be a problem for some other projects though." -Alverik

DADA_universe

I'm trying out with the ClickByHoveringCursorExample script which initially threw off as it was not working for reasons I could not figure out, but I tried again, starting with a backup copy of the project and what I did was to put the script on the first person camera's 'look at' object (using the AC mainCameraVR prefab this time, no VRTK at all). I also attached a mesh filter and mesh render to it, so it could show a cube in the scene. This now works. I am able to trigger hotspots with this by hovering over them without any mouse clicks or any other input. Yet to try with menus, but its probably safe to assume they would work, its just the onerous task of switching over 40 of them to world space that I now have to contend with. I however would like the custom cursor (the cube on the 'look at' empty object) to give feedback to indicate when its hovering over a hotspot. I thought getting it to rotate would give better visual presence than highlighting in this case, so I placed a rotate script I have on it and tried to access the point in the ClickByHoveringCursorExample script, just before where the InteractionA input is called, to get it to rotate for a few seconds (using a coroutine to get the delay) before the hotspot gets triggered.  This does not work, though when I try to do it in the OnSelectHotspot function (without the delay coroutine), it starts to rotate when the hotspot is hovered over, but is kind of choppy and seems to stop the hotspot from being triggered. Any advise please on a more elegant way to achieve what I'm after?

DADA_universe

if (hotspotHoverDuration <= 0f)

                {

                    hotspotHoverDuration = 0f;

                    

                    if (hoverHotspot != null && KickStarter.playerInteraction.GetActiveHotspot() == hoverHotspot)

                    {

                        hoverCursor.GetComponent<toggleRotate>().startTime();

                        StartCoroutine(Wait());

                        KickStarter.playerInput.SimulateInputButton("InteractionA");

                        elementOverIdentifier = "";

                    }

                }

That's where I thought I could get the rotation started, it does not work.

DADA_universe

private void OnSelectHotspot (Hotspot hotspot)

{

hoverHotspot = hotspot;

hotspotHoverDuration = hoverDuration;

 hoverCursor.GetComponent<toggleRotate>().startTime();

        }

This gets some kind of jerky feedback when the hoverCursor hovers over the hotspot, but seems to block the hotspot trigger from executing.

ChrisIceBox

You're showing calls to methods on custom scripts that we have no way of commenting on.  If the Hotspot is being blocked somehow, make sure that your cursor's layer is on Ignore Raycast, and that you're rotating a child object so as not to interfere with the main script object.

DADA_universe

Thanks Chris, I followed your advise with Ignore Raycast and making the rotated object a child object. Rewiring things that way somehow got me rethinking my approach, so I used a coroutine in the object rotation script, created start and stop functions that control the coroutine and accessed these through AC trigger action lists using the object>call event action for the on trigger enter and on trigger exit situations. No more messing around inside the ClickByHoveringCursorExample script. It works as expected. The cube /  VR cursor now rotates okay when hovering over a hotspot, it also gets highlighted for good measure. Both rotation and highlighting get disabled when the cursor is moved away from the hotspot. Screen grabs from the inspector below: 











Relevant functions from the toggleRotate.cs script:

     public void startSpin()

        {

        StartCoroutine(Spin());

        }

        public void stopSpin()

        {

            StopCoroutine(Spin());

        }

    IEnumerator Spin()

        {

        transform.Rotate(0, 0, rotationSpeed * Time.deltaTime);

        yield return new WaitForSeconds(0.1f);

    }

Next challenge is to figure out a way to get hotspots to 'attract' the VR cursor and make it snap to a locked on position after hovering for a few milliseconds. Trying to achieve this because I noticed that it gets a bit tricky getting the VR cursor to lock on to the hotspot manually, especially using a gaze type cursor arrangement like the ClickByHoveringCursorExample script affords.