This page was last modified on 24 January 2020, a better soluation “3. Based on ARFoundation” has been added which is recommended to read.

该页面的最新修改时间为2020年1月26日,新增第三种实现方法“Based on ARFoundation”,推荐跳至此处。

Jump to “3. Based on ARFoundation”


There is an ARKit demonstration named Portal created by Japanese developer Kei Wakizuka. By touching screen, the portal will appear in the real space in the screen with correct perspective and space distortion effect. Through the portal, there is a digital world, but is an immersive one because the audience can also enter the world, and the real world will be behind the portal.

Now there is a variety of similar projects, and most of them are named as Dokodemo Door which is a magical tool in the Japanese manga Doraemon. In the manga, by the Dokodemo Door, Doraemon and his friends can go to anywhere.

Most Dokodemo Door projects are based on SLAM technology, for example, ARKit, the reason is if the audience use Image-Recognization-based AR technology, it is not so easy to enter into the digital world because the audience has to hold the device aiming on the image which will be a terrible experience. But I will keep my experience on both technology, there must be more than one possibility  as Dokodemo Door.

My experiment is based on Vuforia and ARKit, here are two results, and you may see the difference of two design logics:

Feb-07-2018 21-31-11.gif

To create the negative space is the basic step of this kind of AR experience. Now, I only find one method using DepthMask shader. The idea is “a mask object using the Depth Mask shader. This object will be drawn just after regular opaque objects, and will prevent subsequent objects from being drawn behind it.”

1.Based on Vuforia

Updated in 2018.2

1.Components

  1. A target image, which is only necessary toImagine-Recognization-based AR, and it should be a little bit lower than the hole;
  2. A closed 3D object with a hole (I created in C4D) which is used as the entrance for audiences, in my example, it is a cylinder. DepthMask.shader should affect this closed space, to make it as an invisibility cloak to hide other component inside;
  3. An inner box with out top side, it will be the most important part, and may have better construction method if you wanna realize a virtual world, but which is not necessary in my example;
  4. A ball with ShowInside.shader.
未标题-1

2.Shaders

1.DepthMask.shader

 Shader "Custom/DepthMask" {
  
     SubShader {
         // Render the mask after regular geometry, 
         //but before masked geometry and
         // transparent things.
  
         Tags {"Queue" = "Geometry-10" }
  
         // Don't draw in the RGBA channels; just the depth buffer
  
         ColorMask 0
         ZWrite On
  
         // Do nothing specific in the pass:
  
         Pass {}
     }
 }

2.ShowInside.shader, here is the documentation from Unity about how to cull front or back (normally, it is default to cull the back, so you can’t see anything inside a 3D object).

 Shader "Custom/ShowInside" {
     Properties{
         _Color("Main Color", Color) = (1,1,1,1)
         _MainTex("Base (RGB)", 2D) = "white" {} 
     }
 
         SubShader{
         Tags{ "RenderType" = "Opaque" }  
         LOD 100
 
         Pass{
         Cull Front    
         Lighting Off
         SetTexture[_MainTex]{ combine texture }
         SetTexture[_MainTex]
             {
             ConstantColor[_Color]
             Combine Previous * Constant
             }
         }
     }
 }

2.Based on ARKit

Updated in 2018.2

1.Components

  1. 2 closed objects with holes in the same direction and with different sizes: the outside one should be a little bit larger than the inside one, as the outside one’s function is to hide the inside one by DepthMask.shader, and the skybox material should be apply on the inside one to create a scene with depth of field;
  2. A doorframe for the hole to prevent the edge from being too shape. It can be a real doorframe with high quality texture or a black hole with dynamic shader;
  3. A scene with several objects to create spatial sense of hierarchy.
未标题-1

3.Based on ARFoundation

You can get Unity example project here.

Updated in 2020.1

I learnt from these two tutorials:

  1. Guidev’s AR Portal Tutorial with Unity
  2. Pirates Just AR’s How To Unity AR Portal

1.Components

It’s a tutorial for AR Portal, so please make sure you have set up an AR environment in Unity first, including installed ARFoundation, ARKit (or ARCore) in Package Manager, and created AR Camera in the scene.

1.AR Camera

  1. Add Component “Rigidbody” and “Box Collider” to AR Camera
  2. Uncheck “Use Gravity” and check “Is Kinematic” in “Rigidbody”
  3. Check “Is trigger” in “Box Collider”
  4. Set “Box Collider”‘s size to x 0.1, y 0.1, z 2.5
  5. Transform “AR Camera” to prefab ※

2.InnerWorld

  1. Create as usual (it’s not important in this tutorial)
  2. Make sure the materials using in any gameobject in InnerWorld except Sky or other special objects, should use the shader “SpecularStencilFilter.shader” which can be found below.

3.Door

  1. The frame of the door can use any materal but the materials using shader “SpecularStencilFilter.shader” or “PortalDoor.shader” which you can find below
  2. Create a gameobject Plane or Quad and rename it as “Portal”
  3. Add component “Box Collider” on Portal and adjust the size
  4. Select Portal>Box Collider>Is Trigger
  5. Add component “PortalManager” which can be found below
  6. Drag “InnerWorld” into Portal>PortalManager>Inner World
  7. Adjust Portal>PortalManager>Materials, and drag every materials using in the “InnerWorld” in
  8. Add the material using Shader “PortalDoor.shader” on Portal

4.SkySphere

  1. Create gameobject “sphere” and rename “SkySphere”
  2. Add component “Video Player” on SkySphere, and drag your 360 video (which can be easily found in Vimeo) into SkySphere>Video Player>Video Clip
  3. Add the material using Shader “Custom/Sky” on SkySphere (The shader can be found below)
  4. Make sure the Stencil Comp of the material has been chosen “NotEqual”

2.Shaders

1.PortalDoor.shader

// (c) 2020 Tongzhou Yu

Shader "Custom/PortalDoor"
{
	SubShader
	{
		Tags { "RenderType"="Opaque" }
		LOD 100
		Zwrite Off
		ColorMask 0
                Cull off

		Pass
		{
			Stencil{
				Ref 1
				Comp always
				Pass replace
			}

// To further understand Stencil, check the link (EN):
// https://docs.unity3d.com/Manual/SL-Stencil.html
// or (CN)
// https://blog.csdn.net/linjf520/article/details/94867519 

			CGPROGRAM
			#pragma vertex vert
			#pragma fragment frag
			
			#include "UnityCG.cginc"

			struct appdata
			{
				float4 vertex : POSITION;
			};

			struct v2f
			{
				float4 vertex : SV_POSITION;
			};

			v2f vert (appdata v)
			{
				v2f o;
				o.vertex = UnityObjectToClipPos(v.vertex);
				return o;
			}
			
			fixed4 frag (v2f i) : SV_Target
			{
				return fixed4(0.0, 0.0, 0.0, 0.0);
			}
			ENDCG
		}
	}
}

2.SpecularStencilFilter.shader, modification based on Unity’s Built-in Shaders

// (c) 2020 Tongzhou Yu

Shader "Custom/Specular Stencil Filter"
{
    Properties
    {
        _Color("Color", Color) = (1,1,1,1)
        _MainTex("Albedo", 2D) = "white" {}

        _Cutoff("Alpha Cutoff", Range(0.0, 1.0)) = 0.5

        _Glossiness("Smoothness", Range(0.0, 1.0)) = 0.5
        _GlossMapScale("Smoothness Factor", Range(0.0, 1.0)) = 1.0
        [Enum(Specular Alpha,0,Albedo Alpha,1)] _SmoothnessTextureChannel ("Smoothness texture channel", Float) = 0

        _SpecColor("Specular", Color) = (0.2,0.2,0.2)
        _SpecGlossMap("Specular", 2D) = "white" {}
        [ToggleOff] _SpecularHighlights("Specular Highlights", Float) = 1.0
        [ToggleOff] _GlossyReflections("Glossy Reflections", Float) = 1.0

        _BumpScale("Scale", Float) = 1.0
        [Normal] _BumpMap("Normal Map", 2D) = "bump" {}

        _Parallax ("Height Scale", Range (0.005, 0.08)) = 0.02
        _ParallaxMap ("Height Map", 2D) = "black" {}

        _OcclusionStrength("Strength", Range(0.0, 1.0)) = 1.0
        _OcclusionMap("Occlusion", 2D) = "white" {}

        _EmissionColor("Color", Color) = (0,0,0)
        _EmissionMap("Emission", 2D) = "white" {}

        _DetailMask("Detail Mask", 2D) = "white" {}

        _DetailAlbedoMap("Detail Albedo x2", 2D) = "grey" {}
        _DetailNormalMapScale("Scale", Float) = 1.0
        [Normal] _DetailNormalMap("Normal Map", 2D) = "bump" {}

        [Enum(UV0,0,UV1,1)] _UVSec ("UV Set for secondary textures", Float) = 0

// 1/3 Changes:
		[Enum(CompareFunction)] _StencilComp("Stencil Comp", Int) = 3


        // Blending state
        [HideInInspector] _Mode ("__mode", Float) = 0.0
        [HideInInspector] _SrcBlend ("__src", Float) = 1.0
        [HideInInspector] _DstBlend ("__dst", Float) = 0.0
        [HideInInspector] _ZWrite ("__zw", Float) = 1.0
    }

    CGINCLUDE
        #define UNITY_SETUP_BRDF_INPUT SpecularSetup
    ENDCG

    SubShader
    {
        Tags { "RenderType"="Opaque" "PerformanceChecks"="False" }
        LOD 300

// 2/3 Changes:	
		Stencil{
			Ref 1
			Comp[_StencilComp]
		}

        // ------------------------------------------------------------------
        //  Base forward pass (directional light, emission, lightmaps, ...)
        Pass
        {
            Name "FORWARD"
            Tags { "LightMode" = "ForwardBase" }

            Blend [_SrcBlend] [_DstBlend]
            ZWrite [_ZWrite]

            CGPROGRAM
            #pragma target 3.0

            // -------------------------------------

            #pragma shader_feature _NORMALMAP
            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature _EMISSION
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _DETAIL_MULX2
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _SPECULARHIGHLIGHTS_OFF
            #pragma shader_feature_local _GLOSSYREFLECTIONS_OFF
            #pragma shader_feature_local _PARALLAXMAP

            #pragma multi_compile_fwdbase
            #pragma multi_compile_fog
            #pragma multi_compile_instancing
            // Uncomment the following line to enable dithering LOD crossfade. Note: there are more in the file to uncomment for other passes.
            //#pragma multi_compile _ LOD_FADE_CROSSFADE

            #pragma vertex vertBase
            #pragma fragment fragBase
            #include "UnityStandardCoreForward.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        //  Additive forward pass (one light per pass)
        Pass
        {
            Name "FORWARD_DELTA"
            Tags { "LightMode" = "ForwardAdd" }
            Blend [_SrcBlend] One
            Fog { Color (0,0,0,0) } // in additive pass fog should be black
            ZWrite Off
            ZTest LEqual

            CGPROGRAM
            #pragma target 3.0

            // -------------------------------------

            #pragma shader_feature _NORMALMAP
            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _SPECULARHIGHLIGHTS_OFF
            #pragma shader_feature_local _DETAIL_MULX2
            #pragma shader_feature_local _PARALLAXMAP

            #pragma multi_compile_fwdadd_fullshadows
            #pragma multi_compile_fog
            // Uncomment the following line to enable dithering LOD crossfade. Note: there are more in the file to uncomment for other passes.
            //#pragma multi_compile _ LOD_FADE_CROSSFADE

            #pragma vertex vertAdd
            #pragma fragment fragAdd
            #include "UnityStandardCoreForward.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        //  Shadow rendering pass
        Pass {
            Name "ShadowCaster"
            Tags { "LightMode" = "ShadowCaster" }

            ZWrite On ZTest LEqual

            CGPROGRAM
            #pragma target 3.0

            // -------------------------------------


            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _PARALLAXMAP
            #pragma multi_compile_shadowcaster
            #pragma multi_compile_instancing
            // Uncomment the following line to enable dithering LOD crossfade. Note: there are more in the file to uncomment for other passes.
            //#pragma multi_compile _ LOD_FADE_CROSSFADE

            #pragma vertex vertShadowCaster
            #pragma fragment fragShadowCaster

            #include "UnityStandardShadow.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        //  Deferred pass
        Pass
        {
            Name "DEFERRED"
            Tags { "LightMode" = "Deferred" }

            CGPROGRAM
            #pragma target 3.0
            #pragma exclude_renderers nomrt


            // -------------------------------------

            #pragma shader_feature _NORMALMAP
            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature _EMISSION
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _SPECULARHIGHLIGHTS_OFF
            #pragma shader_feature_local _DETAIL_MULX2
            #pragma shader_feature_local _PARALLAXMAP

            #pragma multi_compile_prepassfinal
            #pragma multi_compile_instancing
            // Uncomment the following line to enable dithering LOD crossfade. Note: there are more in the file to uncomment for other passes.
            //#pragma multi_compile _ LOD_FADE_CROSSFADE

            #pragma vertex vertDeferred
            #pragma fragment fragDeferred

            #include "UnityStandardCore.cginc"

            ENDCG
        }

        // ------------------------------------------------------------------
        // Extracts information for lightmapping, GI (emission, albedo, ...)
        // This pass it not used during regular rendering.
        Pass
        {
            Name "META"
            Tags { "LightMode"="Meta" }

            Cull Off

            CGPROGRAM
            #pragma vertex vert_meta
            #pragma fragment frag_meta

            #pragma shader_feature _EMISSION
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _DETAIL_MULX2
            #pragma shader_feature EDITOR_VISUALIZATION

            #include "UnityStandardMeta.cginc"
            ENDCG
        }
    }

    SubShader
    {
        Tags { "RenderType"="Opaque" "PerformanceChecks"="False" }
        LOD 150

// 3/3 Changes:
		Stencil{
			Ref 1
			Comp[_StencilComp]
		}
        // ------------------------------------------------------------------
        //  Base forward pass (directional light, emission, lightmaps, ...)
        Pass
        {
            Name "FORWARD"
            Tags { "LightMode" = "ForwardBase" }

            Blend [_SrcBlend] [_DstBlend]
            ZWrite [_ZWrite]

            CGPROGRAM
            #pragma target 2.0

            #pragma shader_feature _NORMALMAP
            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature _EMISSION
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _SPECULARHIGHLIGHTS_OFF
            #pragma shader_feature_local _GLOSSYREFLECTIONS_OFF
            #pragma shader_feature_local _DETAIL_MULX2
            // SM2.0: NOT SUPPORTED shader_feature_local _PARALLAXMAP

            #pragma skip_variants SHADOWS_SOFT DYNAMICLIGHTMAP_ON DIRLIGHTMAP_COMBINED

            #pragma multi_compile_fwdbase
            #pragma multi_compile_fog

            #pragma vertex vertBase
            #pragma fragment fragBase
            #include "UnityStandardCoreForward.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        //  Additive forward pass (one light per pass)
        Pass
        {
            Name "FORWARD_DELTA"
            Tags { "LightMode" = "ForwardAdd" }
            Blend [_SrcBlend] One
            Fog { Color (0,0,0,0) } // in additive pass fog should be black
            ZWrite Off
            ZTest LEqual

            CGPROGRAM
            #pragma target 2.0

            #pragma shader_feature _NORMALMAP
            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _SPECULARHIGHLIGHTS_OFF
            #pragma shader_feature_local _DETAIL_MULX2
            // SM2.0: NOT SUPPORTED shader_feature_local _PARALLAXMAP
            #pragma skip_variants SHADOWS_SOFT

            #pragma multi_compile_fwdadd_fullshadows
            #pragma multi_compile_fog

            #pragma vertex vertAdd
            #pragma fragment fragAdd
            #include "UnityStandardCoreForward.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        //  Shadow rendering pass
        Pass {
            Name "ShadowCaster"
            Tags { "LightMode" = "ShadowCaster" }

            ZWrite On ZTest LEqual

            CGPROGRAM
            #pragma target 2.0

            #pragma shader_feature_local _ _ALPHATEST_ON _ALPHABLEND_ON _ALPHAPREMULTIPLY_ON
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma skip_variants SHADOWS_SOFT
            #pragma multi_compile_shadowcaster

            #pragma vertex vertShadowCaster
            #pragma fragment fragShadowCaster

            #include "UnityStandardShadow.cginc"

            ENDCG
        }
        // ------------------------------------------------------------------
        // Extracts information for lightmapping, GI (emission, albedo, ...)
        // This pass it not used during regular rendering.
        Pass
        {
            Name "META"
            Tags { "LightMode"="Meta" }

            Cull Off

            CGPROGRAM
            #pragma vertex vert_meta
            #pragma fragment frag_meta

            #pragma shader_feature _EMISSION
            #pragma shader_feature_local _SPECGLOSSMAP
            #pragma shader_feature_local _SMOOTHNESS_TEXTURE_ALBEDO_CHANNEL_A
            #pragma shader_feature_local _DETAIL_MULX2
            #pragma shader_feature EDITOR_VISUALIZATION

            #include "UnityStandardMeta.cginc"
            ENDCG
        }
    }

    //FallBack "VertexLit"
    //CustomEditor "StandardShaderGUI"
}

3.Sky.shader, modification based on Unity’s Built-in Shaders, should be applied on a Sphere gameobject used as the skybox.

// (c) 2020 Tongzhou Yu

Shader "Custom/Sky"
{
    Properties
    {
		_MainTex("Texture", 2D) = "white" {}
			[Enum(CompareFunction)] _StencilComp("Stencil Comp",Int) = 3
    }
    SubShader
    {
        Tags { "RenderType"="Opaque" }
        LOD 100
				Cull Front


        Pass
        {
						Stencil{
			Ref 1
			Comp[_StencilComp]
		}
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag
            // make fog work
            #pragma multi_compile_fog

            #include "UnityCG.cginc"

            struct appdata
            {
                float4 vertex : POSITION;
                float2 uv : TEXCOORD0;
            };

            struct v2f
            {
                float2 uv : TEXCOORD0;
                UNITY_FOG_COORDS(1)
                float4 vertex : SV_POSITION;
            };

            sampler2D _MainTex;
            float4 _MainTex_ST;

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                o.uv = TRANSFORM_TEX(v.uv, _MainTex);
                UNITY_TRANSFER_FOG(o,o.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                // sample the texture
                fixed4 col = tex2D(_MainTex, i.uv);
                // apply fog
                UNITY_APPLY_FOG(i.fogCoord, col);
                return col;
            }
            ENDCG
        }
    }
}

3.Scripts

1.PortalManger.cs, should be applied on PortalDoor component.

  1. MAKE SURE the tag of the camera used in the scene has been set to MainCamera
  2. In the camera’s inspector, “near” of the Clipping Planes should be set to the same value as the one in PortalManger.cs (if(camPostionInPortalSpace.y < 0.01f))
  3. Drag Innerworld which you created into PortalManager>InnerWorld
  4. Set the number of materials used in Innerworld and drag the materials into PortalManager>Materials
// (c) 2020 Tongzhou Yu

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Rendering;

public class PortalManager : MonoBehaviour
{

    public GameObject InnerWorld;

    //This materials matter needs to be optimizated!
    public Material[] materials;

    private Vector3 camPostionInPortalSpace;

    bool wasInFront;
    bool inOtherWorld;

    bool hasCollided;

    // Start is called before the first frame update
    void Start()
    {
        SetMaterials(false);
    }

    void SetMaterials(bool fullRender)
    {
        var stencilTest = fullRender ? CompareFunction.NotEqual : CompareFunction.Equal;

        foreach (var mat in materials)
        {
            mat.SetInt("_StencilComp", (int)stencilTest);
        }
    }

    //Set bidirectional function
    bool GetIsInFront()
    {
        GameObject MainCamera = GameObject.FindGameObjectWithTag("MainCamera");
        Vector3 worldPos = MainCamera.transform.position + MainCamera.transform.forward * Camera.main.nearClipPlane;
        camPostionInPortalSpace = transform.InverseTransformPoint(worldPos);
        return camPostionInPortalSpace.y >= 0 ? true : false;
    }

    private void OnTriggerEnter(Collider collider)
    {
        GameObject MainCamera = GameObject.FindGameObjectWithTag("MainCamera");
        if (collider.transform != MainCamera.transform)
            return;
        wasInFront = GetIsInFront();
        hasCollided = true;

    }

    // Update is called once per frame
    void OnTriggerExit(Collider collider)
    {
        GameObject MainCamera = GameObject.FindGameObjectWithTag("MainCamera");
        if (collider.transform != MainCamera.transform)
            return;
        hasCollided = false;
    }

    void whileCameraColliding()
    {
        if (!hasCollided)
            return;
        bool isInFront = GetIsInFront();
        if ((isInFront && !wasInFront) || (wasInFront && !isInFront))
        {
            inOtherWorld = !inOtherWorld;
            SetMaterials(inOtherWorld);
        }
        wasInFront = isInFront;
    }

    private void OnDestroy()
    {
        SetMaterials(true);
    }

    private void Update()
    {
        whileCameraColliding();
    }

}

发表评论

Fill in your details below or click an icon to log in:

WordPress.com 徽标

You are commenting using your WordPress.com account. Log Out /  更改 )

Google photo

You are commenting using your Google account. Log Out /  更改 )

Twitter picture

You are commenting using your Twitter account. Log Out /  更改 )

Facebook photo

You are commenting using your Facebook account. Log Out /  更改 )

Connecting to %s