Snapping points into screen boundaries in Unreal Engine 4

In the last month or so, I’ve had the privilege to really dive in to the excellent Unreal Engine 4, and I’m quite happy with its flexibility so far. Its documentation gets pretty sparse when the reader strays from the higher level functionality, but the code itself is well written, readable, and walk-able with Intellisense for those times when you just have to trace out what’s happening underneath the hood. But, getting to my point, there are some things which are surprisingly simple to do within a 2D game context, that can be kind of weird and kludgy inside of a state-of-the-art 3D game engine.

The first of these that I ran into on my adventure to create a 2D(ish) side scroller was keeping the player on screen and detecting whether or not enemies were also within the view bounds.

As you might imagine, there are many ways to accomplish both of these things. I’m not sure if the following is the best way, but it’s working for me.

Mimes in invisible prisons

The first solution that came to mind for keeping a player pawn pushed onto the screen, and one that is mentioned numerous times in the forums, is to create collision objects at the screen boundaries, that are swept along with the camera movement.

I tried this out, and it worked OK, for the most part. It’s probably also working OK for many others too - but it felt kind of messy and has some shortfalls. For instance, it is a pain to position the collision blocks for different camera FOVs and screen aspect ratios. Certainly not impossible, but it is a pain. In fact, doing so correctly (to exactly match the view projection) is a significant step towards implementing the method that I’m about to describe. Also, unless the collision boxes on the borders of the screen are aligned to the walls of the view frustum it will only work nicely for a small area of depth. That is, objects in the foreground closer to the camera or far away wouldn’t be hit properly by the boxes.

These are some important catches for my use case, since I would like to have more free-form scrolling with orientation changes and zooming.

So, how can you find the nearest on-screen point for a given viewport and potentially off-screen location? It takes some digging, but it can be done cleanly using the ViewProjectionMatrix data in FSceneView.

Keeping things in perspective

To see how to do this, we have to peek briefly into the process of how all of the vertices in the 3D world are transformed into 2D screen coordinates for rendering. The interested reader can find a very good step by step breakdown of this process on scratchapixel.com but for this article, a quick hand-wavey explanation follows.

The interesting twist in the process of projecting a 3D world into a 2D viewport is that, conceptually, the camera is always in one place, pointing on one direction and it is actually the world that is moved and deformed (for perspective), around it to make it appear that the camera is moving around inside of the world. In the case of a perspective projection of the world, everything that is visible to the camera frustum is transformed into the clip space of ‘cube’.

In the particular case of the Unreal Engine, the camera is positioned at the origin (0, 0, 0) , and looking into the Z axis. From the UE4 Coordinate space terminology page, this is the View Space.

The important property of this coordinate space is that every point on the screen, regardless of depth, is mapped from -1 to 1 on the X and Y coordinates. For example, a mountain far in the distance, that you view on the far left side of the screen will have an X coordinate near -1. The bullet shell also flying off of the left side of the screen very near to the viewer will also have an X coordinate close to -1, even though in world-space the mountain is much further away in the negative x direction relative to the player’s orientation.

So, the trick is that once the position of the object to be limited to screen bounds is transformed into view-space, all that is needed is to clamp its X and Y position to (-1, 1) and then transform it back into world space by using the inverse projection matrix. If an object is snapped back onto the screen in this way, it will be as if its world-space position were moved just back onto screen, keeping the same Z depth relative to the viewer.

Show me the code

Here is the implementation that I ended up using. I made it a part of the PlayerController child class, but I think that it could probably be argued that it doesn’t belong there. It does require access to a LocalPlayer class in order to get the SceneView data for a particular player view, so there is also the caveat that this probably won’t work on a dedicated server unless one takes the trouble to replicate the LocalPlayer on the server-side. I haven’t looked into it, but if anyone has done this, I’d really love to hear about the method.

bool ACosmicDebrisPlayerController::SnapToViewFrustum(const FVector& worldPosition, FVector* outSnapped) {
    // SceneView initialization gently adapted from APlayerController::ProjectWorldLocationToScreen.
    ULocalPlayer* LocalPlayer = Cast<ULocalPlayer>(Player);
    if (LocalPlayer != NULL && LocalPlayer->ViewportClient != NULL && LocalPlayer->ViewportClient->Viewport != NULL)
    {
        // This sceneview initialization should probably be moved into another function.
        // Create a view family for the game viewport
        FSceneViewFamilyContext ViewFamily(FSceneViewFamily::ConstructionValues(
            LocalPlayer->ViewportClient->Viewport,
            GetWorld()->Scene,
            LocalPlayer->ViewportClient->EngineShowFlags)
            .SetRealtimeUpdate(true));
        FVector ViewLocation;
        FRotator ViewRotation;
        FSceneView* SceneView = LocalPlayer->CalcSceneView(&ViewFamily, /*out*/ ViewLocation, /*out*/ ViewRotation, LocalPlayer->ViewportClient->Viewport);

        if (SceneView)
        {
            // Transform our world position into projection coordinates.
            auto ProjPosAug = SceneView->ViewProjectionMatrix.TransformPosition(worldPosition);
            FVector ProjPos(ProjPosAug);
            // Divide by the augmented coord W value.
            ProjPos /= ProjPosAug.W;
            // Clamp position to -1,1 on x and y (corresponds to on-screen)
            ProjPos.X = FMath::Clamp<float>(ProjPos.X, -1, 1);
            ProjPos.Y = FMath::Clamp<float>(ProjPos.Y, -1, 1);
            // Invert the transform back to world space.
            auto AugWorldSnapped = SceneView->InvViewProjectionMatrix.TransformPosition(ProjPos);
            FVector SnapPos(AugWorldSnapped);
            SnapPos /= AugWorldSnapped.W;
            *outSnapped = SnapPos;
            return true;
        }
    }
    return false;
}

In case anyone would like to use/modify it, I release this snippet of code to the public domain.