How to use iOS (Swift) SceneKit SCNSceneRenderer unprojectPoint properly

Typical depth buffers in a 3D graphics pipeline are not linear. Perspective division causes depths in normalized device coordinates to be on a different scale. (See also here.)

So the z-coordinate you're feeding into unprojectPoint isn't actually the one you want.

How, then, to find the normalized-depth coordinate matching a plane in world space? Well, it helps if that plane is orthogonal to the camera -- which yours is. Then all you need to do is project a point on that plane:

let projectedOrigin = gameView.projectPoint(SCNVector3Zero)

Now you have the location of the world origin in 3D view + normalized-depth space. To map other points in 2D view space onto this plane, use the z-coordinate from this vector:

let vp = gestureRecognizer.locationInView(scnView)let vpWithZ = SCNVector3(x: vp.x, y: vp.y, z: projectedOrigin.z)let worldPoint = gameView.unprojectPoint(vpWithZ)

This gets you a point in world space that maps the click/tap location to the z = 0 plane, suitable for use as the position of a node if you want to show that location to the user.

(Note that this approach works only as long as you're mapping onto a plane that's perpendicular to the camera's view direction. If you want to map view coordinates onto a differently-oriented surface, the normalized-depth value in vpWithZ won't be constant.)

After some experimentation, here's what we developed to project a touch point to a given point in the scene for some arbitrary depth.

The modification you need is to compute the intersection of the Z=0 plane with this line, and that will be your point.

private func touchPointToScenePoint(recognizer: UIGestureRecognizer) -> SCNVector3 {    // Get touch point    let touchPoint = recognizer.locationInView(sceneView)    // Compute near & far points    let nearVector = SCNVector3(x: Float(touchPoint.x), y: Float(touchPoint.y), z: 0)    let nearScenePoint = sceneView.unprojectPoint(nearVector)    let farVector = SCNVector3(x: Float(touchPoint.x), y: Float(touchPoint.y), z: 1)    let farScenePoint = sceneView.unprojectPoint(farVector)    // Compute view vector    let viewVector = SCNVector3(x: Float(farScenePoint.x - nearScenePoint.x), y: Float(farScenePoint.y - nearScenePoint.y), z: Float(farScenePoint.z - nearScenePoint.z))    // Normalize view vector    let vectorLength = sqrt(viewVector.x*viewVector.x + viewVector.y*viewVector.y + viewVector.z*viewVector.z)    let normalizedViewVector = SCNVector3(x: viewVector.x/vectorLength, y: viewVector.y/vectorLength, z: viewVector.z/vectorLength)    // Scale normalized vector to find scene point    let scale = Float(15)    let scenePoint = SCNVector3(x: normalizedViewVector.x*scale, y: normalizedViewVector.y*scale, z: normalizedViewVector.z*scale)    print("2D point: \(touchPoint). 3D point: \(nearScenePoint). Far point: \(farScenePoint). scene point: \(scenePoint)")    // Return <scenePoint>    return scenePoint}

This is my solution for getting the exact point in 3d Space.

// If the object is in 0,0,0 point you can usefloat zDepth = [self projectPoint:SCNVector3Zero].z;// or myNode.position.//zDepth = [self projectPoint:myNode.position].z;NSLog(@"2D point: X %f, Y: %f, zDepth: %f", click.x, click.y, zDepth);SCNVector3 worldPoint = [self unprojectPoint:SCNVector3Make(click.x, click.y,  zDepth)];SCNVector3 nearVec = SCNVector3Make(click.x, click.y, 0.0);SCNVector3 nearPoint = [self unprojectPoint:nearVec];SCNVector3 farVec = SCNVector3Make(click.x, click.y, 1.0);SCNVector3 farPoint = [self unprojectPoint:farVec];float z_magnitude = fabs(farPoint.z - nearPoint.z);float near_pt_factor = fabs(nearPoint.z) / z_magnitude;float far_pt_factor = fabs(farPoint.z) / z_magnitude;GLKVector3 nearP = GLKVector3Make(nearPoint.x, nearPoint.y, nearPoint.z);GLKVector3 farP = GLKVector3Make(farPoint.x, farPoint.y, farPoint.z);GLKVector3 final_pt = GLKVector3Add(GLKVector3MultiplyScalar(nearP, far_pt_factor), GLKVector3MultiplyScalar(farP, near_pt_factor));NSLog(@"3D world point = %f, %f, %f", final_pt.x, final_pt.y, worldPoint.z);