When I do ssh -V I get the following, indicating I do in fact have openssh installed:

$ ssh -VOpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f 6 Jan 2014

If you don't see OpenSSH plus a version number, like I do, then you must not have the openssh-client installed, and as you insinuated, you must be getting the ssh binary from some other ssh application.

To see all of your packages installed which have ssh in them, pipe your dpkg --list output to grep using this command:

dpkg --list | grep ssh

...as I have done below. You can see I have openssh-client, openssh-server, openssh-sftp-server, and ssh-add installed, all of which are part of OpenSSH:

$ dpkg --list | grep sshii  libssh-4:amd64                                        0.6.1-0ubuntu3.3                                                               amd64        tiny C SSH libraryii  libssh2-1:amd64                                       1.4.3-2ubuntu0.1                                                               amd64        SSH2 client-side libraryii  openssh-client                                        1:6.6p1-2ubuntu2.13                                                            amd64        secure shell (SSH) client, for secure access to remote machinesii  openssh-server                                        1:6.6p1-2ubuntu2.13                                                            amd64        secure shell (SSH) server, for secure access from remote machinesii  openssh-sftp-server                                   1:6.6p1-2ubuntu2.13                                                            amd64        secure shell (SSH) sftp server module, for SFTP access from remote machinesii  python-paramiko                                       1.10.1-1git1build1                                                             all          Make ssh v2 connections with Python (Python 2)ii  ssh-askpass-gnome                                     1:6.6p1-2ubuntu2.8                                                             amd64        interactive X program to prompt users for a passphrase for ssh-addii  ssh-import-id                                         3.21-0ubuntu1                                                                  all          securely retrieve an SSH public key and install it locallyii  sshfs                                                 2.5-1ubuntu1                                                                   amd64        filesystem client based on SSH File Transfer Protocolii  sshpass                                               1.05-1                                                                         amd64        Non-interactive ssh password authentication

Here is a screenshot so you can see the red-colored ssh findings:

Also be aware that OpenSSH consists of a whole bunch of ssh-related binary utilities, NOT just the ssh command. See: https://en.wikipedia.org/wiki/OpenSSH. Some of them are as follows:

The OpenSSH suite includes the following command-line utilities and daemons:

• scp, a replacement for rcp
• sftp, a replacement for ftp to copy files between computers
• ssh, a replacement for rlogin, rsh and telnet to allow shell access to a remote machine.
• ssh-add and ssh-agent, utilities to ease authentication by holding keys ready and avoid the need to enter passphrases every time they are used
• ssh-keygen, a tool to inspect and generate the RSA, DSA and Elliptic Curve keys that are used for user and host authentication
• ssh-keyscan, which scans a list of hosts and collects their public keys
• sshd, the SSH server daemon

References:

1. https://en.wikipedia.org/wiki/OpenSSH
2. https://askubuntu.com/questions/423355/how-do-i-check-if-a-package-is-installed-on-my-server

"Does simply having the ability to use ssh on a Linux machine mean openSSH is installed?"

Answer: No. If you want to connect to a linux machine or server through SSH, the server you want to connect to must have the openssh-server package installed and configured. The computer you're using to connect to the server does not have to have openssh-server installed. It only needs the openssh client.

You can use Debian's package management system to confirm whether the openssh-server is installed:

sudo apt version openssh-server

If you see nothing printed out on the console, then you do not have it installed.

You can also use the dpkg:

sudo dpkg -l openssh-server