Mesos applications, why not use Marathon for everything?

Apache Mesos is a 2-level scheduler. The purpose of a framework is to provide the intelligence of high-level scheduling. Marathon provides the ability to schedule a task in the cluster, queue that task for scheduling and re-queue tasks that have failed. It is great at keeping long running processes running. It is like the init of the datacenter. As such, it is commonly used to make sure other frameworks are up and running such as Kubernetes-Mesos or Jenkins.

There are many applications for which this level of scheduling is insufficient. Marathon can and often is used for running things like Apache Kafka, however this often falls short in many failure modes. Additionally, Marathon doesn't care if task runs multiple times on the same node, however running multiple Kafka nodes on the same slave is a bad idea. Using Hadoop as another example (since you referred it), HDFS has several types of nodes that need to be managed; NameNode, DataNode and JournalNode. Marathon does not know the order to start these in, or if these can be co-located on the same node or not. It doesn't know how to scale this application. The HDFS framework manages that intelligence.

As far as scheduling efficiency, I'm not sure that is the goal. Apache Mesos is a 2-level scheduler for a reason. It is a highly efficient 2-level scheduler. The value of 2-level scheduling is to abstract the type of concerns I described above to a higher-level scheduler (which is termed by Mesos as frameworks). Marathon is still a great way to schedule and ensure high availability to other frameworks.