Is a container engine an entire layer between OS and applications?
No.
Is a container engine another application running next to other applications on top of OS?
This definition is better.
Scott McCarty has the following slide in one of his presentations:
Image may be NSFW.
Clik here to view.
A bit of history follows which might help with terms like docker daemon, containerd, runc, rkt...
from: CoreOS documentation:
Image may be NSFW.
Clik here to view.Prior to Docker version 1.11, the Docker Engine daemon downloaded container images, launched container processes, exposed a remote API, and acted as a log collection daemon, all in a centralized process running as root.
While such a centralized architecture is convenient for deployment, it does not follow best practices for Unix process and privilege separation; further, it makes Docker difficult to properly integrate with Linux init systems such as upstart and systemd.
Since version 1.11, the Docker daemon no longer handles the execution of containers itself. Instead, this is now handled by containerd. More precisely, the Docker daemon prepares the image as an Open Container Image (OCI) bundle and makes an API call to containerd to start the OCI bundle. containerd then starts the container using runC
Further reading: