Is there anything like shm_open() without filename?
A library to solve the problem
I managed to write a library that provides the simple interface:
int shm_open_anon(void);
The library compiles without warnings and successfully runs a test program on Linux, Solaris, MacOS, FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and Haiku. You may be able to adapt it to other operating systems; please send a pull request if you do.
The library returns a file descriptor with the close-on-exec flag set. You can clear that flag using fcntl()
on all supported operating systems, which will allow you to pass the fd over exec()
. The test program demonstrates that this works.
Implementation techniques used in the library
The readme of the library has very precise notes on what was done and what wasn't done for each OS. Here's a summary of the main stuff.
There are several non-portable things that are more or less equivalent to shm_open()
without a filename:
FreeBSD can take
SHM_ANON
as the pathname forshm_open()
since 2008.Linux has a
memfd_create()
system call since kernel version 3.17.Earlier versions of Linux can use
mkostemp(name, O_CLOEXEC | O_TMPFILE)
wherename
is something like/dev/shm/XXXXXX
. Note that we are not usingshm_open()
at all here --mkostemp()
is implicitly using a perfectly ordinaryopen()
call. Linux mounts a special memory-backed file system in/dev/shm
but some distros use/run/shm
instead so there are pitfalls here. And you still have to shm_unlink() the temp file.OpenBSD has a
shm_mkstemp()
call since release 5.4. You still have toshm_unlink()
the temp file but at least it is easy to create safely.
For other OSes, I did the following:
Figure out an OS-dependent format for the name argument of POSIX
shm_open()
. Please note that there is no name you can pass that is absolutely portable. For example, NetBSD and DragonFlyBSD have conflicting demands about slashes in the name. This applies even if your goal is to use a named shm object (for which the POSIX API was designed) instead of an anonymous one (as we are doing here).Append some random letters and numbers to the name (by reading from
/dev/random
). This is basically whatmktemp()
does, except we don't check whether our random name exists in the file system. The interpretation of the name argument varies wildly so there's no reasonable way to portably map it to an actual filename. Also Solaris doesn't always providemktemp()
. For all practical purposes, the randomness we put in will ensure a unique name for the fraction of a second that we need it.Open the shm object with that name via
shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600)
. In the astronomical chance that our random filename already exists,O_EXCL
will cause this call to fail anyway, so no harm done. The0600
permissions (owner read-write) are necessary on some systems instead of blank0
permissions.Immediately call
shm_unlink()
to get rid of the random name. The file descriptor remains for our use.
This technique is not quaranteed to work by POSIX, but:
- The
shm_open()
name argument is underspecified by POSIX so nothing else is guaranteed to work either. - I'll let the above compatibility list speak for itself.
Enjoy.
No, there isn't. Since both System V shared memory model and POSIX shared file mapping for IPC require operations with a file, there is always need for a file in order to do mapping.
mmap()
withMAP_ANON|MAP_SHARED
is great but instead of a file descriptor it gives a pointer. The pointer doesn't survive over an exec boundary and can't be sent to another process over a Unix domain socket like file descriptors can.
As John Bollinger says,
Neither memory mappings created via
mmap()
nor POSIX shared-memory segments obtained viashm_open()
nor System V shared-memory segments obtained viashmat()
are preserved across anexec
.
There must be a well-known place on the memory to meet and exchange information. That's why a file is the requirement. By doing this, after exec
, the child is able to reconnect to the appropriate shared memory.
This use of
tempfilename
is difficult to do portably and reliably. The interpretation of the filename (what the namespace is, how permissions are handled) differs among systems.
You can have mkstemp
create a unique filename in /dev/shm/
or /tmp
and open the file for you. You can then unlink
the filename, so that no other process can open this file, apart from the process that have the file descriptor returned from mkstemp
.
mkstemp()
: CONFORMING TO 4.3BSD, POSIX.1-2001.