Shared memory takes gigabytes

I have problems with huge RAM consumption by an unknown entity. It is not listed in top, but the amount of memory available decreases over time and turning off all programs does nothing. Currently, it's 3.6 GiB, but it's slowly getting worse. Probably as a result, the system is not capable to swap out much memory, after swapping out about 1.5 GiB of memory, it proceeds to thrashing.

I have used the free -m command to get this result:

 total used free shared buff/cache available Mem: 7875 3517 388 3595 3969 481 Swap: 9536 500 9036 

It seems that shared memory takes 3.5 gigabytes.

This is what I get when I run df -h:

udev 3,9G 0 3,9G 0% /dev tmpfs 788M 80M 709M 11% /run /dev/sdb6 23G 18G 4,1G 81% / tmpfs 3,9G 168M 3,7G 5% /dev/shm tmpfs 5,0M 4,0K 5,0M 1% /run/lock tmpfs 3,9G 0 3,9G 0% /sys/fs/cgroup /dev/sdb1 356M 76M 281M 22% /boot/efi /dev/sdb8 37G 33G 2,0G 95% /home /dev/sda2 746G 269G 477G 37% /data /dev/sda3 184G 132G 43G 76% /lindata tmpfs 788M 84K 788M 1% /run/user/1000 /home/dugi/.Private 37G 33G 2,0G 95% /home/dugi 

/proc/meminfo/ contains this, it seems that it's really something in shared memory:

MemTotal: 8064776 kB MemFree: 318880 kB MemAvailable: 493992 kB Buffers: 36784 kB Cached: 3960388 kB SwapCached: 32776 kB Active: 3732236 kB Inactive: 3539036 kB Active(anon): 3562316 kB Inactive(anon): 3381848 kB Active(file): 169920 kB Inactive(file): 157188 kB Unevictable: 732 kB Mlocked: 732 kB SwapTotal: 9765884 kB SwapFree: 9253044 kB Dirty: 516 kB Writeback: 0 kB AnonPages: 3269552 kB Mapped: 493840 kB Shmem: 3670056 kB Slab: 264136 kB SReclaimable: 136840 kB SUnreclaim: 127296 kB KernelStack: 23296 kB PageTables: 82464 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 13798272 kB Committed_AS: 20782644 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 831488 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 1783952 kB DirectMap2M: 6498304 kB DirectMap1G: 0 kB 

However, when I list the stuff in shared memory using ipcs --human, I get only a few hundreds of megabytes of deleted memory:

------ Message Queues -------- key msqid owner perms size messages ------ Shared Memory Segments -------- key shmid owner perms size nattch status 0x00000000 268959744 dugi 600 386,8K 2 dest 0x00000000 265879554 dugi 600 512K 2 dest 0x00000000 557059 dugi 600 16M 2 dest 0x00000000 6258692 dugi 600 1K 2 dest 0x00000000 271974405 dugi 600 512K 2 dest 0x00000000 31784966 dugi 600 16M 2 dest 0x00000000 819207 dugi 600 16M 2 dest 0x00000000 274464776 dugi 600 512K 2 dest 0x00000000 28704777 dugi 600 512K 2 dest 0x00000000 274694154 dugi 600 512K 2 dest 0x00000000 264372236 dugi 600 512K 2 dest 0x00000000 272465933 dugi 600 4M 2 dest 0x00000000 145522702 dugi 600 16M 2 dest 0x00000000 274563088 dugi 600 7,6M 2 dest 0x00000000 113934353 dugi 600 74K 2 dest 0x00000000 215908370 dugi 600 66,2K 2 dest 0x00000000 35913747 dugi 600 512K 2 dest 0x00000000 274595860 dugi 600 7,6M 2 dest 0x00000000 149946389 dugi 600 512K 2 dest 0x00000000 273842198 dugi 600 128M 2 dest 0x00000000 115015703 dugi 600 29,3K 2 dest 0x00000000 150044696 dugi 600 64M 2 dest 0x00000000 143425561 dugi 600 32,5K 2 dest 0x00000000 150011930 dugi 600 1M 2 dest 0x00000000 149749788 dugi 600 512K 2 dest 0x00000000 149848094 dugi 600 64M 2 dest 0x00000000 149815327 dugi 600 1M 2 dest 0x00000000 166363168 dugi 700 2,2M 2 dest 0x00000000 166395937 dugi 600 384K 2 dest 0x00000000 166428706 dugi 600 512K 2 dest 0x00000000 166461475 dugi 700 53,5K 2 dest 0x00000000 166625316 dugi 700 376,6K 2 dest 0x00000000 166658088 dugi 700 13,5K 2 dest 0x00000000 166690857 dugi 700 88,7K 2 dest ------ Semaphore Arrays -------- key semid owner perms nsems 

If I use ipcs -m | awk '{sum+=$5} END{print sum}', it says there's 366790104 bytes, which is 366 MB. This is far less than what it occupies.

I am using Ubuntu 16.04 upgraded to 17.04. The problem was there already before the upgrade. Programs I use almost constantly are QtCreator, Vivaldi, evince, gcc, and the usual ones like gnome shell, nautilus or gedit.

The memory is never swapped out, I can't google a way to find the cause or just to delete it once per day or force swap it or something, I don't want to restart my computer several times per week. Any ideas what can be the problem?

EDIT: I have seen it suck up about 120 megabytes of memory for no known reason. I had Vivaldi (that makes extensive use of shared memory) shut down at that time because I was suspecting it of using causing the problem. The only thing I did at that time is that opened up Firefox for the first time after last restart (Firefox alone cannot be the cause because I didn't run it before those gigabytes got filled).

EDIT #2: Restarting the X-server removes the stuff from shared memory, but that isn't a solution because the X-server is exactly the thing I don't want to restart (gnome-shell --replace doesn't help). If I remember the lessons about shared memory correctly, this means that the memory is grabbed by something related to the X-server or Gnome-shell that correctly frees it on exit, but never erases it while running (some sort of broken garbage collection?).

Edit #3: I wonder if it's related this:

1 Answer

Maybe smemstat will show the memory hogger(s). Use:

sudo apt-get install smemstat sudo smemstat