The linux kernel, in general, caches things to memory to make them available. These things take up memory, but are marked also as expendable (so the memory can be recovered if another program needs them).
So, it can be good to kill things that do not properly use memory or are not needed (though that too is not strictly necessary, as seen in the above link), but not having "Free Memory" is not something be concerned about on Linux in general.
It is much faster (and more efficient) to pull those needed things from memory than from the machine's normal storage, and if other programs do not need the memory, there is no need to recover it.