I have already told you Linux tries to cache everything in memory even your process finishes linux allocates free memory to other processes
No matter what you do you will see 640MB being used, what's the point having 640MB if you only use 100MB? Linux tries to make good use of all of them