Running software remotely or in the background

Introduction

If you're doing heavy-duty computation with Matlab/Maple/Mathematica or your own programs, chances are you have to wait a long time (perhaps days) for your computation to finish. But you don't want to sit in front of a lab computer for days, or tie up your office computer. There are ways to put tasks in the background (so that you can run other programs at the same time on the same computer, or log out and return later to log in and get your results), and/or put them on a remote machine. However, be forewarned: it is very easy to take up too much CPU time or memory, thus making a computer sluggish or downright unusable for other people.

Commands

Before launching background/remote tasks, you must:

• Learn to use ssh to connect to remote systems.
• Learn to use top or ps, kill, and renice to monitor and manage processes you've started.
• Learn to use &, nice, nohup, and ulimit to launch processes in a desired way.
• For interactive (non-batch) programs: know about ssh -Y, and learn either vnc or screen.

Just to give you an idea of what these things are, a few words appear below. However, the descriptions below are intentionally succinct (and probably come across as rather vague), because there is simply no substitute for reading official manpages. (Nor is there any excuse for not having done so.)

top

This command displays a list of active processes and how much memory and CPU time each process is using, who owns it, and so on. If a machine is sluggish, top will tell you whose fault it is! Often top is used by others to determine that your process is in their way, and then they might send you an unfriendly e-mail message. You can use top yourself to ensure your processes are behaving nicely. From within the top display, you can even kill/renice your running processes. Hit ? for some help, but ultimately be sure to read the manpage: man top. (By the way: if you spot someone else's runaway process eating CPU, or someone else's spurious process that might be benign but probably shouldn't be there, either situation may be due to naivety or due to a system bug, so be sure to inform the other user immediately by e-mail. They are the only ones who can correct the situation, and usually they don't know about it unless you inform them.)

ps

Makes a list of running processes along with additional information. A common invocation is ps -aux, which lists a lot of extra information. Another useful invocation is ps -aux -forest, which draws lines to indicate which process spawned which process. (The separate command pstree does something similar.) For more information: man ps.

kill

Lets you kill one of your processes. You must know the process ID. You can see this in the top display, or you can get a list using ps -aux. Instead of using kill as a separate command, you can access it inside top, which you might find easier. Nonetheless, you'll need to learn about kill signals, so read man kill. Generally try to kill something with signal 15 first and wait a few seconds. Ultimately hit it with signal 9. If that doesn't work, then that machine has become unstable and you must report the problem by contacting the computer support staff. If it's a lab machine, you might put a note over the keyboard, so no one else logs in and loses work when the pending crash occurs.

nice

This starts a task with a nice value, which means the task will do a better job sharing the CPU when you try to run other programs or when other users log in to the same computer. It is absolutely essential that all heavy-duty tasks and all non-interactive remote tasks be nice. Read the manpage: man nice.

renice

Forgot to start something using nice? Use this to go back and make it nicer. You can also do this inside top. Read the manpage: man renice.

&

Putting this after a command puts that command in the background, so that you can continue to give other commands. Note that & alone does little else: for example, it doesn't make the background process share resources (for that, see nice). Furthermore, if you log out, your background task might get killed automatically (to avoid that, see nohup). Finally, if it is a command that creates windows (e.g., Mathematica&), it will create them even from the background. In fact, most people use & specifically when starting programs like Mathematica so that they don't lose their prompt. (Try it both ways and the previous sentence will make more sense.) Since & is not itself a command, but part of the shell language, there is no separate manpage; instead, you have to read the shell manpage: man bash.

nohup

To put something in the background so that it doesn't die when you logout, start it with nohup. (You also need the "&" for the background part.) You can only do this with text-only batch programs. In other words, the programs shouldn't create windows, they can't rely on keyboard input, and they must send all output to a file. So you can use this with your own C/C++ programs, but you can't use it with most software packages. Go read man nohup. These days, with screen available, the only reason to use nohup is because it is easier to learn than screen, but the latter lets you do so much more.

ulimit

To prevent a program from using too much time or too much memory, you can set limits in a shell, and all programs started from that shell will have those limits imposed. Use ulimit -a to get a list of limits. To confirm you understand what is going on, set various values ridiculously low and try to run your program to make sure it gets stopped by the operating system. Then quit that shell, open a new shell, and set more reasonable limits. Go read man ulimit.

ssh

From within a shell, you can use ssh user@remote to obtain a shell on a remote system. Then you can issue commands (such as many of the ones described on this page) on the remote system. If you use ssh -Y user@remote then there will also be interactive X forwarding, meaning that programs launched on the remote system can open windows on your local system (assuming you are logged in to a graphical X desktop). You might also want to read about using ssh to obtain remote shell access to your network account.

screen

This creates an environment in which you can start an interactive text-only program, such as GAP or Macaulay, or even Mathematica in its text-only non-notebook mode, but you can log out and it keeps going. It does not automatically make things nice, so you still have to use that as well. Go read man screen. Then you can log in to, say, chivo, start a screen, then inside that screen you can do, say, nice Singular, type some Singular commands that will take days to complete, hit "control-A d", log out, a day later log back in to chivo, do screen -r to reconnect, and you'll be right back in Singular. It's very cool.

VNC

This is a graphical version of screen. You can create an entire virtual X desktop and connect to it from home, office, anywhere you can access a web browser. You'll have to read man vncserver and man vncviewer. You might want to read about using vnc to obtain remote desktop access to your network account.