script records everything in a session: things you type and things you see. It even records color; so if your command prompt or program output contain

The project, backed by a stealth-mode start-up called Qumranet, uses a technical and cultural approach that has quickly drawn powerful allies -- including Red Hat and Linux founder Linus Torvalds.

That success is only a first step in KVM's push to make a mark in virtualisation. But it signals significant influence over the technology, which is spurring a top-to-bottom revamp of the computing industry through its ability to make a single machine behave like many.

But does the world need another virtualisation option? EMC subsidiary VMware rules the roost today. Microsoft is working on a project, called Viridian, that is set to debut in roughly a year. And numerous open-source allies already have focused attention on an open-source rival called Xen. While KVM delivers some new options and competition, it also brings new complications.

"In the near term, KVM will cause some pain because of the market confusion and developer dilution it will cause," said Illuminata analyst Gordon Haff. "But in the longer run, better technical options can only be good for Linux and open source."

KVM, which stands for "Kernel-based Virtual Machine", provides a new Linux-based mechanism for splitting a single physical computer into multiple virtual machines. It's going up against another approach, which uses a low-level software "hypervisor" to perform the same virtualisation function.

The industry is scrambling to adopt virtualisation for a range of reasons: so that groups of inefficient servers can be replaced with a fewer machines; so software can be tested in harmless partitions; and ultimately, so data centers packed with computers can fluidly adjust to shifting priorities.

Industry players such as Novell and IBM say they're watching to see how well KVM fares. But Brian Stevens, the chief technology officer of dominant Linux seller Red Hat, believes KVM is viable.

"There's a year of work, I'd guess, to really make it at parity to where Xen is today...But I think it's going to happen," Stevens said. "The (open-source programming) community is really going to gravitate (to KVM) much more so than (to) Xen."

Qumranet has funding from Sequoia Capital and Norwest Venture Partners, but Chief Executive Benny Schnaider is mum on the company's business plan. In an interview, he said only that Qumranet is "not planning to make money by selling or supporting KVM."

The KVM project got started in early 2006, Schnaider said. That's about the same time that Moshe Bar left XenSource, the Xen commercialisation start-up that he co-founded. Bar, who now is Qumranet's chief technology officer, declined to comment for this story.

Qumranet is based in Santa Clara, Calif., with research and development in Israel. (Qumran is an ancient settlement near the caves where the Dead Sea Scrolls were found.) The start-up has more than 30 employees, most of them engineers, Schnaider said. Given that fewer than a dozen are working on KVM, according to lead programmer and Qumranet employee Avi Kivity, it's a good bet that the company has other technology in the works.

Kivity introduced the world to KVM with an October 19 posting to the Linux kernel mailing list. His patch updated Linux so that higher-level software could take advantage of hardware virtualisation features built into the latest processors from Intel and Advanced Micro Devices. The result: Other operating systems, including Microsoft Windows, can be "guests" running on a Linux host foundation, on newer hardware.

KVM's approach differs from that of Xen, which governs access to hardware using a combination of a lightweight "hypervisor" foundation and a privileged operating system, which is typically Linux.

KVM's method is conceptually closer to one of two approaches used by VMware -- the "hosted" model used in the free VMware Server and Player products. In that model, guest virtual machines run atop a copy of the operating system. In the second VMware approach, used in the higher-end ESX Server product, a full-featured, heavyweight hypervisor governs access to underlying hardware.

Unlike Xen additions to Linux, the KVM patch slipped nearly instantly into the mainstream kernel maintained by Torvalds and a group of deputies.

"We did things the Linux way," Kivity said in an interview. "I am a longtime lurker on the Linux kernel mailing list, so I know what's important to the kernel maintainers and tried to get things right the first time. Where I got things wrong, I fixed them quickly."

He introduced KVM with source code, not words. "Kernel maintainers only take you seriously if the first word in a message is 'PATCH,'" Kivity said.

Using script to record terminal sessions

Most sys admins know the importance of keeping an action log where various tasks, configuration changes, etc. are kept. Simple logs indicating, "I did this" or "John did that" may be sufficient in some organizations, but for some, full transcripts of changes made are desired. Doing a copy-and-paste of terminal output can be tedious at best, so one answer is to use a little-known program called script, which is part of the util-linux package on most Linux distributions.

script records everything in a session: things you type and things you see. It even records color; so if your command prompt or program output contains color, script will record it.

To use script, simply execute:

$ script

By default, it writes to a file called typescript in the current directory. From then on, everything you type will be recorded to that file. To write to a different file, simply use script /path/to/file.

When you're done, type exit. This will close down the script session and save the file. You can now examine the file using cat or any other program.

The downside of using script is the fact that it records all special characters, so your output file will be full of control characters and ansi escape sequences. This can be avoided by using a very Spartan shell with script:

$ SHELL=/bin/sh PS1="$ " script

When using script, don't use interactive programs or programs that manipulate the screen, like vi or top. They will ruin the output of the session. Otherwise, any command line programs you use and the steps you take to accomplish a task will be recorded. If you do need to edit a file in the transcript, consider exiting the script session and restarting it after the file edit with script -a, which will append the new session to the old session.

Learn the power features of zsh

The Z Shell (zsh) is a power-shell that is not often used by many Linux users. The reason for this is that most Linux distributions install, and make default, the bash shell. zsh is packaged for virtually every Linux distribution and installation is usually an apt-get, urpmi, or yum away.

One of the great features of zsh is tab-completion; it also handles all the logistics of tab-completion and is extremely easy to implement, just by adding two lines to your ~/.zshrc file:

autoload -U compinit
compinit

The compinit function is what loads the tab-completion system by defining a shell function for every utility that zsh is able to tab-complete. By using autoload, you can optimize zsh by telling it to defer reading the definition of the function until it's actually used, which speeds up the zsh startup time and reduces memory usage.

Using the setopt command, you can configure over 150 different options that impact how zsh works. For instance:

setopt autocd

The line above will allow you to change directories simply by typing the name of the directory (no need to use cd). Or, you might wish to use more powerful globbing or pattern matching features, which can be done by adding the line below to ~/.zshrc:

setopt extended_glob

The various zsh options that can be set with setopt are documented in the zshoptions manpage:

$ man zshoptions

Note that the ~/.zshrc file is sourced for both interactive and login shells. If you want, to set options for when zsh is run non-interactively (i.e., via a cronjob), then you'll want to add those to ~/.zshenv.

Another nice feature with zsh is how it handles prompts. These can be custom or they can be loaded via zsh's prompt system, which contains a number of "stock" prompts that might be suitable. For instance, to use the prompt system enter:

autoload -U promptinit
promptinit
prompt fire

To list the available fonts, on the command-line, enter "prompt -l". To define your own prompt, use the $PS1 variable, but zsh uses different format specifiers than bash, so a nice prompt might look like:

PS1=$'%{\e[1;32m%}%n@%m%{\e[0m%}:%B%~/%b >%# '

The resulting prompt looks like:

joe@odin:~/ >%

with the user and hostname in bright green.

Set up Logical Volume Manager in Linux

Logical Volume Manager (LVM), is a mechanism to create virtual drives out of physical drives. These virtual (or logical) drives can then be manipulated in interesting ways: They can be grown or shrunk, and they can span more than one physical disk. An LVM in and of itself is exciting because it allows you to turn a number of disks into one massive volume, but it becomes even more interesting when throwing RAID into the mix.

Using LVM with a RAID-1 mirroring system provides large devices with redundancy. This is important because if one drive in an LVM volume set dies, it could leave your data in an inconsistent (or entirely gone) state. Using LVM over RAID is really no different than using LVM on a physical disk; instead of adding physical volumes to your LVM set, you would add md devices, using /dev/md0 rather than /dev/hda1.

To begin, creating an LVM set from physical disks is quite easy. The following commands will get you started. This also assumes you're using a fairly recent Linux kernel; most distributions have LVM available for install.

# modprobe dm-mod
# vgscan
# fdisk /dev/hda

The first step is to partition the drive. You don't have to give the entire drive over to LVM if you don't want to. Create a partition, say hda1, and give it the type 8e, which is for Linux LVM. Do the same with the second disk (say, hdb). After that, execute:

# pvcreate /dev/hda1
# pvcreate /dev/hdb1

These commands will make the partitions available to LVM for use. The next step is to create the volume group:

# vgcreate data /dev/hda1 /dev/hdb1

This will create a volume group called "data" and assign /dev/hda1 and /dev/hdb1 to it. If you wish to add a third drive to the group later, you would use vgextend data /dev/hdc1. To obtain information on your volume group, use vgdisplay and the volume group name. For information on the physical volume, use pvdisplay. You need to use vgdisplay to find out how many physical extents are available for use. Here, we'll assign them all to one large logical device:

# vgdisplay data | grep "Total PE"
# lvcreate -l 10230 data -n files

The number of physical extents available was 10230, and all were assigned to the logical volume "files." Now you can format, manipulate, and mount the volume just as any other device, except the device name will be /dev/data/files or /dev/mapper/data-files:

# mke2fs -j /dev/data/files
# mkdir -p /srv/files
# mount /dev/data/files /srv/files

There is a lot more information about manipulating and maintaining LVM volumes (http://aplawrence.com/Linux/lvm.html) out there, and it would be wise to become familiar with it (http://tldp.org/HOWTO/LVM-HOWTO/), but it's quite straightforward to create your first LVM volume.

10 things you should do to a new Linux PC before exposing it to the Internet

1: Your purpose

Linux, like Microsoft Windows, is simply a computer operating system. When I talk to friends or co-workers who are embarking on the Linux experience for their initial time, this is the first point I stress. Linux in itself is not a magic wand that can be waved and make all sorts of computing problems disappear. While Windows has its own set of problems, so too does Linux. There is no such thing as a perfect or completely secure computer operating system. Will the machine be a desktop computer or a server; purpose is a key to understanding how to initially install and configure your Linux PC.

2: Installation

Unlike Windows, Linux does not present itself as a "server" version or as a "desktop" version. During a typical installation of Linux the choice is yours as to exactly what software you wish to install and therefore exactly what type of a system you are constructing. Because of this, you need to be aware of the packages that the installation program is installing for you. For example, some distributions will configure and start a Samba server or a mail server as part of the base install. Depending upon the purpose of your Linux PC and the security level you are prepared to accept, these services may not be needed or desired at all. Taking the time to familiarize yourself with your distributions' installer can prevent many headaches and/or reinstalls down the road.

3: Install and configure a software firewall

A local software firewall can provide a "just in case" layer of security to any type of network. These types of firewalls allow you to filter the network traffic that reaches your PC and are quite similar to the Windows Firewall. The Mandriva (http://wwwnew.mandriva.com/) package called Shorewall (http://www.shorewall.net/) along with a component of the Linux kernel called Netfilterprovides a software firewall. By installing and configuring Shorewall during the installation process, you can restrict or block certain types of network traffic, be it coming to or going out from your PC.

To access and configure your firewall for Mandriva simply run the mcc (or Mandriva Control Center) command from a command prompt or, depending upon your graphical environment, you may be able to access the Mandriva Control Center from your base system menu. In the security options, select the firewall icon and you will be presented with a list of common applications that may need access through your firewall. For example, checking the box for "SSH server" will open port 22 needed by the Secure Shell server for secure remote access. There is also an advanced section which will allow you to enter some less commonly used ports. For example, entering "8000/tcp" will open port 8000 on your PC to TCP-based network traffic.

Blocking or allowing network traffic is one layer of security, but how do you secure a service that you do allow the Internet or your intranet to connect to? Host based security is yet another layer.

4: Configuring the /etc/hosts.deny and /etc/hosts.allow files

In the preceding section we looked at the example of opening the Secure Shell service to network traffic by opening port 22 on our firewall. To further secure this server from unwanted traffic or potentially hackers, we may wish to limit the hosts or computers that can connect to this server application. The /etc/hosts.deny and /etc/hosts.allow files allow us to do just that.

When a computer attempts to access a service such as a secure shell server on your new Linux PC the /etc/hosts.deny and /etc/hosts.allow files will be processed and access will be granted or refused based on some easily configurable rules. Quite often for desktop Linux PC's it is very useful to place the following line in the /etc/hosts.deny file:
ALL: ALL

This will deny access to all services from all hosts. It seems pretty restrictive at first glance, but we then add hosts to the /etc/hosts.allow file that will allow us to access services. The following are examples that allow some hosts remote secure shell access:
sshd: 192.168.0.1 #allow 192.168.0.1 to access ssh
sshd: somebox.somedomain.com #allow somebox.somedomain.com to access ssh

These two files provide powerful host based filtering methods for your Linux PC.

5: Shutoff or remove non-essential services

Just like Windows there can be services running in the background that you either don't want or don't have a purpose for. By using the Linux command chkconfig you can see what services are running and turn them on and off as needed. Services that are not running don't provide security holes for potential hackers and don't take up those precious CPU cycles.

6: Secure your required services

If your new Linux PC has some services that will receive connections from the Internet make sure you understand their configurations and tune them as necessary. For example, if your Linux PC will receive secure shell connections make sure you check the sshconfig file (for Mandriva it is /etc/ssh/sshd_config) and disable options like root login. Every Linux PC has a root user so you should disable root login via ssh in order to dissuade brute force password crack attempts against your super-user account.

7: Tune kernel networking security options

The Linux kernel itself can provide some additional networking security. Familiarize yourself with the options in the /etc/sysctl.conf file and tune them as needed. Options in this file control, for example, what type of network information is logged in your system logs.

8: Connect the PC to a router

A hardware router is a pretty common piece of household computer hardware these days. This is the front line security to any home or business network and provides multiple PC's to share one visible or external Internet address. This is generally bad news for any hacker or otherwise malicious program that may take a look at your new Linux PC as it blocks any and all network traffic that you don't specifically allow. Home networking routers are just smaller versions of what the big companies use to separate their corporate infrastructure from the Internet.

9: Update

Always keep the software on your computer up to date with the latest security patches should you be running Linux, Windows, BSD or WhoKnowsWhat. Your distribution will release regular security patches that should be applied and are available off the Internet. As with Windows, this should always be your first Internet destination.

10: Other software

Your second Internet stop may be to install some other hardening or system monitoring software.

Bastille-Linux (http://www.bastille-linux.org/) is a program that can be used to "harden" or secure certain aspects of your new Linux PC. It interactively develops a security policy that is applied to the system and can produce reports on potential security shortcomings. On top of that it is a great tool to use for learning the in and out of securing your Linux PC.

Tripwire (http://sourceforge.net/projects/tripwire) is a software package that monitors your system binaries for unauthorized modifications. Often a hacker may modify system binaries that may be useful in detecting a system intrusion. The modified programs would then report false information to you allowing the hacker to maintain his control over your system.

Linux Command Line Tips

Command line skills are something you pick up over time. When something needs to be done, you work out how, and from then on you know how to do it. Few take the time to systematically learn the ins and outs of the tools at their disposal, however, and so may not be aware of all the possibilities of even the most basic utilities. In this tip we'll be looking at one of the staples of any shell toolbox: the find utility.

As the name suggests, find is a program for searching your disk for files and directories satisfying a given criteria. By default, it starts in the current directory and traverses down to all lower directories. Find is able to search based upon a number of different file attributes and also perform actions on the results, usually running some program for each result.

Let's take a look at a few examples: firstly, to find all html files in the current directory or lower you would use:

find -name "*.html" -type f

Now, this command has two tests, the first, "-name", is used to test against each filename in the search. If you need this to be case insensitive, use "-iname" instead. The second test is "-type" which is used to specify the kind of thing you are interested in. The "f" says we are looking for regular files, however we could have used "d" for directories or "l" for symbolic links, for example. A full list of options can be found on the find manpage.

In this example we don't specify a location since we were looking in the current directory. You can start the search in any directory (or directories) on your system, for example if we know that the html files will be in one of two places then we could make the search quicker and more accurate by specifying a start point:

find /var/www /home/nickg/public_html -name "*.html" -type f

This now searches in the Web server root, my home html root, and their subdirectories. Hopefully this will mean we get what we're looking for and not erroneous files like the Web browser cache, or html help files.

Find traverses down all subdirectories by default, but you can control this behaviour by specifying the maximum depth. If in the previous case you wanted only to search those two directories but go no further you can add "-maxdepth 1" to the command. Setting the max depth to 0 means that only the files given on the command line will be tested. Similarly you can set a minimum depth, so that you can avoid files sitting in the root.

Another use for find is to search for files belonging to a given user. So to search for all files belonging to me on my system I use:

find / -user nickg

The same thing can be done for searching based upon group, using the "-group" test.

The next category of tests relates to time, allowing you to search for files based on when they were last created, accessed or modified by using "-ctime", "-atime" and "-mtime" respectively. For example to find files created in the past day:

find -ctime -1

Note the "-" before the 1, this means that we're looking at a range backwards from the current day. If you need more precision you can use the by minute variations, "-cmin", "-amin" and "-mmin". If you've just made a mistake and you're not sure which files may have been affected, it's easy to narrow things down using find:

find -mmin -5

The standard action for find to perform on the files is to print out the filename, which is why if you've been following along you've ended up with these long lists of filenames. This is perfect if you want to use this data as input into another filter, but if you need a little more information about the results you can get find to give you the same sort of output as you'd get from ls -l:

find -user nickg -iname "*.html" -ls

Which will give permission and time info.

Lastly, you can get find to run any program on each result by using the "-exec" action. The following program will delete all files in your home directory with the extension ".tmp":

find -name ".tmp" -exec rm {} \;

The two braces ("{}") are replaced by each filename matched, and the escaped semicolon is needed to tell find when the command is finished. Find is often used in combination with chmod to quickly change file permissions over a large set of files, or with grep and sed to selectively find or modify text using regular expressions. This is just the tip of the iceberg where find is concerned, by using it as the input to a script you can automate time consuming tasks, such as cleaning up files that have not been accessed in a year, or automatically backing up modified files. This kind of power means that find remains one of the best tools a Linux user has at their disposal.