Sunday, February 8, 2004

Report Printing - Class Descriptions

Introduction

Printing a document programmatically is quite involved. Using the ReportPrinting library presented here, youll be able to print reports of many sections, with very little code.

Figure 1 - Part of a sample report

The reports are comprised of text sections (such as the title "Birthdays", and the other paragraphs), grids of data from a database (more specifically, from a DataView object) and many other primitives. Since the initial version of this document, I've already exteneded the framework to handle images (from .Net Image class), boxes (similar to the CSS box implementation) and more are coming every week.

Two important classes are introduced in this article.

  • ReportDocument - a class that inherits from PrintDocument and greatly assists in printing tabular reports consisting of data from a DataTable.
  • PrintControl - a control that simplifies the process of guiding the user through the printing process.

The section on the ReportDocument includes many helper classes to define sections, columns, and text styles. The PrintControl section includes a brief description of some of the dialogs used in the printing process.

Report Document Classes

There are several classes introduced into the ReportPrinting namespace. They work together for the printing of the above report (in addition to all the .NET Framework base classes that are used). Here is a quasi-UML diagram that shows the relationship between these classes. An open triangle is generalization (i.e. it points to the super-class in the inheritance chain). The black diamonds are composite (i.e. shows that one class instantiates members of another class). The dashed-lines are dependency (i.e. it uses the class).

(note 18-Sep-03, ReportSectionText is now SectionText, ReportSectionData is now SectionTable)

Figure 2 - UML diagram of classes

ReportDocument

ReportDocument extends from PrintDocument and is customized for printing reports from one or more tables of data. A ReportDocument object is the top-level container for all the sections that make up the report. (This consists of a header, body, and footer.)

The ReportDocument's main job is printing, which occurs when the Print() method is called of the base class. The Print() method iterates through all the ReportSections making up the document, printing each one.

The strategy design pattern is employed for formatting the report. An object implementing IReportMaker may be associated with the ReportDocument. This IReportMaker object is application specific and knows how to create a report based on application state and user settings. This object would be responsible for creating sections, associating DataViews, and applying any required styles through use of the TextStyle class. It will generally use the ReportBuilder class to assist with the complexity of building a report.

ReportSection

ReportSection is an abstract class that represents a printable section of a report. There are several subclasses of ReportSection, including ReportSectionText (which represents a string of text) and ReportSectionData (which represents a printable DataView). There are also container sections (which derive from SectionContainer class, which in turn derives from ReportSection). These containers hold child ReportSection objects (also known as subsections) to be printed. Lets take a quick look at how this might work with an example.

In the sample report shown at the top of this article, there is a paragraph of text followed by a table of data. (There are actually two paragraphs of text, one of which is a heading. Plus there is a page header, but well ignore all that for now.) We would create a ReportSectionText object to print the paragraph of text and a ReportSectionData object to print the table of data. To add both of these ReportSections to the ReportDocument, we must create a container. We would create a LinearSections container to hold these two sections. This container is then made the body of the ReportDocument. When the document is printed, the section container will first print the ReportSectionText, and then below that, it will print the ReportSectionData. Simply simply printing each section below the preceding one will result in the finished report. But there are many other ways to set up these classes.

SectionContainer

This abstract class defines a container of sections. There are two types provided with the framework: LinearSections and LayeredSections.

LinearSections

The LinearSections class is a subclass of SectionContainer, which is a subclass of ReportSection. Therefore, the LinearSections can be thought of as "a printable section of a report." However, it is also a container of one or more sections.

As its name implies, it lays sections out linearly -- that is, in a row or in a column. A property named Direction specifies if this container will layout sections going down the page (typical) or across the page (not as typical).

(see

Layouts for more information about how this class works.)

LayeredSections

The LayeredSections class is also a subclass of SectionContainer, which is a subclass of ReportSection. Therefore, the LayeredSections can be thought of as "a printable section of a report." It is also a container of one or more sections.

The child sections of a LayeredSections object are all painted on top of one another (creating layers). The first section added to a LayeredSections object is the bottom layer.  Subsequent ReportSection objects added to the LayeredSections object will be shown on top of each other.

(see Layouts for more information about how this class works.)

SectionText

The SectionText prints a string to the page. Two public properties are used to setup this section. Text is used to specify the string to print. TextStyle, described later, sets the font, color, alignment and other properties for the how the text is printed.

It is interesting to note that the string specified for this section can be just one word, or many paragraphs of text.

SectionTable

The SectionTable prints a table of data.  It uses a DataView object (from the .Net System.Data namespace) as the source of data.  It then uses a series of ReportDataColumns to provide the fomatting details.  These ReportDataColumns are similar to the DataGridColumnStyle class. Table wide formatting includes setting Header, Row, and Alternating row TextStyle's, along with setting the width and margins.

ReportDataColumn

The ReportDataColumn provides the necessary information for formatting data for a column of a report. For every column to be presented within a section of data, a new ReportDataColumn object is instantiated and added to the ReportSection. At a minimum, each column describes a source field from the DataSource (that is, a column name from the DataView) and a maximum width on the page.

The ReportDataColumn can be setup with its own unique TextStyle for both header and normal rows. Therefore, each column's data can be formatted differently (e.g. an important column could be bold and red). The TextStyle is also used to set the horizontal alignment (justification).

TextStyle

The TextStyle class allows styles and fonts to be added to text selectively, allowing default styles to be used when not explicitly set. All styles (except for the static TextStyle.Normal) have another style as their "default" style. Until a property is set (like bold, underline, size, font family, etc), a TextStyle object always uses the corresponding value from its default (or parent) style.

For example, a new style can be defined using Normal as its default, but setting bold.

TextStyle paragraphStyle = new TextStyle(TextStyle.Normal);
paragraphStyle.Bold = true;

It will have all the same properties as TextStyle.Normal, except it will be bold. A later change to Normal (such as below) will have the effect of increasing the size of both styles (Normal and paragraphStyle).

TextStyle.Normal.Size += 1.0f

ReportBuilder

ReportBuilder assists with the building of a report. This class is the main interface between your code and the ReportPrinting library. In many cases, you will never explicitly create any of the above objects.  Instead, the ReportBuilder will create them for you.

To instantiate a ReportBuilder, you must provide the ReportDocument to be built. Then you can call its various Add methods to sequentially add pieces to a report document.

Example:
The following example shows the creation of a report using the ReportBuilder. The following methods would be part of a class that implements IReportMaker (this is from example1 in the sample project).

private DataView GetDataView()
{
    DataTable dt = new DataTable("People");
    dt.Columns.Add("FirstName", typeof(string));
    dt.Columns.Add("LastName", typeof(string));
    dt.Columns.Add("Birthdate", typeof(DateTime));

    dt.Rows.Add(new Object[] {"Theodore", "Roosevelt", new DateTime(1858, 11, 27)});
    dt.Rows.Add(new Object[] {"Winston ", "Churchill", new DateTime(1874, 11, 30)});
    dt.Rows.Add(new Object[] {"Pablo", "Picasso", new DateTime(1881, 10, 25)});
    dt.Rows.Add(new Object[] {"Charlie", "Chaplin", new DateTime(1889, 4, 16)});
    dt.Rows.Add(new Object[] {"Steven", "Spielberg", new DateTime(1946, 12, 18)});
    dt.Rows.Add(new Object[] {"Bart", "Simpson", new DateTime(1987, 4, 19)});
    return dt.DefaultView;
    }

public void MakeDocument(ReportDocument reportDocument)
{
    // Clear the document
    reportDocument.ClearSections();

    // create a data table and a default view from it.
    DataView dataView = this.GetDataView();

    // create a builder to help with putting the table together.
    ReportBuilder builder = new ReportBuilder(reportDocument);
    
    // Add a simple page header and footer that is the same on all pages.
    builder.AddPageHeader("Birthdays Report", String.Empty, "page %p");
    builder.AddPageFooter(String.Empty, DateTime.Now.ToLongDateString(), String.Empty);

    builder.StartLinearLayout(Direction.Vertical);

    // Add text sections
    builder.AddTextSection("Birthdays", TextStyle.Heading1);
    builder.AddTextSection("The following are various birthdays of people who "
        + "are considered important in history.");

    // Add a data section, then add columns
    builder.AddDataSection(dataView, true);
    builder.AddColumn ("LastName", "Last Name", 1.5f, false, false);
    builder.AddColumn ("FirstName", "First Name", 1.5f, false, false);
    builder.AddColumn ("Birthdate", "Birthdate", 3.0f, false, false);
    // Set the format expression to this string.
    builder.CurrentColumn.FormatExpression = "{0:D}";

    builder.FinishLinearLayout();
        
}

IReportMaker

IReportMaker is an interface used to implement the strategy design pattern. An object that implements IReportMaker can be added to a ReportDocument. When the document is about to be printed, it automatically calls the single method MakeDocument(). The above example shows an implementation of that method to print a one-page report.

For example, you could have an application that can print either detailed reports or a shorter overview. The logic to make each of these reports would be located in separate classes.  Each class would implementing the IReportMaker interface. Your print dialog could have a "Print What" combo box to allow the user to select the type of report, and use the selection in the combo box to associate the correct implementation of IReportMaker with the ReportDocument.

Print Dialogs

The print dialog guides the user through the printing process. Most applications have some options that affect what is printed and how it is printed. Most windows applications customize the standard PrintDialog, adding an additional section at the bottom for various options. There are articles on extending the standard PrintDialog using MFC, but Ive yet to find anything for .NET. If someone creates a .NET control that looks like a standard PrintDialog and could easily be added to new Forms to create a customized PrintDialog or knows of some other way to extend the functionality of the .NET PrintDialog, please let me know.

PrintControl

To make printing easy for my applications, I created this very basic control that can be dropped onto any form. It gives the user options to setup, preview, submit (ok) or cancel. Providing a preview button and a page setup button on a print dialog are not standard in the windows interface, but I wish they were. So this control provides that functionality to your print dialog. Note, you can still provide access via a File menu (File | Print Preview, File | Page Setup).

Figure 3 - PrintControl user control

The control uses the following dialogs associated with printing:

  • PrintPreviewDialog
  • PageSetupDialog
  • PrintDialog

To use the print control, place it on a form. Set the Document property to a valid PrintDocument. (it doesnt have to just be the ReportDocument described earlier). Thats it!

You can customize a few things with the following properties:

  • ShowStatusDialog - The progress of the print job is shown in a status dialog. Default is true.
  • PrintInBackground - Indicates that printing should be done in the background. Default is true.
  • Printing - This event is raised prior to printing. It allows user code to setup for printing. (This is useful for dumping data from the GUI to a helper class, for instance).

Print Dialog with PrintControl

A sample form with a PrintControl is shown below. This dialog allows a user to select tables to print from the Northwind sample database.

Figure 4 - A dialog to prompt user for print settings and give them a chance to preview and setup the page.

Revision History

1-Sep-03 : Original article posted.

18-Sep-03 : Names of some classes changed.

C# Screensaver

This screensaver will display just about any media on your computer (pictures, video files, and audio files), thus it is "Super Cool".

It also can count down the number of days, hours, minutes, and/or seconds until some special day. It paints a message on the screen with the countdown.

Other features include: rotating files, a "do not play list", ignoring thumbnails, muting audio, multiple directories of media, and many others.

Download source code here
Download installer here

Installation

  1. If you have not already done so, install the Microsoft .NET Runtime from Microsoft Windows Update
  2. If you have not already done so, install DirectX 9.0 from Microsoft's DirectX to insure full functionality. If you choose not to install this, video files will not play properly. Microsoft points out that DirectX cannot be uninstalled, so you may want to make a Windows Savepoint (XP only).
  3. Download the latest Setup .msi file
  4. Run the Setup .msi file.
  5. Enable the screen saver on the display properties dialog. (Right-click on the windows desktop, select properties, select the Screen Saver tab, from the Screen Saver pull-down menu, select SuperCoolScreenSaver, then hit the Settings button.)

Usage

Since it is a GUI app, most dialogs are hopefully self-explanatory.

Enabling

To enable the screensaver, like any other, goto Display Properties Dialog > Screen Saver > Choose SuperCool

To setup the screensaver, either click the Settings button or choose Start Menu > All Programs > SuperCoolScreenSaver > Settings (note you can also run the screensaver from the StartMenu to use it just as a slide show).

Settings

There are four global controls:

  1. Set the amount of time (in seconds) that each picture or text message is display on the bottom of the Dialog. 
  2. 2. Run will give you a preview of the screen saver.
  3. 3. Cancel will quit without saving changes (including any pictures you rotated in "test" mode).
  4. 4. Ok saves all changes and exits the program.

There are also six tabs within the Settings Dialog.

Message

This tab allows you to setup a message to be displayed on the screen.  Type a message into the text box and check the "Show Text" to enable this feature.

You can use four special fields to countdown to some furtue date (e.g. New Year's, your birthday, vacation from work!)  The following strings have special meaning:

(days)
The total number of days until your chosen day.
(hrs)
The total number of hours until your chosen day.
(mins)
The total number of minutes until your chosen day.
(secs)
The total number of seconds until your chosen day.

Set the date and font through extra dialogs which are invoked by the two buttons on the dialog.  Random Colors, if enabled, will display the message in randomly chosen colors.

Pictures

Use this dialog page to setup the options for showing pictures. First be sure to enable "Show Media", as this must be enabled to show any media.

Then check the directory or directories that contain your media files.

If include all subdirectories is selected, files from the directory you check along with all subdirectories will be used.  If this is not selected, then you can individually select each directory to search for media.

File Types

This page allows you to select the types of files to be displayed.  By default, only jpg and jpeg are enabled.  Either click on the individual extensions to enable / disable or use the following buttons:

  • Select All will enable all supported file types
  • Clear All will disable all supported file types (no files will be displayed)
  • Select All Non-Music will enable images and video files, but not music files.

As you can see, there is quite a variety of supported file types.

Advanced

There are several options for the order in which files are displayed.  By default, it should sort them cronologically with the most recent first (which means you'll see your pictures in reverse order!)  If you want some variety you can enable Random Order or to just start at a random spot of all you pictures chooese Start At A Random Point.

Allow Keyboard To Advance turns on additional functionality while the screensaver is running. Press F1 while it is running for a list of commands.  Note, you must then use Escape or mouse movement to exit the screensaver.

Optionally, the screensaver will display filenames at the bottom of the screen.

Showing color information is rather useless.  It shows percentage of each color (red, green and blue) that makes up a displayed image.

You can setup a filter to block small pictures from being displayed.  For instance, setting the slider to 200 will show only pictures whose width and height are both larger then 200 pixels.  This prevents small thumbnails from being displayed, since they don't generally look very good fullscreen!

You can also have video/audio files muted (or optionally, muted between certain times of day - useful if you have your computer running all night and wish to not be awakened by media playing).

Rotated Pictures

This page will list all of the images which you have rotated.  That is, when the screensaver is running (and if you have enabled Allow Keyboard to Advance), hitting "R" will rotate the picture - but it won't touch the file.  If you'd like to update the files on the harddrive, then select the files (jpeg only) and click Fix Files.  You can also remove selected items off the list here and save the list to a text file for other programs to possibly parse (or for you to go through by hand and rotate).

Hide / Show will hide and show the preview window.

Removed Files

This page lists all of the files which you have chose not to display.  That is, when the screensaver is running (and if you have enabled Allow Keyboard to Advance), hitting "D" or "Delete" will place a file on a "Do Not Show" list.  You can see that list here and save it to a text file for parsing by another script (for instance, to delete them all).  If you'd like the file to be played again, select it and click Remove Selected.  This simply removes them from the list - it does NOT remove them from your harddrive.

To X or not, for RedHat linux

Sometimes it's useful to have an X GUI on your linux box, sometimes it just consumes resources. And sometimes you just want to ssh in and use xterm from a pc. I finally sat down and figured out the ideal setup for me, and this may help you as well. Especially if you are using linux primarily as a server, but still want the X GUI for days when it's your workstation.

Background

As I'm sure you are aware, linux can run with X (the graphical display) and use desktop managers such as Gnome, etc. This document is not a howto on X, GUIs, or the likes. This document assumes you already have a functioning linux system up and running with X, and want to turn that off and on as needed.

Run level services

Linux has up to 9 runlevels that it can run in. Each runlevel determines a set of features that are to be installed or used. There are six interesting runlevels, which are described in the file /etc/inittab as follows:

# Default runlevel. The runlevels used by RHS are:
#   0 - halt (Do NOT set initdefault to this)
#   1 - Single user mode
#   2 - Multiuser, without NFS (The same as 3, if you do not have networking)
#   3 - Full multiuser mode
#   4 - unused
#   5 - X11
#   6 - reboot (Do NOT set initdefault to this) 

Runlevels 3 and 5 are the two that are used for an "up and running" system. As indicated, only runlevel 5 has the X11 GUI. Thus, by changing the runlevel from 5 to 3, we can disable X11 GUI. And changing from 3 to 5 we can enable the X11 GUI. The following two commands do that:

su -
(type root password)
init 3

Changes to runlevel 3, thus disabling the GUI.

(still as root)
init 5

Changes to runlevel 5, thus enabling the GUI.

Other services

On my install, I found that in runlevel 3, I did not have the samba or web servers enabled. I'm not sure if that's just RedHat 8 distro, or common to many distro's. But here's how to fix it.

We want to see exactly what services are enabled and disabled differently between the runlevels. The directory /etc/rc.d has directories for each runlevel as shown below.

[root@devx]# cd /etc/rc.d
[root@devx]# ll
total 64K
drwxr-xr-x    2 root     root         4.0K Aug 15 05:03 init.d/
-rwxr-xr-x    1 root     root         2.3K Jul 14  2002 rc*
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc0.d/
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc1.d/
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc2.d/
drwxr-xr-x    2 root     root         4.0K Aug 15 04:48 rc3.d/
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc4.d/
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc5.d/
drwxr-xr-x    2 root     root         4.0K Jul 24 12:15 rc6.d/
-rwxr-xr-x    1 root     root          220 Jul 10  2001 rc.local*
-rwxr-xr-x    1 root     root          22K Aug 22  2002 rc.sysinit*
[root@devx]#

The init.d directory contains all the scripts that are used to start and stop processes. A look in a directory such as rc5.d reveals a link back to a script in init.d for each process that should be killed or started. This is a small sample of the listing for rc5.d:

(lines shortened by removing owner/group)
[root@devx]# ll rc5.d
total 0
lrwxrwxrwx    1    19 Feb 25 17:19 K05saslauthd -> ../init.d/saslauthd*
lrwxrwxrwx    1    16 Jul 22 14:58 K12mysqld -> ../init.d/mysqld*
lrwxrwxrwx    1    20 Feb 25 17:49 K15postgresql -> ../init.d/postgresql*
lrwxrwxrwx    1    13 Feb 25 17:25 K20nfs -> ../init.d/nfs*

lrwxrwxrwx    1    15 Feb 25 17:22 S05kudzu -> ../init.d/kudzu*
lrwxrwxrwx    1    18 Feb 25 17:24 S08iptables -> ../init.d/iptables*
lrwxrwxrwx    1    14 Feb 25 17:22 S09isdn -> ../init.d/isdn*
lrwxrwxrwx    1    17 Feb 25 17:19 S10network -> ../init.d/network*
lrwxrwxrwx    1    16 Feb 25 17:18 S12syslog -> ../init.d/syslog*

Links that start with K are processes that are killed (the indicated scripts are run with a parameter stop) and those that start with S are started (the scripts are run with the parameter start). To find out any differences between runlevels, do the following:

[root@devx]# diff rc3.d rc5.d
Only in rc3.d: K15httpd
Only in rc3.d: K35smb
Only in rc5.d: S85httpd
Only in rc5.d: S91smb
[root@devx]# 

This indicates that runlevel 3 stops httpd (web server) and smb (samba server), while runlevel 5 starts both of these. To make these runlevels consistent, do the following:

mv rc3.d/K15httpd rc3.d/S85httpd
mv rc3.d/K35smb rc3.d/S91smb
[root@devx]# diff rc3.d rc5.d
[root@devx]# 

Now, when you change to runlevel 3 via the "init 3" command, you won't lose these services.

Starting X from runlevel 3

You can start the X11 GUI from runlevel 3 with the following command:

2% startx

Once started, you can exit from the X11 GUI by simply logging out (using the RedHat menu), and you'll be back to your nice memory saving text console.

Default run level

Setting the default runlevel is also easy. Just su to root, and open the file /etc/inittab. Change the following line:

#id:5:initdefault
id:3:initdefault

Conclusion

Now when you boot your machine, it will have simple console prompt. It is still multiuser, so you can Alt+F1, Alt+F2, etc. to separate console windows. You can ssh / telnet into the box, and even bring up xterms on another workstation. Finally, you can start the X11 GUI as needed on the console by simply logging in and running 'startx'. Logging out of the GUI brings you right back to the nice console windows.

Fixing the Backspace Key for Exceed and RedHat Linux

This is really old information back from 2004, so it may not work anymore

This is perhaps the shortest article on this site. One simple step that can save hours of frustration using Exceed on RedHat linux when the backspace key doesn't work:

In Xconfig>(Keyboard) Input, edit the keyboard file you have selected. e.g. us.kbf.

Clear the "Shifted" mapping for the backspace key. This will restore the functionality to the backspace key.

Sharing a Printer with Samba

Disclaimer: I have no idea if this post is still applicable. Technology has changed so much since I wrote this 4 years ago, but just in case it's useful to somebody somewhere......

When I first shared my printer with the default RedHat 8 smb.conf, I could print from windows but had an annoying "Access denied: " error message and could not see jobs in the printer queue. If you're reading this page, first find out what print system you are using: LPRng is the one used for Redhat 7 and 8, CUPS is the prefered system now and is used in RedHat 9. I suggest going with CUPS.

This is the best site I've for setting up CUPS / Samba: http://www.buberel.org/linux/cups-samba.php

If you are using LPRng, then the following lines must be added to the smb.conf file.

  • The three commands (print, lpq and lprm) allows the windows Printers dialog to update the printer queue (list and remove print jobs).
  • The last line "use client driver = yes" eliminates a windows error of "Access denied".

[global]

# (obviously a lot of other stuff not shown here)

# The print command by which data is spooled to a printer under Linux. print command = lpr -r -P%p %s # The print command by which job queue information (printer status) # can be obtained. lpq command = lpq -P%p # The print command by which unwanted print jobs can be deleted # from the queue. lprm command = lprm -P%p %j [printers] comment = All Printers path = /var/spool/samba browseable = no # Set public = yes to allow user 'guest account' to print guest ok = no writable = no printable = yes use client driver = yes

Snapshot backups

Disclaimer: I haven't used this technique in over 4 years, so I have no idea if this is still possible or works as advertised. It should, but I haven't kept up on Linux and there could be changes to some of the underlying technologies...

Mike Rubel found that using rsync on linux makes for a powerful snapshot-style backup tool. Using a script or two and calling with cron, you can have hourly, daily, weekly and monthly snapshots of your files.

Background

For background information, check out his page mikerubel.org. Another utility that looks very complete (and based on rsync as well) is rsnapshot.

Quick introduction

We'll setup some drives, two scripts, and a few files indicating the directories that are to be backed up. The amount of time spent on this last step is proportional to how much you want to trim the size of your backups.

We'll setup a destination hard drive to hold snapshots. They'll look something like this:

drwxr-xr-x    5 root     root         4.0K Aug 15 22:00 daily.0/
drwxr-xr-x    5 root     root         4.0K Aug 14 22:00 daily.1/
drwxr-xr-x    5 root     root         4.0K Aug 13 22:00 daily.2/
drwxr-xr-x    5 root     root         4.0K Aug 12 22:00 daily.3/
drwxr-xr-x    5 root     root         4.0K Aug 16 02:00 hourly.0/
drwxr-xr-x    5 root     root         4.0K Aug 16 00:00 hourly.1/
drwxr-xr-x    5 root     root         4.0K Aug 15 22:00 hourly.2/
drwxr-xr-x    5 root     root         4.0K Aug 15 20:00 hourly.3/
drwxr-xr-x    5 root     root         4.0K Jul 31 22:00 monthly.0/
drwxr-xr-x    5 root     root         4.0K Jun 30 22:00 monthly.1/
drwxr-xr-x    5 root     root         4.0K Jun  5 22:00 monthly.2/
drwxr-xr-x    5 root     root         4.0K Aug  9 22:00 weekly.0/
drwxr-xr-x    5 root     root         4.0K Aug  2 22:00 weekly.1/
drwxr-xr-x    5 root     root         4.0K Jul 26 22:00 weekly.2/

As you can see, we'll have access to snapshots from various points in time over the past three months. More importantly, if you need to restore a backup for a crashed system, the most recent snapshot (hourly.0) is a complete image. No need for restoring a full and several partial backups.

The secret that makes this work is the way file systems store files and links to those files on a hard drive. Files that haven't changed are stored once, with hard links from each of the snapshot directories. Files that do change will be copied, and given a new link in hourly.0, while allowing all previous snapshots to continue linking to the old version. See the links under Background for more information.

A new harddrive

For optimum protection, the backups should be stored on their own harddrive. I'll quick step you through the process of installing a new ide harddrive, creating a read/write partition for root access, and creating an NFS readonly share for everyone to see.

Installing new drive

For a quick overview (assuming IDE, no RAID):

Install the drive physically. Make note if it is primary / secondary and master / slave.

As root, use the command fdisk /dev/hd[abcd] to create a linux ext2 partition (id 83) and write the table to disk. ([abcd] should just be one letter, where a is primary-master, b is primary-slave, c is secondary-master, d is secondary-slave. I'll use b for the rest of this example.)

mke2fs -j /dev/hdb1will install la journaled file system on the first partition of the primary-slave drive. A journaled file system is preferred because it can recover from error much more easily.

Mounting for root access

Create a directory to mount the backup snapshots - someplace only root has access. I suggest /root/mounts/backups. To mount the drive, add the following line to: /etc/fstab

/dev/hdb1               /root/mounts/backups  ext3    ro             0 0

And mount it:

mount /root/mounts/backups

And test that you can read and write files to the mount point.

Mounting readonly for everyone else

Based on advice from Mike Rubel (see links above), we'll make an nfs readonly share (for example /var/backups) for users to see the snapshots.

Start by adding the following to /etc/exports
/root/mounts/backups localhost(secure,ro,no_root_squash)

Next make sure nfs and portmap are both installed and running. (Either check the scripts from /etc/rc.d/rc3.d/ and /etc/rc.d/rc5.d/ to insure that they are installed or use the RedHat GUI for "Server settings | Services"to enable these.)NFS will have to be restarted to get the changes to/etc/exports. (You can do this from the GUI Server settings | Services | NFS | Restart or the command /etc/init.d/nfs restart;)

Unfortunately, we cannot mount/var/backups with a simple entry in fstab, since the nfs share will not be available as the fstab file is read during bootup. Instead, I suggest adding the following line to /etc/rc.d/rc.local- as this is the last file parsed during bootup. Thus, nfs will be running which is a pre-requisite.

mount -o ro localhost:/root/mounts/backups /var/backups

Check that everything works. Make sure that you cannot add or edit files in the /var/backups directory. Try remounting the /root/backup_snapshots directory read/write with the following:

% mount -o remount,rw /root/mounts/backups
% mount
(see a listing of all mount points, make sure /root/mounts/backups is read/write
% mount -o remount,ro /root/mounts/backups
(to put it back to readonly)

Setting up the scripts

Download and install

The next step is to download and install the scripts. I installed mine at /etc/backups.

After downloading and unzipping, you should have two new directories: /etc/backups/scripts and /etc/backups/excludes I'll describe each of these later.

Windows sharing

Decide the directories you'd like to have backed up. I'm assuming you'll want to back up files from both a windows pc and your linux box. To backup your windows pc, you'll have to first share out the necessary directories or drives and mount them with samba (alternatively, you can use ssh and rsync directly, but I think that takes a bit more work). I made a new user on windows (winuser below) with a simple password. I gave this user the most minimal permissions, and then I shared out my C drive readonly for just this user. If you are comfortable with this, add the following to /etc/fstab. The only security problems I see with this is that anybody who has access to your linux box now has readonly access to your windows shares. If you have a better idea for samba sharing by machine (similar to nfs) leave a comment.

//windows/c_share /root/mounts/c smbfs username=winuser,password=pass,
  uid=username,gid=group0 0

You can leave out the password, but then you will be prompted for it on a reboot, which is annoying.

Customize scripts

We'll first setup the scripts, then the excludes.

There are two scripts. First is make_snapshot.bash, written by Elio Pizzottelli based on original work by Mike Rubel. Second is run_backups.bash which I wrote as a wrapper. For both scripts, you'll have to setup the paths to required executables. For run_backups.bash, it would be as follows:

MAKE_SNAPSHOT=/etc/backups/scripts/make_snapshot.bash;
LS=/bin/ls;
ECHO=/bin/echo;
AWK=/usr/bin/awk;

We'll look at the remaining steps for run_backups.bash

  1. Setup the backup device and mount point. The following assumes second IDE harddrive mounted in /root/backup_snapshots
    BACKUP_DEVICE=/dev/hdb1;
    BACKUP_MOUNT_POINT=/root/mounts/backups;
  2. Setup the number of hourly, daily, weekly, and monthly snapshots. Also, setup the day of week and day of month to perform weekly and monthly snapshots. A few notes, the NUMBER_OF_HOURLY must be 3 or greater or else no backup will ever occur. The name hourly is somewhat wrong here because hourly can actully be every 2 hours, every day, or whatever frequency you setup cron to run the script. (I use every 2 hours). Because of a requirment of make_snapshots.bash, the NUMBER_OF's must be 3 or greater for that category (hourly, daily, weekly or monthly) to be valid.
    # Number of each backup
    # To be valid, the NUMBER_OF values
    # must be greater than or equal to 3.
    # Setting to 0, 1, or 2 will prevent that backup from occurring.
    # HOURLY must be run in order to get anything.
    # Note that, however, you can have this entire script only run once
    # a day by cron, hence "HOURLY" becomes "DAILY"
    NUMBER_OF_HOURLY=4;
    NUMBER_OF_DAILY=3;
    NUMBER_OF_WEEKLY=3;
    NUMBER_OF_MONTHLY=3;
    
    # Pick the day of week and day of month for those backups
    # 0 disables, 1=Monday, 7=SUNDAY
    WEEKLY_BACKUP_DAY=7;
    # 0 disables, otherwise choose a number
    # 01-31, although better not to go above 28 if you want
    # backups in february
    MONTHLY_BACKUP_DAY=01;
                    
    NUMBER_OF_HOURLY=4;
    NUMBER_OF_DAILY=3;
    NUMBER_OF_WEEKLY=3;
    NUMBER_OF_MONTHLY=3;
  3. Now to setup the shares to be backed up. There are four arrays which must be setup. For each backup, we specify the source directory, the destination directory (which will be under the BACKUP_MOUNT_POINT specified earlier), an EXCLUDES file (which we'll discuss next) and any other options for the rsync program. This is an example of a backup of all of the linux box (source directory is /). The destination is a directory linux (which will be /root/mounts/backups/linux). Note: we'll use excludes later to prevent backing up of directories and files that shouldn't be backed up, including the /var/backups directory and /root/mounts directories. We point to a file which has this infomation at /etc/backups/excludes/linuxroot_exclude.
    #set up backup 1 - linux box
    SOURCE_DIR[0]=/;
    DEST_DIR[0]=linux;
    EXCLUDES[0]=/etc/backups/excludes/linuxroot_exclude;
    OPTS[0]="";
  4. Let's look at another example, for a windows share:
    #set up backup 2 - windows box
    SOURCE_DIR[1]=/root/mounts/c;
    DEST_DIR[1]=win;
    EXCLUDES[1]=/etc/backups/excludes/windows_exclude;
    OPTS[1]="";               
  5. Finally, make sure to setup the NUMBER_BACKUPS to the number of backups defined. (e.g. 2 if one linux and one windows)
    NUMBER_BACKUPS=2;

Customize excludes

As mentioned earlier, the excludes allow you to specify directories and files to be included and excluded from a backup. Their syntax can be somewhat tricky, so you'll need to reference rsync documentation.

Backup 1 - Linux

The idea is simple. In the backup number 1 above, we specified that we wanted all of the linux file system backed up (by using / as the path). There are, however, only a few directories which are important to me. /home, /var/www, /etc, and /root to name a few. This is the excludes file in /etc/backups/excludes/linuxroot_exclude that I use to get those directories. (The notes in parenthesis are not really in the file, but are here for your explanation)

+ /var/         (include the directory /var
               the trailing / means it will only match a directory, not a file
               it includes all subdirectories as well)
+ /var/www/     (include the directory /var/www)
- /var/*/       (now exclude all other directories under /var
               the order was important of the above statements.  Excluding all
               var subdirectories first would invalidate the include of /var/www)
- /var/*        (and exclude all files under /var)
- Pictures/     (exclude any directory Pictures)
- manual/       (exclude any directory manual)
+ /root/        (include /root directory)
- /root/mounts/ (exclude /root/mounts directory -
               since we don't want to backup the backups!)
+ /etc/         (include /etc/ directory)

- *~            (exclude any file ending with ~ )
- /*/           (exclude all other directories under / )
- /*            (exclude all other files under / )
          

Backup 2 - Windows

Here's the excludes file for the C drive of my pc

+ /IISRoot/                   (include a directory)
+ /projects/                  (include a directory)
- tivo.bak                    (exclude a really big file)
+ /Documents and Settings/    (include a directory)
- Application Data/           (now exclude Application Data directory)
- Local Settings/             (and exclude Local Settings directory)
- My Music/                   (and a few other directories that don't fit)
- My Pictures/
- My Videos/
- My Slide Shows/
- ZIPFILES/
- nobackup/
- *.mp3                       (and don't back up any mp3 file)
- /*/                         (exclude any other directory at the root level)
- /*                          (exclude any other file at the root level)

Backup 3 - Outlook Express

I made another backup, in addition to the two shown above. This is a specific backup for outlook express mailboxes. It seems OE decides to update the time stamps on all folders every few minutes. This causes a lot of problems for backing up. In fact, just having a really big inbox getting updated with a few new messages every hour can consume a bit of drive space on the backups since an entirely new copy will be made for 4 hourlies + 3 dailies + 3 weeklies + 3 monthlies. So it might be best to just backup the address book or else only save one extra copy of the inbox every day or so. But here's my setup:

SOURCE_DIR[2]=/root/mounts/c;
DEST_DIR[2]=win_oe;
EXCLUDES[2]=/etc/backups/excludes/outlook_exclude;
OPTS[2]="size-only";

Note the size only option here. This is passed to rsync, so that it only judges a changed file by size (and not date, ownership, etc). And here is the corresponding excludes file

+ /Documents and Settings/
+ /Documents and Settings/username/
- /Documents and Settings/*/
+ /Documents and Settings/username/Application Data/
- /Documents and Settings/username/*/
- Deleted Items.dbx
- microsoft.public*     (exclude some newsgroups I'm on)
- /*/
- /*

Automate

To get this all to backup regularly, simply add an entry into the root crontab.

su - root
(password)
crontab -e
(then in editor)
0 0-23/2 * * * /etc/backups/scripts/run_backups.bash

Note: You'll want to insure that the above scripts are secured to just root (owner root:root, permissions 755 or such) since you don't want somebody else changing them to do arbitrary things! While I'm on the subject of security, it is also a good idea to change the permissions of files in the /root/mounts/backups folders for your linux backup.a You may not want somebody having read access to all of /etc or /root or anything else backed up.

Results

I've been using it for months without a hitch. Please don't use any information I've posted against me or my systems. Finally, I can't guarantee it to work for you. I've had some issues with PC files being copied over even if they haven't changed, so it's not perfect. But it has also come in quite handy on more than one occasion when trying to undo changes that I didn't mean to code and other files. And if the really unfortunate were to happen, such as a system crash, it's good to know there a second copy.

Good luck, and let me know of any comments on the above.

Tutorial 3 - Types and parameter passing primer

This primer explains three different concepts.

  • Value types and reference types
  • Passing by value and passing by reference
  • Garbage collection (brief overview)

Background

I recommend the following sites for more thorough explanations (especially Jon Skeet's two pages) or just another viewpoint.

Value Types and Reference Types

Every object in C# (and the .NET environment) is one of two types: Value or Reference. I'll pull an ASCII art from Chris Brumme's BLOG.

              System.Object
                 /       \
                /         \
         most classes   System.ValueType
                            /       \
                           /         \
                  most value types   System.Enum
                                       \
                                        \
                                       all enums

Value Types

The section "most value types" in the above picture includes most primitives like int, float, and double. Structs (see the MSDN struct tutorial) are also value types. A struct is a "light-weight" class. It is declared with syntax very similar to that for declaring a class. But there are some differences in the way it is used since it is a value type. One struct that is commonly used, but sometimes overlooked as a value type, is DateTime. When a value type is created, a spot in memory is reserved for it and the contents are placed there.

MSDN Link: list of value types

The following commands declare and assign a variable x. In line 1, the variable x is declared as an int, which is a value type. This creates a spot in memory for the variable x sized to hold an int. In line 2, the value 37 in placed in that spot. Thus we say x has a value of 37.

1          int x;
2          x = 37;
The same two lines can be expressed as:
    int x = 37;

Some value types use the new keyword to call a constructor for setting an initial value. The following command creates a spot in memory to hold a DataTime struct, and sets the value of that spot in memory to 2000-01-01.

    DateTime dt = new DateTime(2000, 1, 1);  // Jan 1, 2000

Reference Types

The section "most classes" in the above picture includes the reference types. This includes all user defined classes (like our Airplane class from before). It also includes the string class.

Creating a reference type object requires the new keyword. The following command declares a variable airplane1, which holds a reference to an Airplane object on the heap. The Airplane object on the heap contains the data such as "Boeing" and "747". The variable airplane1 is similar to a C++ pointer in that all it really holds is an address (or reference).

    Airplane airplane1 = new Airplane ("Boeing", "747");

The following line of code does not actually create a new Airplane object. It simply creates a variable (airplane1) that could hold a reference to an Airplane object.

    Airplane airplane1; // no object created

Differences between value and reference types

There are a few key differences between value types and reference types.

  • Reference types are always stored on the heap. Value types can be stored on the stack (for local variables) or on the heap (generally for non-local variables, like static variables and member variables of a type that is already on the heap). See Memory in .NET - What goes where? by Jon Skeet for a more thorough explanation.
  • The value of a reference type variable can be null. That is, a reference type variable doesn't have to point to a valid object. Value type variables, on the other hand, cannot have a null value.

Analogy for value and reference types

I like to use the following analogy to understand value types and reference types. (I presented this earlier in Classes, objects and methods).

Reference types are like balloons and value types are like balls. A variable is like your hand. When you hold a balloon, you use a string to hold onto it. This string is a reference to the balloon. This is similar to how a variable holds a reference to a reference type object. When you hold a ball, you hold the actual ball in your hand, no strings. This is similar to how a variable holds a value type object. It holds the actual object (or value), no references and no strings attached.

We will continue this analogy later.

Passing by Value and Passing by Reference

This is an important concept in C# and .NET. I highly recommend either of the two pages listed above on Parameter passing.

There are two ways of passing variables to a method:

By value
Passes the value that a variable holds. Think of it as passing a copy of a variable.
By reference
Passes a reference to a variable. Think of it as passing the actual variable itself, as opposed to simply whatever value it contains.

The default is to pass by value. The keywords ref or out are required before a parameter to pass by reference.

The above statements always hold for all variables: value type and reference type. You either pass the variable's value or a reference to the variable. There is a lot of confusion due to the similarity in terms value type, reference type and passing by value and passing by reference. To alleviate this confusion, we'll look at each of the following four combinations:

  1. Passing value types by value
  2. Passing value types by reference
  3. Passing reference types by value
  4. Passing reference types by reference

Passing value types by value

FunctionA (below) simply increments a number by 10. I've declared it static which means that it is not part of a specific object, but rather part of the class itself (even though the class isn't shown). This has no bearing, however, on the principles demonstrated below.

public static void FunctionA (int x)
{
    x += 10;
}

public static void Main () 
{
    int myInt = 37;
    Console.WriteLine ("Before calling function, myInt = " + myInt);
    FunctionA (myInt);
    Console.WriteLine ("After calling function, myInt = " + myInt);
    Console.ReadLine();
}

In this example, Main creates an int (myInt) and prints the value to the screen. It then passes myInt, by value, to FunctionA. This means that FunctionA gets a new copy of the value of myInt. When FunctionA increments x by 10, it does not change the value of myInt since it is changing its own copy. Thus, the result is as follows:

Before calling function, myInt = 37
After calling function, myInt = 37

Analogy

To continue the balloon / ball example... Calling a function is like handing a balloon or ball over to another friend. In this example, we are talking about value types, which are analogous to balls Passing by value doesn't have a perfect analogy since it is like passing a copy. Thus, it would be like passing a copy of the ball that is in your hand over to a friend. You keep your original ball and your friend can't touch it.

Passing value types by reference

FunctionB (below) also increments a number by 10. However, by using the keyword ref when declaring the parameter int x, FunctionB indicates that the int must be passed by reference. The C# compiler requires that we also use the ref keyword when calling FunctionB so that it is obvious that we know what we are doing. We'll see why in a second.

public static void FunctionB (ref int x)
{
    x += 10;
}

public static void Main () 
{
    int myInt = 37;
    Console.WriteLine ("Before calling function, myInt = " + myInt);
    FunctionB (ref myInt);
    Console.WriteLine ("After calling function, myInt = " + myInt);
    Console.ReadLine();
}

In this example, Main creates an int (myInt) and prints the value to the screen. It then passes myInt, by reference, to FunctionB. This means that FunctionB gets a reference to myInt, not a new copy myInt's value. When FunctionB increments x by 10, it also changes the value of myInt (since x more or less points to myInt). Thus, the result is as follows:

Before calling function, myInt = 37
After calling function, myInt = 47

Using ref allows a function to change the contents of a variable passed in, which can lead to tricky-to-debug systems and spaghetti-code. The C# compiler requires that ref be used on both the declaration of FunctionB and the call of FunctionB (within Main) so that it is immediately obvious that variables passed into FunctionB may not have the same value when FunctionB returns.

Analogy

To continue the balloon / ball example: In this example, we are still talking about value types, which are analogous to balls. Passing by reference is like passing your ball over to your friend. Then when your friend is finished (the function returns) you get it back, along with any changes he made to it.

Passing reference types by value

FunctionC (below) uses the reference type StringBuilder. I chose StringBuilder because it is a simple, easy to use class that meets the requirements of being a reference type. Any other class would behave the same here.

//Note, StringBuilder is part of System.Text namespace.
public static void FunctionC (StringBuilder x)
{
    x = new StringBuilder ("Hello Universe");
}

public static void Main () 
{
    StringBuilder sb = new StringBuilder ("Hello World");
    Console.WriteLine ("Before calling function, sb = " + sb);
    FunctionC (sb);
    Console.WriteLine ("After calling function, sb = " + sb);
    Console.ReadLine();
}

In this example, Main creates a new StringBuilder instance with an initial value of "Hello World" and assigns it to the variable sb. Thus, sb holds a reference to our new StringBuilder object. FunctionC is called and sb is passed by value. This means that the reference to the StringBuilder object is being passed by value. FunctionC gets the value of the reference (or in other words, it gets a new copy of this reference). At the start of FunctionC, printing x would print "Hello World". The variable x is assigned to a new StringBuilder object with a value of "Hello Universe". However, this does not affect sb. After calling the function, sb still prints "Hello World".

Results

Before calling function, sb = Hello World
After calling function, sb = Hello World

Analogy

To continue the balloon / ball example: In this example, we are now talking about reference types, which are analogous to balloons. Remember that you only hold a string to a balloon, not the actual balloon. Passing by value is like getting a new string, connecting it to your balloon, and then giving this new string to your friend. If your friend cuts the string or connects it to a different balloon (as is done in the above example), it has no affect on your string.

Passing reference types by reference

FunctionD (below) also uses the reference type StringBuilder.

//Note, StringBuilder is part of System.Text namespace.
public static void FunctionD (ref StringBuilder x)
{
    x = new StringBuilder ("Hello Universe");
}

public static void Main () 
{
    StringBuilder sb = new StringBuilder ("Hello World");
    Console.WriteLine ("Before calling function, sb = " + sb);
    FunctionD (ref sb);
    Console.WriteLine ("After calling function, sb = " + sb);
    Console.ReadLine();
}

Again in this example, Main creates a new StringBuilder instance with an initial value of "Hello World" and assigns it to the variable sb. Thus, sb holds a reference to our new StringBuilder object. FunctionC is called and sb is passed by reference. This means that the reference to the StringBuilder object is being passed by reference. The variable x will hold the same reference as sb holds in Main. When the variable x is assigned to a new StringBuilder object with a value of "Hello Universe", it also affects sb. After calling the function, sb prints "Hello Universe".

Results:

Before calling function, sb = Hello World
After calling function, sb = Hello Universe

Analogy

To continue the balloon / ball example: In this example, we are talking about reference types, which are analogous to balloons. Remember that you only hold a string to a balloon, not the actual balloon. Passing by reference is like handing the string in your hand over to your friend. When your friend is finished (i.e. at the end of the function) he hands it back to you. If your friend cuts the string or connects it to a different balloon (as is done in the above example), then you have a new balloon at the end of your string. What a nice surprise!

Parameter Passing Summary

We've discussed passing by value and passing by reference for both value types and reference types. I should point out that when dealing with reference types, it doesn't matter how you pass the object, the calling function will always be able to make changes to the object itself. In the following example, we are passing by value, however sb and x both point to the same object, so x.Append() will affect sb.

//Note, StringBuilder is part of System.Text namespace.
public static void FunctionE (StringBuilder x)
{
    x.Append(" from me!");
}

public static void Main () 
{
    StringBuilder sb = new StringBuilder ("Hello World");
    Console.WriteLine ("Before calling function, sb = " + sb);
    FunctionE (sb);
    Console.WriteLine ("After calling function, sb = " + sb);
     Console.ReadLine();
}

Results:

Before calling function, sb = Hello World
After calling function, sb = Hello World from me!

Garbage Collection

To summarize, I'll use the words of Jeffrey Richter:

The garbage collector checks to see if there are any objects in the heap that are no longer being used by the application. If such objects exist, then the memory used by these objects can be reclaimed

Analogy

When you let go of a string, the attached balloon floats away. Even if two balloons are tied together, they will both float away if they are not tied down. The garbage collector is a small plane that goes around collecting these balloons. Note, the plane doesn't have any set schedule or order for collecting balloons, it just happens as needed. (Ok, there are some rules about when it can happen, but not when it will happen).

When the garbage collector needs resources, it will find all of the objects that are not "tied down" and collect them. There are a few more advanced topics when it comes to Finalizers and the IDisposable interface. We'll get into that later.