1. Summarization This movie is according to a true story in a published book “Twelve Years a Slave”. The story took place in the 1840s and Solomon Northup was the main character in the story. At the start of the story, Solomon was a free man. He was an expert violin player. He had a wonderful family with two kids, and owned a house in the New York State. But unfortunately, all his family members’ skin color was black, which finally brought them a disaster with no exception. They were kidnapped by two whites and then transported to the south. Solomon got a new name, Platt. The couple was bought by a decent slaver, Mr. William Ford, but the boy was chosen by another slaver and the little girl was kept by the slave seller. So the children were apart from their parents, especially their mother, Eliza. Eliza then fell into despair and was crying almost each past second. The mistress couldn’t stand her any longer, so Eliza was hung dead. Few days later, because of fighting against William’s chief carpenter, Platt was almost hung dead. In order to protect his slave, Ford transferred Platt to another slaver, Edwin Epps. Epps was a hard man. He whipped his slaves a lot. Platt sometimes got whipped because he couldn’t pick enough cotton. It’s pretty hard. There was a white among them, called Armsby. It appeared like he was compassionate to the Negros. So, Platt asked him whether he could send a letter to the north for him. Unfortunately, Armsby cheated him. Few years later, there was another white man worked in Epps plantation, called Bass. Well, this guy seemed like that he believed white and black alike. He even tried to persuade Master Epps. Platt told Bass his sorry experience and begged him for help. This time, it worked. So, after twelve years of being a slave, Solomon was free again. 2. Good Sentences a. Gentlemen, whoever moves that nigger is a dead man. b. You have no claim to his life. c. This conversation concerns what is fact and what’s not. Then it must be said, that there is no justice, or righteous in this slavery. d. Laws change, man. Universal truths are constant. A plain and simple fact that what is true and right is true and right for all. White and black alike. 3. Feelings This story happened right before the Civil War. This background makes me think about Uncle Tom’s Cabin. I read that book when I was in primary school, and that book had impressed me a lot. I just couldn’t understand why Tom didn’t run away at that age. But, I did respect Tom pretty much. He just showed me what royal is. And thanks the writer for writing such a novel to show us the black’s lives under the damned slavery. Well, this movie does the same things for us, too. It shows us some good men among the white like Mr. Bass. It shows us some bad guys, like Master Epps. It also shows us some people who used the black to exchange their better life although they knew it’s unjust, like Master Ford and Mr. Armsby. Martin had said that “All men are created equal”, I reckon that this must be the point of this movie. Twelve Years A Slave

 

第二篇：A Retrospective on Twelve Years

Proceedings of LISA '99: 13th Systems Administration Conference

Seattle, Washington, USA, November 7–12, 1999

A R E TR O S P E C T I V E O N T W E L V E Y E A R S

O F L I S A P R O C E E D I N G S Eric Anderson and Dave Patterson

THE ADVANCED COMPUTING SYSTEMS ASSOCIATION

? 1999 by The USENIX AssociationAll Rights ReservedFor more information about the USENIX Association:Phone: 1 510 528 8649FAX: 1 510 548 5738Email: office@usenix.orgWWW: Rights to individual papers remain with the author or the author's employer.

Permission is granted for noncommercial reproduction of the work for educational or research purposes.

This copyright notice must be included in the reproduced paper. USENIX acknowledges all trademarks herein.

ARetrospective on Twelve Yearsof LISA Proceedings

Eric Anderson–University of California, BerkeleyDave Patterson–University of California, Berkeley

ABSTRACT

We examine two models for categorizing tasks performed by system administrators. Thefirst model is the traditional task based model.The second model breaks tasks down by thesource of the problem. Wethen look at the historical trends of the last 12 years of LISAproceedings based on these models. Finally,weanalyze some of the more important tasks doneby system administrators and propose future research in those areas. Our hope is that some ofthe academic rigor in analyzing research can be brought to systems administration withoutlosing the practicality that makes the research valuable.

Introduction

System administrators don’thave a lot of time

for introspection of their field. So work is repeated

and new administrators, or people trying to do

research on system administration, don’tknow where

to start. Toprovide a starting point, we have examined

the last twelve years of LISA proceedings and have

categorized the papers in two separate ways. One cate-

gorization is by problem causes, and has the advantage

that it will apply to any task in system administration.

The second categorization is the traditional task break-

down, which shows us where past research has been

focused.

In addition to categorizing the papers, we exam-

ine the trends in the categorization over the last twelve

years. Wefind that some tasks were solved, and then,

due to external changes, needed more work. Wealso

find that some tasks have had a remarkable amount of

effort focused on them without a complete solution.

We then examine in more detail the more popular

areas of research both to gain historical understanding

and to consider future directions.

So that others can more easily build on our work,

we make the complete set of data including both cate-

gorizations, brief summaries, and bibliographic infor-

mation available on the web from http://now.cs.berke-

ley.edu/Sysadmin/categorization/.

The next two sections examine the two models,

and then the fourth section shows historical trends.

The fifth section focuses in on a number of important

tasks and examines each in detail. The final section

provides a few brief conclusions.

AModel of Tasks

The traditional approach for categorization is to

group related papers by the task each targets. Wedid

this for all of the papers. Wecontinued the aggregation

process starting with the list of tasks having at least

two papers and built a hierarchy of tasks as shown

1999 LISA XIII – November 7-12, 1999 – Seattle, WAbelow.The categories are sorted by popularity; thepaper count is shown in brackets; ties are brokenalphabetically.There were a total of 342 papers, and64 separate categories.?Services [75]?Backup [28]?Mail [20]?Printing [11]?News [5]?NFS [4]?Web[3]?DNS [2]?Database [2]?Software Installation [57]?Application Installation [32]?OS Installation [14]?User Customization [8]?Software Packaging [3]?Monitoring [44]?System Monitoring [14]?Resource Accounting [6]?Data Display [5]?Network Monitoring [5]?Benchmarking [4]?Configuration Discovery [4]?Host Monitoring [4]?Performance Tuning [2]?Management [40]?Site Configuration [27]?Host Configuration [7]?Site Move [4]?Fault Tolerance [2]?Miscellaneous [40]?Trouble Tickets [9]?Secure Root Access [8]?General Tool [6]?Security [6]?File Synchronization [4]?Remote Access [3]?File Migration [2]?Resource Cleanup [2]95

ARetrospective on Twelve Years of LISA Proceedings Andersonand Patterson

committee may also have affected the papers accepted

based on their views of what should be in the confer-

ence, or because of a limited selection of available

papers. Finally,some papers may be missing because

companies consider the information to be proprietary.

We believe that the classification is useful, but keep-

ing the weaknesses in mind will help prevent us from

drawing incorrect conclusions.

AModel of Problem Sources

Asecond model based on the source of a prob-

lem is shown in Figure 1.The source of the problem

is labeled on the edges leading out from the center (the

happy state) of the state transition diagram. The edges

leading back in to the center represent tasks performed

to return the system to a happy state. The model was

derived in part from the time surveys, which indicated

that administrators spent about a third of their time on

each of these tasks.

This model is fairly general and hence is able to

cover all types of things done by administrators. Either

administrators are trying to improve people (training)

or trying to improve machines. If they’re trying to

improve the machines, it’seither because the

machines need to do something different (configura-

tion management) or because they need to get back to

doing what they used to do (maintenance).

Examination of the Different Categories

Configuration management tasks will remain so

long as people change how they want to use the sys-

tem. Only by freezing how the system is used can we

eliminate configuration management tasks. Even a

simple appliance like a toaster has a few configuration

tasks (plugging it in, adjusting the amount of toasting).

The tasks have been simplified by limiting choices;

adding choices inherently increases complexity.

Maintenance tasks may be eliminated by build-

ing systems that recover from internal faults. If a

maintenance task can’tbeeliminated (for example,

purchasing replacement hardware), the goal should be?Management [35]?Accounts [23]?Documentation [4]?Policy [3]?User Interaction [3]?White Pages [2]?Networking [19]?Network Configuration [9]?LAN [4]

?WAN[4]?Host Tables [2]?Improvement [18]?Self Improvement [7]?Models [5]?Software Design [4]?Training Administrators [2]?one paper on topic [19]We can see that there are many different types oftasks, but a few subjects are very popular: Backup,Mail, Application Installation, Site Configuration, andAccounts. Wecan also see that there is a remarkableamount of variability among the various tasks, notincluding the single-paper topics.The taxonomy is useful because it helps todescribe a skill set necessary for system administra-tors. Wecan see which areas system administratorshave focused most of their efforts on, examine whichareas have been successfully solved, and identify areasneeding more work. Since this taxonomy is derivedfrom papers, for completeness, it should be combinedwith tasks from time surveys [Ande95, Kols92] andinterviews.There are some potential concerns about this cat-egorization. The simplest of which is that there wereerrors in the classification. There were about 350papers, so a few errors probably occurred in classifica-tion. Furthermore,while the first author worked as asystem administrator both at SURAnet, and atCarnegie Mellon University,hehas clearly not person-ally performed all of the tasks described. The program

Figure1:System state transition diagram. Edges out indicate problems that occur making the system less usable.

Edges in indicate tasks performed by system administrators to restore the functionality of the system.

96 1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and Patterson

to make the task schedulable, rather than forcing an

administrator to deal with the problem immediately.

Reducing the number of interrupt-style tasks should

lead to improving system administrator effectiveness.

Training tasks may be transferable out of the

organization and into the schools. Users could be

trained in the tools they will be using, and administra-

tors could be trained in system administration. Earlier

education would mean people would only have to

learn the specifics of a site rather than the general

knowledge. Alternately,the various tools that are

being used could be improved to reduce the need for

training. Researchers in Human Computer Interaction

have been looking at this for some time, and havemade a number of strides, but more work remains.

Historical Trends of the Conference

Given the two models, we can look at how the

papers in the conference have fit into the models over

the course of time. This will help us see if things have

been changing from previous conferences.

Task Model Trends

Figures 2 and 3 show the papers over the last

twelve years categorized by the Task Model. For com-

pleteness, we show all of the papers that were shown

in the hierarchical categorization.

We can see that some tasks, such as backup,

application installation and accounts alternated

between very heavy and light years. This probably

Application Installation [32]ARetrospective on Twelve Years of LISA Proceedingsindicates some amount of duplicated effort in the veryheavy years. In some cases (application installation,OS installation), this pattern indicates that good solu-tions have not been found, and people are still makingnew,slightly different attempts. In other cases(backup, accounts), it indicates that there was somechange in the external world that caused previoussolutions to stop working. For example, backup was atask that was successfully solved in the past, but withdisk capacity and bandwidth growing faster than tapecapacity and bandwidth, it has returned as a problemof dealing with larger scale.We can see that some tasks, such as printing andtrouble tickets, have received a little bit of work fairlysteadily.This pattern is probably a good sign, as itmeans that slow and steady progress is being madewithout too much duplication of effort.Mail alternated between the steady work and theheavy work models.Initial work was fairly steadyuntil the explosion of the Internet increased the size ofmailing lists, and commercialization resulted in prob-lems with SPAM.Similarly,some tasks, such as system monitoringand network configuration, see punctuated bursts ofactivity.This pattern probably indicates that the prob-lem occurred simultaneously due to some externalchange such as sites scaling up, or new applications. Itwould be nice if there were some way for differentpeople to coordinate their work as they simultaneouslydiscover new problem areas. This would reduce theamount of duplicated work, and probably alsoBackup [28]Accounts [23]Site Configuration [23]Mail [20]OS Installation [14]System Monitoring [14]Printing [11]Network Configuration [9]Trouble Tickets [9]Secure Root Access [8]User Customization [8]

LISA87LISA88LISA89LISA90LISA91LISA92LISA93LISA94LISA95LISA96LISA97LISA98LISA99

Figure2:Breakdown of number of papers/conference/category for categories with at least eight total papers. Sorted

by popularity of a category,ties broken alphabetically.Height of a box (and the number inside) indicates num-ber of papers. Total number of papers in a category is shown in brackets after the category name. Remainder ofcategories are shown in Figure 3.

1999 LISA XIII – November 7-12, 1999 – Seattle, WA97

ARetrospective on Twelve Years of LISA Proceedings improve the resulting solution as it will deal with theidiosyncrasies of multiple sites.

It is not clear what we can learn from the taskswith fewer papers. In a few cases, we can infer thatcertain areas did not become problems until fairlyrecently.Web is an obvious example; configurationdiscovery,LAN, WAN, and NFS problems also appearto have only become problems recently.

Source Model Trends

Figure 4 shows the papers over the last twelveyears categorized by the Source Model.

We can see that the number of training taskpapers has been remarkably small, and in fact, further

Host Configuration [7]Self Improvement [7]

General Tool [6]

Resource Accounting [6]

Security [6]Data Display [5]

Models [5]

Network Monitoring [5]

News [5]

Benchmarking [4]

Configuration Discovery [4]

Documentation [4]File Synchronization [4]

Host Monitoring [4]

LAN [4]NFS [4]Site Move [4]Software Design [4]

WAN [4]Policy [3]

Remote Access [3]Software Packaging [3]

User Interaction [3]

Web [3]DNS [2]Database [2]Fault Tolerance [2]File Migration [2]Host Tables [2]

Performance Tuning [2]Resource Cleanup [2]Training Administrators [2]

White Pages [2]

LISA87

LISA88

LISA89

LISA90

LISA91

LISA92

LISA93

LISA94

LISA95

LISA96

Andersonand Patterson

examination of the papers in those categories indicatesthat they are mostly papers on improving the skills ofadministrators. The one oddity is LISA93, in which athird of the papers were on many different trainingissues. Some of the training papers cover softwaredesign issues for administrators, others suggest how toimprove interactions with other administrators, usersor managers. A few of the training papers cover howto train new administrators, but surprisingly none ofthe papers cover training users to take better advan-tage of software or provide better problem summaries.Training is an area where some work should be done,although it is more difficult to analyze because itinvolves the variability of people.

LISA97LISA98LISA99

Figure3:Continuation of Figure 2 for categories with 2-7 papers overall. Sorted by popularity of a category,ties

broken alphabetically.This figure is included for completeness, care should be taken in drawing conclusionsgiven the small number of papers.

98 1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and Patterson

We can also see that maintenance tasks comprisethe second largest fraction of papers. Unfortunately,interrupt-style maintenance tasks contribute greatly toadministrator stress. Beyond simply eliminating main-tenance tasks by having systems automatically repairthemselves, we should strive to convert maintenancetasks to schedulable tasks. If systems were designed tooperate in degraded mode, then administrators wouldnot have to respond immediately every time a problemoccurred, but could instead work on related tasks atthe same time.

Finally,wecan see that configuration manage-ment tasks are the most prevalent of the papers, whichis reasonable and unsurprising given that many taskseventually require some change in configuration. Con-figuration tasks generally lead to results which can bemore easily described in a paper than results from theother two categories.

Examination of Important Tasks

We now examine the important tasks performedby system administrators in more detail. Wesumma-rize the area, examine the research history,and pro-pose directions for future research. Many of the direc-tions would make good papers for future LISA confer-ences. In the research history,wereference some ofthe better papers on each topic, so that readers willknow where to look for additional information.

SoftwareInstallation: OS, Application, Packagingand Customization

Software installation covers the problems ofmanaging software installed on computers. There areARetrospective on Twelve Years of LISA Proceedingsfour sub-categories of software installation: OperatingSystem (OS) Installation, Application Installation,Software Packaging, and User Customization. Operat-ing system installation deals with the problem of tak-ing the raw machine and putting the operating systemon it so it can boot. Application installation is theaddition of optional (non-OS) packages to a machine.Software packaging is the step of creating an instal-lable package. User customization happens when usersneed to change the way the software operates.Research HistoryOS installation usually puts files in specificplaces and has limited support for multiple versions onasingle machine. Research into operating systeminstallation has taken a cyclic path. In the very begin-ning, the OS was installed by either cloning a disk andthen putting it in the new machine, or by booting thenew machine offsome other media (e.g., floppy disk,network) and then copying an image to the local harddrive. Those solutions were then modified to supportcustomization of the resulting installation and easierupgrades [Zwic92, Hide94].The tools were thenscaled to allow fast installation across the entire enter-prise [Shad95]. By then large-scale PC OS installationneeded to be supported, and the cloning solution[Troc96] reappeared.Application installation usually puts packagesinto separate directories, and uses symbolic links tobuild composite directories, so multiple versions areeasily supported, and programs can be beta tested eas-ily before being made generally available. ApplicationConfiguration [218]Maintenance [94]Training [30]

LISA87LISA88LISA89LISA90LISA91LISA92LISA93LISA94LISA95LISA96LISA97LISA98LISA99

Figure4:Breakdown of number of papers/conference/category.Sorted by popularity.Height of a box (and the

number inside) indicates number of papers. Total number of papers in a category is shown in brackets after thecategory name.

1999 LISA XIII – November 7-12, 1999 – Seattle, WA99

ARetrospective on Twelve Years of LISA Proceedings Andersonand Pattersoninstallation has had many more papers written on it

than OS installation, probably because vendors didn’t

supply tools to install additional applications. The ini-

tial solution was to build packages in separate directo-

ries and link them into a common directory [Manh90,

Coly92]. These tools were then extended to support

customization per host [Wong93]. Recently,the

caching and linking pieces were untangled and refined

into separate tools [Couc96b, Bell96].

Relatively few papers have been written on soft-

ware packaging, probably because most of the appli-

cation installation tools use source code trees rather

than binary packages. These papers cover the patching

of software for different host types, and the subse-

quent generation of installation packages [Stae98].

The papers on user customization cover two sep-

arate areas of customization: Selecting which pack-

ages are accessed by a user [Furl96, Will93], and cus-

tomizing application behavior [Elli92]. The package

selection tools started as simple shell scripts that

adjusted environment variables to enable packages,

and later were refined to work faster and more flexi-

bly.The customization tools have dealt with different

aspects of making it easer to control the behavior of

programs and have been targeted at beginning users.

An Alternative Breakdown

There have been a remarkable number of papers

in this area, many of which seem like slight variations

of each other,which makes us wonder if the problem

has been broken down poorly.Wetherefore propose a

different breakdown into the following five pieces:

Packaging, Selection, Merging, Caching, and End-

User Customization.

The distinction between installing applications

and the operating systems is probably unnecessary and

ahistorical artifact. Some of the OS installation papers

supported some limited number of additional pack-

ages, and recent OS installation programs [Hohn99]

can install most of the packages available on the net.

However,the distinction in functionality that was

found in the OS/Application split still remains.

Packaging

Software packaging appears to be a mostly

solved problem. There have been a few papers in the

LISA conference on it, and the freely available Unix

systems have associated packaging tools. Comparing

these tools might pave the way to a single multi-plat-

form tool.

Packaging usually binds pathnames into an

application. This can limit how packages can be

merged later (e.g., two versions both believe they own

/usr/lib/package). Some packages allow environment

variables to override pathname choices. Exploring the

performance and flexibility of the different choices

could help improve existing tools.

Selection

Package selection is part of all OS/Application

installation tools.The key pieces for a selection tool

100 are the need for per-machine flexibility and the needto support multiple collections. Both programmaticand GUI interfaces should be supported so that thetool is both easier to use and scriptable. The selectiontool could then be integrated into some of the existingtools as a uniform front-end.MergingMerging packages remains a hard outstandingproblem. Many tools just ignore the problem. A fewhave a configuration file to specify which packageoverrides another when conflicts occur.Merging ismost difficult when packages are inter-related, as isthe case with Emacs, Perl and Tcl with their variousseparate extensions; Tex/LaTeX; X windows with var-ious applications that add fonts and include files; andshared library packages.One unsatisfying solution is to pre-merge pack-ages during packaging so that there are no inter-rela-tions between packages. A modular solution wouldneed to handle merging of files, for example generat-ing the top level Emacs info file, or the X windowsfont directory files. Some programs include searchpaths, which might make the merging easier to handle,others require the execution of a program in the finalmerged directory.If multiple versions need to be supported simul-taneously,there is a more substantial problem. Sup-porting the cross product of all possibilities is notpractical. However,there is no clear easy solution.Quite a bit of thought will be needed to find an ade-quate solution.CachingCaching to the local disk is beneficial for bothperformance and for isolating clients from server fail-ures. Caching is a semi-solved problem.Some filesystems cache onto local disk to improve performance(e.g., AFS, CacheFS, Coda). In general, cachingmerely requires mounting the global repository some-where different and creating symlinks or copies asappropriate. There have been tools written to do justthis [Couc96b, Bell96], and many of the general soft-ware installation tools have included support forcaching [Wong93]. Making the caching fully auto-matic and fine grained will probably require someamount of OS integration.End User CustomizationEnd user customization has only been slightlyexamined. A few tools help users dynamically selectthe packages they want to use [Furl96]; most havefixed the choice on a per-machine basis. One old paperlooked at how users customized their environment[Will93]. It would be nice for this area to be resur-rected for research. Programs are becoming increas-ingly complex, especially as they add GUI interfaces,but the ease of customizing the programs has not keptup. Work in this area would require a large amount ofinterviewing users to determine what they would liketo customize.1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and Patterson

Backup

Backup addresses four separate, but related prob-lems: User Error,Independent Media Failure, Corre-lated Media Failure (e.g., Site Failure, SoftwareError), and Long Term Storage. All the solutions arebased on some type of redundant copy,but the particu-lars of each are different. Damage due to user errorcan be reduced by online filesystem snapshots. Inde-pendent media failure can be remedied by techniqueslike RAID. Correlated media failure requires use ofadditional uncorrelated media (e.g., Off-site tape,remote duplicates with different software). Finallylong term storage requires very stable media, and aneasily read format. Consider how many people canstill read data written on punchcards, or even 9-tracktape. Most of the focus in backup has been on inde-pendent media failure, usually by creating copies ontape, although people have looked at the other issues.Research History

Research on backup has passed through manystages. The first was correctness: Does the right dataget written? [Zwic91b] Are backups happening regu-larly and on schedule? [Metz92] Do restores work?Having achieved correctness, research turned to scal-ing backup solutions to the enterprise. The solutionwas staging disks so that backups could stream to tape

[Silv93]. Having solved the correctness and scalabilityproblems, research on backup paused. But then theonward march of technology reintroduced scalabilityas a problem. Disk bandwidth and capacity are startingto outstrip tape bandwidth and capacity leading tosolutions requiring multiplexing of disks and tapes

[Pres98].

FutureDirections

Restores seem to be a somewhat overlooked partof the backup problem.Most backup papers deal ingreat detail with formats of dump tapes, scheduling ofbackups, streaming to tape. However,they usuallyonly write a few paragraphs on the subject of restores,often ignoring the time taken to restore data. Thewhole purpose of backup is so that when somethinggoes wrong, restores can happen!We would like adiscussion of restore difficulty and measurements ofrestore performance in future papers. When somethingfails, there is a cost in lost productivity in addition tothe direct cost of performing the repair.

Examining technology trends and technologyoptions would help identify future backup challengesbefore they occur.The technology involved has rea-sonably predictable future performance in terms ofbandwidth, latency,and capacity.Somewhat weakerpredictions can be made about the growth in the stor-age needs of users. Given this information, a predic-tion can be made about the required ratio of hardwarein the future. In addition, alternatives to tape backupsuch as high capacity disks and writable cds/dvds maybecome viable in the future. One advantage of randomaccess media is that data can be directly accessed offthe backup media to speed up recovery.

1999 LISA XIII – November 7-12, 1999 – Seattle, WAARetrospective on Twelve Years of LISA ProceedingsBackup by copying to remote sites is very differ-ent from traditional approaches. A few companies aredealing with the possibility of a site failure by per-forming on-line mirroring to a remote site over a fiberconnection. It may be possible to decrease the requiredbandwidth by lowering the frequency of the updates,so that this approach is practical for people unable topurchase a dedicated fiber.Backups also present special security concerns.Abackup is typically an unprotected copy of data. Ifanyone can get access to backup tapes, they can readcritical data. How can encryption be used to solve thesecurity problem?Will encryption enable safe webbackup systems?Another interesting question is how to handlebackup for long-term storage. Some industries havelegal requirements to retain documents for a long(indefinite) time. There are two related problems.First, media needs to be found which is stable enoughto last a long time.Second, it seems wise to rely onconversion to a common format because it is neverclear what software will still work in 20-50 years.How can these two concerns be integrated into abackup solution?Configuration: Site, Host, Network, Site MoveConfiguration tasks are modification to the setupof hardware and software so that the environmentmatches the requirements of a particular organization.These tasks can range from simply installing theappropriate exports and resolv.conf files to compli-cated tasks like migration from an MVS platform to aUNIX one.Research HistoryThe first few LISA conferences included manypapers which summarized their site’sconfiguration.Research then forked in two directions.Some paperslooked at how to store and extract configuration infor-mation from a central repository,either using availabletools such as SQL [Fink89], or by designing their ownlanguage [Roui94a]. Other papers looked at using alevel of indirection to make configuration changestransparent to users [Detk91].The great growth spurt in the computer industrylead to complete site moves, either as part of a merger,separation, or just to handle growth [Schi93]. Simi-larly,the great amount of research in this area ledsome people to examine the question, ‘‘What proper-ties of site design make it easier to administer?’’[Trau98]. Recently,a mobile user base causeddynamic network re-configuration to become a prob-lem [Vali99].Categorization CommentaryThis is probably the weakest categorization. Theoriginal intent was that host configuration would coverhost issues, network configuration would cover net-work issues, and site configuration would cover globalsite issues. However,the line between host and site is101

ARetrospective on Twelve Years of LISA Proceedings Andersonand Pattersonat best blurry.Wetherefore believe that someone

should re-examine the papers in these areas, and see if

they can find a better categorization.

FutureDirections

The key to host configuration seems to be having

acentral repository of information that is then pushed

or pulled by hosts. Most of the papers did some vari-

ant of this. Two areas remain to be refined:First,

someone should analyze exactly what information

should be in the central repository,and how it can be

converted to the many different types of hosts in use.

Second, someone should write a tool to automatically

create the repositories so that the start-up cost to using

aconfiguration tool is lower.

Site configuration tools vary widely,probably

because of the different requirements at each site (e.g.,

awall street trading firm vs. a research lab). [Evar97]

surveyed the current practices, and [Trau98] studied

the best practices for certain environments. Combining

these two directions by identifying the best practices

based on the requirements of a site would help all sites

do a better job of configuration.

Network configuration is a fairly recent topic, so

proposing directions by analyzing the papers is risky.

However,wecan still look at analogies to previous

work. First, we want to build abstract descriptions of

the system. Second, the models should be customiz-

able; early configuration tools didn’tsupport much

customization, so later ones had to add it.Third, a

survey paper,analogous to [Evar97] would help iden-

tify the problems in network configuration research.

Accounts

Managing user accounts at first seems very sim-

ple. But further examination indicates that there are

additional subtleties because an account identifies

users, and therefore has lots of associated real world

meaning. Therefore, authentication, rapid account cre-

ation, and managing the associated user information

become important.

Research History

Accounts research started with the goal of sim-

plifying the account creation process. Scripts were

designed that automated the steps of accumulating the

appropriate information about users, adding entries to

password files, creating user directories, and copying

user files [Curr90]. Because the scripts were site-spe-

cific, they were able to do better error checking. Once

creating accounts became easy,accounts research

paused until enough people needed accounts that scal-

ability became a concern. Sites with thousands of

accounts, usually schools, needed to create lots of

accounts quickly because of high turnover in the user

population. Their solutions tended to have some sort

of central repository storing account information

(often an admissions’ database), with complementary

daemons on client nodes to extract the needed parts of

the database [Spen96]. Some of the recent papers

102 considered auxiliary details such as limiting accountsto certain hosts, account expiration, and delegatingauthority to create accounts [Arno98].FutureResearch OpportunitiesSurveying account creation practices would helpidentify why no tool has evolved as superior despitemany papers on this subject. Webelieve this isbecause of unrecognized differences in the require-ments at each site. With all the requirements explicitlydescribed, it should be possible to build a universaltool.Arelated topic is the examination of specificissues related to account creation. For example, manyof the papers ignored the question of how to limitaccounts to specific machines. Is a simple grouping aswas done for host configuration sufficient, or is somesort of export/import setup needed?Sharing accountsacross administrative boundaries within an organiza-tion will make this problem even more difficult.Another specific issue is delegation of accountcreation. The one tool to do this [Arno98] assumed allthe employees were trusted to enter correct accountinformation. Clearly this solution will not work at allsites. There may be synergy with the secure rootaccess papers that looked at delegation.MailElectronic mail has been one of the drivingapplications on the Internet since its inception. Thismakes it unsurprising that it ranks extremely high onthe list of applications. It is the highest of the applica-tions that are used by end-users on a regular basis.There is a vast amount of email, traveling around theworld-wide network, leading to a lot of effort in inter-operability and scalability.Research HistoryVery early research in mail targeted interoper-ability between the wide variety of independentlydeveloped mail systems. This research and the reduc-tion in variety over time, combined with SMTP as astandard mail interchange protocol, solved the interop-erability problem. Research then turned to flexibledelivery and automating mailing lists [Chap92].There was then a brief pause in the research. However,as the Internet continued to grow,research on scalingdelivery of mail both locally and in mailing lists[Kols97] was needed. At the same time, commercial-ization caused SPAM to become a problem [Hark97].FutureResearchThe biggest remaining problem is dealing withSPAM. The correct solution is probably dependent ontrading offdifficulty in being reached legitimatelywith protection from SPAM. Some possibleapproaches are: acceptance lists with passwords, a listof abusers that are automatically ignored (this is beingdone), a pattern matcher for common SPAM forms,and receive-only/send-only addresses. Finding a goodsolution will be challenging.1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and Patterson

Scalability and security still need some work.Scalability of mail transport and mail delivery may bepossible by gluing together current tools into a clus-tered solution. Both problems partition easily.Han-dling more types of security threats also remains open;[Bent99] has done some initial work securing MTA?MTAtransfers.

Monitoring: System, Network, Host, Data Display

Monitoring solutions help administrators figureout what is happening in the environment. There areproblems of system, network and host monitoring, andthe associated problem of data display.Monitoringsolutions tend to have two variants: instantaneous andlong term.

Research History

Research in monitoring has progressed along anumber of axes. First, there has been work in monitor-ing specific sources from file and directory state[Rich91] to OC3 links [Apis96]. Simultaneously,generic monitoring infrastructure [Hard92, Ande97a]has been developed. Finally,asthe amount of dataavailable has increased, some work on data displayhas been done [Oeti98a].Categorization Commentary

The categorization here was by the type of thingbeing monitored (host, network system). Perhaps abetter classification would be by the axes described inthe research history.FutureDirections

There has been a lot of work on gathering datafrom specific sources, but in most cases, the overheadfor gathering data has been high, so the interval is usu-ally set in minutes. Reducing this overhead is impor-tant for allowing finer grain monitoring [Ande97b]. Inaddition, we would like to vary the gathering intervalso that the overhead of fine-grain gathering is onlyincurred when the data would be used. In addition tojust gathering the data, having a standard form forstoring the data efficiently would be very useful. Com-bining these two issues should lead to a nice universaltool with pluggable gathering modules.

Data analysis and data reduction have notreceived nearly the attention they deserve. The datacollection techniques are only useful if the data can beused to identify problems. But beyond averaging time-series data, very little automated analysis has beendone. An examination of methods for automated anal-ysis, for example, looking at machine learning tech-niques, could prove fruitful.

Data visualization has started to get some exami-nation in the system administration field. There is avast amount of literature on various forms of visual-ization in the scientific computing field. Webelievethat a survey of existing techniques would lead totools that allow visualization in system administrationto be both more effective and more scalable.

1999 LISA XIII – November 7-12, 1999 – Seattle, WA

ARetrospective on Twelve Years of LISA ProceedingsPrinting

Printing covers the problems of getting print jobsfrom users to printers, allowing users to select print-ers, and getting errors and acknowledgements fromprinters to users.Research History

Early research in printing merged together thevarious printing systems that had evolved [Flet92b].Once the printing systems were interoperable, printingresearch turned to improving the resulting systems,making them easier to debug, configure, and extend[Powe95]. As sites continued to grow,scaling theprinting system became a concern, and recent papershave looked into what happens when there are thou-sands of printers [Wood98].FutureDirections

Printing research seems to be in fairly goodshape. Scaling print systems is still not completelydone, debugging problems and selecting the rightprinter is still challenging. Perhaps printer selectioncould be done by property (e.g., color,two sided).Finally,the path for getting information from printersback to users has not been well examined. A notifica-tion tool to tell users the printer’s status, such as printjob finished or out of paper,would be useful. The noti-fication tool might also help in debugging printingproblems.

Tr ouble Tickets

Trouble ticket tools simplify the job of acceptingaproblem report, assigning the problem report to anadministrator,fixing the problem, and closing theproblem’sticket. Trouble ticket systems usually have afew methods for getting requests into the system (e-mail, phone, GUI), and provide tools for querying andadjusting the requests once they are in the system.Research History

Trouble ticket systems began as email-only sub-mission tools with a centralized queue for requests[Galy90]. Later,the systems were extended so thatusers could query the status, and tickets could beassigned to particular administrators [Kobl92]. Thesystems were improved to support multiple submis-sion methods such as phone [Scot97] and GUI, and tosupport multiple request queues [Ruef96].FutureDirections

There seems to be a fair amount of overlap in theresearch on trouble tickets. Many of the tools werecreated from scratch, only occasionally building onthe previous research. Examining the existing toolsshould identify the different requirements that haveled to all these systems and to a more general tool.

Asecond direction to extend trouble ticket sys-tems would be to build in a knowledge of the requesthandling process. [Limo99] examines the process ofhandling problem reports, but doesn’tpropose tools. Atrouble ticket system supporting the process would bequite valuable.

103

ARetrospective on Twelve Years of LISA Proceedings Andersonand Patterson

SecureRoot Access

Secure root access is the general problem of pro-viding temporary privileges to a partially trusted user.Many actions need to be taken as root, and giving outthe root password is clearly a poor decision.Thequestions then are how to give out privileges, how totrack their use, and how to retain some amount ofsecurity.

Research History

Research in secure root access has gone downtwo separate paths. One path has been to examine howto provide secure access to commands within a host.This has gone through many iterations, slowly addingin more complex checking of programs and arguments[Mill99, Hill96]. The other has been to provide secureaccess remotely [Ramm95].FutureDirections

The unfortunate effect of having the two separatepaths of research is that neither handles all the prob-lems easily.The remote tools are more flexible, butharder to configure, and don’tsupport logging well.The local tools have a more natural interface, but don’thave as much power to provide partial access. Com-bining these two paths of research should lead to amore powerful and flexible tool.

Asecond direction to consider is toward provid-ing finer-grain access control. [Gold96] did this bysecurely intercepting system calls.Further work couldlead to having something like capabilities in the OS,allowing very precise control over the access grantedto partially-privileged users.

Conclusions and Analysis

We have categorized all of the papers in theLISA conference according to two separate models.We have made the categorization available so that oth-ers can examine our choices, correct mistakes, or pro-vide better categorizations. Hopefully this paper willencourage people to think differently about the fieldand problems that it presents, and as a result build bet-ter tools and processes.

We would like to see other people examine someof the other conferences that may publish relevantpapers. The USENIX general conference, SIGCOMM,and SANS are a few places to start looking. There islikely to be some useful information present in thoseconferences which was not covered in this paper.

We have examined the historical trends of theLISA conference according to the two models. Thishas helped us see that some areas are under served,and some are probably over-served. Wecan also seethe bursty nature of research in system administration(probably because the same problem occurs to every-one at the same time). As a result we recommend thatacentral clearinghouse of problems be created tofacilitate collaboration and improve the resulting tools.

Finally we examined some of the important taskareas. Based on our analysis, we have proposed a104

number of papers to be written. Webelieve that thissort of analysis should be performed every few years.The Database community gets together and decideswhich areas of research were successful, and whichrequire more work [Silb91, Silb96]. Their reports havehelped their community show their results and focustheir efforts. Hopefully this analysis of system admin-istration will help do the same for ours.

Acknowledgements

We would like to thank Evi Nemeth, Kim Kee-ton, Drew Roselli, Aaron Brown, and David Oppen-heimer,and the anonymous reviewers for their com-ments on the paper.Their comments have improvedboth the ideas and the readability of the paperimmensely.This work was supported by DARPAunder grant DABT63-96-C-0056.

References

The entire database of categorized papers isavailable from http://now.cs.berkeley.edu/Sysadmin/categorization/ .

[Ande95] EricAnderson. ‘‘Results of the 1995 SANS

Survey’’;login:,October 1995, Vol20, No. 5,http://now.cs.berkeley.edu/Sysadmin/SANS95-Survey/index.html; Found weak correlation betweennumber of machines and number of admins;many important tasks.[Ande97a] EricAnderson and Dave Patterson,

‘‘Extensible, Scalable Monitoring for Clusters ofComputers,’’Proceedings of the Eleventh Sys-tems Administration Conference(LISA ’97), SanDiego, California pp 9-16; http://now.cs.berkeley.edu/Sysadmin/esm/intro.html; Tool for monitor-ing and displaying cluster statistics.

[Ande97b] JenniferM. Anderson, Lance M. Berc, Jef-frey Dean, Sanjay Ghemawat, Monika R. Hen-zinger,Shun-Tak A. Leung, Richard L. Sites,Mark T.Vandervoorde, Carl A. Waldspurger,and William E. Weihl. ‘‘Continuous Profiling:Where Have All the Cycles Gone?’’16th ACMSymposium on Operating Systems Principles,Saint-Malo, France, /pub/DEC/SRC/technical-notes/abstracts/src-tn-1997-016.html .[Apis96] JoelApisdorf, k claffy,Kevin Thompson,

and Rick Wilder,‘‘OC3MON: Flexible, Afford-able, High Performance Statistics Collection,’’Proceedings of the Tenth Systems AdministrationConference(LISA ’96), Chicago, Illinois, pp97-112, /NA/Oc3mon/; HW&SWfor monitoring and analyzing traffic on anOC3 link.

[Arno98] BobArnold, ‘‘Accountworks: Users Create

Accounts on SQL, Notes, NT,and UNIX,’’Pro-ceedings of the Twelfth Systems AdministrationConference(LISA ’98), Boston, Massachusetts,pp 49-61.1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and Patterson

[Bell96] JohnD. Bell. ‘‘A Simple Caching File Sys-

tem for Application Serving,’’Proceedings of theTenth Systems Administration Conference(LISA’96), Chicago, Illinois, pp 171-179; Automaticcaching of applications from a remote server tolocal disk.

[Bent99] DamienBentley,Greg Rose, and Tara

Whalen. ‘‘ssmail: Opportunistic Encryption insendmail,’’Proceedings of the Thirteenth Sys-tems Administration Conference(LISA ’99),Seattle, Washington.

[Chap92] D.Brent Chapman. ‘‘Majordomo: How I

Manage 17 Mailing Lists Without Answering‘-request’ Mail,’’Proceedings of the Sixth Sys-tems Administration Conference(LISA ’92),Long Beach, California, pp 135-143, ftp:///pub/majordomo.tar.Z .

[Coly92] Wallace Colyer and Walter Wong, ‘‘Depot:

ATool for Managing Software Environments,’’Proceedings of the Sixth Systems AdministrationConference(LISA ’92), Long Beach, California,pp 153-162, ftp://export.acs.cmu.edu/pub/depot/;Build merged tree by copy/link from packages;conflict resolution by package preferences.

[Couc96b] AlvaL. Couch, ‘‘SLINK: Simple, Effec-

tive Filesystem Maintenance Abstractions forCommunity-Based Administration,’’Proceed-ings of the Tenth Systems Administration Confer-ence(LISA ’96), Chicago, Illinois, pp 205-212,ftp://ftp.cs.tufts.edu/pub/slink; Flexible sym-link-ing/copying for merging software repositories.

[Curr90] DavidA. Curry,Samuel D. Kimery,Kent C.

De La Croix, and Jeffrey R.Schwab,‘‘ACMAINT:AnAccount Creation and Mainte-nance System for Distributed UNIX Systems,’’Proceedings of the Fourth Large InstallationSystems Administrator’s Conference(LISA ’90),Colorado, pp 1-9.

[Detk91] JohnF. Detke. ‘‘Host Aliases and Symbolic

Links -or-How to Hide the Servers’ RealName,’’Proceedings of the Fifth Large Installa-tion Systems Administration Conference,(LISA’91), San Diego, pp 249-252; Use host aliases &symbolic links to allow mount points & serversof exported FS to move w/o needing clientchanges.

[Elli92] RichardElling and Matthew Long. ‘‘user-

setup: A System for Custom Configuration ofUser Environments, or Helping Users HelpThemselves,’’Proceedings of the Sixth SystemsAdministration Conference(LISA ’92), LongBeach, California, pp 215-223, ftp://ftp.eng.auburn.edu/; Extension to Modules [Furl91] sys-tem, menu driven script to select applications &configure apps.

[Evar97] RémyEvard. ‘‘An Analysis of UNIX System

Configuration,’’Proceedings of the EleventhSystems Administration Conference(LISA ’97),San Diego, California, pp 179-193; Examination

1999 LISA XIII – November 7-12, 1999 – Seattle, WAARetrospective on Twelve Years of LISA Proceedingsof current configuration practices at nine differ-ent sites.[Fink89] RaphaelFinkel and Brian Sturgill. ‘‘Tools forSystem Administration in a Heterogeneous Envi-ronment,’’Proceedings of the Workshop onLarge Installation Systems Administration III(LISA ’89), Austin, Texas, pp 15-29; Relationalstructure stores host, file information. Tables canbe generated at runtime. Schema describes rela-tion structure & constraints. Query languagequeries & executes.[Flet92b] MarkFletcher,‘‘nlp: A Network PrintingTool,’’Proceedings of the Sixth Systems Admin-istration Conference(LISA ’92), Long Beach,California, pp 245-256; Centralized print serverdatabase, uses lpd protocol to transfer files.[Furl96] JohnL. Furlani and Peter W.Osel, ‘‘AbstractYourself With Modules,’’Proceedings of theTenth Systems Administration Conference(LISA’96), Chicago, Illinois, pp 193-203, /; Per-user flexible configuration ofaccessible packages.[Galy90] Tinsley Galyean, Trent Hein, and EviNemeth, ‘‘Trouble-MH: A Work-Queue Manage-ment Package for a >3 Ring Circus,’’Proceed-ings of the Fourth Large Installation SystemsAdministrator ’sConference(LISA ’90), Col-orado, pp 93-95.[Gold96] IanGoldberg, David Wagner,RandiThomas, and Eric A. Brewer,‘‘A secure environ-ment for untrusted helper applications: confiningthe wily hacker,’’Sixth USENIX Security Sympo-sium, Focusing on Applications of Cryptography,San Jose, California, http://www.cs.berkeley.edu/?daw/janus/ .[Hard92] DarrenR. Hardy and Herb M. Morreale,‘‘buzzerd: Automated Systems Monitoring withNotification in a Network Environment,’’Pro-ceedings of the Sixth Systems AdministrationConference(LISA ’92), Long Beach, California,pp 203-210; Central monitoring server,remotemonitoring daemons, paging on problems, userscan put in notification, filtering, and escalation.[Hark97] RobertHarker,‘‘Selectively RejectingSPAM Using Sendmail,’’Proceedings of theEleventh Systems Administration Conference(LISA ’97), San Diego, California, pp 205-220,/sendmail/anti-spam; Con-figuring sendmail to reject spam messages.[Hide94] ImazuHideyo. ‘‘OMNICONF – Making OSUpgrads and Disk Crash Recovery Easier,’’Pro-ceedings of the Eighth Systems AdministrationConference(LISA ’94), San Diego, California,pp 27-31; Calculate a delta between two configu-rations, store the delta, apply it later.[Hill96] BrianC. Hill, ‘‘Priv: Secure and FlexiblePrivileged Access Dissemination,’’Proceedingsof the Tenth Systems Administration Conference(LISA ’96), Chicago, Illinois, pp 1-8, ftp://ftp.105

ARetrospective on Twelve Years of LISA Proceedings Andersonand Patterson

ucdavis.edu/pub/unix/priv.tar.gz; Secure abilityto run programs as root with flexible commandchecking.[Hohn99] DirkHohndel, ‘‘Automated installation of

Linux systems using YaST,’’Proceedings of theThirteenth Systems Administration Conference(LISA ’99), Seattle, Washington.

[Kobl92] DavidKoblas, ‘‘PITS: A Request Manage-ment System,’’Proceedings of the Sixth SystemsAdministration Conference(LISA ’92), LongBeach, California, pp 197-202; Users can querydatabase of open tickets, centralized assignmentof new tickets, request editing tool.

[Kols92] RobKolstad, ‘‘1992 LISA Time Expenditure

Survey,’’;login:;Administrator time spread overmany tasks.[Kols97] RobKolstad, ‘‘Tuning Sendmail for Large

Mailing Lists’’Proceedings of the Eleventh Sys-tems Administration Conference(LISA ’97), SanDiego, California, pp 195-203; Configuringsendmail to increase performance.[Limo99] ThomasA. Limoncelli. ‘‘Deconstructing

User Requests and the 9-Step Model,’’Proceed-ings of the Thirteenth Systems AdministrationConference(LISA[Manh90] KennethManheimer,Barry A. Warsaw,

Stephen N. Clark, and Walter Rowe, ‘‘TheDepot: A Framework for Sharing SoftwareInstallation Across Organizational and UNIXPlatform Boundaries,’’Proceedings of theFourth Large Installation Systems Administra-tor ’sConference(LISA ’90), Colorado pp 37-46.[Metz92] MelissaMetz and Howie Kaye, ‘‘DeeJay –

The Dump Jockey: A Heterogeneous NetworkBackup System,’’Proceedings of the Sixth Sys-tems Administration Conference(LISA ’92),Long Beach, California, pp 115-125, ftp://ftp.cc.columbia.edu/.

[Mill99] Todd Miller,Dave Hieb, JeffNieusma, Garth

Snyder,etal., ‘‘Sudo: a utility to allow restrictedroot access,’’/sudo/ .[Oeti98a] Tobias Oetiker,‘‘MRTG – The Multi Router

Traffic Grapher,’’Proceedings of the TwelfthSystems Administration Conference(LISA ’98),Boston, Massachusetts, pp 141-147, http://ee-staff.ethz.ch/?oetiker/webtools/mrtg/3.0/ .

[Powe95] PatrickPowell and Justin Mason, ‘‘LPRng –

An Enhanced Printer Spooler System,’’Proceed-ings of the Ninth Systems Administration Confer-ence(LISA

[Pres98] W.Curtis Preston, ‘‘Using Gigabit Ethernet

to Backup Six Terabytes,’’Proceedings of theTwelfth Systems Administration Conference(LISA ’98), Boston, Massachusetts, pp 87-95.[Ramm95] KarlRamm and Michael Grubb, ‘‘Exu – A

System for Secure Delegation of Authority on anInsecure Network,’’Proceedings of the NinthSystems Administration Conference(LISA ’95),Monterey,California, pp 89-93, ftp://ftp.duke.106

edu/pub/exu; A tool for providing fine-grain rootaccess via authenticated, privileged scripts.

[Rich91] KennethRich and Scott Leadley,‘‘hobgob-lin: A File and Directory Auditor,’’Proceedingsof the Fifth Large Installation Systems Adminis-tration Conference(LISA ’91), San Diego, pp199-207, ftp://cc.rochester.edu/ftp/pub/ucc-src/hobgoblin; list of files/dirs + attributes => model.Checks for correctness, autogenerated from tar orls listings.[Roui94a] JohnP. Rouillard and Richard B. Martin,

‘‘Config: A Mechanism for Installing and Track-ing System Configurations,’’Proceedings of theEighth Systems Administration Conference(LISA’94), San Diego, California, pp 9-17, ftp://ftp.cs.umb.edu/pub/bblisa/talks/config/config.tar.Z;Update target machines using rdist + make withmaster repository in CVS, look for changed fileswith tripwire.

[Ruef96] CraigRuefenacht, ‘‘RUST:Managing Prob-lem Reports and To-Do Lists,’’Proceedings ofthe Tenth Systems Administration Conference(LISA ’96), Chicago, Illinois, pp 81-89, ftp://ftp.cs.utah.edu/pub/rust; Manages trouble ticketreports via e-mail.[Schi93] JohnSchimmel, ‘‘A Case Study on Moves

and Mergers,’’Proceedings of the Seventh Sys-tems Administration Conference(LISA ’93),Monterey,California, pp 93-98, How the mergerof SGI & Mips was handled, physical move &computer configuration issues.[Scot97] PeterScott, ‘‘Automating 24x7 Support

Response ToTelephone Requests,’’Proceedingsof the Eleventh Systems Administration Confer-ence(LISA ’97), San Diego, California, pp27-35, Phone system for receiving problemreports and paging people.

[Shad95] MichaelE. Shaddock, Michael C. Mitchell,

and Helen E. Harrison, ‘‘How to Upgrade 1500Workstations on Saturday,and Still Have Time toMow the Yard on Sunday,’’Proceedings of theNinth Systems Administration Conference(LISA’95), Monterey,California, pp 59-65; Tools &processes used to quickly upgrade an entire site.[Silb91] Avi Silberschatz, Michael Stonebraker,and

Jeffrey D. Ullman, ‘‘Database Systems: Achieve-ments and Opportunities,’’Communications ofthe ACM,34(10), pp 110-120.[Silb96] Avi Silberschatz, Michael Stonebraker,and

Jeffrey D. Ullman, ‘‘Database Research:Achievements and Opportunities into the 21stCentury,’’Stanford Technical Report, http://elib.stanford.edu/ as CS-TR-96-1563.[Silv93] Jamesda Silva and ólafur Gu?mundsson,

‘‘The Amanda Network Backup Manager,’’Pro-ceedings of the Seventh Systems AdministrationConference(LISA ’93), Monterey,California, pp171-182, ftp://ftp.cs.umd.edu/pub/amanda; Net-work backup by staging to a holding disk &streaming to tape, flexible scheduling.1999LISA XIII – November 7-12, 1999 – Seattle, WA

Anderson and PattersonARetrospective on Twelve Years of LISA Proceedings

[Spen96] HenrySpencer,‘‘Shuse: Multi-Host Account

Administration,’’Proceedings of the Tenth Sys-

tems Administration Conference(LISA ’96),

Chicago, Illinois, pp 25-32, Centralized account

management for rapid account creation.

[Stae98] CarlStaelin, ‘‘mkpkg: A software packaging

tool,’’Proceedings of the Twelfth Systems

Administration Conference(LISA ’98), Boston,

Massachusetts, pp 243-252, http://www.hpl.hp.

com/personal/Carl_Staelin/mkpkg; Automatically

generate manifest & install scripts; backend

makes package.

[Trau98] SteveTraugott and Joel Huddleston, ‘‘Boot-

strapping an Infrastructure,’’Proceedings of the

Tw e l f t hSystems Administration Conference(LISA

’98), Boston, Massachusetts, pp 181-196.

[Troc96] JimTrocki, ‘‘PC Administration Tools:

Using Linux to Manage Personal Computers,’’

Proceedings of the Tenth Systems Administration

Conference(LISA ’96), Chicago, Illinois, pp

187-192; Installation of DOS/Windows using

Linux boot disk.

[Vali99] PeterValian and Todd K. Watson, ‘‘NetReg:

An Automated DHCP Network Registration Sys-

tem,’’Proceedings of the Thirteenth Systems

Administration Conference(LISA ’99), Seattle,

Washington.

[Will93] CraigE. Wills, Kirstin Cadwell, and William

Marrs, ‘‘Customization in a UNIX Computing

Environment,’’Proceedings of the Seventh Sys-

tems Administration Conference(LISA ’93),

Monterey,California, pp 43-49; Study of how

users customize their environment (copied from

friends, then changed).

[Wong93] Walter C. Wong, ‘‘Local Disk Depot – Cus-

tomizing the Software Environment,’’Proceed-

ings of the Seventh Systems Administration Con-

ference(LISA How to cache packages onto the

local disk in the depot [Coly92].

[Wood98] BenWoodard, ‘‘Building An Enterprise

Printing System,’’Proceedings of the Twelfth

Systems Administration Conference(LISA ’98),

Boston, Massachusetts, pp 219-228, http://pasta.

penguincomputing.com/pub/prtools .

[Zwic91b] ElizabethD. Zwicky,‘‘Torture-testing

Backup and Archive Programs: Things You

Ought to Know But Probably Would Rather

Not,’’Proceedings of the Fifth Large Installation

Systems Administration Conference(LISA ’91),

San Diego, pp 181-189; Static & dynamic tests

for many backup programs (dump, cpio, tar,etc.)

shows many problems.

[Zwic92] ElizabethD. Zwicky,‘‘Typecast: Beyond

Cloned Hosts,’’Proceedings of the Sixth Systems

Administration Conference(LISA ’92), Long

Beach, California, pp 73-78, ftp://ftp./

pub/packages/typecast .

1999 LISA XIII – November 7-12, 1999 – Seattle, WA107

108 1999LISA XIII – November 7-12, 1999 – Seattle, WA