Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

October 28 2011

What's New in CFEngine 3: Making System Administration Even More Powerful

CFEngine is both the oldest and the newest of the popular tools for automating site administration. Mark Burgess invented it as a free software project in 1993, and years later, as deployments in the field outgrew its original design he gave it a complete rethink and developed the powerful concept of promise theory to make it modular and maintainable. In this guise as version 3, CFEngine stands along with two other pieces of free software, Puppet and Chef, as key parts of enterprise computing. Along the way, Burgess also started a commercial venture, CFEngine AS, that maintains both the open source and proprietary versions of CFEngine.

Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing a book for O'Reilly on CFEngine 3. I talked to him this week about the recent new release of the open source version (3.2.4) in tandem with a new commercial release of CFEngine 3 Nova (version 2.1.3). Here's are excerpts of what he has written to introduce CFEngine 3.

CFEngine 3 is fine-tuned to the features and design that make it possible to automate very large numbers of systems in a scalable and manageable way. CFEngine 3 is also very lightweight--its binaries normally use less than 30MB of disk space, it requires a single TCP port to communicate among servers and clients, and it has been designed to be very resource-efficient. CFEngine 3 can run on everything from smartphones to supercomputers.

CFEngine 3 is different from many other automation mechanisms in that you do not need to tell it what to do. Instead, you specify the state in which you wish the system to be, and CFEngine 3 will automatically and iteratively decide the actions to take to reach the desired state, or as close to it as possible. Underlying this ability is a powerful theoretical model known as Promise Theory, which was initially developed for CFEngine 3, but which has also found other applications in Computer Science and in other fields such as Economics and Organization.

This allows you to develop building blocks for complex promises that remain readable and manageable because the lower-level components are encapsulated. Each promise represents the desired state of certain parts of the system. At the lowest level, these are some of the things that you can express to CFEngine 3 as desired states:

  • "Make sure file /foo/bar contains line xyz"

  • "Make sure user foobar exists/does not exist"

  • "Make sure process foo is/is not running"

    At a higher level of abstraction, you can encapsulate CFEngine 3 operations and express high-level desired states:

  • "Make sure all web servers have Apache installed"

  • "Make sure all root accounts have the same, centrally-designated password"

  • "Make sure parameters EnableDNS and AllowRoot are disabled on all sshd configurations"

    And at an even higher level, you can express top-level desired states like these:

  • "Configure host xyz as a database server"

  • "Create a new cluster of VMs to use as web servers"

    So what's in the new versions? CFEngine 3 Nova includes:

    • System monitoring extensions, which extend the monitoring capabilities of CFEngine 3 Community (to monitor system state such as CPU load, number of processes and network connections, disk utilization, etc.) to allow for defining custom monitors for any type of information.

    • Support for manipulating virtual machines on Xen, VMware ESX, and KVM.

    • Native Windows support.

    • Flexible searching of reports in a brand new scalable interface that supports thousands of hosts on a single hub.

    • Improved machine learning and anomaly monitoring for diagnostics and capacity planning. Additional sensors have been added to detect operating system performance and behavioral trends, especially on Linux kernels.

    • The NoSQL document-oriented database MongoDB, used instead of MySQL for all storage on Nova's Mission Portal.

    • Generic JSON return values so that users can customize the interface and JQuery framework of the Mission Portal. This allows direct access to data in a way that makes higher levels of scripting more effective.

    CFEngine 3 Community also includes a large number of improvements, all of which are in Nova too:

    • A vastly improved bootstrapping process, which makes it easy to get new CFEngine 3 servers and clients up and running with very little manual configuration.

    • Support for environments, which are a way of grouping hosts according to arbitrary definitions. This makes it very easy to define, for example, "development," "testing," and "production" environments for CFEngine 3 policies.

    • The new cf-report command, available in both Community and Nova, which allows extraction of data and generation of reports from the command line. It can produce reports both about the behavior of the current CFEngine 3 environment (policies, hosts, etc.) and about internal information, such as a CFEngine 3 syntax summary.

    • Many performance and concurrency improvements and bug fixes.

    • Several new functions and parsing improvements, including and(), not(), and or() functions, to ease writing of complex class expressions.

    • A new and improved Emacs mode for editing CFEngine 3 policy files.

    Velocity Europe, being held Nov. 8-9 in Berlin, will bring together the web operations and performance communities for two days of critical training, best practices, and case studies.

    Save 20% on registration with the code RADAR20

    August 30 2011

    Four short links: 30 August 2011

    1. Data Monday: From PC to Tablet (Luke Wroblewski) -- some great stats here. Sales of Apple's iPad pulled in 30% more than all of Dell's consumer PC business in just the first half of the year.
    2. Munki -- munki is a set of tools that, used together with a webserver-based repository of packages and package metadata, can be used by OS X administrators to manage software installs (and in many cases removals) on OS X client machines.
    3. Crustache (GitHub) -- a fast C implementation of the Mustache templating engine. (via Hacker News)
    4. Minecraft Cube in Real Life -- clever hardware hack with projection and Arduino sensing.

    July 29 2011

    Happy SysAdmin Appreciation Day!

    Sysadmin DayToday is the 12th Annual System Administrator Appreciation Day. If you are reading this (or any other) page, sending a message, watching a video, reading an email, or doing anything else that touches the web ... you can thank a SysAdmin.

    To all of you that care so much about building & running the infrastructure that we depend on every day... thank you. You are exceptional people, and you are doing work that matters.

    Here's a video of Tim O'Reilly talking about how he came to learn how awesome SysAdmins are.

    Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
    Could not load more posts
    Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
    Just a second, loading more posts...
    You've reached the end.

    Don't be the product, buy the product!