Since last weekend, I spent my evenings manually moving content on deninet. It many ways, it was an exhausting process, but it gave me something to do while I suffer through the last two seasons of Star Trek: Voyager. Let me chart how I arrived at this point. 

Over vacation time in December, I decided to conduct a little experiment. I would attempt to upgrade the current deninet website from Drupal 6 to Drupal 7. When I had attempted this a year ago, it did not go well. The upgrade process crashed spectactually, leaving me with a lump of a database. Thankfully I was doing this on a local copy, and not the live site. I had speculated that in the interviening year, Drupal would have ironed out a lot of the edge cases I originally ran into. So, I copied off the site and tried again. To my astonishment, it worked. 

This created a dilemma. The previous year I had created a new deninet website by manually copying content from the version based on Drupal 6 to the new one based on Drupal 7. When all the features I needed to run the site were available, I'd switch to the new site. I had maintained the D7 fork from time to time, manually importing nodes and to make sure the two sites where in sync. At the time, this seemed the best way to go. I would lose things like comments and break all the URLs, but the end would be a good move. 

Except that the new site never seemed to get anywhere. 

I had reservations about the URLs and the comments. While I could bury my feelings, claming it was for "the greater good", it still bothered me. There was the bigger issue that I really wasn't getting anywhere with the D7 site. It sat and languished, while I couldn't resolve the issue. 

After successfully upgrading the site, I began to wonder if I could take a more fluid approach. I used Pathauto to create URLs that would be the same for both the current D6 site, as well as a D7 site -- upgraded or new. The biggest problem was images. 

Yesterday I had a wonderful contact of firsts. It was:

• My first contact outside the US
• My first contact with Canada
• My first contact on 20 meters
• My first contact on PSK31

Until later that day, my only radio is my beloved Yaesu FT-817ND. It's a low power rig that is intended for backpacking purposes, featuring a maximum output of 5 watts and an internal battery. It's an adorable little rig, but the total output wattage is tiny compared to typical Ham Radio rigs. 

Let's compare my FT-817 to my newest rig that arrived only hours later -- an Icom 707. This older rig was originally released in 1993, 11 years before the 817ND! Nevertheless, the power output for the 707 is common to many rigs manufactured today, 100 watts.Even with a modest antenna, you can go across the country easily on 100 watts. With luck and the Ionosphere on your side, you can go around the world! With only 5 watts, however, things become much more tricky. You need to consider your mode carefully.

A mode is the method of encoding information in a radio signal. The most basic one you're probably familiar with through movies, Morse Code. This method turns the transmitter (or more specifically, the power amplifier connected to the antenna) on and off. Like binary, it has two "digits", a short "dit" and a long "dah". By encoding each letter of the alphabet as a series of dits and dahs, you can exchange information. Since the signal is either fully on, or fully off, and the frequency is constant, Hams like calling this mode CW, or Continuous Wave. CW is the mode proper, while Morse is merely a method of encoding the information. 

Other modes you're already familiar with, AM and FM. These are voice modes. Explaining these requires a bit more imagination. Think of your voice as a sine wave. Radio waves can also be thought of as a sine wave. For your voice, the length between the peaks of wave is much, much bigger than the radio wave. So much so, you can superimpose your voice wave by varying the power output of the transmitter. This is how AM, or Amplitude Modulation works. FM, or Frequency Modulation works by varying the frequency of the radio signal proportionally with your voice sine wave. FM was created to avoid noise generated by the atmosphere, most of which causes spikes in amplitude. 

While voice modes are easily approachable, they are rarely used in QRP work. Voice modes have a hidden cost, Sidebands. The original, unmodulated radio signal produced by the transmitter still makes it to the antenna even when you're talking! This bit of the signal, the Carrier Wave, doesn't contain any information. Instead, by a quirk of combining waveforms, the signal spreads out below and above the carrier wave proportional to the spoken frequency. These are Sidebands. FM has huge sidebands, since it's frequency changes slightly. While AM's are smaller, the two sidebands are mirrors, containing the same vocal information. A more efficient modification of AM is called Single Side Band. This mode cuts out one of the sidebands and the carrier wave. This means that what is transmitted has much more punch and travels further than it would otherwise. This makes SSB a good mode for low power work.

The ultimate mode for QRP work is still CW. Since there is no superimposed audio wave, the sidebands are tiny. All the punch is put into a very narrow range of frequencies, meaning each watt carries the signal that much further. The problem is that CW is a dying art. It's no longer a required skill to get an Amateur Radio license. While this was done to lower the barrier to entry for prospective Hams, the ubiquity of computers has popularized Morse Code Decoders -- software that can listen to a CW signal and print out the text. Likewise, Morse Encoders allow you to use a keyboard to type letters, making the computer translate them into dits and dahs for you. 

The problem is, that feels like cheating to me. If you're going to do CW, you should break out a straight key, or an iambic paddle and cut out the silicon middleman. And while that is my plan, after I practice thoroughly, I'm still facing the problem of a poor antenna and a low power rig. Enter PSK31.

PSK31 stands for Phased Shift Keying, 31 Baud. In many ways, it's like Morse Code. It encodes letters in an equivalent of dits and dahs, but by shifting the phase of the signal. Going back to our imagined sine wave, if you're at a peak, PSK31 will suddenly shift to a valley, with no transition in between. By doing this rapidly, we can encode letters in a series of timed phase reversals. The nifty thing is that this is not done with the radio signal! Instead, PSK31 is encoded as an audio tone! Why do this? PSK31 was designed to be simple, and bare bones. Requiring new transmitters to be build just to support it would hamper adoption. Instead, PSK31 is produced (and received) by a computer, and sent to a standard SSB transceiver. Unlike transmitting a full human voice on SSB, PSK31 only results in a very narrow range of audio, this means the sidebands are very narrow. As narrow or more narrow than CW! This means all the power is packed into a tiny bit of frequency, making each watt count. 

Now here comes the really nifty part: PSK31 conversations are often all sent and received with the transceiver tuned to the same frequency. Multiple people, having multiple conversations, at the same time, on the same frequency. How? Recall that PSK31 is encoded as audio first. By telling the PSK31 software to produce audio at one tone or another, multiple people can occupy the same frequency. It's like a room full of people each with a whistle. Only two people in the room have the same kind of whistle that produces the same tone. Since they're only listening for the same tone they themselves are producing, they can ignore everyone else. 

Want to try it yourself? Really! No radio required! Download and install Digipan on your computer. If you're on Linux, it runs just fine in Wine! Now, plug in a microphone and start up the software. Go to Configure -> Sound Card and set things appropriately. Then put on this Youtube video while having the mic pointed at your computer speakers. Nifty!

In a typical Ham setup, however, a special interface is required to interconnect the computer in the transceiver in a safe way. Too much audio power from the receiver can damage your computer's sound card. Too much from the computer, can damage the transmitter. I recently purchased a SignaLink USB. The nice thing about this unit is it's only a USB sound card to the computer. It works in Linux just fine.

While I made my 560 mile contact fairly easily, there's a lot more I can do. My antenna was not properly tuned, resulting in poor efficiency. Not all of my 5 watts ended up "in the air". I think with some additional work and tuning, I can make it much, much further. 

Click here to read Part 1.

Click here to read Part 2.

Click here to read Part 3.

Understanding how Drupal gets from a page URL to page content is a complex topic involving many moving parts. In previous parts we've covered how Drupal initializes itself, we haven't discussed how a requested URL is routed within the CMS to the code that produces the web page. Handling this is the beating heart of Drupal -- the Menu System. Before we get to that, however, we need discussed modules. We glossed over them in the last part, but here we dive into detail.

Module Loading

It occurred to me after writing the last part of this series, I didn't go into more than high-level detail of drupal_bootstrap(). One of the biggest questions I had when I began this exploration was "How are modules loaded?" Coming from a self-taught C and C++ background, loading external code seemed a black art. Thankfully, PHP makes this very simple.

Apart from very simple applications or sample code, most programming projects are broken up into multiple files. C and C++ have a pre-processing directive called "#include". This isn't actual code, but a command sent to the compiler. What I failed to realize as I moved on to other languages -- especially those like PHP that are interpreted rather than complied -- is that PHP's method is actual code, and that's the key to how Drupal loads modules.

PHP has several ways to include a dependent file, include(), include_once(), require(), and require_once(). The difference between includes and requires is if an error is tossed if the file cannot be included. The *_once's prevent a file from being imported twice, thereby causing definition conflicts. You quickly learn that require_once() is the version you'll use the most. Another astonishing fact to C/C++ devs new to PHP is that all of these statements can accept variables.

How does Drupal load module code? Like this:

include_once($variable_of_module_file_path);

That's the core of it. You'll notice that the include_once() statement is used instead of the require_once(). This is because Drupal does not want to give you a White Screen of Death if a module's files have been misplaced. Drupal does use require_once() when loading other core files like bootstrap.inc or common.inc. After all, if those are missing, you're really in trouble!

It Can't be that Simple

You're right. It's not. While the basic idea of loading module code is an include_once() inside a foreach loop, the actual process is a bit more complicated. As we learned in the last part of this series, drupal_bootstrap() does not load all modules at once but in two distinct steps. During the DRUPAL_BOOTSTRAP_VARIABLES phase, several modules are loaded that are required for the bootstrap process to complete. If you're stepping through a default installation of Drupal, there are three bootstrap modules: devel, dblog, and overlay. The process looks like this:

  drupal_bootstrap()
    _drupal_bootstrap_variables()
      module_load_all($bootstrap = TRUE)
	module_list($bootstrap = TRUE)
	  Loop through list of bootstrap modules (devel, dblog, overlay)
	    drupal_load($type="module", $name)

We already know the first two, but the third, module_load_all() is new. This function gets a list of modules by calling module_list(). The $bootstrap parameter isn't used by module_load_all(), but is passed to module_list(), instructing it to return only bootstrap modules.

With an array of modules in hand, drupal_load() is called for each element. It checks to see if the file was already loaded, and then gets the module file name. A lot of care is taken in getting this file name and making sure that it's correct, and points to actual PHP code.

The module loading process is the same for non-bootstrap modules (including contrib modules) as it is for bootstrap modules. During the DRUPAL_BOOTSTRAP_FULL phase, a default installation of Drupal will load block, color, comment, contextual, dashboard, dblog[skipped], devel_generate, field, field_sql_storage, field_ui, file, filter, help, image, list, menu, node, number, options, overlay[skipped], path, rdf, search, shortcut, system, taxonomy, text, toolbar, update, user, devel[skipped], and standard.

Notice that in the above list, the bootstrap modules are drawn into the global module loading process. Thankfully, Drupal already knows that these modules have been loaded, and skips the process. The psudocode looks like it did earlier:

  drupal_bootstrap()
    _drupal_bootstrap_full()
      module_load_all()
	module_list($bootstrap = FALSE)
	  Loop through list of contrib modules
	  drupal_load($type="module", $name)

What about Hooks?

Loading modules isn't the only thing that's seemingly magic about modules. Module developers rely on a huge API of hooks into the Drupal process in order to add features, modify display, perform access restrictions, and so on.

It really comes down to a function called module_invoke(). It takes two parameters, the module name, and the hook to call. How it works its magic to call a function is via the PHP call_user_func_array() statement. This statement doesn't require a pointer or anything fancy to the function to invoke, just the function's name! Drupal modules implement individual hooks by creating functions with the moduleName_hookName() signature. This is very clearly represented in module_invoke():

  module_invoke($moduleName, $hookName)
    Get any additional unnamed $arguments as an array
    If the module implements the hook
      return call_user_func_array($moduleName . '_' . $hook, $arguments)

The module_invoke function actually can take more than the two named parameters thanks to PHP. These additional parameters are packaged up as array and passed to call_user_func_array(), which in turns passes the contents of the array as parameters to the hook implementation. This, and the fact that module_invoke() returns the result of the invoked hook, make it a smart function invoker cabable of working across modules.

Often in Drupal core, you do not simply invoke a single hook implementation, but call all the implementations of a hook at a single point. Handling this is a sister function to module_invoke(), module_invoke_all(). It's used the same exact way as module_invoke(), but lacks the $moduleName parameter.

Menu Routing

While modules provide Drupal's extensibility, Menu Routing does the real work of the CMS. At first blush, you may think that the "menu" refers to something like the primary and secondary links, or even the navigation block. After all, all of those are called "menus" in Drupal's own UI!

Repeat after me: Menu Routing has nothing to do with Menus, everything to do with Routing.

Page routing, specifically. The Menu [Routing] System is the core component of Drupal that takes a URL of the requested page and returns a viewable HTML web page. That is, it takes the part of the url after "?q=" in the follow example:

http://example.com/index.php?q=the/requested/page

In the above example, the page part of the URL is "the/requested/page", and this is exactly what the menu system handles. It matches up the page URL with the PHP function needed to generate the web page, and executes the function. When coding a module, the Menu System is often the first thing you'll code after an *.info. file. It's perhaps the most approachable piece of API in the entire project, which is why I bring it up here:

  function motleymod_menu(){
    $items[my/motley/page] = array(
      'page callback' => 'motleymod_my_page', 
    );
    return $items;
  }
  
  function motleymod_my_page(){
    return "<p>This is my motley page.</p>";
  }

The above code would work relatively unchanged as far back as Drupal 5.0. We have two functions, the first implements hook_menu(), and the other generates and returns the page content. You'll notice that the first function defines and returns an array with one element with the key 'my/motley/page'. As you might have guessed, this is the page URL! The content of the element is itself an array containing one element, 'Page Callback'.

The Page Callback parameter instructs Drupal what PHP function to invoke to generate the page. In this case, motleymod_my_page(), which returns raw HTML to display in the body section of the webpage. Drupal takes care of everything else -- headers, blocks, footer, and all.

But That's Not the End of the Story

If you think the above example was a little spare, you're right. Let's look at a more complex example of hook_menu():

  function motleymod_menu{
    $items[my/motley/page] = array(
      'page callback' => 'motleymod_my_page',
    );
    
    $items[admin/config/motley] = array(
      'title' => "Configure Motley Module!",
      'page callback' => 'drupal_get_form',
      'page arguments' => array('motleymod_admin_settings'),
      'access arguments' => array('administer motleymod'),
      'type' => MENU_NORMAL_ITEM,
      'file' => 'motleymod.admin.inc'
    );

    return $items;
  }

Whoa! What the heck is all that stuff? We see our original entry for 'my/motley/page', but now there's an additional entry for 'admin/config/motley'. This time, with a lot more parameters than just Page Callback:

• Title is the title of the page to display. It's used both in the <title> tag of the generated page, as well as in the <h1> tag in the body section of the page.
• Page Callback is the the magic parameter. It tells Drupal what function to call!
• Page Arguments specify the arguments to pass to the function specified in the Page Callback parameter.
• Access Arguments specify the access permissions (under admin/people/permissions) the current user must have in order to access the page.
• Type is the kind of menu item this item represents. More on that later.
• File tells Drupal in what file to find the function specified in the Page Callback parameter.

Again, we have a Page Callback, but something seems screwy: It's set to drupal_get_form() -- a function provided by Drupal itself. How the heck does that work? Over the history of Drupal, it was discovered that many module's Page Callbacks returned pages with similar structures. Furthermore, pages started falling into classifications of pages. Content pages usually have a title and a bit of text to display. Settings pages usually have a title and a form with a submit button. Drupal-provided page generators, like drupal_get_form(), handle a lot of rendering work for you, making module developer's lives a little easier. In this case, the function provides a module setting page.

menu_execute_active_handler()

Now that we have a framework, let's tie it back to where we were in our debugger. After drupal_bootstrap(), the last function called in index.php is menu_execute_active_handler(). Without any arguments, the function looks up the page URL, and then digs into the Menu System.

The first thing that it does is check if the site is "offline", that is, in Maintenance Mode. Administrators set this mode under admin/config/development/maintenance, and it temporarily blocks all other visitors save the Admin from accessing the site. This allows the Admin to perform site updates safely. If the site is offline, it doesn't matter what page was requested, all results are sent to _menu_site_is_offline(). This function checks to see if the user is the Admin, if so, normal page routing resumes.

The next thing menu_execute_active_handler() does is check if the Menu System needs to be rebuilt. This is where hook_menu() is invoked. Normally, Drupal 7 stores menu routing information in a database table -- menu_router. It does this because performing a database query is actually faster than searching through all implementations of hook_menu() to find a match. During a rebuild, the contents of menu_router are thrown out. Drupal then locks the database temporarily and recurses through each enabled module and invokes hook_menu(). This results in a huge array of menu items like we saw in the code sample above. This is then committed to the menu_router table.

After that, things get much more straightforward. The appropriate menu item is found, and the function specified in the Page Callback is called passing any Page Arguments as necessary. The result of the Page Callback is then sent to the user's browser for display.

Thirsty Yet?

After all we've covered, it's easy to simply accept that how Drupal is written is the best way it can be written. I know that when I started using Drupal, it was after I tried and failed to create my own content manager. "Let me learn from those more experienced than I!" This, however, blinded me to one of the biggest problems Drupal is facing today.

Drupal is heavy.

It used to be that each time a page was requested, you could assume that the entire page was requested. Graphics, HTML, and all. In the new world of refresh-less web apps, however, use AJAX requests to grab content or save information. Instead of sending the entire page, lighter packages of JSON formatted data are exchanged between the user's browser and the web server. While Drupal can handle these requests, it must still go through the entire initialization and routing process it would for a full web page. Worse yet, it must load all modules into memory with each request -- even if that module is not needed in that request.

I discovered this myself when I attended Crell's presentation during the Twin Cities Drupal Camp introducing the Web Services and Context Core Initiative. The goal of WSCCI (pronounced like the drink) is to ultimately replace the heavy drupal_bootstrap() and menu_router() system with a more agile core. Intead of Drupal being just a first-class CMS, it will be a REST server with a first-class CMS on top.

It's an ambitious but, I believe, necessary evolution of the platform. While targeted for Drupal 8, I have my doubts it will be fully realized until Drupal 9.

Unless we help, that is.

What's Next?

We've reached the end of index.php, but not of our series. There's still a lot more to explore, but the going gets more treacherous from here.

Click here to read Part 1.

Click here to read Part 2.

When looking at Drupal code, it can be hard to know where to start. Dive into any of the directories in a standard installation, and you'll find file after file of PHP code. In the root directory alone, you'll find five files! Where do we start? Determining this is quite simpler than it seems; all you have to do is look at the URL of any Drupal site. 

http://example.com/node/1234

The above is a typical URL for a Drupal site. Most Drupal sites use a feature called "Clean URLs". This removes a lot of the nuts and bolts of the address that helps us understand it. Without Clean URLs enabled, the above address looks like this:

http://example.com/?q=node/1234

Okay, that's a little helpful, but it doesn't tell us what we want to know. What file amongst all the files in Drupal do we look at? It turns out that many websites will look for a default file in the site's root directory if one isn't specified in the URL. In the Web 1.0 days, this was "index.html", for modern PHP websites this is "index.php". 

Four Lines to Rule Them

When we open of index.php, one might expect a lot of code. Instead (aside from comments) you'll find four lines:

define('DRUPAL_ROOT', getcwd());

require_once DRUPAL_ROOT . '/includes/bootstrap.inc';

drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

menu_execute_active_handler();

The first two lines are very straightforward. The first line defines a constant, DRUPAL_ROOT, containing the file path to the Drupal installation. This constant is then used in the second line, to load bootstrap.inc in the includes/ subdirectory. Despite the file extension, bootstrap.inc contains PHP code. Drupal uses the *.inc extension to prevent files from being executed directly. This way, someone cannot plug in http://example.com/includes/bootstrap.inc and get back a valid page.

That leaves two more lines, and that's where all the fun starts. At this point we've only defined the DRUPAL_ROOT constant and imported an include file. We have yet to execute any real code. Any Drupal page request can be thought of as a two step process (if you're a core dev, insert your groans here): Initialize the Drupal environment, and then handle a page request. The third line of code, drupal_bootstrap() does the initialization -- it forges a connection to the database, loads modules, loads needed data into memory, and prepares everything for the final line of code in index.php.

The menu_execute_active_handler() function handles the page request. This is the part that takes the ?q= half of the URL and produces a web page for the visitor to see. This is actually a new function in Drupal 7. Drupal 6 preferred to bury the request handling within drupal_bootstrap(). Separating the two makes things much, much cleaner.

Making Invocation Easier

In the last article in this series, I mentioned that invoking the debugger was somewhat tricky. Eclipse assumes that any code you're debugging is in a project and not external files. It turns out that the PHP Development Toolkit actuall anticipates this problem and solves it rather easily. 

1. Create a new PHP project by using the File menu:
   File -> New -> PHP Project
2. In the provided field, enter a Name of your choice that is unique within your workspace.
3. Under the Contents box, select the Create Project at existing location (from existing source) radio button. 
4. Click the Browse button adjacent to the radio button. Select the root of your Drupal install. 
5. Click Finish.

This creates a new project in Eclipse importing all of Drupal's files. This mode of project creation does not create a copy of the files within the Eclipse workspace, but rather creates references to the actual files on the web server. Thus, any changes you make to the files in the Eclipse UI affect Drupal's actual files. This is great for our purpose of exploration, but can problematic. Remember, don't hack core! 

Once the project is created, invoking the debugger is easy, easy, easy. Double-click index.php in the project explorer, then start the debugger with Run -> Debug As -> PHP Web Page.

Booting Up Drupal

Once you've started a new debugging session, we can dig into drupal_bootstrap(). The function looks a little circuitous at first: An array of phases is initialized, and then a while loop begins, iterating through a seven step process in which key subsystems and dependents of those subsystems are initialized in the correct order. Here's how it boils down:

• DRUPAL_BOOTSTRAP_CONFIGURATION -- Do some basic setup and load settings.php.
• DRUPAL_BOOTSTRAP_PAGE_CACHE -- If page caching is enabled, and the request is asking for a cached page, return the page.
• DRUPAL_BOOTSTRAP_DATABASE -- Connect to the database.
• DRUPAL_BOOTSTRAP_VARIABLES -- Load variables and enabled modules required for bootstrap.
• DRUPAL_BOOTSTRAP_SESSION -- Load the user's session from the DB. If the request isn't from a logged in user, return an anonymous user.
• DRUPAL_BOOTSTRAP_PAGE_HEADER -- Invoke hook_boot(), send default HTTP headers.
• DRUPAL_BOOTSTRAP_LANGUAGE -- Initialize the language system for translations. 
• DRUPAL_BOOTSTRAP_FULL -- The phase name is better thought as "Last", rather than "full". Load all enabled modules. Invoke hook_init().

Why the added complexity? Why not just use a static series of functions? The initialization process can be very complicated. For example, you may be in the middle of one phase, like DRUPAL_BOOTSTRAP_PAGE_CACHE, and you'll need to load something from the database. The code that's initializing the page cache will invoke drupal_bootstrap(DRUPAL_BOOTSTRAP_DATABASE). Furthermore, each phase is dependent on each preceding phase -- you cannot do the DRUPAL_BOOTSTRAP_VARIABLES phase without doing DRUPAL_BOOTSTRAP_DATABASE, DRUPAL_BOOTSTRAP_PAGE_CACHE, DRUPAL_BOOTSTRAP_CONFIGURATION. This is used by index.php itself when it calls drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL). Since DRUPAL_BOOTSTRAP_FULL is the last phase in the list, it is dependent on all previous phases. 

Each phase of the bootstrap process has a lot of code behind it. While I could write an article for each, there's so much more Drupal to cover. We haven't even served a page yet! That's what we'll do next when we cover menu_execute_active_handler().

My gender-inverted version of the famous Babylon 5 "Fasten then Zip" scene from the episode Babylon Squared.

Garibaldi: Mind if I ask you a question?
Sinclair: Sure.
Garibaldi: Okay, it's morning. You get ready to go to work. You put on your bra. Do you clasp in the front and then twist it around or just clasp it in the back?
Sinclair: What kind of question is that?
Garibaldi: Look, we got two hours to kill...
Sinclair: Forget it!

Sinclair: Why do you wanna know?
Garibaldi: Why do I wanna know? Because I think of these things sometimes. I was getting dressed this morning - then I stopped and started thinking about it. Does everybody do it the same way? Does it have to do with cup size? Band size? Wrist strength?
Sinclair: Do you think about this stuff a lot?
Garibaldi: Yeah.

Sinclair: *sighs* Clasp, then twist. You?
Garibaldi: No twist, just clasp.
Sinclair: Huh.

Garibaldi: Do you put your arms through the straps first?
Sinclair: We're done.