svn copy http://localhost/svn/ll/trunk/php http://localhost/svn/ll/branches/phpMVC

The branch development was adopted, thus requiring merging the code back into the trunk. Overall, using a branch on just the subdirectory I was working on made the development and merging easier. Commits by others could be retrieved by an update without needing a merge, minimizing disruptions during coding.

I screwed up the merge as follows:

• svn merge -r 11:15 http:../branches/phpMVC – on the basis of a stop-on-copy command that started at 15 and ended at 11.  But other files in the branch had been modified up to revision 20.  So I will use the form svn merge -r 11:HEAD from now on.
• I kept issuing the svn merge and svn switch  commands in the root directory instead of the php subdirectory.  These missteps were reported by SVN and I recovered, but I did scare myself a few times.

Also, I modified a file in a css subdirectory which I could not commit to the branch because it was not within the scope of the branch. Next time I will create the branch on the entire project and then svn switch only the subdirectory I’m working on. This leaves the option of switching other subdirectories if the work expands.

YaST isn’t good at all facets of Apache configuration so some editing of conf files may eventually be needed. If manual editing is required, why not use it exclusively? Because once you go manual, you can’t switch back to YaST when major changes are needed. I like the fact that if I install a new web application that needs several additional Apache modules, I can use YaST to enable them in a couple of minutes, often without having to learn anything about them.

I also like the fact that using YaST to configure Apache will give me a configuration closely matched to thousands of other openSUSE Apache servers. This makes it likely that I will get a functional, robust httpd set up at the start. Then I can incrementally add modules and customizations as needed, testing as I go.

Following is a YaST-based recipe to install a straightforward httpd configuration with Perl, PHP, and mod_userdir enabled. Subversion, WebDAV, and virtual host support are also added (using some conf file editing). Finally, the added bookkeeping steps that allow YaST and hand-edited conf files to co-exist are presented.

Part I: Content-Independent Configuration

Part 1 installs a functional (but empty) httpd server with all modules and parameters properly configured. Connections to the site-specific content are discussed in Part 2.

1. OpenSUSE scatters Apache conf files among /etc/apache2 and its conf.d, sysconfig.d and vhosts.d sub-directories. Keep a copy of the original conf files to determine what YaST has done and add source code control repositories to manage manually edited files:
   • cd /etc/apache2
   • mkdir ORIG; cp -p *.conf ORIG
   • mkdir RCS
   • Repeat for the three “.d” subdirectories.
2. Start YaST/ Network Services/ HTTP Server:
   • Turn on php, perl, python and/or ruby in a special configuration page that only appears once (one can still turn these on/off later, but it’s easier to do here).
   • Accept defaults in the Server Wizard (5 pages), except choose Start When Booting on page 5.
   • Click Advanced Configuration.
   • In the Server Modules tab, enable dav, dav_fs, info, rewrite and status.
   • Needed to Add Module for dav_svn but the YaST screen would not accept input, so I hit Finish (will add in Step 3).
     
   • The server starts. http://localhost just says “It works!” instead of the normal Apache welcome page. Didn’t like this so I copied index.html from an old SUSE 10.0 installation.
   • Read the comments in all of the /etc/apache2/conf.d/*.conf files, some of which are controlled by IfDefine blocks. For my installation, I learned I needed to add the SVN_DOC and SVN_VIEWCVS flags (next step).
3. Start YaST/ System/ sysconfig Editor:
   • Select Network, WWW, Apache2.
   • Highlight APACHE_MODULES. Add dav_svn into the Modules line. Note: Don’t hand-edit dav_svn into /etc/apache2/sysconfig.d/loadmodule.conf, as YaST will just remove it next time it runs.
   • Highlight APACHE_SERVER_FLAGS. Type “SVN_DOC SVN_VIEWCVS” into the text field.
   • Finish.
4. A diff of ORIG/default-server.conf with the one YaST created shows that YaST removed the mod_userdir section, changed indents, and added ServerName and ServerAdmin. Thus I chose the original distribution .conf as my starting point:
   • mv default-server.conf{,.YAST}
   • cp ORIG/default-server.conf .

   • ci -l default-server.conf
   • Edit with a text editor and copy the ServerName and ServerAdmin directives from the YaST version into default-server.conf.
   • ci -l default-server.conf
5. Restart YaST/ Network Services/ HTTP Server:
   • Select the Server Modules tab and verify that the dav_svn module appears.
   • Exit or Abort.
   • Note: from this point forward, if YaST/HTTP Server is re-run and Finish is clicked, default-server.conf and /etc/sysconfig/apache2 will be overwritten. Capture a snapshot before running or retrieve original from RCS archive afterward.
6. Check the syntax of all configuration files and reload httpd:
   rcapache2 restart-hup
7. Browse http://localhost/server-info and server-status. Verify all desired modules are present and running.
8. Browse //localhost/manual and localhost/svn-manual to see documentation.

The content-independent phase of installation is complete.

Part II: Serving Local Site Content

To prepare for Part 2, load all content files that are outside of the server document root (e.g. Subversion repositories, WebDAV directories, and user public_html areas). I keep my Subversion repositories under /home/wwwrun/svn, giving them the same owner and group as the httpd process.

1. Verify that browsing a http://localhost/~<username> directory works.
2. Subversion repository configuration goes in the /etc/apache2/conf.d/subversion.conf file. I piggybacked the WebDAV configuration into this file rather than creating a separate file: subversion.conf. Also, I add Basic Auth to ViewVC in subversion.viewvc.conf (see earlier blog entry for an explanation).
3. Use htpasswd2 to create the basic authentication password file.
4. Create a /var/lock/apache2 directory (user wwwrun, group www).
5. rcapache2 configtest
6. rcapache2 restart-hup
7. Browse http://localhost/svn/<reponame>. Username and password are requested and the repository appears. Run RapidSVN from a Windows machine to verify connectivity.
8. Edit /srv/viewvc/viewvc.conf to add root or root_parents, default root, and mailto.
9. Browse http://localhost/viewvc , username and password are requested, then the ViewVC page appears. Click on the “ViewVC Help” hyperlink and the documentation appears.
10. Using Konqueror, browse webdav://localhost/fs and a folder view of the WebDAV folder appears.
11. See this blog entry for details of connecting to WebDAV via WindowsXP.

All of my local Subversion, WebDAV, and web page content is now available via Apache.

Part III: Virtual Hosts

Finally I’ll add virtual hosts for my various development, prototype, and staging areas. The YaST HTTP Server GUI offers a screen to do this, but I chose to start with the as-delivered file just as with default-server.conf.

1. Create a new conf file in /etc/apache2/vhosts.d (e.g. LLvhost.conf in my case). I copied the vhost.template as the starting point.
2. I defined four virtual servers:
   • localhost – contains only a ServerName statement to serve as the default server.
   • llxxx – three name-based servers that have document roots in various code development sub-directories. Each of these gets a separate set of error and access log files.
3. Uncomment the NameVirtualHost *:80 line in /etc/apache2/listen.conf.
4. Add the three name-based server names to /etc/hosts as synonyms for localhost.
5. rcapache2 configtest
6. rcapache2 restart-hup
7. Browse http://llxxx and the virtual host content appears. Verify that log information appears in the correct file pair in /var/log/apache2.

Part IV: Configuration Control and Maintenance

The Apache server installation is now complete. But adding source code control is necessary to safely re-run YaST/ Network Services/ HTTP Server Configuration in the future. Whenever HTTP Server Configuration is ended with Save or Finish, any of the conf files may be overwritten, obliterating prior manual edits. Source code control allows us to manage and resolve this conflict.

Using the unaltered conf files in the ORIG directories as references, perform a diff of all of the conf files to learn which have been customized. Any altered conf files that were hand-edited should be placed under configuration control (e.g. CVS, Subversion, RCS). Consider controlling the YaST-altered files as well.

I maintain five files in RCS: default-server.conf, mod_userdir.conf, subversion.conf, subversion.viewvc.conf, LLvhost.conf. After saving any new HTTP Server Configuration session, I use rcsdiff to learn what changes have been made to these files. Review the changes and either restore the last version from RCS or check in a new revision as appropriate.

Comments:

• I’m using AuthType Basic for svn, viewcvs, and filesystem DAV access. It provides some security, and is also necessary for WindowsXP, which apparently will not connect to a Web Folder with no authentication.
• The viewCVS location needs its own authentication, since this tool accesses the SVN repository directly using the file: protocol. In doing so, it bypasses the http server /svn location and its associated authentication.
• The SVNParentPath is convenient because new SVN repositories can be added without editing the conf file. The downside is that different users can’t be restricted to particular repositories or given different access privileges.
• ViewCVS has an additional configuration file /srv/viewcvs/viewcvs.conf that must contain the roots of all repositories, plus a few other items.

• Gave my httpd workstation a reserved IP address in the dhcp server and then forwarded http requests to that address.
• Created dynamic DNS accounts at both no-ip.com and DynDNS.org. The ZyXEL was able to update both without a hitch. Settled on no-ip.com.
• Both no-ip.com and the ZyXEL router support port shifting, so I had no-ip shift 80 to 5555 and then had the ZyXEL shift back from 5555 to 80. This worked but with problems: 1) the actual IP address showed up in the browser, and 2) PROPFIND and the other WebDAV methods were rejected by no-ip on port 5555.  So port translation is out for now.
• Installed RapidSVN on WindowsXP after deciding I didn’t want the TortiseSVN commands showing up in every right-mouse click. RapidSVN worked perfectly after removing the port shifting described above.

* = svn:keywords=Date Rev Author Id

This turns on keywords for all files, and I don’t have to make a new entry when I start working with a new type of source file.

However, I realized that with this rule, .jpg and .gif files will have keywords turned on and could trigger a substitution if say $Id$ appeared as part of the binary data. So I went into my image directories and issued:

svn propdel svn:keywords *

Given subversion’s current config capabilities, I’ll always have to remember to perform a manual step. Currently, I have to remember to do the propdel for all new binary files. With the normal suffix rules, I’d have to add a new rule for each new source file type. I’d probably forget the first time and have to do a propadd for a bunch of files.

Subversion should extend the auto-prop rules to allow matching on text and binary file types; then I could change my * rule to text files only.

In the branch working dir:
svn log -v --stop-on-copy

This reports the list of revisions since the branch was created. This range of revisions is used as an argument to the merge command.

Create a clean trunk working dir and this issue these commands:
svn merge -r 11:15 http://../branches/css-1024px
svn diff -r 15 http://../branches/css-1024px
svn commit

Next, switch my development and reference working directories back to pointing at the trunk:
svn switch http://localhost/svn/ll/trunk

Lastly, there were a bunch of tmp files in my working directory, so I tried out svn cleanup which removed them.

svn copy http://localhost/svn/ll/trunk http://localhost/svn/ll/branches/gui
cd wkgDirectory
svn switch http://localhost/svn/ll/branches/gui

Everything worked as advertised and the copy added only 5% to the repository size, since files aren’t duplicated until a modification is committed. The documentation says that individual directories and even files can be switched. Directories worked but individual files turned into working directories?

I’m up and running under Subversion and am declaring the conversion from CVS as complete.

As detailed on 14 Nov, the lack of httpd authentication prevents svn updates from getting an author assigned. So now I’m installing authentication per the Subversion instructions:

• htpasswd2 -cm /etc/apache2/basicHTTPpasswd bozzie
• Cut & pasted the Auth{Type,Name,User} directives from the instructions into the <Location> /svn section of my default-server.conf file. Cut & pasted the Require valid-user only for write actions, also from the instructions.

Re-created the repository and imported. The import challenged for a password and the newly checked-in revision finally has an author.

SVN is ready for use, I think.

ViewCVS (ViewVC)

SuSE installs this package in /srv/viewcvs. Finally found some installation instructions by downloading the current version from the Tigris web site and ad-libbing from the viewVC instructions.

Summary of mods to default-server.conf:

• Alias /viewcvs-docroot /srv/viewcvs/templates/docroot
• ScriptAlias /viewcvs /srv/viewcvs/www/cgi/viewcvs.cgi
• Plus viewcvs.conf must be edited.

ViewCVS presents a serviceable graphic interface to SVN, which I imagine is even better in the current viewVC version. There is also a graphing interface of some sort that I haven’t looked into yet.

Next, the output of svn diff is a non-standard format, apparently used for creating patch files. I specified /usr/bin/diff via the .subversion/config diff-cmd option. No change! Finally discovered that svn diff passes a “-u” argument to diff, resulting in a non-default report format. Wrote a Perl wrapper to substitute the –normal option and call this wrapper via my .subversion config entry. Finally, the svn diff output now looks like a normal diff command.

Next, the SVN log says “(no author)” for files imported using Apache. I finally found the answer in the svn FAQ:

“If you allow anonymous write access to the repository via Apache, the Apache server never challenges the SVN client for a username, and instead permits the write operation without authentication. Since Subversion has no idea who did the operation, this results in a log like this (no author).”

So now I will add authentication.