Mason and I talked a bit about using Subversion to store our web projects. I had originally thought it would be best for each web project to be stored in its own repository (Palmer LTER web is stored elsewhere from CCE LTER web). However, we ultimately decided to stick with the many projects to one repository approach.

This decision comes easily. We already have a repository set up, so there’s no need to go about creating new ones and configuring apache to recognize them. Additionally, none of our projects are on a large enough scale to warrant being separated. Some of the projects may share some common code, and it would easier to merge like components of different projects within a single repository (opposed to distributed repositories). Lastly, it’s easy to migrate projects into seperate repositories should we ever want change to the one-project-per-one-repository approach.

The main issue we discussed is how to store documents: docs, pdfs, imgs, etc… basically files that don’t really need to be versioned. Our solution is to store all such files in a /docs folder that sits at the root level of a given web site. This folder and its contents will not be imported into the repository. When checking out the project, the docs folder will be missing. It will either need to be copied over from another working area, or a symbolic link can be created assuming both working areas are on the same server.

This solution implies that some images may fall under the docs folder. This makes sense, especially if these images are not on the peripheral, but serve a purpose to provide information, be it group photos or mapping grids. Thus, some images embedded into pages’ content will not be versioned.

Some integrated web apps (photo gallery, iforum, etc.) store files at a path deeper than the web root. Any such folders and their contents should also be excluded from the repository. In this case, a README file should be created explaining which folders are missing.

Most of our projects have a config.php file. This file should not be versioned since its settings are usually pertinent only to the local working copy. However, we should commit a config-sample.php file, which is a copy of config.php, but with all sensitive data removed (e.g. database connection settings). A global README file should contain instructions for creating the config.php file:

e.g. Rename config-sample.php to config.php and enter the settings.

A global README file (which rests at the webroot) should also contain information on which folders are being left out of the repository. Each distributed web app can also contain a README file at its root for more in-depth explanation on the information related to that project.

That’s all for now. I’ll add more to this as we make progress loading our projects back into Subversion. The first tests will include the IOD Personnel admin code, and the CCE LTER site.

This week I finished porting the Palmer site into the new theme/framework. From Mason’s advice, I ported most of the files over from the Production area since Mason had previously used a link checker to originally move only the necessary files into the Production area, trimming it down significantly from the Dev area.

I ported practically every page manually. This allowed me to address some usability issues on certain pages, clean up some html markup, and most importantly, detect broken links and missing files. We have a few missing files in the Production area because the link checker we used failed to follow symbolic links. In addition, some documents may not have been detected, partically if they were referenced by a different protocol (file:/// vs. http://). These issues have been fixed for the New site.

The New Palmer site has reached a somewhat stable point. We have succeeded in restructuring the file system so to be parallel with the navigation setup of the site. It is now time (particularly for Karen) to fine-tune the structuring. Some folders and files may be moved around as Karen continues to add more content. However, the framework of the new file structure should help guide us to place and locate documents in a semantically meaningful way.

I would eventually like to run a link checker on the New site as part of the fine-tuning process. Mason used a fairly good one, nice interface and all that… except it’s Windows only. Plus it’s limited to 500 links w/o a registered license. We can use that link checker as a Plan B. I may try to find a Mac link checker next week. Best bet is probably to check versiontracker.com.

We will ultimately throw this site into Subversion. However, we need to address some issues before hand, particularly concerning images and documents. The Palmer site is loaded with distributed docs, pdfs, txts, and other documents. Is it worth loading these into svn and having them eat up space? I still beleive they should not g0 in the repository. This leaves a couple options:

1. Create a central location to store images and documents, like pal.lternet.edu/images and pal.lternet.edu/docs
   This method requires an extra global variable in php to store the permanent URI of these locations. In this case, that would be pal.lternet.edu
   This method also implies that we could somewhat mirror the file structure inside these folders. For instance, a pdf referenced in the sci-research page could be located under /docs/sci-research
2. Store images and documents in the both Development and Production areas. I am against this idea, mainly because it requires double-maintenance of any changes made to these files. It also makes the site less extensible. Should someone else want to checkout a 3rd copy from the repository, they would also need to copy over these folders from one of the two areas.

A lot of these documents are embedded deep into the multiple tiers of the file system. To sweep thru the system and move them all to a central docs folder would be quite tedious. Not only does it involve the physicall moving of the object, it also requires to make changes in the html to all hyperlinks that refer to that document. However, a workaround for that may be to create symbolic links to the documents’ new resting place so we could leave the hrefs at peace. If we choose to leave the files where they sit now, this corresponds to Option #2. It makes it easier on us by removing the burden of extra work, but it results in duplicated files spread thinly throughout the site.

***

I’ve also worked on some datacat stuff this week. Yesterday, I recoded the html for the datasets details page, so it fits well with the new template, and today I recoded the html for the attributes form. In addition, I’ve also been working on the IOD Personnel page.

Jotting down some notes and to-dos for the IOD personnel app:

- Jerry and I enabled http-based authentication yesterday over an SSL connection to provide extra security. These measures are in place to ensure that only privileged users can access and edit a user’s profile, especially since we need a secure place to add and view users keyed by their UCSD Employee #.

- We still need to establish a direct connection to the BLINK database. Currently I am using an indirect route: Pulling data from Wayne, who in turn is pulling from BLINK. Wayne is only pulling a subset of fields, exluding ones we’d like to have (alt. phone, lab phone, mobile phone, title, etc.).

- Once we can pull all the data we need from BLINK, we need to create lookup tables to match abbreviated values to their actual names. Michelle has already sent an email containing the mappings that belong in the lookup tables.

- For the IOD people who have R.A.B (Research Activities) profiles, we need to harvest those profile links and add them to their IOD profile.

- We should create a “print view” of the personnel, one for the users to download, and perhaps one with extended info for michelle to use.

- The emails should be removed from the list view on the People page to prevent robots from harvesting them. I’d prefer to do this once we have a valid field to take it’s place, like Title.
[UPDATE 1/27] - Replaced the Email column with Building. Ultimately this should be Title, but until we have the database completely populated, this will work fine.
[UPDATE 1/27] - Changed Building to Location and appended the room # if it exists. This works under the assumption that we will replace this column with Titles before Michelle may start adding multiple room numbers for people.

- Last names are in CAPS in BLINK, but should preferably be in Uppercase here.
[UPDATE 1/26] - I wrote a script called name.php which converts all LAST NAMES to just have Upper Case First Letters. This is done using two php functions in conjuction with each other:
$name = ucwords(strtolower($str))
I also modified the syncing script to call these functions when pulling the last name from BLINK.

- Should the phone number format change? Currently it’s (xxx)xxx-xxxx.

- How should we implement the People Search feature?

To anyone else involved with this project, feel free to update or comment on this post. We can experiment with using it as a running log to track the progress.

Scroll down to the bottom of this page (or any page in this site if this page is too long). You will find two logos added in the footer section: UCSD and SIO.

I just added these. Up until now, we had no logos on this site, except for the OI logo embedded into the header graphic.

Big deal, right?

Well yeah, it is.

Go to Google and search for ‘Ocean Informatics’ (with or without quotes). We are currently the 4th site returned in the results. Not as impressive as being in the top 3, I know… but at least we are now on the first page! For a while, we were sitting buried under layers of irrelevant results.

So what does this have to do with the logos? Well, I remember learning at point (from Wayne I think) that any publicly accessable page in the ucsd domain should sport a ucsd logo somewhere. Our site has failed to do that since August when I first set up the blog. That is, until now.

I first noticed the missing logos as a concern yesterday. I was searching for info on Subversion use with web-related projects, and I stumbled upon my recent blog entry from late last year. Treating myself as an ‘outside visitor’, I noticed that I learned very little about Ocean Informatics, particularly the Who, Where, and Why. The only hint came from the ucsd.edu domain, which would lead me to guess this was a university-related site.

Thus, in putting the logos on the site, we help to clarify the Who (somewhat) and the Where. Now, maybe we should revise the site (the content, not the theme!) to give a bit friendlier welcome to outside visitors, especially if our blog posts are showing up in Google searches.

Maybe the next step is a visitor logger…..

In moving this thread away from email and into the blogging realm, I have a couple announcements regarding updates to the OI powerpoint figure.

I added 4 semi-transparent white circles, one in each quadrant, for the text the lay on. This should help the text stand out as it is added in powerpoint. The new files are available under oceaninformatics/figs_ppt/06OI_OVKarinKnorr/

Photoshop file: oi_ov_bg2.psd
Corresponding jpg: oi_ov_bg2.jpg
The latest ppt version: BakerOI_OV6.ppt

Note:
Karen, if you wish to added the content from our previous emails to the blog for archival purposes, we can do so in this post… all we need to do is edit this.

Last week I began moving the current Palmer LTER site into WordPress, mainly as an experiment to see how well WordPress works as a CMS-lite solution. I had already moved the CCE LTER site into WordPress successfully (being a much much smaller site compared to Palmer). The CCE site performed smoothly under WordPress 1.5. For the Palmer site, I downloaded and installed the newly released WordPress 2.0.

The result was not impressive.

After migrating a dozen pages, I noticed that WordPress 2.0 performs awfully slow, even compared to version 1.5. There were a lot of lag time (wait) between page navigations. Though WordPress 2.0 contains some improved features in the administrative area, the cons of the sluggish performance outweigh the pros of the beefed-up admin interface.

At this point I had 3 options:

1. Continue with the migration to WordPress 2.0. Maybe the slow performance is a transient issue that may go away (a future release of WP 2.1?, or a server-related issue, etc.)…. this however is a fat chance.
2. Downgrade back to WordPress 1.5. Afterall, the CCE site runs fine using it, where performance is much faster.
3. Throw out WordPress completely. I considered this briefly, and it quickly made sense. The Palmer site will be easier to maintain if we go back to the good ol’ file structure, and ignore a CMS-type solution completely. I’ll explain why a little later…

First, my reasons for originally choosing WordPress:

• Success with Ocean Informatics site - We’ve been using WordPress for our Ocean Informatics blog/home site since August, and it is undoubtely a success. We’ve already established a grounding of understanding and learned the intricacies of WordPress including plugins, hacks, and themes.
• CMS-Lite Solution - We’ve spent the last year and a half (though not recently) exploring various open-source CMS applications, including PostNuke, Mambo, and Xoops. Though each of these programs excel in providing various features for building a community portal, they required too much overhead for our simple Ocean Informatics group. We needed something lighter. Thus, I played around with the idea that WordPress could function as a CMS-lite solution: ability to post static pages in addition to blog posts, hierarchical ordering of categories, etc. (I still believe that WordPress is a great CMS-lite platform.)
• Pretty Themes - The most superficial reason of them all, but maybe also the most important?
  Once I downloaded and tweaked (changed colors, graphics, etc.) the Connections theme for WordPress, I felt a lot more comfortable. Building the navigation and sidebar menus became fun. The pages became colorful but not too distracting. The page title was pronounced, and it was enjoyable to read the page content…. basically it was a really well-designed theme. And I didnt have to do any design work! (Except the tweaking of course). Because all the visualization work is done for me, it relieves me of the burden of having to design the website, so I can focus more on coding applications and re-structuring the file system.

Why the Palmer site works better with a file structure:

• What to do with files? The Palmer site contains many non-php, non-html files, like docs, pdfs, images, and text files. All of these need a place to be stored and be accessable with an associated URI (web link). Using a file structure, we can logically place such files in the semantically meaningful folders, nested appropiately according to the site’s navigation scheme. This method works much better than in WordPress, where the basic solution is to throw all of these files into a centralized folder, thus flattening the semantic layout and the URI construction of the file’s path.
• The Connections theme can work outside of WordPress. It’s basically just a stylesheet, a few images, and a few php files (header, footer, etc). Even though the site no longer lives in WordPress, it can look exactly the same.
• Performance is much much faster… even faster than WordPress 1.5. Navigation between pages is lightning fast compared to within WordPress, where it stores all page content in the database. Thus, whereas WordPress takes some time to process the code and query the DB, the file structure solution simply retrieves the php file which has all the contented embedded inside.

What we lose when we move away from WordPress:

• No Web Editor - All editing takes place in the file, usually using a unix editor like vi or emacs. This means that anyone who wishes to edit a file must contain some knowledge of html. This also limits the users who can edit files to those who have server accounts or another means of access (ftp). The advantage of WordPress was its built-in web editor which generates some html for you, making it easier for non-html users to add and update content.
• Blog, RSS, etc. - WordPress is mainly a blogging platform, with rss publishing and aggragation. If we ever decide to setup a Palmer web blog, we can still use WordPress. The rest of the site, however, would stay outside.

Just why does the Palmer site need to be moved anyway?

The Palmer site doesn’t just need to be moved… it needs to be completely re-organized. The file-structure behind the Palmer site is ridiculously out-0f-sync with the web site’s navigation scheme. This is confusing because the path you would take to navigate to an item on the web site differs from the path you would take to find the same item on the hard drive. A web site with a good navigation system is one that mimics its directory structure. The Palmer site is suffering from 10+ years of data build-up and low maintenance, creating a messy file system with lots of noise and little logic.

The experiment to move the Palmer site into WordPress was a first attempt at rebuilding the file structure. Because we have abandoned that attempt, we are simply rebuilding the directory structure from the ground-up, creating a much cleaner file system and navigation scheme.

What about the CCE site?

The CCE site is being moved out from WordPress as well. Although it worked a lot better with WordPress 1.5 than Palmer did with version 2.0, it is in our best interests to keep these 2 LTER sites as related to each other as we can, for ease of development and maintenance. Additionally, the CCE site also performs faster outside of WordPress.

When will these sites be ready?

The CCE site is basically done. Mason may still be working on fine-tuning some specific applications (IForum, etc.). You can compare the 3 sites using the links below. CCE new is the newest revision of the site (using the file system). Compare it the WordPress site. It’s the same theme (with a revamped nav-bar).

CCE site: http://cce.lternet.edu
CCE wordpress: http://ccelter-dev.ucsd.edu/wordpress
CCE new: http://ccelter-dev.ucsd.edu/new

The Palmer site, being much larger, will need more time. Hopefully by the end of this month, but depending on what other projects come up, it could be even later. I spent the last two days alone migrating the entire Datacat app from the Palmer dev site to the Palmer new site.

Pal site: http://pal.lternet.edu
Pal dev: http://pallter-dev.ucsd.edu
Pal wordpress: http://pallter-dev.ucsd.edu/wordpress
Pal new: http://pallter-dev.ucsd.edu/new

Future plans I’d like to implement is to throw these sites into Subversion, using a ‘2-way door’ method. The dev area and the production area would contain a checkout of the repository. Anytime someone makes an update (in either the dev or the production area), they should simply commit that change to the repository. Likewise, to update either area, you simply update from the repository. Nothing else fancy here… no other users or working-spaces, and definitely no trunks or branches! I feel this may be the simplest way to incorporate subversion into our workflow.

We may also want to merge (normalize) some of the code shared betweem the Palmer and CCE files, notably the templating files, and a common library of php functions. Both files are using the exact same template, differed only by the stylesheet. Any differences like these (stylesheet, site title, etc.) can be stored in the site-specific config file. Using a shared library helps ensure the sites are in sync with each other, mainly from the developer’s point-of-view. Mason and I have already noted that slight additions we make to the Palmer code-base are not in CCE, and vice versa. While this is typically a minor (if yet insignificant) impact, it requires extra manual labor to ensure that both sites are kept up-to-date with the latest additions of new functions, etc.

I don’t buy many peripherals (no pda or mp3 players) - a much used small camera and mini voice recorder complete my digital toolkit along with my mac powerbook. I purchased recently a Mac iPod for reasons I’m still trying to figure out:

-it had a 60GB disk meaning it could now serve as the archive for an ethnographic body of work accumulated in 2002
-it was demoed in use for me at the LTER CI meeting without any reference to music functionality which previously seemed to have backgrounded other functions important for me such as calandar sync, remote mount availability on borrowed laptop, and disk archive options.
-it’s ability to make available in color selected photo archives
-the availablility of video viewing
-the value for gaining experience with alternative portable modularized functions

Some items noticed in using the iPod
-no zoom capability with photos (common on most cameras) is a severe viewing limitation
-the Mac iPod video format is ‘ipod mp4’ so a video converter application is required to transform from standard formats (quicktime (.mov), realplyer (mpg4), and windows media player (.wmv)) to the iPod mp4. Although a number of free converters are available online, to minimize number of applications on my Mac, I purchased for $30 the upgrade of QuickTime to QuickTime Pro with its built-in converter. Interesting sidenote, I was not able to purchase the QTPro key unlock at the UCSD bookstore (last copy they received and sold was more than a year ago) nor from local Apple stores.My only option was to purchase it online from the MAC store - which worked flawlessly. I have tried to keep my own online transaction sprawl to a minimum - one company - but now I’m on record with two companies (amazon and apple).

So far I have used the iPod to ask others a question from a photo, to share a photo collection, and to demo a short video. To be continued …

We are planning to establish a mechanism that provides a central location for participants to exchange data files - and thus meet the particular need of establishing a procedure to submit data files into a community database. In the past we have done this by providing a group of users individual logins and passwords as well as group privileges to log-on to ftp to the central computer. This works well for those who are somewhat familiar with unix but introduces barriers for those unfamiliar with unix commands, shared spaces, and the concept of group file privileges.

Four alternatives to exchanging files include: 1) ftp, 2) subversion, 3) remote mount, and 4) WebDAV. The scope of the project pervasive availability would be: a) involves fairly small number of people (PAL 10; CCE 25); b) involves some remote geographic locations such as New York or Oregon; and c) includes desire to ensure some security through defined groups/accounts.

First thought is that subversion is not highly transparent for most users. We haven’t yet discussed the security arrangements for group use of ftp. We have, however, successfully used the remote mount option as a file exchange mechanism in some of our early iOcean supported collaborative work, ie with PAL and CCE projects. This approach makes use of three protocols with a fair amount of network overhead (afp for mac; smb for pc; nfs for unix).

Webdav is an approach JRW has been investigating through the Apple’s .mac endeavor with iDisk (remote mounts). WebDAV is the open source underlying environment that supports desktop disk mounts. In addition, it uses a single protocol (http) across all platforms and can likely be part of existing authentication schemes (she says hopefully). Webdav does not allow for versioning, but to me this seems a minor issue as I’m comfortable with engaging the user in naming their own files rather through an automated mechanism. This technology is being used by macworld’s .mac file sharing service which makes it a bit of a mystery as to why we haven’t heard about more groups locally or in our communities making use of it.

iDisk creates an icon on your desktop, allowing you to drag and drop files just as with any other type of hard drive. Xdrive and MySpace have similar features but both these technologies are available only to PC.

From a community point of view, the WebDAV (http://www.webdav.org) defines itself as “WebDAV stands for “Web-based Distributed Authoring and Versioning”. It is a set of extensions to the HTTP protocol which allows users to collaboratively edit and manage files on remote web servers.” Dav is more general than an API, it is a protocol, an extention of the http protocol. Subversion uses the WebDAV protocol and has a WebDAV interface.

Articles:
http://ftp.ics.uci.edu/pub/ietf/webdav/intro/webdav_intro.pdf
http://www.cs.ucsc.edu/~ejw/papers/kim_oss04.pdf

Last night I upgraded this blog to WordPress 2.0, which was released at the end of last year. Though it contains some nice improvements, particularly in the user/admin interface, the performance was less than spectacular. The pages took much longer to load; I suspected that some of the plugins may be at fault. Sure enough, upon disabling all plugins, the site ran faster. However, since we rely on some of the plugin functionality, I decided to downgrade the site back to version 1.5.2.

It may be best to wait a few months before attempting to upgrade again. This will allow time for other developers to create plugins that work better with WordPress 2.0.

The main plugin at fault was “Extended Live Archives”, which provides the cool AJAX viewer for browsing through posts on the Archives page.

A question now looms whether we should upgrade the CCE site to WordPress 2.0. I figure that since it’s primary use is as a CMS and not a blog, it uses less plugins and thus performance may not be hindered. Additionally, I have been porting the Palmer site into WordPress 2.0, and the site seems to run OK… no worse than the CCE site runs on WordPress 1.5.2.

Read this article more about what’s new in WordPress 2.0.

A few notes from my experiences already:

• The admin interface is more colorful and uses AJAX in some places for ease of usability, most notably when Deleting an item, and also receiving a Confirmation Flash Notice at the top of the page after having successfully completed some action.
• It is much easier to upload images and files to a post. The Write Post page contains an embedded file uploaded/browser for inserting images and files directly into your posts. All uploaded files (by default) are stored and organized in /wp-content/uploads/(year)/(month)/ where year and month reflect the current date of the upload.
• In FIREFOX and IE (I assume), there is a Rich Text Editor available when writing posts. This is particularly useful for those with little or no html knowledge, yet are comfortable with a WYSIWYG (What you see is what you get… e.g. Microsoft Word). editing environment. SAFARI on the Mac still lacks some Javascript functionality, and thus does not have the Rich Text Editor available. This however is not all that bad…
• …Using the Rich Text Editor screws up Relative Links. The Rich Text Editor detects relative links (if you edit directly in html mode) and will convert them to absolute links… incorrectly (it converts the link using the wrong query path). Safari is advantageous because it allows you to save posts with relative links intact. This is extremely noteworthy, particularly for the CCE and Palmer sites which have a lot of static pages that may link to each other. If we ever move a site to a new location or domain, relative links will keep working while absolute links will break….
• Thus, as I have been working on porting the Palmer site, I’ve been using a hybrid approach. I use Firefox to do the bulk of the admin work since it contains the Rich Text Editor. However, I use Safari whenever I need to create a relative link as part of a Page’s content. It’s kind of a pain, but it gives the best of both worlds. Hopefully the guys at WordPress will fix this Rich Text Editor (bug, feature, etc… which is it?) on a later version.

I hope this wasn’t too confusing.

Overall I think WordPress 2.0 is a nice improvement and is fairly intuitive, but it definitely has its kinks. Knowing and discovering the tricks like those above will help. Since the OI site is primarily used as a blog and relies on some cool plugins, I thnk keeping it at version 1.5.2 is okay for now. However, the Palmer site runs fine on WordPress 2.0, so there is no reason to revert it back. The CCE site should probably be upgraded to 2.0 as well.

FYI, the link to the CCE and Palmer WordPress development sites:
http://ccelter-dev.ucsd.edu/wordpress
http://pallter-dev.ucsd.edu/wordpress

Quick note: All the sites are using the same theme, each one tweaked differently. I added a dark-blueish background to this site to help the pages stand out better.

Ever since I’ve opened commenting to the public (anonymous users) last November, we’ve been getting some spam comments at random posts. Lately, the number of spam comments has been increasing. Fortunately, pretty much all have them had been placed in the Moderation Queue because they contained at least one common comment spam word. I’ve been getting emails each morning for the last couple weeks telling me there are comments awaiting moderation. (WordPress provides a nice interface for moderating comments and marking them as spam).

To prevent the noise in my email and in WordPress, I have decided to once again close public commenting, so that only registered users can post comments. Note that anyone can still register if they want to.

If our blog starts seeing higher usage again and attracting more anonymous users, then we can re-open public commenting. However, since this is mainly an internal blog, used primarily by our small group, there is little need to open our doors when nobody is appearing, and at the expense of combating spam every day.