Lyle's Perl Blog

Finishing off the spring cleaning

2010-06-10T03:00:23Z

Before I start off on some new Perl posts, I thought I'd write about some of the useful things I came across when doing a spring clean of my disks. I decided it was about time to clean out all my old disks, backups, folders, etc, into a nice ordered filing system. I ended up with 2 root folders, one for work and another for personal stuff. At the same time I upgraded my PCs and laptop to be quad core and use Windows 7.

Firstly a warning, there are some very well made counterfeit copies of Windows 7, hollograms and all. Unless you have a genuine copy of compare them side by side, it's easy to buy a fake. Unfortunately I did :( Here is my post to Microsoft about it.

Some of my personal files I wanted to keep private, I searched for open source options and ended up using AESCrypt. There was another interesting one that created an encrypted drive, but for now I'm happy to put files in a zip and encrypt that file.
]]> CheckPlaces by Andy Halford. It quickly checks your bookmarks for errors, 404's, etc.

I develop on Windows/IIS then test on Linux/Apache. I've always found doing it this way round leads to less problems. I was able to export/import my IIS sites with simple commands.
Although this left me to re-create the application pools. Oddly it included all the handler mappings, but did not populate the ISAPI and CGI Restrictions accordingly.

IIS 7 on Windows 7 doesn't include an SMTP server, I settled on hMailServer I'd used it before and had no problems.

Another issue I've had is that my web designer always saves his locally and not to the server (despite repeated instructions to do so, yes Phil, I'm talking about you! :P ). When he does save things to the server they often get out of sync with the local copies he works on. This leads to issues as you can probably imagine.

The solution I chose is SyncToy. Now he can save locally, and the files will automatically sync with the server at the end of the day (that is if he manages to save them in the right place, lol). This also acts as a useful backup.

I've switched to AceFTP freeware so we can easily use synctoy to share the .ftp files with FTP details for all the sites we work on

Lyle

First Perl 6 experiences part 2

2009-04-09T13:04:45Z

First Perl 6 experiences part 2

When I first looked at this Sunday night I promised myself I'd spend no more than an hour or two on it. Well that went right out of the window :/

My initial aim was to get it working on Vista and run as CGI on IIS. This didn't work. I've spent the last 3 days playing about with it and testing out a load of things to find out why. Eventually with the help of the good people on IRC #perl6 and #parrot we found the problem and came up with a fix. Here are some of the details...

Following on from my last post, the people of the perl6-compiler@perl.org mailing list pointed me to a Rakudo build fix so I was able to build perl.exe by running mingw32-make. This has now been patched into Rakudo on github so if you are following this you shouldn't have the issue I had.

Playing with Rakudo on the command prompt I made my first hello world script 'saved as hello.p6':-

#!c:/temp/rakudo/perl6.exe
say( 'hello' );

And ran it with 'perl6 hello.p6'.

Following that I prepared a version for CGI that included the content-type header:-

#!c:/temp/rakudo/perl6.exe
say( "Content-Type: text/html\r\n\r" ); # say adds the last \n
say( 'hello' );

I checked this on the command prompt to make sure it was outputting properly and it was. Now to setup with IIS.
]]> Setting up Rakudo with IIS

Open up IIS7 services manager. Then select the website from the left panel (for my it's under COMPNAME->Sites->Default Website). In the center panel there is a link for 'handler mappings', double click.
Top right click on 'Add script map' and fill in:-
Request Path: *.p6
Executable: C:\temp\rakudo\perl6.exe "%s" %s
Name: Rakudo

When I clicked OK a message popped up asking "Do you want to allow this ISAPI extention?" I clicked Yes.

Now we *should* be able to run Rakudo cgi scripts through IIS. But copying my test script to c:\inetput\wwwroot\cgi-bin and trying to run through the browser localhost/cgi-bin/hello.p6 gave me a 502 error. At this point I tried a lot of different scripts such as:-

#!c:/temp/rakudo/perl6.exe
$*ERR = open("C:/inetpub/wwwroot/cgi-bin/err.txt", :w);
my $crlf = "\x[0D]\x[0A]";
$*OUT.say( "Content-Type: text/html$crlf$crlf" );
$*OUT.say( 'hello it works' );
$*ERR.say( 'got to end' );

Which still gave me the error, but when I check err.txt it was having 'got to end' written to it, so the script was actually running... To convince myself I wasn't going mad I installed Apache and tested through that as well:-

Setting up Rakudo with Apache

I know they are working on a mod_perl6, but right now I just want it to work through cgi. I installed Apache 2.2 on port 8080 and updated my httpd.conf with the line:-

AddHandler cgi-script .cgi .pl .pl6 .p6

I also updated my htdocs and cgi-bin folders to be at c:\htdocs and c:\htdocs\cgi-bin. I restarted Apache, put my hello.p6 script in c:\htdocs\cgi-bin and ran it through the browser http://localhost:8080/cgi-bin/hello.p6. Bingo! That worked as expected. So at least I'm not going crazy, the hello.p6 test script is working, so why isn't IIS displaying it's output?

Installing November

At this point I was thinking maybe it was worth trying a more detailed cgi script to see what happened. I grabbed a copy of november from github. It didn't look like anyone had tried installing november on Vista, so I joined the november-wiki@googlegroups.com mailling list and kept them updated as to my progress. I ended up submitting 4 patches and 1 new script to get it running on Vista command prompt. But still IIS and Apache weren't working :(

IRC #perl6

I was going round in circles and not getting any further. I contacted the ActiveState mailing list thinking Jan might be able to shed some light, but unfortunately he was away for the week. Seemed a cautious pestering of the people on the #perl6 IRC channel was in order.

I said hi and spoke to PerlJam, Moritz_, jnthn and Masak. I felt a bit more compfortable when Moritz_ said "Lyle: ah, that was you... does Rakudo build on vista without modifications now?". TimToady was on there as well, but I'll have to build up a bit of courage before I speak to him...

I told them the things I'd tried and they suggested trying some more. Nothing worked. One error that kept coming out was "src\io\api.c:233: failed assertion 'pmc'". jnthn suggested writing a parrot PIR hello world cgi script and seeing if that failed as well. Then we'd know if it was a Rakudo or Parrot issue. (looking back at the IRC logs jnthn had an incling it was something to do with IO and STDIN which proved almost spot on in the end).

I managed to get november working on Apache when I finally realized it wasn't picking up the libraries. Still no further with IIS. Masak also suggested writing a PIR hello world script...

Parrot PIR hello world script

I searched online and found a deceptively simple script (saved as hi.pir):-

.sub main :main
    print "Hello world!\n"
.end

I updated it for CGI:-

#!c:/temp/rakudo/parrot/parrot.exe
.sub main :main
    print "Content-Type: text/html\r\n\r\n"
    print "Hello world!\n"
.end

Setting up Apache and IIS for parrot in exactly the same way I did for Rakudo except for .pir files and the c:/temp/rakudo/parrot/parrot.exe executable. I found similarly this worked for Apache and not IIS. This told me it was definitly a parrot issue. Time for more help...

IRC #parrot

I joined this channel and updated them as to the problem and the things I'd tried. They suggested trying a few more things. I ended up with a PIR script that tried to write out to a file but kept failing with "src\io\api.c:455: failed assertion 'pmc'". I tried getting IIS to run parrot with trace enabled -t1 but that just changed the error slightly "src\io\api.c:603: failed assertion 'pmc'". I also tried a few hacky things to try and get the parrot output but they didn't work. Infinoid offered his assistance, and guided me through using MinGW's gdb program to attach to the running parrot process and add a breakpoint. This was all well above my head by Infinoid was very patient and told me exactly what i needed to do.

The basic process was:-

Update hi.pir with a sleep 30 command near the top
Run through IIS
As it hangs use Task manager to grab the PID
Run gdb parrot.exe
Type "break Parrot_io_putps" from within gdb
Type "cont" until we got to a break point that had the null value we were looking for
Then type "bt" to dump some info about it

As soon as this was done Infinoid saw what the problem was in win32.c. Parrot was trying to open STDIN and failing. When IIS invokes a CGI script it doesn't allow STDIN unless there is POST data. Infinoid wrote a patch and I even got to edit a little bit of C code myself :)

Success!

After much torment parrot worked on IIS through cgi. I was able to rebuild Rakudo (had to do a 'mingw32-make clean', then 'mingw32-make') and get Perl 6 working as well. Much happiness :)

I couldn't help but feel like I'd made a different (albeit very small). The problem in parrot meant none of the parrot based languages were working on IIS through CGI, not just Perl 6. Now it's all sorted. Thanks again to everyone.

Lyle

First Perl 6 experiences

2009-04-05T23:38:47Z

First Perl 6 experiences

Ok. I've been telling myself I can look at Perl 6 after I've finished my big project... But I just can't wait. I'm going to have a go at installing it on my Vista 32 laptop, and getting it working through CGI with IIS7.

Getting Rakudo

The Rakudo site has an easy looking guide.

I'm still very new to git, so rather than faf about trying to remember how to use it, I'm just going to download the tarball from:-
http://github.com/rakudo/rakudo/tree/master

Unzipped it to c:\temp and renamed the long rakudo-rakudo-xxxxxxxxx folder to just 'rakudo'.

cd c:\temp\rakudo
perl Configure.pl --gen-parrot

Error:
Generating Parrot ...
C:\Perl\bin\perl.exe build/gen_parrot.pl

The system cannot find the path specified.
Checking out Parrot r37869 via svn...
'svn' is not recognized as an internal or external command,
operable program or batch file.
]]> Perl 6 guide last year that gave instructions on how to get subversion and svk setup.

After skipping through, I downloaded:-
svn-win32-1.6.0.zip
svn-win32-1.6.0_dev.zip
svn-win32-1.6.0_pl.zip
from:-
http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=8100

Then extracted them to c:\temp\svn. I don't think I need to set this up with Apache to get this working. So for now I'll just update my path to point to the svn binary folder:-

set PATH=%PATH%;c:\temp\svn\svn-win32-1.6.0\bin

Now retry the Rakudo build:-

cd c:\temp\rakudo
perl Configure.pl --gen-parrot

Ok. Got a lot further this time but still failed. Useful parts of the output:-

Since you're running this program, you obviously have Perl 5--I'll be pulling
some defaults from its configuration.

...blah...

inter::progs - Determine what C compiler and linker to use...Compilation failed with 'cl'

So it's getting default from Perl 5. As I'm running ActivePerl which comes pre-compiled and without a compiler, this is the problem. ActivePerl is built with MSVC++. I can see 3 options:-

1) Installing MSVC++
2) Install Strawberry Perl that comes with a compiler
3) Install a separate compiler and get the parrot install to use this instead of MSVC++

Hmm. I do have a copy of MSVC++, but it takes ages to install and I don't fancy waiting for it. Installing Strawberry Perl would be easy, but it might effect my current ActivePerl and scripts I run on this system. I'll try option 3.

Googling ActivePerl MinGW seems that these two work well together and have done since 2005. I'll try just installing MinGW and see if that's enough.

I downloaded the Automated MinGW Installer and ran it selecting a full install to c:\MinGW.

Add this to the path and retry the Rakudo install:-

set PATH=%PATH%;c:\MinGW\bin

cd c:\temp\rakudo
perl Configure.pl --gen-parrot

That seems to have been enough. Parrot it now building.

Righty then. Looks like it all went ok. Finishing with the message:-

You can now use 'mingw32-make' to build Rakudo Perl.
After that, you can use 'mingw32-make test' to run some local tests,
or 'mingw32-make spectest' to check out (via svn) a copy of the Perl 6
official test suite and run its tests.

But running mingw32-make is giving me the error:-

makefile:332: *** commands commence before first target. Stop.

Checking the makefile, line 332 is:-
perlcritic -1 --profile tools\util\perlcritic.conf $(CRITIC_FILES)

If I type perlcritic in the command shell then I get a command not recognised error. So I'm guessing perlcritic is a requirement that you need to install before trying to build Rakudo.

ppm install Perl-Critic

Hmmm. Still getting the same make error, even with a working perlcritic command...

I'm no Makefile expert, I can't see what's wrong. I've subscribed to the perl6-compiler@perl.org mailing list to see if they can help :/

Update: The guys on the mailing list got back to me with a solution the same night :) They've also updated the master tree with the fix, so anyone reading this should find it works first try.
I'll do another blog post when I've had a bit more of a play...

Lyle

Perl patching experiences part 2

2009-03-02T03:40:05Z

Perl patching experiences part 2

Success with patching Email::Stuff! I saw that it was a part of the Perl Email Project. I got in touch with them about the Email::Stuff bug on their mailing list then followed up on IRC (irc.perl.org #email).
Ricardo SIGNES was good enough to help me investigate if this was an issue with Email::Stuff or one of the underlying Email::modules. He confirmed this was a stuff issue, and also pointed out a couple of other bugs.
I offered to supply patches to fix all these bugs. RJBS pulled Email::Stuff into github and gave me a fork to work on (which for me also meant getting a github account and learning the basics of git). A few hours later I had a fully working fork fixing the header loss issue, unused clone issue, and also multipart/alternative issue. I went further to write unit tests to check that these were working properly and ensure they continued to do so in the future. I also updated the docs a little, although I should probably have documented the new conforming functionality a bit more.
With the minor patch I submitted to Params::Util getting applied but giving me no mention in the changelog. This time I was very chuffed to see my name appear in the credits :)

Patched Email::Stuff 2.07 now on cpan ;)

Clone issue fixed, RT BUG fixed and other problems Ricardo found all fixed.

Lyle

Perl patching experiences part 1

2009-02-25T01:28:06Z

Perl patching experiences part 1

Recently I looked at submitting code fixes and posted a quick guide. Since that post I managed to find and fix a bug in Params::Util, submitting the appropriate patch. Adam Kennedy (the maintainer of the module) took it further to write the appropriate unit tests to make sure the problem never came about again.

This has made me realize that patches can be more than just code and typo fixes. An ideal patch would include code fix, unit tests for the problem and possibly documentation updates. It could even be code replacement, or alternative versions.
]]>
My latest plugin for CGI::Application is an email module which currently isn't much more than a wrapper around Email::Stuff. When choosing it 2 things put me off. One was the promblem with Pure Perl Params::Util which is now fixed. The other was that is uses Clone which is an XS only module. Upon further inspection, Clone is loaded, but ever actually used. I guess this was a feature Adam was planning but didn't implement. I asked him about it and he said it wouldn't be an issue to remove Clone.

It's been a couple of weeks and the Email::Stuff on CPAN still declares Clone as a dependency. So my CGI::Application::Plugin::Email module currently looks like it doesn't have a Pure Perl option (which is the aim for all of my CGI::Application modules).

I know Adam runs a lot of projects and is very busy. Given his very fast responce to fixing Params::Util when I submitted a patch. My current thinking is that I can give him a worthwhile update to Email::Stuff (more than just a removal of "use Clone;") then it'll be worth updating the CPAN version.

Taking a look at the RT, there is a bug that has been in there for over a year. I'm hoping if I fix this then I'll be able to also fit in the Clone removal.

Results to follow ;)

Lyle

Writing a CPAN Task

2009-02-08T21:30:59Z

Writing a CPAN Task

Last week I was looking into CPAN Bundles and created my first Bundle for the PerlCertifiedHosting project. As a follow up this week I'll create a Task, which is the successor to Bundle.

After listing a load of CGI::Application modules in my Bundle::CertHost it seems it would make a lot of sense if CGI::Application had it's own Bundle or Task that I could just link to.

http://search.cpan.org/dist/Task/lib/Task.pm
]]>
First Draft

### In Makefile.PL
use inc::Module::Install;

name         'Task-CGI-Application';
abstract     'Install the most common CGI::Application modules';
author       'Lyle Hopkins ';
version_from 'lib/Task/CGI/Application.pm';
license      'perl';

# All the things we need for CGI::Application
requires     'CGI::Simple' => 0;
requires     'FCGI' => 0;
requires     'CGI::Fast' => 0;
requires     'CGI::Application' => 0;
requires     'CGI::Application::Plugin::ActionDispatch' => 0;
...lots more...
requires     'Titanium' => 0;

WriteAll;

### In lib/Task/CGI/Application.pm
package Task::CGI::Application;

use strict;
use vars qw( $VERSION );
BEGIN {
    $VERSION = '0.01';
}#BEGIN

1;

That was pretty easy. This is the first time I've used Module::Install so I had to grab that module first "ppm install Module::Install". Also I had to update the MANIFEST to include the Module::Install files that are put in the inc/ folder.

### In MANIFEST
Changes
Makefile.PL
MANIFEST
README
t/Task-CGI-Application.t
lib/Task/CGI/Application.pm
inc/Module/Install.pm
inc/Module/Install/Base.pm
inc/Module/Install/Can.pm
inc/Module/Install/Fetch.pm
inc/Module/Install/Makefile.pm
inc/Module/Install/Metadata.pm
inc/Module/Install/Win32.pm
inc/Module/Install/WriteAll.pm

nmake dist
perl \UnixUtils\tarfixer.pl -i.bak Task-CGI-Application-0.01.tar.gz

Ready to go up to CPAN. That was easy :)

Lyle

Writing a CPAN Bundle

2009-01-31T04:15:49Z

Writing a CPAN Bundle

I decided to take an hour out from my big project today to make my first CPAN Bundle for the PerlCertifiedHosting Project. Last year I posted to B&BPM asking how this was done as when I looked at bundles on CPAN I couldn't see anything special. David Cantrell was good enough to respond, pointing me to:-
http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Bundles
Which explains it. I was looking for some special code or something, when after all it's just a list of modules in the contents section.

First draft:-
]]>
$VERSION = "0.01";

1;

__END__

=head1 NAME

Bundle::CertHost - A bundle to install PerlCertifiedHosting.com module requirements

=head1 SYNOPSIS

Linux:-
perl -MCPAN -e 'install Bundle::CertHost'

Windows:-
ppm install Bundle-CertHost

=head1 CONTENTS

Bundle::DBI - Start DB section

DBD::mysql

... to many to list in the blog post

=head1 DESCRIPTION

This bundle provides the CPAN module requirements for PerlCertifiedHosting.com.

=head1 AUTHOR

Lyle Hopkins Ewebmaster@cosmicperl.com>

=cut

The list is already very long. While wondering how to make this bundle easier to maintain I came across Bundle::Math, which is a Bundle that only lists other Bundles in it's namespace. This seems an excellent way of breaking up the modules into groups. Although I'll save this for next time.

Next steps will be making a TASK:: version and splitting up Bundle::PerlCert into groups. Also I'll need to make a script (or collection of scripts) to automate this process.

Lyle

How to submit Perl patches?

2009-01-24T19:36:33Z

How to submit Perl patches?

It's about time I learnt to submit proper patches. In the past I've just emailed module authors spelling mistakes and code fixes. This isn't good for the authors as then they have to scan through and manually edit the modules themselves. Thus making it more time consuming for them, and less likely for them to apply your fix. The proper way to do with is by supplying a diff which can be patched into the module. I've been sending lots of typo fixes to Mark Stosberg recently as I've been reading up on CGI::Application. I've promised him the next ones I send will be proper patches. Although as it turns out, my first patch wont be for CGI::App typos.

This post is a continuation of my last post, which you'll need to follow for the example to work.

Let's go through the process for the bug I found in the CGI::Kwiki test suite when ran under Windows.

First let look at the current test file (C:\temp\CGI-Kwiki-0.18\t\test.t):-

]]> use lib 'inc';
use Test::More;
use Cwd;
use File::Path;

plan(tests => 9);

my $cwd = cwd;
rmtree('t/kwiki');
ok(mkdir 't/kwiki', 0777);
ok(chdir 't/kwiki');
ok(system("PERL5LIB=../../blib/lib;../../blib/script/kwiki-install") == 0);
ok(-f 'config.yaml');
ok(-d 'database');
ok(-f 'database/HomePage');
ok(-f 'database/KwikiFormattingRules');
ok(-f 'database/KwikiHelpIndex');
ok(chdir $cwd);

I can see the problem starts on line 12 with the call to system.
ok(system("PERL5LIB=../../blib/lib;../../blib/script/kwiki-install") == 0);

On Windows it's done a little differently. First I'll make a backup of the current test.t file called test.t.bak.
Now I'm changing that line to:-
if ( ( $^O eq 'MSWin32' ) || defined( $ENV{'OS'} ) ) {
    ok(system("SET PERL5LIB=../../blib/lib") == 0 && system('..\..\blib\script\kwiki-install') == 0);
}#if
else {
    ok(system("PERL5LIB=../../blib/lib;../../blib/script/kwiki-install") == 0);
}#else

On Linux that single call to system has set the PERL5LIB environment variable and then executed the kwiki-install script. For windows I've had to break it up into 2 system calls, use SET for the environment variable and fixed the Windows file path \'s, hence ' and not " otherwise I'd have to use \\. Internally Perl with convert / to \ when on Windows, but when you are talking to the OS directing with system you need to use \.

I've ran "nmake test" to see that this is working and it is. Brill :)

So now I have the new working test.t file and the old test.t.bak file.

Time for diff

Luckily diff and patch come as a part of UnxUtils that I described how to download and setup in my previous post.

First I need to change the file names, as I don't want my patch to be for test.t.bak, but for the original test.t. So I've renamed test.t (that has my working code) to test.new and test.t.bak back to test.t. (I had to name my new file test.t earlier in order for "nmake test" to work).

Next I open a command prompt and navigate to the C:\temp\CGI-Kwiki-0.18\t folder.
Now I run the diff program with the -u option for "Unified format", this is the diff format that most patch programs want.

diff -u test.t test.t.new

It gives me the output:-

--- test.t   Sat Jan 24 16:45:54 2009
+++ test.t.new   Sat Jan 24 17:06:14 2009
@@ -9,7 +9,12 @@
rmtree('t/kwiki');
ok(mkdir 't/kwiki', 0777);
ok(chdir 't/kwiki');
-ok(system("PERL5LIB=../../blib/lib;../../blib/script/kwiki-install") == 0);
+if ( ( $^O eq 'MSWin32' ) || defined( $ENV{'OS'} ) ) {
+    ok(system("SET PERL5LIB=../../blib/lib") == 0 && system('..\..\blib\script\kwiki-install') == 0);
+}#if
+else {
+    ok(system("PERL5LIB=../../blib/lib;../../blib/script/kwiki-install") == 0);
+}#else
ok(-f 'config.yaml');
ok(-d 'database');
ok(-f 'database/HomePage');

You can see the line numbers and the - before the line that is removed, and the + before the lines that need to be added. Before and after there are 3 lines of matching text. These are used so the patch program can still apply the diff if the line numbers don't match up.

The output looks good to me, so I'm going to run the command again and save the output to a file called test.patch:-

diff -u test.t test.t.ew > test.patch

Almost there. Now that I have my diff that explains how to apply my patch to test.t I need to test that it actually does it's job properly.

Time for patch

Patch is the program that'll apply the diff. I'm going to use it now to check that my patch file works properly. I'll be using the -b option so that the original file is backed up before it's patched.

patch -b test.t test.patch

Seems to have worked. The backup file is named test.t.orig, checking test.t I can see the patch has been applied correctly. If I wanted to remove the patch I can by running the command above with the -R option instead of -b. Or of course I could just restore the test.t.orig file.

Right, final test.

cd \temp\CGI-Kwiki-0.18
nmake test

Brilliant! So I've found a bug, fixed it, created the patch, and tested the patch out. Now I just need to get the patch to the module author and hope he applies it...

DOH! Just check the RT (Request Tracker) for Kwiki, it appears someone posted this as a critical bug 4 years ago! Yikes!:-
http://rt.cpan.org/Public/Dist/Display.html?Name=CGI-Kwiki
http://rt.cpan.org/Public/Bug/Display.html?id=2550

Aghh well. They didn't seem to provide a patch then, maybe if I submit a new bug with my patch it'll get used.

Lyle

Building perl modules from source on windows

2009-01-24T06:25:19Z

Building perl modules from source on windows

This guide is to cover building and installing Pure Perl modules on Windows for use with ActivePerl. It doesn't cover XS Perl modules that use C libraries. I'll leave that for another post :)

You'll want UnxUtils from sourceforge and nmake from Microsoft. If you haven't already installed these, follow these steps:-
]]> UnxUtils.zip

2. Unzip to a temp folder, for some reason the folder you want is in temp\usr\local\wbin. Move this wbin folder into your c:\ root to make c:\wbin, then rename to something sensible like c:\UnixUtils. (yes I added the 'i').

3. Update your path environment variable to include this folder on the end. In Vista32 in go Start->(Right click on)Computer->Properties->Advanced System Settings, then click the Advanced tab at the top of the window, then the Environment Variables button near the bottom. Finally under 'System Variables' scroll to find Path, select then click the edit button. Scroll to the end of the list and add ";c:\UnixUtils". Click Ok and Ok again. You may need to reboot for this to take full effect.

4. Now you need to grab nmake from Microsoft.
(this link is subject to change, you may need to search their site). Move the downloaded nmake15.exe file to c:\windows, then run it. It'll extract the nmake.exe executable.

Now we've got everything we need to build and install the Perl module sources.

From my last post I'll use CGI::Kwiki as an example.

Grab the CGI::Kwiki sources from CPAN, extract them to a temp folder, such as c:\temp. (If you find you can't extract this, get hold of WinRAR)

Open a command prompt, navigate to the temp Kwiki folder you extracted eariler ("cd \temp").
Now run Makefile.PL to create the makefile ("perl Makefile.PL").
Now run "nmake" to build it, then "nmake test" to run the modules test suite.

Hmm... Seems CGI::Kwiki 0.18 fails on some tests. That'll be why ActiveState don't have it as a ppm.
Looking closer at the tests it's failing because the test is trying to set a Linux style environment variable.

I think all that's needed is the test to be updated to check if the OS is windows, and set the environment variable Windows style.

I'm going to run "nmake install" to install this module as it does look like it's actually working, the problem is with the tests.

Although I would like to fix this problem so that Kwiki gets automatically built into a ppm by ActiveState. Which brings me onto my next blog post...

Lyle

Taking a look at Kwiki

2009-01-23T21:55:24Z

Taking a look at Kwiki

A few months ago I was looking at Perl Wikis. I had a terrible experience trying to get SocialText installed on CentOS and ended up using TWiki for 3 sites.

I was impressed by TWiki, but there are a few things that nag me about it. I don't like having to setup redirects from a would be index.html to /twiki/bin/view/Main/WebHome. My designer has been moaning at me that getting the designs he wants into it is hard. For smaller Wiki's, it does seem a bit like overkill. So what about the other end of the Perl Wiki scale?

I've recently been looking at redesigning a site that uses Kwiki. So let's take a closer look.

Kwiki - "A Quickie Wiki that's not too Tricky"

Well at the moment the Kwiki site seems to be down most the time. Not quite sure what's happening there.
Luckily you can get it on cpan:-
http://search.cpan.org/dist/CGI-Kwiki
(Make sure you get the old CGI::Kwiki from cpan, and not the new Kwiki distro. You should only download Kwiki from the Kwiki.org site)

I had a quick look on ppm and CGI::Kwiki wasn't there. Having a quick look at CPAN testers I got the feeling it was a problem with the tests rather than Kwiki itself. Which leads me onto my next 2 posts...

Lyle

Useful firefox search hack

2009-01-16T15:09:54Z

Useful Firefox search hack

I read of a really useful Firefox hack the other day which I've been
using an awful lot since. It's saved me a lot of time, but I can't for
the life of me remember where I read about it... I'm sure it was a book
review or something? :-\

Anyway, in Firefox do:-
Bookmarks->Organise Bookmarks
Then click on 'Bookmarks Menu', right click on the right pane and select
'New Bookmark'.

Call it 'CPAN search', put the location as
'http://search.cpan.org/search?query=%s&mode=all' and most importantly
set the keyword to 'cpan' (This is why we're adding through Organise
Bookmarks rather than just add bookmark).

Now you can easily search cpan. Just type 'cpan MODULE' in the address bar.

You can of course use this same hack for any number of sites that have
the search term in the query string.

Lyle

Making .po and .mo files

2009-01-16T14:37:06Z

Using CGI::Application::Plugin::I18N

Making .po and .mo files.

Now the the module is released to CPAN I've added a i18n guide on how to make .po and .mo files. Here is a cut down version for my blog:-

Creating .po and .mo files

The .po files are the editable language packs. The .mo files are compiled versions that are a bit faster to use.

I've found a good little app for creating .po and .mo files:- http://www.poedit.org It's cross platform and gives you a nice GUI to work from. Of course you can always edit the .po files direct and generate the .mo files yourself.

]]> Getting the sample files

CGI::Application::Plugin::I18N includes some sample files you can use to test out your .po and .mo files. For this blog post I'll try and make things a bit more general.

Using Poedit

Download and install Poedit. When you open for the first time it'll ask for your name and email, don't worry this is just for stamping the .po files with your details.

Click the File menu, then Preferences and select the Parsers tab. You'll see that there is a Perl parser, select this and then click the Edit button.

You'll see that the list of file extensions only has *.pl, extend this so that is contains *.pl;*.pm;*.cgi and any other extensions you use for your applications Perl files. Click OK and OK again.

Now we are ready to create our first catalog. Click File->New catalog. The details I entered:- Project name: Demo Team: po testers Team email: not@now.com Language: English Country: United States Charset: UTF-8 Source code charset: UTF-8

Then select the Paths tab, click the little square graphic for new item then input the path to your script files, I've done /tmp/potest

Then select the Keywords tab and add localtext the same way you added the path. If you aren't using my I18N module, then you'd input the name of the function or method that generals the local text, such as maketext, gettext, loc, etc.

Click OK to finish, a dialog will ask you where to save your .po file. I selected /tmp/potest/I18N/en-us.po.

At this point Poedit will search in the paths you selected for files that contain the keyword you input. For me it'll be searching for .pl, .cgi and .pm files and looking for localtext. It'll extract all the text it finds, and provide a list of strings that need translating. You should get a window titled Update summary appear with a list of all the strings. Click OK.
(If it doesn't find any you'll get an error)

If you change any of your files to include new localized strings then all you need do is click on the globe icon at the top and it'd parse the files again for new strings.

Enter in your translations. Now click the save icon. The .mo compiled version of this file is generated automatically on each save.

Lyle

Packaging CPAN modules from Windows in Linux Style

2009-01-08T02:24:25Z

Packaging CPAN modules from Windows in Linux Style

You're a CPAN author, but against the trend you develop on Windows rather than Linux.
This is fine, helps a lot with your modules cross platform compatibility, but when you come to package your modules for CPAN you hit a snag. Windows doesn't have tar or gzip, or even make... You can get nmake as a download from Microsoft, you could use .zip rather than .tar.gz, but this isn't going to make the Linux buffs very happy.

The solution is pretty simple, luckily there is a package called UnxUtils (yes no i in Unx) available on sourceforge. This has Win32 ports of gzip, tar and make. Although you'll still need nmake from Microsoft as you'll find make won't read windows paths properly.

Here is a quick step by step:-

]]> Module::Starter rather than h2xs as it gives you a better starting point.

2. Download UnxUtils.zip

3. Unzip to a temp folder, for some reason the folder you want is in temp\usr\local\wbin. Move this wbin folder into your c:\ root to make c:\wbin, then rename to something sensible like c:\UnixUtils. (yes I added the i).

4. Update your path environment variable to include this folder on the end. In Vista32 in go Start->(Right click on)Computer->Properties->Advanced System Settings, then click the Advanced tab at the top of the window, then the Environment Variables button near the bottom. Finally under 'System Variables' scroll to find Path, select then click the edit button. Scroll to the end of the list and add ";c:\UnixUtils". Click Ok and Ok again. You may need to reboot for this to take full effect.

5. Now you need to grab nmake from Microsoft.
(this link is subject to change, you may need to search their site). Move the downloaded nmake15.exe file to c:\windows, then run it. It'll extract the nmake.exe executable.

6. You are ready to build your distribution :) Open a command prompt and navigate to your modules folder that you created in step 1. Run:-
perl Makefile.pl
nmake test
nmake dist

7. It's important to note, that although at this point you'll have what seems to be a perfectly acceptable .tar.gz file for your module. If you upload this module to PAUSE you'll get an error because it contains world writeable files and folders. This is because tar perceives windows files permissions as being equivalent to open Linux file permissions.

When I first hit this snag I thought I was going to have to write a script to fix it, but luckily someone else has already done it :) Visit the PerlMonks node 731935 and save the code to c:\UnixUtils as tarfixer.pl (use the Download code link at the bottom, don't just copy and paste).

8. Now you can fix your distribution with perl c:\UnixUtils\tarfixer.pl -i.bak Module-Name.tar.gz
Hopefully bart will be good enough to upload this script to CPAN and create a nice tarfixer.bat file for us.

Voilà! A nice Linuxy .tar.gz Perl module built on Windows.

Lyle

Giving CGI::Application internationalization (I18N) part4

2009-01-07T00:28:08Z

Giving CGI::Application internationalization (I18N)

Part 4

After a bit of debugging I finally have the first release of the module ready. All the methods are documented, I've done some manual testing but I still need to bulk out all of the automated tests. I also need to writing documenation on how to create the .po and .mo files to use with this, but I've come across a very useful cross platform app called Poedit which makes it a lot easier.

I'm probably over eager to upload this to CPAN, as it's only version 0.01 but I'm keen to get it up.

With a bit of luck I'll have it uploaded tonight. It's also given me some good material for my next post "Packaging CPAN modules from Windows in Linux Style", as I'm having to do this and I've seen this come up in forums from time to time.

Module code below:-

]]>

=head1 NAME

CGI::Application::Plugin::I18N - I18N and L10N methods for CGI::App

=head1 SYNOPSIS

    use CGI::Application::Plugin::I18N;

Within your setup, cgiapp_init, cgiapp_prerun or specific runmode routine add
the line

    $self->i18n_config();

Or

    $self->i18n_config( %options );

%options are the same as for Locale::Maketext::Simple. If none are passed the
following default are used:-

    %DEFAULT_OPTIONS = (
        Path        => "$RealBin/I18N",
        Style        => 'gettext',
        Export        => '_maketext',
        Decode        => 1,
        Encoding    => 'locale',
    );

$RealBin being the folder from which the executed cgi script is running.
B properly!>

For instance if you wanted to use maketext style markup in your lexicons you
would use the line:-

    $self->i18n_config( Style => 'maketext' );

Then use the I method to localize text:-

    print $self->localtext( 'Hello World!' );

=head1 DESCRIPTION

This module is a wrapper around C by Audrey Tang.
It extends the C object with variour methods to control the
localization of text. A L is provided with the aim to fill in the gaps.

=head1 Methods

=head2 i18n_config

Runs the initial configuration of C and runs it's
import within your calling objects namespace (Your CGI::App class)

=head2 localtext_langs

Sets the current language for localtext output. Usage:-

    $self->localtext_langs( LIST );

LIST must consist of valid language tags as defined in RFC3066. See
C for more details.
If LIST is ommited then the method will attempt to figure out the users locale
using C.

This method will also return the list of language tags as an array reference.

    my $langtags = $self->localtext_langs( LIST );
    print @$langtags;

=head2 localtext_lang

This method returns the currently selected language. This is the tag that was
actually available for use, after searching through the localtext_langs list.
This is the name of the module used in your MyAPP::I18N::XXX namespace (where
XXX is the name of the lexicon used)

    my $lexicon = $self->localtext_lang;

=head2 localtext_lang_tag

This method returns the RFC3066 language tag for the currently selected
language. This differs from the above method which would most likely return
I for American English, whereas this method would return I.

    my $langtag = $self->localtext_lang_tag;

=head2 localtext

This is the method that actually does the work.

    print $self->localtext( 'Hello World!' );

=head1 FAQ

=head2 How does it all work?

I kept a blog on how I put this module together and all the material I looked
through in order to understand internationalization.
L

=head2 What is a Lexicon?

Think of it as a kind of hash. Where the text you use (usually english) has a
corrosponding value in the local language. So the 'Hello world' under a German
lexicon would have the value 'Hallo welt'.

=head1 Thanks to:-

C - The module this one was heavily based on

C - Making it possible

C - Doing all the hard work

C - Providing the framework

And all others I haven't yet mentioned.

=cut

use strict;
use warnings;
use Carp;

use FindBin qw($RealBin);

use I18N::LangTags ();
use I18N::LangTags::Detect;

require Locale::Maketext::Simple;

use vars qw ( $VERSION @ISA @EXPORT @EXPORT_OK $RealBin %DEFAULT_OPTIONS );

require Exporter;
@ISA = qw(Exporter);

@EXPORT = qw (
    i18n_config
    localtext_langs
    localtext_lang
    localtext_lang_tag
    localtext
);

@EXPORT_OK = qw(
    i18n_config
);

$VERSION = '0.01';

%DEFAULT_OPTIONS = (
    Path        => "$RealBin/I18N",
    Style       => 'gettext',
    Export      => '_maketext',
    Decode      => 1,
    Encoding    => 'locale',
);

sub i18n_config {

    my $self = shift;
    my $class = ref $self || $self;

    local %DEFAULT_OPTIONS = %DEFAULT_OPTIONS;
    if ( @_ ) {
        my %newoptions = @_;
        my @valid_options = ("Class","Style","Export","Subclass","Decode","Encoding","Path");
        foreach my $key (keys %newoptions) {
            unless (grep (/^$key$/, @valid_options)) {
                croak( "Invalid option: $key" );
            }#unless
        }#foreach
        %DEFAULT_OPTIONS = (%DEFAULT_OPTIONS, %newoptions);
    }#if

    my $evalcode = qq~
        package $class;
        Locale::Maketext::Simple->import( \%CGI\::Application\::Plugin\::I18N\::DEFAULT_OPTIONS );
    ~;
    eval $evalcode;

    if ( $@ ) {
        croak( qq~Couldn't initialize i18n, error "$@", code "$evalcode"~ );
    }#if

}#sub

sub localtext_langs {
    my $self = shift;
    my @langs = @_;
    if (@langs) {
        $self->{__I18N_LANGS} = \@langs;
    }#if
    else {
        ### Get CGI query object
        my $q = $self->query();
        $self->{__I18N_LANGS} = [
            I18N::LangTags::implicate_supers(
                I18N::LangTags::Detect->http_accept_langs(
                    $q->http('Accept-Language')
                )
            ),
            'i-default'
        ] unless $self->{__I18N_LANGS};
    }#else
    no strict 'refs';
    &{ ref($self) . '::_maketext_lang' }( @{ $self->{__I18N_LANGS} } );
    return $self->{__I18N_LANGS};
}#sub

sub localtext_lang {
    my $self = shift;
    my $class = ref $self || $self;

    my $lang = ref "$class\::I18N"->get_handle( @{ $self->localtext_langs } );
    $lang =~ s/.*:://;

    return $lang;
}#sub

sub localtext_lang_tag {
    my $self = shift;
    my $class = ref $self || $self;

    return "$class\::I18N"->get_handle( @{ $self->localtext_langs } )->language_tag;
}#sub

sub localtext {
    my $self = shift;
    $self->localtext_langs unless $self->{__I18N_LANGS};
    no strict 'refs';
    return &{ ref($self) . '::_maketext' }( $_[0], @{ $_[1] } ) if ( ref $_[1] eq 'ARRAY' );
    return &{ ref($self) . '::_maketext' }(@_);
}#sub

1;

Giving CGI::Application internationalization (I18N) part3

2008-12-30T23:43:40Z

Giving CGI::Application internationalization (I18N)

Part 3

Lets look at the remaining functions of Catalyst::Plugin::I18N and what they are doing.

sub languages {
    my ( $c, $languages ) = @_; ### Getting the catalyst object and languages to set to it.
    if ($languages) { $c->{languages} = $languages } ### Setting the objects languages property if languages passed in
    else {
        $c->{languages} ||= [
            I18N::LangTags::implicate_supers(
                I18N::LangTags::Detect->http_accept_langs(
                    $c->request->header('Accept-Language')
                )
            ),
            'i-default'
        ];
    } ### Otherwise detects users language tags using I18N::LangTags::Detect and make sure all superordinates are included using I18N::LangTags::implicate_supers (if the objects languages property is empty)
    no strict 'refs';
    &{ ref($c) . '::_loc_lang' }( @{ $c->{languages} } ); ### Call Locale::Maketext::Simple's loc_lang function from the calling objects namespace setting the languages
    return $c->{languages};
}
]]> sub language {
    my $c = shift;
    my $class = ref $c || $c; ### Getting the objects package name

    my $lang = ref "$class\::I18N"->get_handle( @{ $c->languages } ); ### Bipassing Locale::Maketext::Simple and calling Locale::Maketext's get_handle routine directly, returning a language handle object.
    $lang =~ s/.*:://; ### Which is then stripped to give you the name of the lexicon that is actually being used.

    return $lang;
}

sub language_tag {
    my $c = shift;
    my $class = ref $c || $c;

    return "$class\::I18N"->get_handle( @{ $c->languages } )->language_tag; ### Similar to above, but returning the language tag name as opposed to the
}

At first these two functions appeared to be needlessly creating a new language handle to get the required info. I thought Locale::Maketext::Simple must already have a language handle it's using, why not just access that. But when I looked deeper at the Locale::Maketext::Simple code, it seemed that handle wouldn't be easy to access at all, if even possible...

*loc = \&localize; ### Make the glob of loc a reference to the subroutine localize

sub localize {
    my $c = shift; ### Get catalyst object
    $c->languages; ### Sets the language on each call??? Seems a bit much...
    no strict 'refs';
    return &{ ref($c) . '::_loc' }( $_[0], @{ $_[1] } )
      if ( ref $_[1] eq 'ARRAY' ); ### Returns the output of the _loc routine (Locale::Maketext::Simple version of maketext), automatically dereferences array references passed as the second arguement
    return &{ ref($c) . '::_loc' }(@_); ### Returns the output of the _loc routine (Locale::Maketext::Simple version of maketext)
}

Here is the bit that actually displays the text. It links up the glob so that loc can be used as well as localize (that's why they've renamed Locale::Maketext::Simple's loc export to _loc).

Again none of these things are exported as they would need to be in CGI::App so I'm assuming Catalyst does this by default. Here are my routines:-

sub localtext_langs {
    my ($self, $langs) = @_;
    if ($langs){
        $self->{__I18N_LANGS} = $langs;
    }#if
    else {
        ### Get CGI query object
        my $q = $self->query();
        $self->{__I18N_LANGS} = [
            I18N::LangTags::implicate_supers(
                I18N::LangTags::Detect->http_accept_langs(
                    $q->http('Accept-Language')
                )
            ),
            'i-default'
        ] unless $self->{__I18N_LANGS};
    }#else
    no strict 'refs';
    &{ ref($self) . '::_maketext_lang' }( @{ $self->{__I18N_LANGS} } );
    return $self->{__I18N_LANGS};
}#sub

sub localtext_lang {
    my $self = shift;
    my $class = ref $self || $self;

    my $lang = ref "$class\::I18N"->get_handle( @{ $self->localtext_langs } );
    $lang =~ s/.*:://;

    return $lang;
}#sub

sub localtext_lang_tag {
    my $self = shift;
    my $class = ref $self || $self;

    return "$class\::I18N"->get_handle( @{ $self->localtext_langs } )->localtext_lang_tag;
}#sub

sub localtext {
    my $self = shift;
    $self->localtext_langs unless $self->{__I18N_LANGS};
    no strict 'refs';
    return &{ ref($self) . '::_maketext' }( $_[0], @{ $_[1] } ) if ( ref $_[1] eq 'ARRAY' );
    return &{ ref($self) . '::_maketext' }(@_);
}#sub

I'll export all these by default so the CGI::App object can have them as methods.

Again I have to stress that all this is completely untested. My next post should have some tests, and most likely fixes...

Lyle