1.  Dump the schema only.  This will  not contain any data, procedures or triggers, but only create info.  This will allow you to easily import your original schema later on.

mysqldump -uroot -proot -d --skip-triggers myDatabase > myDatabase_schema.sql

The -u specifies your username, and must be followed by your username e.g. -uroot. This is the same with your password, -proot.

The -d tells mysqldump to not dump any data in the schema and the –skip-triggers will skip all triggers and procedures.

The myDabase is the name of your database and the > myDatabase_schema.sql is where you want to dump it.

2. Dump the triggers/procedures. This will only contain the triggers and procedures contained within your database.

mysqldump -uroot -proot -d --no-create-info myDatabase > myDatabase_triggers_procs.sql

Here we have a –no-create-info to tell mysqldump that no create statements must be allowed, thus eliminating any schema sql and again the -d parameter to skip all data.

3. Dump the data only without triggers and create statements.

mysqldump -uroot -proot --skip-triggers --no-create-info myDatabase > myDatabase_base_data.sql

Finally we can dump the data only. The –skip-triggers skips all triggers and procedures, whils the –no-create-info skips sql create statements.

This will allow you to have 3 different sql files:

• myDatabase_schema.sql (only contains the schema)
• myDatabase_triggers_procs.sql (only contains triggers/procs)
• myDatabase_base_data.sql (only contains data)

Now to import.

This is fairly trivial.

All you have to remember is to import your data in a sequence.

First the schema, then the triggers and procedures and then finally the base data.  The schema will create your tables, the triggers and procs can then be added to your tables and finally the tables can be populated with data.

1. Import schema

mysql -uroot -proot myDatabase < myDatabase_schema.sql

2. Import triggers/procs

mysql -uroot -proot myDatabase < myDatabase_triggers.sql

3. Import base data

mysql -uroot -proot myDatabase < myDatabase_base_data.sql

And your done.

This might start out a bit scary, especially if you have only heard about it, but never done it yourself. There is a couple of good reads on the net, so you might want to start out by familiarising yourself with this exciting way of doing things.

Have a look at http://en.wikipedia.org/wiki/Test-driven_development or Google it.

What is test driven development?

Test driven development (TDD) is more a change in coding philosophy than anything else. Some of us might say: “…yip, I’ve written a couple of unit tests when I had some spare time at the end of my project.”

To be honest, this is a bit of the wrong way round method of doing things, kind of like building a house by putting on the roof first. I think Martin Fowler describes it best when he pointed out these three components.

• Write a test for the next bit of functionality you want to add.
• Write the functional code until the test passes.
• Refactor both new and old code to make it well structured.

TDD is a branch of Agile development which was based upon the first stages of extreme programming, but has really become a coding methodology that changed the way in which a lot of projects are being done.

What do I need to get started?

UML

It is always good to have a basic Unified Modeling Language (UML) diagram of how you would like your objects/classes to work and behave, but believe me, after you analyzed it through testing, you would do a fair amount of refactoring.

Here we have some pretty UML for a very basic Message object which must create and delete messages. The Message implements a Message_Store object for message storage.

PHPUnit

Now before you even attempt to write a line of code – go to http://www.phpunit.de/.

This will become your friend in days/months/years to come and you will learn to love it.

Background:

PHPUnit was created by a German fellow called Sebastian Bergmann and has grown into a full fledge PHP testing framework. It was based upon SUnit, which was developed by the father of extreme programming, Kent Beck, to accommodate testing in Smalltalk.

Installing:

The easiest way to install PHPUnit would be via your PEAR Installer.

First you must register PHPUnit PEAR channel with your local PEAR environment.

pear channel-discover pear.phpunit.de

Now you can install PHPUnit to your local PEAR packages

pear install phpunit/PHPUnit

That should be it for now – PHPUnit should hopefully be installed on your system. According to the PHPUnit website the directory it is located in is /usr/lib/php/PHPUnit, but I instead found it inside /usr/share/php/PHPUnit on my Ubuntu installation. In all likelyhood it should be located inside the same directory as your PEAR installation.

Let the testing commence.

“Whoa, whoa, whoa!”, most developers would cry. “How can you possibly test code you haven’t written yet?”, they all sing together.

Here is where the mind shift starts. You should have a basic understanding of how your code should work/behave after you have done the UML for your class. So why not use the way you think your code should behave in your tests.

Your test class

First things first. If you have a look at your UML diagram you would see that there is a dependency between the Message class and Message_Store class.

You should now write the test for how your Message object should behave.

//Include the base class for PHPUnit in order for your tests to work
require_once 'PHPUnit/Framework/TestCase.php';

//Include the Message class you are testing
require_once 'Message.php';

//Your Message Test Class
class MessageTest extends PHPUnit_Framework_TestCase
{

//Create a private class variable for re-use
private $oMessage;

/*
The setUp class function is native to PHPUnit and is executed infront of each
test function.  This will help you do things such as create a new object to work with before each new test.
*/
protected function setUp()
{
    $this->oMessage = new Message();
}

/*
The tearDown class function is reverse of the setUp() function and is executed after each test.
This will help you do things such as setting objects to a NULL value for re-use.
*/
protected function tearDown()
{
  $this->oMessage = null;
}

/*
First you test the construction of your Message object.
This isn't really necessary, since you know that a PHP constructor is suppose to work  , but is just done
for completeness sake.
$this->assertType() tests to see if the object you created is of the type 'Message'.
*/
public function testMessageConstructed()
{
    $this->assertType('Message', new Message());
}

/*
Now you start testing all the public methods inside your class.  Since you are going to test the way you want
your code to work in, you will only be allowed to test the public accessible methods.  The tests will still cover any
private methods which is called from the public method tested.
This is kind of a double-barrel test. Since you can prove that the setStore is working if the getStore returns the
correct object.
$this->assertSame() will test if two different variables are of the exact same type
*/
public function testSetAndGetStore()
{
    $oPDO = new PDO('mysql:host=localhost;dbname=localdb', 'username', 'password');
    $oMessageStore = new Message_Store_Pdo($oPDO);
    $this->oMessage->setStore($oMessageStore);
    $this->assertSame($oMessageStore,$this->oMessage->getStore());
}

/*
You will now test if the save function will return a variable of bool type, either true if correctly saved or
false if incorrectly saved
*/
public function testSave()
{
    $oPDO = new PDO('mysql:host=localhost;dbname=localdb', 'username', 'password');
    $oMessageStore = new Message_Store_Pdo($oPDO);
    $this->oMessage->setStore($oMessageStore);
    $this->assertType('boolean',$this->oMessage->save('Hello'));
}

/*
You will now test if the delete function will return a variable of bool type, either true if deleted or
false if not deleted.
*/
public function testDelete()
{
    $oPDO = new PDO('mysql:host=localhost;dbname=localdb', 'username', 'password');
    $oMessageStore = new Message_Store_Pdo($oPDO);
    $this->oMessage->setStore($oMessageStore);
    $this->assertType('boolean',$this->oMessage->delete(1));
}

}

What does our test class tell us?

That’s it – we now have a couple of basic tests to cover our Message class. The important thing about writing tests is that you should cover all public functions used in the class you are testing. At the moment these tests shouldn’t run, but it will give you a clear idea of where to start and is great for debugging whilst writing code.

Already there is a bit of a discrepancy of what is happening in the unit tests and UML. The way we test our code, is the way we would like our objects to behave. In the UML we assumed that the the message store could be part of the message object, but as our tests showed us – there might be a more elegant way using a storage object, which would allow for more pluggable, loosely-coupled code.

If we passed a storage object into the Message object itself, after a while the message object will become bloated and unusable, since now you have to cater for your different storage objects within the Message object. For e.g. create a saveToFileSystem function and a saveToDatabase function, where some functions might become redundant when using a specific storage object.

So instead of passing a PDO storage object directly to the message, you could have a Message_Store object which interfaces to your various storage objects, such as a PDO or filesystem etc.

Thus turning:

$oMessage = new Message();
$oPDO = new PDO('mysql:host=localhost;dbname=localdb', 'username', 'password');
$oMessage->setStore($oPDO);

Into:

$oMessage = new Message();
$oPDO = new PDO('mysql:host=localhost;dbname=localdb', 'username', 'password');
$oMessageStore = new MessageStore($oPDO);
$oMessage->setStore($oMessageStore);

By creating a Message_Store class, rather than building extra code onto your Message object, you reverse any dependencies that might occur within the Message object. All this means is that for you can now create a message object, and pass a interfaced storage object to it, thus allowing your code to become more flexible, without building layers of complexity itself into the Message object.

Refactor your code.

Now update your UML to include an interface to the Message_Store object.

Let’s build our message class according to the tests and updated UML diagram.

class Message
{

    private $oMessageStore;

    public function __construct($oMessageStore=Null)
    {
        $this->oMessageStore = $oMessageStore;
    }

    public function setStore($oMessageStore)
    {
        $this->oMessageStore = $oMessageStore;
    }

    public function getStore()
    {
        return $this->oMessageStore;
    }

    public function save()
    {
        return $this->oMessageStore->create($this);
    }

    public function delete($sMessageId)
    {
        return $this->oMessageStore->delete($sMessageId);
    }

}

Now your Message class is starting to look elegant. Now you can create the interface.

interface Message_Store_Interface
{
public function __construct($oStorage);
public function create($oMessage);
public function delete($sMessageId);
}

You can now implement the interface in the various Message store objects.

class Message_Store_Pdo implements Message_Store_Interface{

    private $oPdo;

    public function __construct($oPdo)
    {
        $this->oPdo = $oPdo;
    }

    public function create($oMessage)
    {
        $sSql = "insert into message_store (message)
        values (?)";
        $oSql = $this->oPdo->prepare($sSql);
        $oSql->bindParam(1, $oMessage->sMessage);
        $oSql->execute();
        return (bool) $this->oPdo->lastInsertId();
    }

    public function delete($sMessageId)
    {
        $sSql = "DELETE FROM message_store
        WHERE message_id = ?";
        $oSql = $this->oPdo->prepare($sSql);
        $oSql->bindParam(1, $sMessageId);
        $oSql->execute();
        return (bool) $oSql->rowCount();
    }

}

The advantages of Test Driven Development and Unit Testing

Running the unit tests

Now the big moment has arrived. Time to run your test. Assuming there is no syntax errors the test should execute fine.
Run this by going to your command prompt and enter the following

phpunit tests/MessageTest.php

If you saved the tests to MessageTest.php in the tests directory, everything should execute fine and the following would be displayed.

dewald@dewald-desktop:/var/www/mobi$ phpunit tests/MessageTest.php
PHPUnit 3.3.14 by Sebastian Bergmann.

....

Time: 0 seconds

OK (4 tests, 4 assertions)
dewald@dewald-desktop:/var/www/mobi$

You have compiled your tests and ran it succesfully. Whoohoo – if your tests gave an error, just follow the debug messages and fix your code. Easy as that.

You can also create reports to see if you have achieved acceptable code coverage by running.

phpunit --coverage-html /tmp/tests/ tests/MessageTest.php

The reports should can be found in the /tmp/tests/ directory and should look something like this:

In the End

In the beginning of the article I wrote that Martin Fowler said the following about Test Driven Development:

Write a test for the next bit of functionality you want to add.

We wrote all our test for the functionality of our Message object in the way which we would like our object to behave. We then test the results of that object.

Write the functional code until the test passes.

We wrote the functional code for the classes to make it work as we determined through our tests.

Refactor both new and old code to make it well structured.

After writing the tests, we realised that our code has some short-comings and dependencies, which we could reverse in-time, since we saw the way it was supposed to work in unit tests meant refactoring our UML and inherently we would have needed to refactor our code.

What now?
TDD doesn’t come naturally. Especially if you were used to years of a different kinds of development techniques. It comes through practice and experience.

I’ll suggest you read through the PHPUnit manual and examples, as to see how flexible the testing framework can be, also as a start, write some unit tests for your old code, and you might be surprised just how efficient you could have been.

Enjoy.

so for this, i’ve decided on DeviceAtlas.  just head on over to DeviceAtlas and open a developers account – you will get a one year developer’s license to play around to see how cool it is.

after registering – click on the downloads link – then go to the php example and download the source files.  after de-tar-and-un-zipping the file, just extract the contents.  at the moment all we are really interested in, is the Mobi/Mtld/DA directory and its contents – also create a json directory and drop the Sample.json file inside.

dump everything into a web directory somewhere.

create a deviceatlas.php file in the working directory root and put the following inside.

//includes the DeviceAtlas API

include 'Mobi/Mtld/DA/Api.php';//fetch the previously populated apc cache instance into a variable

$tree = apc_fetch('tree');

//Get the properties from a variable containing a JSON tree loaded from cache

$properties = Mobi_Mtld_DA_Api::getProperties($tree, $_SERVER['HTTP_USER_AGENT']);

//Output the phone properties for your viewing pleasure

print_r($properties);

‘hhmmm, that’s it?’, you might ask, and if you were clever you would have noticed something strange in the code above.

yip, that is pretty much it – but you also need caching.  the apc_fetch(‘tree’) command in the code above actually fetches the json file already dumped into a ‘tree’ cache instance in PHP APC (Alternative PHP Cache).

why PHP APC, you may also ask.  well, besides using the overstated memcached (you might want to read this), there are some methods to the madness.  with PHP APC, the cache is shared on a server.  it might not be accessible from other servers, but this is not really a con in my mind.

if you wish to scale your application later one and add another server, there is going to be some issues, you might say – but there will also be issues if you have millions of users hit one central instance of DeviceAtlas.  so, having one instance of DeviceAtlas per server might not be too much of an issue.  also when it comes to updating the json file, you might want to consider the single point of failure issue of just having one server.

so have a couple of servers, and just write some scripts to update the single instances of DeviceAtlas, rather than updating one, already busy server which still needs a cache update.

but, back to business.

first you have to install PHP APC.  i found this handy guide on the net, which helped me install it on my debian system and integrate it with apache and php5 – there is also a guide at the bottom for fedora based machines, so don’t fret my linux loving friends

you can create a php script to run the following – which should cache your DeviceAtlas tree into PHP APC.

//ads the sample json data from DeviceAtlas

apc_add('tree', Mobi_Mtld_DA_Api::getTreeFromFile("json/Sample.json"));

from the first code example, the output might look something like this (depending on your phone)

Array
(
[wmv] => 0
[vendor] => Nokia
[mobileDevice] => 1
[memoryLimitMarkup] => 357000
[memoryLimitDownload] => 61440
[midiMonophonic] => 1
[midiPolyphonic] => 1
[mpeg4] => 1
[3gpp] => 1
[drmOmaForwardLock] => 1
[drmOmaCombinedDelivery] => 1
[drmOmaSeparateDelivery] => 1
[markup.xhtmlMp10] => 1
[markup.xhtmlBasic10] => 1
[image.Jpg] => 1
[image.Png] => 1
[amr] => 1
[mp3] => 1
[aac] => 1
[h263Type0InVideo] => 1
[gprs] => 1
[edge] => 1
[image.Gif87] => 1
[uriSchemeTel] => 0
[qcelp] => 0
[hsdpa] => 0
[amrInVideo] => 1
[3gpp2] => 0
[displayColorDepth] => 18
[model] => N70
[https] => 1
[image.Gif89a] => 0
[umts] => 0
[displayWidth] => 176
[displayHeight] => 208
[markup.xhtmlMp11] => 0
[markup.xhtmlMp12] => 0
[csd] => 0
[hscsd] => 0
[id] => 204255
[_matched] => NokiaN70
[_unmatched] => -1/5.0609.2.0.1 Series60/2.8 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Link/6.3.1.13.0
)

you can view the attribute explanations here

now you can use the phone’s specific properties to your advantage.  you also might want to look at the user agent switcher add-on for firefox for testing purposes.

that’s it.  now every once in a while you can just update the json file, delete the current cache (http://www.php.net/manual/en/function.apc-delete.php) and bob’s your device detecting, apc caching uncle.

/*
* Abstract Unifier class to select objects from different sources.
*/

abstract class childObjectUnifier
{
    protected $currentObjectLocation;

    public function __construct()
    {
        $this->currentObjectLocation = get_class($this);
    }

    public function getData()
    {
        switch ($this->currentObjectLocation)
        {
             case 'currentServerLoginObject':
                    return $this->currentServerLoginFunction();
                    break;

             case 'remoteServerLoginObject':
                    return $this->remoteServerLoginFunction();
                    break;

             default:
                    die('Current user load not available');
        }
    }
}

/*
* Login Object which exists on the current server.
* This object could still be hit on the current server if the current user load is not high.
* Better utilizes the current server.
*/

class currentServerLoginObject extends childObjectUnifier
{
    public function currentServerLoginFunction()
    {
         return 'Some function on the current server';
    }
}

/*
* This login object could exist remotely, for e.g. a restfull/soap/xml-rpc service
* which could be hit if the current server load is climbing.
* Could possibly use a dedicated 'login' server if user load is exceedingly high on current machine.
*/

class remoteServerLoginObject extends childObjectUnifier
{
    public function remoteServerLoginFunction()
    {
        return 'Some function on the remote server';
    }
}

/*
* The game theory pattern which decides which objects to instantiate based on logical
* decisions made by solid data.
* In this example the current user load.
*/

class GameTheoryPattern
{
    public $currentObject;

    function __construct($iCurrentUserLoad)
    {
        $this->currentObject = $this->baseObjectUponStrategicData($iCurrentUserLoad);
    }

    private function baseObjectUponStrategicData($iCurrentUserLoad)
    {
         switch ($iCurrentUserLoad)
         {
             case $iCurrentUserLoad < 1000:
                    return new currentServerLoginObject();
                    break;

             case $iCurrentUserLoad >= 1000 :
                    return new remoteServerLoginObject();
                    break;

             default:
                    die('Current user load not available');
        }
    }
}

//GetCurrentUserLoad
$iCurrentUserLoad = 1000;

//Create Login Object based on the Game Theory Pattern
$oLogin = new GameTheoryPattern($iCurrentUserLoad);

//Call an function from the unifier class
echo $oLogin->currentObject->getData();

?>

It should be pretty self explanatory – due to the high level comments

feel free to add suggestions or comments – since i’m pretty sure there could be a better way, but for now, just to convey the message it should be sufficient.

as a developer, we kind of try and convince ourselves that developing for mobile and developing for a desktop browser is kind of the same thing.  but we all know that this is a stalling technique for the inevitable, since invariably it becomes a whole different field of play.

every little feature you add, every little flow created and every branch of navigational hierarchy is a challenge on its own.

enter the game theory pattern

this is was where my idea of a game theory patterns came into play.  game theory is a branch of applied mathematics, most notably used in social sciences (economics, biology, engineering etc.).  it is also used a bit in computer science, although referred to in that field as artificial intelligence.

thanks wikipedia

game theory allows someone to apply a bit of mathematics to capture data from a situation and use that data to build a strategic base, which allows you to make decisions to determine your success.

take for example morning traffic – you have 2 routes to take to work – one a bit longer than the other, but the longer route is less likely to be busy.

what do you do?  most of us will take the pro’s and con’s of both routes and use that to base our decision upon.

game theory works exactly, well almost, the same.  except you will probably use a bit of math to finalize your decision (average speed, distance etc.)

so why not use game theory in development, especially in a field like mobile, which is very limiting and very frustating at first.  but the good kind of limitation, the kind of limitation that could possibly create innovation.  the guys behind a phenomenon like twitter decided to limit people to 140 characters.  and despite this limitation, people have found innovative ways to increase the effect of that 140 character limited textual communication.

game theory would allow us to mathematically decide on feature sets, the use of algorythms, objects and could even change the flow of an application.

say on an example mobile application we have 3 major features – chat, activity, content sharing.

which one do we choose as our killer feature, and which would probably make the application more bloated and more problematic to use.  why not use the main variable in the equation to help you decide – the user.

i often secretly chuckle to myself, when a system gets developed for one purpose and all the users start using it for a totally different end result, no one really every thought of, or intended for.  you could use this to your advantage with game theory.

a simple case could be made using application flow as an example.  we have a menu with chat, activity feed and shared content.  which one is more important, which one would you make easier to access for the user.  by simply adding a click count to each feature, you could sort a menu by popularity, rather than obvious choice as this will invariably change someday.

if users decide this month, chat is the next best thing, let chat be number one on your menu – and i know some user interface expert will maintain that consistency is important, but if users consistently choose chat as their most important feature, what would the issue be?

this would change the way you develop and architecture software as well.  since you know chat is gaining in popularity according to your available data, then you should be able to utilize servers better for your chat functionality.  write scaling scripts which would use databases sharding.  and on the other hand, allow your game theory pattern to simplify things – why use a 40 slave/master instances of a database if all you need is one.  eliminate unnecessary overheads in that way – more layers makes for more complexity, makes for more overhead, and will eventually kill your application.

i’ve still got to convince a couple of people to maybe try this game theory pattern approach to development and if, by chance, martin fowler reads this article:

‘please martin, consider this pattern as a topic for a new book! ’

who know maybe even me?

-  coolest guy on the internet

there is a great online community which supports mobile development called mobiforge which could be utilised to answer all of your mobile related questions.

but as a start i will point out some important steps/considerations to take/make:

user-centered design

before you start on any work, you have to realise that by doing a mobile site, the most important part of your equation is the user and his phone.  it is easy to develop/design a site which looks absolutely amazing in a pc browser, but fails to even load within a phone browser.

you have to make sacrifices in design/layout to suite the user of the site and understand the limitations your user might face with his specific mobile device.  people accessing a site with a lower end phone usually wants quick textual feedback about a certain subject, rather than big glossy images that fill your screen.  once you understand what the user need – and design according to those needs, then you will be ready to continue.

keep your site simple

it is easy in a web browser to go back in your history and traverse navigation, but on a mobile it is a bit different.  so why confuse the mobile user with 50 levels of navigation, where a simple three leveled hierarchy navigational solution would be more than sufficient

and remember that there is no need to overcomplicate things by adding features that is nice to have, rather than have to have.  it is a bad practice on mobile or pc.

prototype it

a while back i read a blog post written by phil barret, an user-experience guru , who used prototyping to do the layout of his new home using newspapers.  this is probably the best analogy you can use to describe the importance of prototyping your designs to control the flow of your site.

design for the right device

group mobile devices into specific groups, for e.g. low, middle and high.  where your low end would be your nokia 2610, middle would be sony ericsson w890 and the rest, high end devices, such as the iphone, palm and other feature rich pda devices.

it would make sense to deliver more text-based content to lower end phones and more graphic rich sites to high end phones.  and make sure your features apply to the phone’s capabilities.  you shouldn’t give someone the option of uploading images if his phone doesn’t support it.  this could be achieved by using something like tera-wurfl.

standardise

make sure you comply with the industry standards.  for e.g. using XHTML-MP as your markup language and something like WCSS as your presentation markup.  specify your doctype, character encoding and use well formed html code.  you can visit the oma for more information.

this gives you more flexibility and would cut down on rendering issues in your mobile browser.

this is only very basic advice, and by no means do i advocate this as being the most correct approach, but it should give you some direction and hopefully a bit of a mind shift when it comes to mobile development.

some usefull links:

• wap 2 specifications – http://www.wapforum.org/what/technical.htm
• XHTML-MP – http://www.openmobilealliance.org/tech/affiliates/wap/wap-277-xhtmlmp-20011029-a.pdf
• comparison between XHTML-MP and XHTML basic – http://pc.dev.mobi/?q=node/119
• mobile web best practices – http://www.w3.org/TR/mobile-bp/

there is however a bit of an issue with web based mvc frameworks, and in my opinion a large freaking elephant that no one seems to talk about.  how can i call a controller from within a view and assign an action to it – and is it correct to do so?

and before the quiet whispers start, i’d like to note that this does not necessarily go against mvc principles.  if you think about it – the whole point of mvc is to seperate business logic, application logic and the view (presentation logic) – and i’ve confirmed this with a couple of conversations with other developers.  but on the other hand, if you feel that i have overstepped a boundary – please do let me know if there is a better, or more correct way of doing this.

now lets give a practical problem:

i have a view template, lets call it home, which consists out of a header, navigation, content and footer.  now my content is pretty much controlled by the controller and whatever the view calls to display.  but say for example something you wish to partially include a login form in the top header.

how do i actually use a controller to change the way the view behaves, without building the logic into all the controllers which i access.  now in order to overcome this – we can attach a controller, lets call it log controller, to the view.  this controller will have the business logic to decide whether or not a login or a logout form must be generated.

now the obvious would be to include a partial view, but then the issue arrives where you let your view start making logic decisions, instead of presentation. on the other hand, you can use the current controller to help you decide this, but if you ask me, that might be a bit of overkill and lots of extra and duplicate coding.

the solution:

in zend framework, i found the solution to be a bit easier than in kohana, but still fun to figure out.

zend framework:

okay, so i have my home template, which is controlled by my home controller.  on my home template in my header i have a form which would change depending on if the user is logged in or not, which i would like to be controlled by my log controller.

so, in my view i would do the following call:

<?php echo $this->action('log','generateForm') ?>

which would go to the log controller and call the method generateForm() – so in generate form i will be able to decide whether the user is logged in or not and assign the corresponding result to the view assigned to the controller, in the case above it would most likely be the generateForm view.

there you go – without having to overcomplicate things or duplicate code, you can assign controllers as part of your presentation logic with a simple call.  i have however not yet figured out how to pass variables to the action method you are calling, but as soon as i have an update i will let you know.

kohana:

in kohana the task is  a bit more complicated – luckily i found in the underbelly of the kohana forums an user with the same problem – which came up with the following solution -

<?php
class Dispatch_Core{

protected $controller;

public static function controller($controller)
{
$controller_file=strtolower($controller);

// Set controller class name
$controller = ucfirst($controller).'_Controller';

if(!class_exists($controller, FALSE))
{
// If the file doesn't exist, just return
if (($filepath = Kohana::find_file('controllers', $controller_file)) === FALSE)
return FALSE;

// Include the Controller file
require_once $filepath;
}

// Run system.pre_controller
Event::run('dispatch.pre_controller');

// Initialize the controller
$controller = new $controller;

// Run system.post_controller_constructor
Event::run('dispatch.post_controller_constructor');

return new Dispatch($controller);
}
public function __construct(Controller $controller)
{
$this->controller=$controller;

}

public function __get($key)
{
if($key=='controller')
{
return $this->$key;
}
else
{
return $this->controller->$key;
}
}

public function __set($key,$value)
{
$this->controller->$key=$value;
}

public function __toString()
{
return $this->render();
}

public function render()
{
return (string) $this->controller;
}

public function __call($name,$arguments=null)
{
if(method_exists($this->controller,$name))
{
return $this->method($name,$arguments);
}
return false;
}

public function method($method,$arguments=null)
{
if(!method_exists($this->controller,$method))
return false;

if (method_exists($this->controller,'_remap'))
{
// Make the arguments routed
$arguments = array($method, $arguments);

// The method becomes part of the arguments
array_unshift($arguments, $method);

// Set the method to _remap
$method = '_remap';
}

ob_start();

if(is_string($arguments))
{
$arguments=array($arguments);
}

switch(count($arguments))
{
case 1:
$result=$this->controller->$method($arguments[0]);
break;
case 2:
$result=$this->controller->$method($arguments[0], $arguments[1]);
break;
case 3:
$result=$this->controller->$method($arguments[0], $arguments[1], $arguments[2]);
break;
case 4:
$result=$this->controller->$method($arguments[0], $arguments[1], $arguments[2], $arguments[3]);
break;
default:
// Resort to using call_user_func_array for many segments
$result=call_user_func_array(array($this->controller, $method), $arguments);
break;
}

// Run system.post_controller
Event::run('dispatch.post_controller');

if($result!=NULL)
{
$result=ob_get_contents();

ob_end_clean();
}

return $result;
}

}

now the above you can save under your library directory in your application for ease of use as something like dispatch.php.

so to solve the problem of the login form – in my default home template i’ve added <?php echo $this->template->partial = new View('log/generateform');?>

in the generateform view i called i will use the following:

<?php
//call the log controller
$dispatch = Dispatch::Controller('log');
//call the generateForm method
$dispatch->method('generateForm');
//return the result from the template to the view
echo $dispatch->template;
?>

the above will call the  log controller and the method generateForm, in the controller, you can specify the template, for e.g. <?php public $template = 'kohana/partial/navigation';?> and then in the action / controller assign any values to be used by the template view/presentation.

oh, and one more cool thing with the above is that you can also call the method using parameters for e.g. <?php $dispatch->generateForm(array('title'='login'));?>

recap

as i said in the beginning, if anyone feels that i’ve overstepped the mvc divide, please let me know, i’m sure that with the assigning of a controller/action from within a view is totally acceptable, since only your presentation logic is affected by it really, and the controller still decides on what is available for the view to pull, without having adjust your controllers to cater for everything.

and if someone knows of a quicker method in kohana, also do let me know.

as a short term solution to this i will show you how to make your own web 2.0 social network in no time.

feeling confused?

no need to be – just follow these easy steps:

1.  get a funky name

use this cool site to generate that cutting edge and hip web 2.0 company name – one that will make silicon valley startups look like a bunch of geeks with ancient 2008 macbooks – http://www.dotomator.com/web20.html and register it – before someone else takes it.

i chose tagvine – it sounds like something me and my friends should belong too.

2.  make them jealous with a shiny logo

take your funk-a-licious company name and generate a web 2.0 logo of format using http://creatr.cc/creatr/ – remember it looks more professional and web 2.0ish with a bright beta tag in the top right corner.

3.  lunch

after a hard mornings worth of work – you’ve probably worked up a mighty appetite.  so take your macbook and head over to the nearest trendy coffee shop and surf some net.

4.  socialize your brand

now that the hard work is finished – head over to http://www.ning.com/ – just enter your company name – register your url, choose some colors and themes, upload your logo and voila! – a shiny, trendy web 2.0 company is born -> tagvine

5.  monetize your brand

no need – just stand back – and watch your super awesome web 2.0 social company generate some theoretical dollars.  i’ll suggest taking that beefy macbook of yours and head over to that trendy coffee shop and have another tall mocha-chocha-chino.

as everyone knows the new web 2.0 motto is – we will build it and they will come

so i’ve been looking into solr the last couple of days.  solr is relatively new in the arena and probably outshined a bit in popularity by other search engines such as lucene and nutch.  “but why solr?”, you may find yourself asking.

Well solr has a couple of tricks up the sleave – which is likely due to the fact that its a fresher version of the old, dare i call it legacy, search engines.

some of the features of solr includes:

• highly scalabe java search server (as i will try to show you through this article)
• works with the lucene search library (tried and tested)
• you can update your engine with xml using a kind of lightweight restful service.
• can parse html, openoffice, microsoft office suite documents, pdf’s etc using lucene parsers.
• custom tokenizer, filter and analyzer steps for control over indexing and query processing
• extremely rich indexing of fields and metadata, including numbers
• can combine fields for fulltext-type searching
• tuned for high performance even when updating
• spell checkers included, etc, only 2 name a couple of features.

click here to view a more complete list.

solr currently have api’s for ruby, php, python, json, forrest/cocoon.

the obvious elephant that is missing from the list above is the fact that solr is not compiled with a crawler of some sort.  luckily some friendly open source guys already released a guide for integrating nutch with solr for an all round experience.

you might also want to look into localsolr – which enables geographical and spatial searches with solr

now… where to begin:

first things first – since solr is java based, it has to be wrapped in a java container of some sorts.  i choose the friendly open source tomcat as a basis.  i installed tomcat 5.5 using this helpful guide as a reference.  read a bit more than what is required and make sure you understand the concept of using web applications and the configuration of tomcat.

after tomcat is installed, its time for solr.  and as luck would have it – a solr installation in tomcat is probably one of the easiets most straight forward ways of doing it – and i just followed this handy guide, provided by solr, as a reference.

now after solr is up and running, you should be able to access your admin screen via the following url http://127.0.0.1:8180/solr

that would be assuming your tomcat port is 8180 and solr is installed under the directory solr in your web applications.

now after installing solr – and accessing the admin screen you should – you should be able view your configuration and perform a basic search.  to update the schema and import data into solr is a whole other post on its own, and as soon as i figure it out, you will be the first to know – but for know we are going to focus on replication and scalability.

look at the following diagram (done by me using gliffy)

as you can see above – there is 3 main levels

1. load balancers (wrapped in a monitoring service – keepalived which is optional)
2. solr slave instances
3. solr master

you can see that it isn’t the most original idea in the book, but it works.  especially if you should run it within a cloud where your instances could almost be infinite.

searching with solr

lets start from the bottom up – at the base – you have the solr master – which would be the solr instance where all your data importing from xml happens.

above that is the slave instances, which is will replicate their indexes from the master.  the slaves will in turn handle all search request from the load balancers which sits at the top.  this reduces load on the master, which will be used as the primary indexer.

also the load balancers will be wrapped in a monitoring system, which will alert you in case something goes wrong, and also reduced the risk of a single point of entry failure.  this monitoring system is really optional, since the load balancer i will be using has built in failover checks.  but nevertheless it is good practice to look into a monitoring system like keepalived for your architecture.

now for the specifics – to update the index of your master is almost a trivial exercise – since solr has a kind of a restful approach of doing this -

#the following would be in a .sh file
FILES=$*
URL=http://localhost:8180/solr/update

for f in $FILES; do
echo Posting file $f to $URL
curl $URL --data-binary @$f -H 'Content-type:text/xml; charset=utf-8'

#send the commit command to make sure all the changes are flushed and visible
curl $URL --data-binary '<commit/>' -H 'Content-type:text/xml; charset=utf-8'

so – if you save the above script as for e.g. post.sh, make it executable (chmod a+x post.sh), then you could run it with a xml dataset (./post.sh dataset.xml), and it will update the index using its semi restful interface.  all you have to do is change your url in the above script to the solr instance you wish to update.

then comes the replication of data to the slaves – this is done by using scripts in the bin directory of your solr installation.  here is the descriptions of these scripts from the solr websit.  we would most likely be interested in the snapshooter – which takes a snapshot of the current master index – from there you would use the snappuller – which would pull the latest snapshot from the master index and update the index.

after you’ve mastered these scripts, you can store run them as cron jobs every so often and update the slave indexes.  i’ve also read on there site, that in the new release they would be compiling replication as a feature into solr, where you can actually just tweak a couple of configuration settings, and it should replicate without external tools, so just keep a lookout for the new solr version.

load balancing with haproxy

so as a load balancing solution i chose haproxy, which offers high availability and balancing for tcp and http based applications.  haproxy does not deliver content such as apache, neither does it do caching in a way squid does it, but it’s small, simple and works very well.

to install haproxy:

mkdir /opt/haproxy
cd /opt/haproxy
wget http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.15.7.tar.gz
gunzip haproxy-1.3.15.7.tar.gz
tar -xf haproxy-1.3.15.7.tar
cd haproxy-1.3.15.7
make
cp ./haproxy /usr/bin/haproxy

then create a file called haproxy.cfg that contains the following data:


global
log 127.0.0.1   local0               #logs all haproxy info to local0 log
log 127.0.0.1   local1 notice    #logs all notifications to local1 log

daemon                                     #specifies haproxy to run as a deamon instance
maxconn         4096                 # total max connections (dependent on ulimit)

defaults   #setup some default values
log            global
mode       http
option      httplog
option      dontlognull

clitimeout        60000       # maximum inactivity time on the client side
srvtimeout        30000       # maximum inactivity time on the server side
timeout connect   4000        # maximum time to wait for a connection attempt to a server to succeed

option            httpclose     # disable keepalive (HAProxy does not yet support the HTTP keep-alive mode)
option            abortonclose  # enable early dropping of aborted requests from pending queue
option            httpchk       # enable HTTP protocol to check on servers health
option            forwardfor    # enable insert of X-Forwarded-For headers

balance roundrobin            # each server is used in turns, according to assigned weight

stats enable                  # enable web-stats at /haproxy?stats
stats auth        admin:pass  # force HTTP Auth to view stats
stats refresh     5s        # refresh rate of stats page

listen myloadbalancer 127.0.0.1:8888 #where the loadbalancer should listen for requests
server slave1 192.168.1.6:8180 #a slave
server slave2 192.168.1.6:8180 #another slave

–

and that is it.  now all you do is run:

/usr/bin/haproxy -f haproxy.cfg

and bob’s your uncle – just go to http://127.0.0.1:8888/haproxy?stats to verify that it is running (the username is: admin and the password: pass – as specified in the config file above)

just note – if you get binding errors, make sure the port you specify in the config file is open, and not in use by something like apache, tomcat, yaws etc.

now if you have all your slave instances running a version of solr – you should be easily able to connect to them in a round robin fasion – for e.g. connecting to http://127.0.0.1:8888/solr – you can confirm this in the syslog or with the above stats link.

monitor that architecture, with keepalived

last, but not the least would be the enabling of the load balancing monitor – keepalived was suggested to me as a viable option, however, with haproxy  you can have do failover checks and run multiple loadbalancers under a shared ip so this isn’t really a necessary part, but as explained earlier, it is good practice to run a monitor like keepalived on your system.

conclusion

because it is late, and i still have to proofread this sucker, i haven’t really got around to keepalived, although – do yourself a favor and read up on it – i know i certainly will have a cup of java and research it a bit more.

so – there you go – you have a fully load balanced system – running a search engine, which should scale horizontally with your hardware buying budget.