All about Magento E-commerce Store.......MagentoForum: September 2011

Friday, September 16, 2011

Magento Development Environment


Learning Magento can range from frustrating to infuriating for the first time user. Here's a few tips to ease the burden of assimilating an entirely new system.

Turn on Logging

While the Magento Community Edition doesn't have much default logging, it does have a solid logging system in place. Anywhere you can execute PHP in Magento you can do something like
Mage::Log($variable); 
Magento will output the contents $variable to a log. Arrays and Objects are auto-expanded and pretty printed, allowing you quickly zoom in on the portion of the data structure you're interested in. I've found using this in combination with Mac OS X's Console.app to be a much smoother experience than dumping objects to the browser, particularly given how often Magento objects store references to a good portion of the system (resulting is large object data dumps).
Logging is turned off by default, to change that
  1. In the Magento Admin, go to System->Configuration
  2. In the left column, click on Developer (under Advanced)
  3. If it's not expanded, Click on Log Settings
  4. Select Yes from the "Enabled" drop down
  5. Click on Save Config
Unfortunately, Magento won't create your log files for you. If they don't exist you'll need to create them yourself. By default, log files are stored at
var/log/system.log var/log/exception.log 
If the log folder doesn't exist, create it along with system.log andexception.log. Make sure Apache has write access to both the directory and the files. Check with your system administrator on the best way to do this.
Calls made using Mage::Log will be dropped in system.log. Exceptions will be logged to exception.log

Turn on Developer Mode

If you look in the Magento bootstrap file (index.php) you'll see lines similar to the following
#Mage::setIsDeveloperMode(true); #ini_set('display_errors', 1); 
Uncomment these. In a production system, you'd never want to have your errors display to the browser, but while developing having an errors and warnings thrown immediately in your face is invaluable.
Setting the IsDeveloperMode flag is a big time saver. By default, when Magento discovers an exception, it will log the exception to a report and then forward the user to a page that will display that report. This can be infuriating when you're developing, particularly because Magento redirects in such a way that you can't backup to the page you were just on. Setting the developer mode flag results in the Exception output being sent directly to the browser with <pre> formatting.

Logging Magento’s Controller Dispatch


I'm taking a quick break from the Magento series as I plumb the depth of EAV and get through a crunch phase at The Day Job™. In the selfless interest of helping out the internet and the selfish interest of keeping my front page free of moldy content, I thought I'd pass on a simple Controller dispatch logging patch for Magento. This is mainly useful when you're setting up a new Module and can't figure out why Magento won't see it, or if Magento keeps throwing the CMS 404 page in your face

A Few Caveats

  1. These files are not a true Magento Module. While powerful, creating something like this with Magento's override system would be an order of magnitude harder
  2. Because of the above, I can't make any promises about using this with a anything but Magento version 1.3.2.3
  3. Standard caveats about production servers apply. Use at your own risk, no warranties, etc.

Installing

Take the "Mage" folder (1.3.2.3/Mage) and place it in
app/code/local 
so that your directory tree looks like
local/Mage/Core/Controller/Varien/Action.php local/Mage/Core/Controller/Varien/Front.php local/Mage/Core/Controller/Varien/Router/Standard.php local/Mage/Core/Model/App.php 
Then, enable logging and you should start seeing messages in your system.log that look something like
2009-09-20T16:55:48+00:00 DEBUG (7): Matching router [standard] = Mage_Core_Controller_Varien_Router_Standard  vs. request 2009-09-20T16:55:48+00:00 DEBUG (7): Module: helloworld 2009-09-20T16:55:48+00:00 DEBUG (7): Found Modules: Array (     [0] => Alanstormdotcom_Helloworld )  2009-09-20T16:55:48+00:00 DEBUG (7): Setting Route Name [helloworld] on request 2009-09-20T16:55:48+00:00 DEBUG (7): Passing request into controller Alanstormdotcom_Helloworld_IndexController 2009-09-20T16:55:48+00:00 DEBUG (7): Setting Module Name to: helloworld at Mage_Core_Controller_Varien_Router_Standard::match(248) 2009-09-20T16:55:48+00:00 DEBUG (7): Setting Controller Name to: index at Mage_Core_Controller_Varien_Router_Standard::match(250) 2009-09-20T16:55:48+00:00 DEBUG (7): Setting Action Name to: index at Mage_Core_Controller_Varien_Router_Standard::match(252) 2009-09-20T16:55:48+00:00 DEBUG (7): Calling dispatch method on Alanstormdotcom_Helloworld_IndexController 2009-09-20T16:55:48+00:00 DEBUG (7): Action Controller: Alanstormdotcom_Helloworld_IndexController dispatching with action [index] 
You'll also see a message whenever Magento can't make a Module, Controller, or Action Method match, which is useful when debugging 404 problems.

Magento’s Class Instantiation Abstraction and Autoload


If you're at all conversant with object oriented programming, the following line should look familiar to you.
$customer_group = new Mage_Customer_Model_Group(); 
We're instantiating an instance of the Mage_Customer_Model_Group class. However, if you were to search the Magento codebase forMage_Customer_Model_Group, you'd never find a single expression that looks anything like the one above. You would, however, see a lot code that looks something like this
$customer_group = Mage::getModel('customer/address'); 
By the end of this article, you'll understand the how and why of instantiating classes in Magento, as well as how to include custom Models, Helpers, and Blocks in your own modules.

PHP Autoload

Before we get into the nitty gritty of Magento's class instantiation abstraction, we need to spend a minute or two talking about PHP include files.
Prior to version 5, PHP didn't offer much guidance on how project files should be structured. PHP has a simple concept of including or requiring a file. You pass the include or require statement a string that contains a path to another PHP file
include('/path/to/file.php'); 
and PHP acts as though the code in file.php was part of your main program. There's no automatic namespacing, linking, packaging or any attempt to reconcile the context of the included files.
Over the years this has led every PHP shop and project to develop their own conventions as to how their base system/library code should be included. If two headstrong developers have a difference of opinion on how this should be accomplished, the resulting codebase quickly becomes a rat's nest of redundant includes and requires.
While PHP still doesn't enforce any particular strategy, PHP 5 introduced the concept of autoloading your class files. Developers can create an autoload function that will be called whenever an undefined classes is referenced anywhere in the codebase, with a single argument of the referenced class name.
For example, consider the following simple autoload function
function __autoload($class_name){     require('lib/'.strToLower($class_name).'.class.php'); } 
The team that uses this function stores all its classes in a directory structure that looks something like this
lib/customer.class.php lib/email.class.php 
While not suitable for every project, if used judiciously the autoload concept frees developers from having to care about when and where they include their class files, as well as enforcing/encouraging a consistent naming convention.

Magento Autoload

As you'd imagine, Magento's module system leverages the autoload feature heavily. While tricky to master, once you understand the naming convention, you'll know exactly where to find any Magento class is, as well as know where to place your own classes.
You can split all Magento classes into four parts that we'll call Namespace, Module Name, Class Type, and Name. Consider the The class mentioned above
Mage_Customer_Model_Group Namespace_ModuleName_ClassType_Name 
Namespace
A classes's namespace lets you know who's responsible for creation and maintain of the class, and helps prevent name collisions between modules. All Magento core modules use the Mage namespace, and the recommended convention is to use a version of your company name for your own modules.
ModuleName
All customization of Magento is done through modules, which are a collection of source code and config files that can be loaded into the Magento system. In the example above, the ModuleName is Customer.
Class Type
Without getting into too much detail, there are several broad categories of classes in Magento, including Model, Singleton, Helper, and Block.
Name
Finally, each class should have a unique name that describes its intended use or function. In the above example, this is Group.

clearing the Magento Cache


Just another quick Magento tip as I continue my journey back into the swamps of e-commerce.
Magneto's admin console has a section for managing your cache
(System->Cache Management) which works well, unless you happen to get a bad configuration cached that crashes the system or renders the admin inaccessible.
When that's happened to me I've been able to completely delete the cache folder, and Magento is smart enough to recreate it on the fly
rm -rf var/cache/mage-* 
This will zap all your cached values and allow you to fix whatever it is you broke.

Magento Configuration Lint


The biggest challenge for Ruby and PHP web developers coming to Magento is having to deal with individual module configuration. This is an integral part of of Magento that won't ever go away, but I think there's huge steps we can take to make Magento's configuration less painful for newcomers.

The System you Have

Convention over Configuration is an old industry war, with an unsteady ceasefire in place between the combatants. Each side agrees to treat the other with silent contempt. I don't want to go anywhere near a debate that radioactive, but here's the problem I think we can solve.
Magento's system is, for better or worse, configuration based. Before you can start writing interesting code you need to write a bunch of boring code (XML) to create the framework for your code to live in. In addition to bringing a different set of assumptions to the table, Magento's configuration files are also deeply nested and not always intuitive. Some nodes have contextual importance and others do not. There's probably a handful of people on the planet who know which ones are which. To make matters worse, a partially misconfigured system will run code — until it won't. This can lead to the incredibly frustrating situation where a developer who's learning the system isn't sure if the problem is their own code or if they've misconfigured something.
The end result is a cliff of a learning curve that most time-strapped developers don't have time to climb. Worse, with so much negative feedback early in the learning process, they never get into that groove where they really start learning to work with the system instead of against it.
I don't want to pick a side in the Convention vs. Configuration fight. I want to make Magento's configuration process less confusing for newcomers.

What's a Lint?

The generic term lint comes from a program of the same name. What the original lint program did was scan C code for things that weren't technically wrong (they'd compile), but were likely not what the original programmer had in mind. The combination of C's terse semantics and raw power led to countless situations where a simple typo or failure to think through a function could lead to subtle memory errors that took days to track down. The lint program scans your code and looks for common errors that still compile. This is sometimes called static analysis.
To web developers (or at least those web developers paying attention) the most well known lint like program is Douglas Crockford's JSLint. JSLint lint scans Javascript code for things Doug thinks will lead to errors and/or misunderstood programs. Javascript has some quirks (attempts by the interpreter to "guess" where you wanted a semicolon, scoping issues with for, etc.) that can lead to subtle and hard to track down bugs. JSLint calls these out right away. Again, JSLint will call out code that is valid and will run, but it's code that may lead to confusing bugs or behavior.
Magento's configuration system suffers from the same problem. It's very powerful, but confusing to newcomers. Even those familiar with it can make a quick slip of the fingers leading, once again, to hard to track down bugs. This is why we need a Configuration Lint, and this is why I've built a new system for creating Lint Cases.

How Configlint Works

There are two Magento modules in the Github Project. The first, Configviewer, is the system that contains the lint runner as well as a few sample Lint Cases. The second, Linttemplate, is a module with several empty Lint Cases (some passing, some failing) that you can use to start writing new cases right away.
To get started, let's download and install both the Configviewer and Linttemplate module I prepared for this tutorial. If you're new here and are unsure on how to install modules, I'd recommend some remedial reading.

Running The Configlint

If you've loaded up both modules, head over to the following URL
http://example.com/configlint/ 
This URL is calling the Configviewer's IndexController, which contains all the code you need to run your Lint Cases.

Generated Magento Model Code


This is a quick one for advanced users, and was inspired by a recent question I answered over at StackOverflow.
One of the nice things about a well abstracted system like Magento is the opportunities it presents for meta-programming. Drop the following code in a controller action (yes, you can define an inner function in PHP)
function someAction() {         header('Content-Type: text/plain');         $coupon = Mage::getModel('salesrule/rule')->load(1);                  function outputValue($value)         {             switch(gettype($value))             {                 case 'string':                     echo "'".str_replace("'","\\'",$value)."'";                     break;                 case 'array':                     echo "array(";                     foreach($value as $v)                     {                         outputValue($v);                         echo ",";                     }                     echo ")";                     break;                 case 'NULL':                     echo 'NULL';                     break;                 default:                     echo "'can\'t handle ".gettype($value)."'";             }          }                 echo '$model';         foreach($coupon->getData() as $key=>$value)         {               echo '->set';             echo str_replace(' ', '',ucwords(str_replace('_', ' ', $key)));             echo '(';              outputValue($value);                         echo ')';             echo "\n";         }          exit; } 
Run this code and you'll get some auto-generated code for the creation/updating of a Magentosalesrule/rule Model (assuming you have a Model with an id of 1)
$model->setRuleId('1') ->setName('test') ->setDescription('') ->setFromDate('2010-05-09') ->setToDate(NULL) ->setCouponCode('') ->setUsesPerCoupon('0') ->setUsesPerCustomer('0') ->setCustomerGroupIds(array('1',)) ->setIsActive('1') ->setConditionsSerialized('a:6:{s:4:"type";s:32:"salesrule/rule_condition_combine";s:9:"attribute";N;s:8:"operator";N;s:5:"value";s:1:"1";s:18:"is_value_processed";N;s:10:"aggregator";s:3:"all";}') ->setActionsSerialized('a:6:{s:4:"type";s:40:"salesrule/rule_condition_product_combine";s:9:"attribute";N;s:8:"operator";N;s:5:"value";s:1:"1";s:18:"is_value_processed";N;s:10:"aggregator";s:3:"all";}') ->setStopRulesProcessing('0') ->setIsAdvanced('1') ->setProductIds('') ->setSortOrder('0') ->setSimpleAction('by_percent') ->setDiscountAmount('10.0000') ->setDiscountQty(NULL) ->setDiscountStep('0') ->setSimpleFreeShipping('0') ->setApplyToShipping('0') ->setTimesUsed('0') ->setIsRss('1') ->setWebsiteIds(array('1',)) 

Magento Connect Role Directories


This is less a full fledged article this time, and more "something I always forget and can never find anywhere else, so I'm posting it here to help future me".
When you're packaging up a Magento Extension for distribution through Magento Connect
System -> Magento Connect -> Package Extensions 
You need to create a list of Roles and Path pairs
The first time you do this, it's not entirely obvious what a Role or a Path is. If you expand the Role drop-down, you get a list of different file types.

Magento Base Directories

As platform, PHP has a long history of relying on the base computer's underlying file system as a storage mechanism for various application and system functions. This ties in closely with PHP's practical history of closely modeling the nacent web's file serving architecture for it's own.

In an ideal world, as an end-user or application developer using the Magento system, you wouldn't need to worry about the file system at all. When you're handling user sessions, you don't care if they're stored on disk or in a database, or when you declare a Model class you don't need to worry about requireing the source file into the system. That's because there's an abstraction that sits on top these systems and allows you to not worry about how they're implemented.

Unfortunately, we don't live in an ideal world. There are times when, either out of deadline pressure or lack of an interface, we need to programmatically deal directly with the raw file system in Magento. This article will teach you how to programmatically find the paths of important Magento systems files, which will allow you to build less fragile applications that can withstand system configuration and installation changes.

Finding The Path

If you're worked much with PHP, you know that finding the full file path to any particular file can be a bit of a black art. When you include or require a PHP file, or open a raw file with functions like file orfile_get_contents using a relative file path, there's a set of rules that PHP will use to determine which directories need to be searched.

Because this list of directories (called the include_path is malleable at runtime, if two files with the same name exist in the include path, it's often hard to tell which one is loaded. Consider the following

?
1
2
3
4
5
include('foo.php');
#files system has
#   /lib/foo.php
#   /use/web/phplib/foo.php
#which one loads?

Because of this, it's a common pattern in the PHP to use the magic constants __FILE__ or __DIR__ to find the absolute path to the calling file, and then navigate to the file you want relative to that calling file

?
1
2
$info = file_get_contents(dirname(__FILE__) . '/../data/info.json');
$info = json_decode($info);

This is far from ideal.

First, there's the problem of symlinks. Depending on your web server system, it's not always clear if __FILE__is returning the file's real path, or its path in the context of a symlink. Because of this, you'll often see the use of the realpath function

?
1
2
$info = file_get_contents(realpath(dirname(__FILE__)) . '../data/info.json');
$info = json_decode($info);

While this works, it's not without its problems. As you can see, this will often lead to code that's a little hard to read. Additionally, calls to realpath mean that PHP needs to stat the filesystem to retrieve that information. As traffic to your application grows, these kind of file stats can start to create a performance bottleneck.

Finally, although it solves the problem of ambiguity in PHP's include path, this isn't a 100% portable solution. If we move the code from one PHP file (say, a Model in a module) to another PHP file (say, a Magento phtml template), the number of ..'s we'll need is different, since one file is deeper in the hierarchy than the other.

Sometimes __FILE__ is the only way to get where you want to go. Fortunately, Magento offers another option for commonly accessed folders.

Magento Base Directories

Add the following code an empty controller action or other area of Magento where you can run arbitrary code.

?
1
2
3
4
5
$base_path = Mage::getBaseDir('base');
var_dump($base_path);
 
$etc_path = Mage::getBaseDir('etc');
var_dump($etc_path);

You should see output something like the following

?
1
2
string '/Users/username/Sites/magento1point4.1.dev' (length=43)
string '/Users/username/Sites/magento1point4.1.dev/app/etc' (length=51)

The first folder is the base directory for the entire Magento installation. The second folder is the base directory for Magento's etc folder, which holds configuration files.

In the latest Community Edition release of Magento there are 14 different directories that are considered important enough to be retrievable by this method. The static calls to Mage::getBaseDir are a wrapper/public-api to the getDir method on a Mage_Core_Model_Config_Options object. If you take a look at the Magento Model pseudo-constructor in

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#File: app/code/core/Mage/Core/Model/Config/Options.php
class Mage_Core_Model_Config_Options extends Varien_Object
{
    /* ... */
    protected function _construct()
    {
        $appRoot= Mage::getRoot();
        $root   = dirname($appRoot);
 
        $this->_data['app_dir']     = $appRoot;
        $this->_data['base_dir']    = $root;
        $this->_data['code_dir']    = $appRoot.DS.'code';
        $this->_data['design_dir']  = $appRoot.DS.'design';
        $this->_data['etc_dir']     = $appRoot.DS.'etc';
        $this->_data['lib_dir']     = $root.DS.'lib';
        $this->_data['locale_dir']  = $appRoot.DS.'locale';
        $this->_data['media_dir']   = $root.DS.'media';
        $this->_data['skin_dir']    = $root.DS.'skin';
        $this->_data['var_dir']     = $this->getVarDir();
        $this->_data['tmp_dir']     = $this->_data['var_dir'].DS.'tmp';
        $this->_data['cache_dir']   = $this->_data['var_dir'].DS.'cache';
        $this->_data['log_dir']     = $this->_data['var_dir'].DS.'log';
        $this->_data['session_dir'] = $this->_data['var_dir'].DS.'session';
        $this->_data['upload_dir']  = $this->_data['media_dir'].DS.'upload';
        $this->_data['export_dir']  = $this->_data['var_dir'].DS.'export';
    }

you can see where these directory paths are stored.

Let's quickly review each one

Mage::getBaseDir('base');

This is your main Magento directory. In a default root level application instal, this is the equivalent of the document root.

Mage::getBaseDir('app');

This is your Magento application directory. This is the directory where the final class Mage... application file (Mage.php) is stored.

The default directory (from the Magento base folder) is

 /app/ 

Mage::getBaseDir('code');

This is your Magento code directory. This is base directory for the three Magento code pools (core,communitylocal).

The default directory (from the Magento base folder) is

 /app/code 

Mage::getBaseDir('design');

This is your Magento design package directory. This is the base folder that contains the "design packages" for each Area of Magento, (frontend, adminhtml, and install)

The default directory (from the Magento base folder) is

 /app/design  

Mage::getBaseDir('etc');

The etc folder is where Magento stores system level (as opposed to module level) configuration files. The nameetc is borrowed from the *nix family of operating systems, and Magento's configuration files are all XML based.

The default directory (from the Magento base folder) is

 /app/etc 

Mage::getBaseDir('lib');

Magento's library folder is where non-module based Magento code lives. This include a large amount of the system code which allows Magento to run, as well as a number of third party libraries (including the Zend Framework). The library is also the last code pool Magento will search when attempting to autoload a file.

The default directory (from the Magento base folder) is

 /lib 

Mage::getBaseDir('locale');

The locale folder contains the translation files for the core Magento modules. Magento uses a system similar to GNU gettext to provide translated text. Unique strings are stored as key value pairs in the translation files. They "key" is used to lookup which text should actually be displayed. This means for the english locale you'll see redundant looking strings like

 "Add Option","Add Option" 

It's only when you look at the non-english speaking translation files that the system become obvious

The default directory (from the Magento base folder) is

 /app/locale 

Mage::getBaseDir('media');

Magento's media folder is where media files (images, movies, etc.) related to data (products) is stored.

The default directory (from the Magento base folder) is

 /media 

Mage::getBaseDir('skin');

If the word module is the most used yet least well defined programming term, then the words skin and themeare the most abused in the designer's world. In Magento, the skin folder contains images, CSS and Javascript files used by your themes.

This is not the only folder where you'll find images, CSS or javascript though. This folder is meant for files that are customized per theme.

The default directory (from the Magento base folder) is

 /skin 

Mage::getBaseDir('var');

The var folder is another one borrowed from the *nix world. The var stands for Variable files, and is intended to store files which are expected to change during normal system operations.

The default directory (from the Magento base folder) is

 /var 

Mage::getBaseDir('tmp');

The tmp dir is a temporary directory for safely outputting files into for immediate processing. The operating assumption of the tmp folder is that any developer can write to it and expect their file to stay around for a few minutes, without any expectation that it will be there tomorrow.

The default directory (from the Magento base folder) is

 /var/tmp 

Mage::getBaseDir('cache');

Magento, rather famously, makes heavy use of caching for activities that might bog down the system if they had to be performed every-time a page loads. For example, Layout XML files are merged once, and then the tree is cached so they don't need to be merged again. The cache folder is one place where Magento will store these cached results.

The default directory (from the Magento base folder) is

 /var/cache 

Mage::getBaseDir('log');

Magento's log folder is where is stores the system and exception logs. These logs can be turned on from the Admin Panel's

 System -> Configuration -> Developer -> Log Settings 

section. The apache/web-server user will need write permission on this folder and the files therein.

The default directory (from the Magento base folder) is

 /var/log 

Mage::getBaseDir('session');

During installation you have the option of storing user sessions on disk, or in the database. The session folder is where the user sessions are written out to and read from if you choose to store them in the filesystem

The default directory (from the Magento base folder) is

 /var/session 

Mage::getBaseDir('upload');

There are a number of Admin Panel features which allow you to upload media files (default logos, etc.). Theupload folder is where Magento stores these files.

The default directory (from the Magento base folder) is

 /media/upload 

Mage::getBaseDir('export');

The export folder is where Magento will write out files meant to be viewed and used by system owners. For example, the Data Flow section of the Admin uses this folder to write out its export files.

The default directory (from the Magento base folder) is

 /var/export 

Module Base Directories

So, that covers the system folders that Magento gives you access to. There is, however, one other important static method on the Mage application class class that we should discuss. Give the following a try

?
1
2
3
4
5
var_dump( Mage::getModuleDir('', 'Mage_Core') );
var_dump( Mage::getModuleDir('etc', 'Mage_Core') );
var_dump( Mage::getModuleDir('controllers', 'Mage_Core') );
var_dump( Mage::getModuleDir('sql', 'Mage_Core') );
var_dump( Mage::getModuleDir('locale', 'Mage_Core') ); 

That static method Mage::getModuleDir method allows you to retrieve directory path information that's specific to any module loaded into the system. Like the getBaseDir method, this call is a public-api to code deeper in the Magento system

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#File: app/code/core/Mage/Core/Model/Config.php
class Mage_Core_Model_Config extends Mage_Core_Model_Config_Base
{
    /* ... */
    public function getModuleDir($type, $moduleName)
    {
        $codePool = (string)$this->getModuleConfig($moduleName)->codePool;
        $dir = $this->getOptions()->getCodeDir().DS.$codePool.DS.uc_words($moduleName, DS);
 
        switch ($type) {
            case 'etc':
                $dir .= DS.'etc';
                break;
 
            case 'controllers':
                $dir .= DS.'controllers';
                break;
 
            case 'sql':
                $dir .= DS.'sql';
                break;