Last month I took over support of one project that inspired this post. It is a website with couple static pages, news and galleries that was created with Joomla. The problem was that site owners could not support it. They are not computer savvy and the internal structure of admin panel is way too complex for them to understand.

Funny enough, I would have gone to completely opposite direction with that site design. Based on site requirements, I would install WordPress and code additional functionality. WordPress handles many things better than Joomla:

• Image upload and resize to sane sizes
• Article/news organization
• Simpler admin interface

And a lot of clients do not need anything more – they do not want to resize media prior uploading, or searching for missing plugins to do that resize on upload time. Also, content and organization is tragic in Joomla, and it is really difficult to comprehend for person that does not work with such systems daily.

Now Joomla has its own uses and I would not create complex sites on wordpress. However, how often developers think about customer needs instead of their development speed and “Power” of the application? Many of us want to show off with powerful options, lots of customizability as we cannot brag about nice code inside to the customer. Often it is not what customer needs.

Thus it is important to write down requirements of project before putting everything on most powerful CMS you know. Think what user wants to see and make it easy for user to navigate through admin panel to do the tasks he needs.

2. Ability to integrate various things he learned in different environment. Few programmers learned MVC or frameworks at universities here. However, they have learned FSMs or some other paradigms that can be successfully applied in web development as well. Few try it though (Note: knowing more than one programming paradigm is a bonus).

Sometime we get a new inexperienced guy in a company as a programmer. Typically, we test them beforehand; however they are rarely perfect when started. Sometimes they are very new to programming or have bad habits from previous programming jobs. Thus it is my goal to help them to improve. And sometimes it gets really hard to integrate the guy into company. However there are some criteria that allow recognizing people with potential on becoming programmers and ones that are better left behind.

1. Persistence and knowing how to learn. Even more, urge to try and learn new things. I love programmers that know how to look for information on a problem.  Everyone can do a task they learned, but the best programmers know where to look for an answer and never give up without a good reason. The best way is to give a task they have no experience in and look how they are trying to solve it. If a person gives up prior googling for an answer, he will not be a programmer.
2. Ability to integrate various things he learned in different environment. Few programmers learned MVC or frameworks at universities here. However, they have learned FSMs or some other paradigms that can be successfully applied in web development as well. Few try it though (Note: knowing more than one programming paradigm is a bonus).
3. Ability to create a concise logical algorithm for solving a problem. You can be entry level web programmer without that, but that’s it.  Ability to draw these using some popular schematics like UML state diagram is a bonus.
4. Ability to debug code of other person. A programmer is unable to learn if he can’t dive into code of someone else and understand how it works. Sooner or later you will have to implement a similar algorithm where you will not be able to use that code as it is. This is closely related to ability to create algorithm.
5. Ability to write code other people understands. I loved one programmer that wrote nice functions for everything while coding a trivial task in PHP. Also, he commented a lot. Sadly, his interest was not web programming.

Quite often I am hired to perform modifications in other web projects done by someone else.  This might be an unmaintained website bought ages ago, or when programmers mess up. It is tough to analyze your own code after couple of years from creation. It is much harder to analyze code of someone else. There are so many different programming styles. And most often the code is documented poorly. So, how to do it properly and save the time?

1. Gather all documentation of code if it is available. What framework or library was used? Was a CMS used? If so, you are in luck as these are more or less documented. Not always, though. If the basis used is branded as “software”, the documentation is typically lacking. It is assumed that you don’t need to modify ready to go software.
2. Try to map code on some known framework/pattern/approach. If there is clear separation from business logic, database tables and visual representation, you can try to approach the code using MVC. If there is separation between data processing and representation, then it is template based approach. Maybe the system is hook based? This will help to understand how programmers think. Worst kind of code is when business logic calls representation logic which calls some different business logic and so on. Then it might be a mess, especially if you can’t get why programmers decided to do so. Usually there is a different separation taking place between first and second layers of business logic, though sometimes this can be called “quick hack to output specific info” only.
3. Try to identify file deployment structure and write it down. Where are configuration files, templates, classes located? What is structure based on?
4. Draw a basic request scheme on paper. What files/modules the request passes till result is send to the user? This will help you to better understand the software.
5. Identify places you need to make modifications. The best help is simply stopping execution in various places of file and seeing what data you have at these places. Die(), echo and print_r are the best friends.
6. Stick to commenting out old code and commenting changes you make. Make it easy to undo your changes, and make it easy to understand what you have done.
7. Last, but not least, test everything.

Today I want to talk about really unproductive topic – that is how to code poorly. Or, more exactly, I am going to pick on my unfavorite  piece of social site again, that is Boonex. We will talk about how one should not implement user sessions and how to do it better.

Boonex uses unencrypted cookies to store session data, that is member ID and hashed member password. Each request Boonex checks database for memberID and password combination. This is to ensure that the person would be logged off the system if password is changed by someone else.

Thus, what is the problem with this approach?

Well, what about making hashed passwords save on PC user logs in? These passwords remain available till user explicitly logs out.  And the hash used is a simple md5 hash which is fast one-way function. You cannot reverse it, but it is susceptible for dictionary or brute force attacks. There are databases for such md5 strings as well.

You could partly solve that problem by using a seed with the hashed password. However, it would make the password guessing game more complex but not impossible. Also, you do not need the password for being logged in, just the cookies. And it is quite simple to change values in them with Firefox.

Another problem we had is with effectiveness of this password checking. It is one of multiple requests that are done on each page load. Even when they are fast, they affected boonex performance, which is quite bad anyways. But caching this information makes a security hole open:  ID could be changed and your system could be compromised.

It is much safer to save session information on server side.  PHP native session management is much better than what boonex has to offer, thus you should switch to it in the code (I have demo version of boonex  7 on August, it still has this problem).   It is quite some work, but it is worth it. Or, even better, look for some other social network script.

I had read a post at sitepoints about 16 php 5 frameworks. I will not argue the fact that frameworks are good and you should use one. What saddens me is that the key points of comparison are omitted in the post.

So, how to compare different php 5 frameworks? Let us see.

1. Paradigms used. Although MVC is mainstream now, it is not a single paradigm in php development. There are frameworks that are event based only. Additionally, there are different paradigms for different application layers, for example CRUD for database (model) layer, pretty well supported in php cake. Zend does not have module layer at all, you need to use your own database access, as example.

2. Modularity and reusability of framework. Although MVC provides modularity on controller- basis, additional modularity is required to re-use code between applications. This is important for developing bigger project that share modules.

3. Php version support. Some frameworks require specific versions of php5. This means that you have limited options for hosting.

4. Shared hosting support. You might need that before project kicks in. Some frameworks behave weirdly on shared hosting, as they need specific setup.

5. Proof of concept and maturity. Have you seen applications developed with the framework? Is framework developer community active or dormant? I would shun a framework without some real-world example sites.

Have something to add? Comment bellow!

My blog post is inspired by a meeting where I had a role of mediator between web developing company and customer. The issues were quite simple and common: increasing costs due to neglected specifications and some badly described features.

Customer and web development company are in the same boat, although this fact is often forgotten once it is time to show the work and pay the money. The typical problem is lack of signed specifications. There are several reasons why specifications were not prepared:

1. “The project is too simple for specifications” FALSE – there is no project too simple for specifications
2. “The project is too complex to specify it in one go” FALSE – There should be specifications made for separate parts of the project and compiled into whole document.
3. “xxx development model does not requires specifications because it is superior” FALSE. I hear this quite often from “fans” of Agile-alike development model. Although the whole spec is not prepared, you have to write down what are you making right now.
4. “The customer will not understand the specification” YOUR PROBLEM – customers are not dumb. As long as you use human language, they will understand you. The goal of specification is to explain for both sides what to build and what to expect
5. “They haven’t asked for one!” They need one, though. And in fact, you need a specification too. I have seen customers using awkward terminology that can be very confusing for people outside their company. The web project specification helps in cases client refuses to pay because they expected something else.

Typically, lack of specifications is assisted by pay-per-hour payment model. This is quite useful short term for the company which wants to make money from the customer, but is quite bad long-term strategy. Specification saves time used for redoing project, which leads to customer frustration. And frustrated customer is not a customer that is happy to pay and hire you again.

Furthermore, good explanation of things between client and developer saves lots of time that would be used to develop too complex and completely useless things. During the discussion I mentioned above customer had to explain to developer lead that all they needed for particular attribute was a simple manual checkbox instead of complex automated state machine (and couple other problems). That would not have happened if they took care to update a specification.

For 4 years I work on some image heavy projects. I started clueless. Now, when I look at first works they are not so good, though I have seen far worse implementations of such sites. Such sites have quite a lot common bottlenecks that can be solved using some of the “design patters”.

The top problems in no particular order are as follows:

1. Disk space
2. Throughput
3. Speed 
4. Memory

Although disk space is cheap nowadays, it is still important for couple reasons, namely in shared/vps hosting it is limited, or when considering backup solutions.  So you should not store images bigger than needed, though you will need creating various formats used for site.

These additional formats are needed to save from one of the biggest bottlenecks in the image heavy websites, that is image processing. I have seen lots of websites that scale the images dynamically on each image view. That’s flawed approach. You will need lots of processing power, and the pages will always load slower. Converting images should be done once, either on upload or on specific event.

Should you save original images is up to you, but I would recommend stripping unnecessary information from them and compressing to ~70% quality or so if they are JPEG. That would not impact their visual appearance too much, but will save lots of space. In some cases this is not applicable.

You should never scale images using image tag attributes or styles, or the image will look like crap. Also, this will not save bandwidth.  Always create formats required for site, in the worst case you will regenerate images when you need them.

Another thing related to speed of image site is its disk access. Although there are RAIDs, on very busy sites that’s not enough. So sooner or later you will have to think how to serve images from another server than dynamically generated content (php scripts/db). This can be done in various ways, for example using network disks mapping your infrastructure, though the best way around is using separate subdomain for static content. 

There is one more quite unexpected load -eater as well. That is logs. I would not recommends switching logs off, but you should minimize the amount you write to error log. We managed to reduce loads by 1/3  by just fixing single banner with wrong URL. Use full URIs if you have to. But check the logs after you code, please. 

There is another reason having statical content served on separate sub domain. The reason is memory. A typical apache process takes around ~30-50m. And it serves both complex php scripts and tiny icons used in design. So, using lighthttpd or similar simple and very fast web server for static content is quite useful. Then you can focus on servers disk qualities than memory amount.  It is even useful keeping these 2 webservers running on same physical server through 2 IPs. On the long term you will notice significant load decrease, limited by disk access alone.

It is quite strange that most of the commercial server setups do not take this into account. Most usually, control panels use single web server (aka Apache) for hosting all the contents and you have to setup everything manually.

Boonex is an assembly of social site php scripts including chats, forum and the site itself. As we work on several social sites this year, we have chosen boonex as starting platform to build things upon and kinda regret it now. 

I wrote a post about newscloud a year ago and about pligg. Well, boonex is shittier than pligg and as newscloud as well. Why ? Lets tackle some of its problems

Boonex problem nr1. No coding standard 

Boonex is writen by several people using different technologies. Its main base (Dolphin) is writen in pure php with its own template engine and forum  (Orca) uses xlst. That has large negative impact to integration of forum in site and site in forum. When coding, please use single technology and template system. 

Whats even worser, different parts of dolphins code itself is writen by completely different people, and very in the hurry. So everyone has its own imagination how to interact with different parts of the code. It is very tricky to modify code that way to suit site needs. 

Boonex problem nr2. Template engine and separation of code/design/database

This problem in dolphin/boonex needs a point on its own. Boonex uses custom template system ( that should be called layout system). The blocks of generated html code are pased to specific places in the template. That creates a big headache for programers as they need to search through code for the place where some box is generated. It might be generated in template, or in specific function in one of numerous includes. And so on. Even pligg has better templating than this. 

It is very tricky to rip boonex templating apart, as whole coding is based on such poor programming practice. They would be better off using existing template system like smarty or similar one. 

This leads to problem nr3

Boonex problem nr3. Crapy use of Database

We saw serveral other competing sites launched on boonex, but we did not care much. Why? Becouse when they reach 500-1000 daily visitors they will break apart. The reason for it is very bad programming and use of database. 

For example, boonex uses profile builder which assigns fields to profiles. So the output of profile uses more than one table and is quite inefficient. 

Also, there is a nice 20-30 query overhead on each page display to fetch all the configuration values from table. Silly, isn’t it ? They would be FAR better of using a file for configuring sofware or caching it in php file like it is done in most of the systems. 

Another simple problem. When boonex wants to display a profile being online it additionally check database for its status. But that data was pulled from database already. So 20 useless queries again. 

Resume

I would not suggest using boonex if you want to keep your programmers sane. We have have changed the code almost completely for now, and you will need to do that too.