In Defense of the WordPress Theme Review Guidelines

Filed in Web DevelopmentTags: , ,

Recently, the official WordPress Theme Repository has been the subject of much discussion within the WordPress community, thanks to the formation of a Theme Review team and the implementation of a stricter set of Guidelines for acceptance of Themes into the Repository. Some have complained regarding the unreasonable length of time required to get a Theme through the review process and accepted into the Repository. Other, more vocal critics have decried the new Guidelines as overly strict, stifling innovation, creating a barrier-to-entry for new Theme developers, and actively driving away existing Theme developers.

The most contentious matter is, of course, the Theme Review Guidelines. While some in the community have embraced the Theme Review team's efforts and principle regarding improving the overall standard of quality of Repository-hosted Themes, others have found the Theme Review Guidelines to be "stifling", a "barrier to entry" for new Theme developers, and even "clueless".

So what's really going on? Let's take a look at this most contentious subject: what constitutes quality with respect to WordPress Themes.

Commercial vs. Free Themes

At this point in the development of WordPress and the developer and user community that surrounds the project, free Themes - and especially those hosted in the WordPress Theme Repository - have developed a reputation for being inferior to their Commercial counterparts. As best as I can discern, the advantages that Commercial Themes provide can be summarized as follows:

  • Quality
  • Documentation
  • Support
  • Advanced Features

While much of this list is outside the control of the Theme Review team, the one that the team can most directly (and immediately) impact is, in fact, Theme quality.

Why Quality?

Especially now that Themes can be installed and updated from the Repository, all from within the WordPress admin interface, the Repository is essentially an extension of WordPress itself. Thus, the quality of Themes in the Repository reflects directly upon end users' perception of the quality of WordPress itself.

To be fair, Commercial Themes are not immune to criticism - to which the recent discussion regarding ThemeForest can attest. As Jeff Chandler points out, the criticism against ThemeForest parallels the criticism against the WordPress Theme Repository. Consider the following quotes, as applied to the Theme Repository:

Comment:

I really think the issue is about quality and support. Although there are many very qualified authors on [the repository] putting out high quality, well supported themes, there is also a lot of junk.

[...A] lot of them are great looking. I’ve [downloaded] several for just that reason. But of the 10 of so themes I’ve [downloaded] from [the repository] over the past year, only 1 is actually in use. The rest of them, once I started working with them, ended up being too limiting in one way or another, had serious coding problems, conflicts with plugins or other, major customization roadblocks and very little to no support from the author.

I’m a fairly geeky person who can muddle through figuring out how to customize themes myself, most of the time. But I’m not a coder, so there’s a limit to what I can do. I think there are a lot of people like me. Woo and StudioPress and other premium theme authors provide support for their themes that allow us middle of the road users to get where we want to go. [The repository] mostly doesn’t.

Maybe...the support will improve. But I’m much more leery about [downloading] a theme from [the repository] than I used to be. [The repository] is cheap, but you get what you pay for. I spent more money for a theme from StudioPress and guess what? It worked right out of the gate, I could customize it easily, and any questions I had were promptly answered by the support team.

Comment:

The only thing that is letting [the repository] down is that there’s no quality control. Maybe a quality control system should be put in place or some sort of star rating system, for example one person with a fair bit of theme creation knowledge and who knows how to create a theme properly could examine the themes and give them a rating.

Comment:

[M]ost of the themes there are...crap... poorly coded themes by wannabe themers.

Comment:

This is what I’ve seen with [the repository], and while not a sweeping generalization, it’s too common to be rare:

* Lack of updates. Some of the really successful theme authors update their themes to add features, fix bugs or address new WordPress functions. A whole lot more don’t. The problem with this is that the theme looks great. The demo works. Joe Consumer might not realize that because it uses old functions or does things in a certain way, it’s not going to work with newer WordPress features or some plugins.

* JavaScript Soup. Now, this isn’t a [repository] problem, this is a THEME problem. ...I see this all the time. Hell, my own website...needs to have its JS significantly refactored, I’m not immune — but my stuff also doesn’t break anything else.

* Plugin Incompatibilities – This is usually directly related with problem 1 — lack of updates or fixes.

The big advantage that places like Woo Themes and Studiopress and Thesis have over places like [the repository] is that there is a central point of support contact. If something breaks on a Woo theme, I know they’ll release an update. Their business depends on that happening. If there is a conflict with a popular plugin, they are going to figure out what is going on.

[The repository] isn’t the same way.

Comment:

I won’t personally ever [download] a theme from [the repository] because I’m a theme developer myself. But, I do deal with end users that have gotten burned on more than one occasion by themes from [the repository]. Holding your theme authors to certain levels of coding standards and best practices would be a good start.

Comment:

One thing I agree with is that the quality of themes should be stepped up...

Every one of these comments was describing ThemeForest - but every single one can bbe equally applied to the WordPress Theme Repository. Look hard enough, and you can find identical sentiment expressed about the Repository.

What is Quality?

The difficult question is: what is quality? More importantly to the current discussion: how can the Theme Review team define quality in an objective manner that can be interpreted the same by both Theme developers and Theme reviewers? For example:

ThemeGrade Test Criteria:

  • W3 HTML/CSS Compliance
  • Browser Compatibility (Firefox/Chrome/IE7)
  • Theme Support
  • Image Alignment
  • Clearing Floats
  • Content Width Auto-Adjustment
  • Threaded Comment Support
  • Hierarchical Page/Category Sidebar Lists
  • Sidebar Content Width Auto-Adjustment
  • Special Effects (Features/Functionality)

Another Site's Criteria:

  • Design
  • Ease of Use
  • SEO
  • Features
  • Support
  • Price

Here are some of ThemeForest's common "rejection factors" for their hosted WordPress Themes:

  • Documentation
  • Typography (Aesthetics, Line-Height, Text Alignment)
  • HTML/CSS Validation
  • List Markup
  • Browser Compatibility
  • License

So which of these is correct, or best (or at least closest)? From what I can gather thus far, the consensus opinion regarding Theme quality can be distilled down to this response from Leland, of ThemeLab:

XHTML/CSS validity, general code cleanliness, design not looking like crap, using wp_head/wp_footer hooks, basic stuff I guess

But, how to translate all that into objective Guidelines for performing Theme reviews?

Theme Review Quality Standards

The current Theme Review Guidelines can be summarized into five main areas:

  1. Licensing, Credit Links, and Security
  2. Code and Markup Standards Compliance and Best Practices
  3. WordPress Functionality and Feature Support
  4. WordPress Best Practices
  5. Theme Front-End Display

The current Guidelines are derived from these main areas of concern.

Licensing, Credit Links, and Security

The primary reasons that Themes undergo any review in the first place fall into this category. An astounding number of submitted Themes has non-GPL-compatible licenses or restrictions, SEO/spam footer links, or security exploits of one form or another.

Licensing

The Theme Repository Guidelines have, since the inception of the Repository, stated that Themes hosted in the repository must be compatible with WordPress' GPL licensing. This requirement includes all Theme template files (PHP), style (CSS), markup (HTML), scripts (JS), images, icons, fonts, and anything else bundled with the Theme.

Some Themes indicate that they are licensed under GPL, but knowingly include restrictive notices such as a prohibition against modifying the Theme footer, removing credit links, or modifying/redistributing the Theme. Some Themes unknowingly bundle icons (such as FamFamFam's great icons) or fonts that do not have GPL-compatible licenses.

Whichever the case, one of the most important safeguards provided by the Theme Review Guidelines (and Review) is to ensure that the user freedoms protected by the GPL are not restricted by Themes downloaded from the official Theme Repository.

Credit Links

The visibility and sheer download/usage numbers of Themes hosted by the official Theme Repository have turned the Repository into a magnet for Link Spammers ever since the first Theme was submitted. Users do not want their Themes to be riddled with Link Spam, nor do they want their Themes to be used to advertise for all manner of legitimate and illegitimate websites.

For this reason, the Theme Review Team has established fairly strict, intentionally rigid requirements for public-facing links in Themes, and for the credit-link URLs Theme URI and Author URI, which are displayed by the Repository. In order to ensure fairness to all Theme Developers, the established requirements are as objective as possible - intentionally leaving little room for subjectivity.

Security

Likewise with Link Spammers, the visibility and sheer download/usage numbers of Themes hosted by the official Theme Repository have turned the Repository into a magnet for Black-Hat types looking to propagate security exploits. The nature of Themes (and Plugins, for that matter) cause them to be an attractive attack vector, since a Theme, once activated, has free reign over every part of WordPress and its database. Additionally, poor and/or obsolete coding can create exploitable security vulnerabilities.

For this reason, Themes submitted for inclusion in the Repository must be reviewed for intentional exploits (worms, etc.) and unintentional vulnerabilities.

Code and Markup Standards Compliance and Best Practices

While differences of opinion exist regarding the importance of the correlation, or the extent of code validity/cleanliness required, there does appear to be some consensus within the community regarding the correlation between clean, valid code and Theme quality.  Thus, the Theme Review Guidelines include requirements regarding HTML/CSS validation, PHP errors, JS errors, and the use of deprecated WordPress functions.

HTML/CSS Validation

The ubiquitousness of valid markup (HTML/CSS) has been well-established for some time now. Most developers are not unfamiliar with using the W3 HTML and CSS validator tools to ensure their code conforms to current standards. (And W3 have made this task even easier, with a unified validator: Unicorn.) So, this requirement hasn't been viewed as terribly controversial.

Note that the Theme Review Guidelines do allow for legitimate exceptions. For example, both WordPress core and the Theme Unit Test XML data output markup that does not pass the W3 validator. Some of these errors can be eliminated for testing purposes using the FixPress Plugin. Nevertheless, these types of validation errors are ignored during the Theme review. Also, browser-specific CSS properties for draft CSS3 specs, such as border-radius and box-shadow (e.g. moz-border-radius or webkit-box-shadow) are likewise ignored during the Theme review.

PHP/JS Errors

By far, the most controversial Theme Review requirement is the prohibition against PHP errors, warnings, and notices - particularly with respect to warnings and notices. However, in my experience reviewing Themes, the vast majority of these warnings and notices are for undefined indices (not putting function arguments in quotes), undefined variables, and undefined offsets - all of which are quite easily fixable. Also, while such warnings and notices do not impact the rendering of the site for visitors, they can be incredibly noisy in the site's error logs - which can make searching the error logs in order to find legitimate errors rather difficult.

Finding such PHP errors is as simple as using the Debogger Plugin, and viewing output generated for both the Theme's front-end and admin options pages.

Deprecated WordPress Functions

Probably the most important code-related standard is the prohibition against deprecated functions. Whereas both modern browsers are well-suited to overcoming markup validation and PHP parsers handle errors, warnings, and notices with little impact to the end-user, the use of deprecated WordPress functions represents true obsolescence - and, therefore, both functional issues as well as potential security concerns - within Themes.

Fortunately, deprecated WordPress functions are almost always simple to replace. In fact, WordPress core even outputs the replacement function to use. Finding calls to deprecated WordPress functions is as simple as using the Log Deprecated Notices Plugin (hint: it works for Plugins, too!), and viewing its output in the admin UI, under Tools -> Deprecated Calls.

WordPress Functionality and Feature Support

Since the official WordPress Theme Repository is, essentially, a very public showcase of WordPress Themes, it is important that repository-hosted Themes support core WordPress functionality and features. However, this need must be balanced against the intended use cases for each Theme. The Theme Review Guidelines attempt to maintain this balance by indicating which major core features are Required, Recommended, or Optional.

Required Features

Required features must be included in repository-hosted Themes. Themes will not be accepted into the repository unless they support these features and implement them properly. Currently, the list of Required features is (intentionally) very limited:

  • Automatic Feed Links
  • Widgets
  • Comments

Recommended Features

Recommended features are not required to be included in repository-hosted Themes, but are strongly recommended. Themes will be accepted into the repository with or without support for these features, but if these features are supported, they are required to be implemented properly:

  • Navigation Menus
  • Post Thumbnails
  • Custom Header Images
  • Custom Background
  • Visual Editor CSS

Optional Features

All other features are considered optional, and are not listed in the Guidelines. However, as with the Required and Recommended features, any Optional features that a Theme supports are required to be implemented properly.

WordPress Best Practices

The official WordPress Theme Repository is intended to be used not only by WordPress end users installing Themes for their own use, but is also intended to be used by nascent WordPress developers who want to learn how to develop WordPress Themes - whether for their own use, or for distribution. For this reason (and due to the public visibility and widespread use of repository-hosted Themes), it is important that repository-hosted Themes adhere to some WordPress standards and best practices.

These standards and best practices include support for the most recent WordPress version; the use of core functions, hooks, template tags, and styles; the use of standard Theme template files; and Theme documentation.

Support For Most Recent WordPress Version

Repository-hosted Themes are required to support the version of WordPress current at the time the Theme is submitted for inclusion in the repository. This support includes incorporation of new features and functionality, as well as replacement of newly deprecated functions. In order to give Theme Developers sufficient time to incorporate new features and other changes necessitated by new versions of WordPress, this requirement takes effect one month after each WordPress version release.

Since each new WordPress version release represents potentially major changes to Theme Review Guidelines, the Guidelines will be updated with each WordPress major version release, to include any major revisions to the requirements. A Version-Specific Changes section will be added to the Theme Review Codex page, to highlight any major revisions to the Guidelines.

Use of Core Functions, Hooks, Template Tags, and Styles

In order to facilitate their use as learning tools for new developers, as well as to ensure that they work as well as possible and support as many Plugins as possible, repository-hosted Themes are required to support core WordPress functions, hooks, and template tags. This requirement encompasses both the use of core functionality rather than customized functions, as well as proper implementation of such functionality.

This requirement ensures a consistent experience from Theme to Theme for end users who choose to customize their Themes, as well as ensures that the most common hooks will exist for Plugins that depend on them.

Use of Standard Theme Template Files

All Themes are required to include certain files (index.php, style.css) in order to be considered a "valid" Theme by WordPress when installed. Repository-hosted Themes are required to include screenshot.png in order to display a preview screenshot when displayed in the repository. Repository-hosted Themes are also required to support comments and must implement this feature properly; therefore, they are required to include comments.php. All other Theme template files, if included, must be called by the Theme using the proper template tag.

Theme Documentation

Themes are encouraged to be a minimalist or as complex as the Theme Developer wishes to make them. However, if customizations, configuration options, custom template files, etc. are included (or if the Theme is designed or optimized to work with specific Plugins), the Theme is required to include user documentation explaining how to use all of the Theme's various customizations and options.

Theme Front-End Display

Back-end (e.g. code) reviews do help ensure a certain quality level, but what is most important is the front-end experience. Repository-hosted Themes are expected to provide a certain degree of user-experience consistency for visitors to the sites using them. For this reason, the Theme Review process includes the Theme Unit Tests: Themes are installed on a test site using a standard XML data set, in order to test navigation, layout, readability, HTML tag and CSS rendering, image handling, and various usability criteria.

Arguments and Complaints Against the Theme Review Guidelines

Having explained the reasoning behind the current Theme Review Guidelines, let's examine some of the arguments and complaints expressed regarding the Guidelines. Some of those complaints include:

  • The Guidelines represent a "barrier to entry" for new WordPress Theme developers
  • The Guidelines are too extensive/detailed/specific/strict
  • The Guidelines stifle design creativity/innovation
  • The Guidelines change too frequently

The Guidelines Represent a "Barrier to Entry" for New WordPress Theme Developers

Some have argued that the current Guidelines make WordPress Theme development more difficult for new Developers.

To the contrary, I would argue that, due to the extensive Codex function/tag cross-referencing, the Guidelines actually provide an incredibly useful and educational tool for new Theme Developers. Not only can new Developers learn the more important WordPress functions, template tags, and hooks, they also have direct links to the Codex entries, so that they can learn how to implement them.

The only way that the Guidelines might represent a "barrier to entry" would be if they are poorly written, and difficult to understand. Thus, it is certainly incumbent upon the volunteers who comprise the Theme Review Team to ensure that the Guidelines are clear, concise, and understandable. To that end, we will work continually to ensure that the Guidelines meet this standard.

The Guidelines are Too Extensive/Detailed/Specific/Strict

Some argue that the Guidelines are too strict, and too imposing, with "Required" this and "Required" that listed everywhere.

I believe this complaint represents a misunderstanding of how the Guidelines are written. In order to ensure objective, fair reviews, the Theme Review Team decided that the Guidelines would be written per RFC 2119. This way, the criticality of each guideline would be clearly understood, as required, recommended, or optional. The intent was not to create the perception of being imposing, but rather being as consistent as possible in describing each guideline's criticality. We intentionally decided to limit criticality adjectives to required, recommended, and optional, so as not to introduce potential confusion from similar, synonymous adjectives.

Further, the extent and strictness of the Guidelines hasn't really changed; rather, they have just been stated more explicitly, and enforced more consistently. The Theme Unit Test requirements have changed very little, and many of the rest of the requirements are actually derived from the Theme Unit Tests.

The Guidelines Stifle Design Creativity/Innovation

Some argue that the Guidelines have too many required criteria, and in so doing stifle design creativity and innovation.

I would counter that good innovation is not found in poorly written code, or in "reinventing the wheel". Further, I would counter that the Guidelines provide an excellent baseline from which Developers can focus less on basic Theme functionality, and more on design creativity and innovation.

The Guidelines Change Too Frequently

Initially, when the Theme Review Team was first getting itself organized, and getting the Guidelines ironed out, this complaint may have been considerably more valid. However, at this point, the Guidelines have had no major changes for several weeks (bearing in mind that this whole effort started less than four months ago).

Also, aside from minor clarifications, the Theme Review Team has determined that major revisions to the Guidelines will take place in synchronization with the WordPress release cycle. There are exceptions to this rule; for example, if we discover an egregious security oversight, or an incorrect requirement, we will make an immediate change.

Major changes (if any) will be introduced with each major WordPress version release, and will take effect one month after the WordPress version release. Further, these changes will be summarized in a dedicated "Version-Specific Changes" section in the Guidelines, so that Developers will know what changes may be coming.

If you have read the Guidelines recently, you will also notice that several things indicate that they are "draft" status. This is an attempt to "dry-run" changes to the Guidelines, for input/feedback. Using this approach, developers will generally always have an idea of proposed changes to the Guidelines.

Room For Improvement

Is there room for improvement? Always! And we welcome all constructive criticism and feedback.

As Theme Reviewers, we don't gain anything by making more work for ourselves, by needlessly failing Theme reviews that will only lead to the same Theme being reviewed multiple times. It is in everyone's best interest to have more Themes pass the review process. So, if a specific guideline is incorrect, inappropriate, inaccurate, or otherwise in need of revision, we want to know about it!

But at the same time, at this point, the Guidelines are fairly stable, and fairly reasonable. Generalized complaints about the Guidelines as a whole will be neither terribly helpful nor especially well-received. Try to focus your concerns to specific issues regarding specific requirements, and we would be happy to receive such feedback. In fact, the Guidelines have only been able to be refined to the point that they have currently, thanks to such feedback from Theme Developers.

Feedback

Comments (Comments are closed)

9 Responses to “In Defense of the WordPress Theme Review Guidelines”
  1. chip_bennett says:

    In Defense of the WordPress Theme Review Guidelines – http://www.chipbennett.net/2010/10/20/in… #wordpress