This chapter looks at various contributors to insecure code. These range from cases of simple misinformation to simple forgetfulness. Many common scenarios are shown that can be identified and changed to better practices and habits.
Anti-Pattern #1
Not Matching Data Character Set to Filter Character Set
Mismatches between the character set of the data being parsed and the functions performing the parsing are a systemic, root level problem. If web security, based in the scripting environment of PHP, JavaScript, MySQL, and HTML, is based on how characters are interpreted, then care must be taken from the start to ensure that a string of user-supplied text is comprised of the expected character encoding, and every filter and sanitizer operating on that data set should use the expected character encoding. The rules and the data have to match before anything else, or the rest does not matter.
The practice of not specifying and not ensuring character set uniformity is wide-spread and largely ignored. It is a bad practice perpetuated by numerous examples that rely on function default parameters that do not match actual use. Because of character set differences, code that works correctly in one environment will not nec-essarily work correctly in a different environment of different data sets with differ-ent defaults. For example, when the character set of the web page does not match the default settings of the PHP environment as set by php.ini, or when attacker-supplied data specifically alters a character set to bypass filters that do not enforce character set matching.
This book emphasizes UTF-8, but the security demand remains the same no mat-ter which characmat-ter encoding needs to be used. If ISO 8859-1 needs to be used in the application, then ensure that the text is valid IS0 8859-1 and that all application data filters are set to internally process ISO 8859-1. If Windows-1250 needs to be used, then likewise ensure that all text and filters conform to Windows-1250.
The foundational basis of secure programming rests on properly parsing and exam-ining data. This can only be done when the character set/encoding of the supplied data matches the character set of the filters used. Mismatched data is the root of much evil.
Take action. Explicitly set the character sets used by the application processes.
1. Decide what character encoding should be used.
2. Ensure all internal functions, filters, and structures are configured for the chosen character encoding.
3. Ensure user-supplied data is comprised of the chosen encoding.
4. Convert or drop data that does not conform.
Not Designing with Content Security Policy Anti-Pattern
Content Security Policy 1.0 (CSP) is a W3C Candidate Recommendation. Most major browser vendors have adopted it. CSP is the new weapon against XSS and other client side attacks because it only allows whitelisted scripts to execute. This gives excellent control over security measures. The second, very powerful feature of CSP is that is completely disallows inline javascript to execute. This is the only way to prevent injection attacks. Inline scripts and Javascript event handlers must be relocated to an external file and whitelisted. This can make it difficult to retrofit an application with poor separation of concerns (SoC).
The new best practice, with major security gains, is to architect with CSP from the beginning, and to use SoC effectively. There is an online chapter, Secure Developement with Content Security Policies that covers building PHP/Javascript pages using CSP, available at: http://www.projectseven.net/secdevCSP.htm
One Size Fits All Anti-Pattern
Every engineering circumstance has its own particular need to be addressed.
Sometimes this is response time, sometimes scalability, sometimes enhanced secu-rity. With regard to PDO-prepared statements, there is no doubt that using them is a best practice. The benefit of automated escaping provided to the developer and to the development process is too great to ignore. Forgetfulness is a key component of security holes. The code in this book uses PDO::quote() instead of PDO::prepare() in some places for the purposes of speed optimization. Usually this is the case where high transaction requests are made. The price of using PDO::quote() is the possibil-ity of forgetfulness. When PDO::quote() has been intentionally used, the reasoning for it has been included. Should your reasoning or circumstances be different, use PDO::prepare(). It is a better choice and practice in most cases. Feel free to disagree.
My intent is to give enough information to make an informed choice.
Misinformation Anti-Patterns
Misinformation is a common contributor to security problems. People ask security questions on forums, and in many cases the answers given are based on opinion, or
“I think this works.” These answers may be copied into production applications over
and over until they become de facto standards. This presents a problem for at least two reasons. First, advice given without technical verification is a poor solution. A security solution is something that needs to actually be tested and verified. Opinion is not good enough. Second, a bad habit, once formed, is difficult to break. Copying bad examples leads to bad habits, which is self-perpetuating.
One important example is this. A simple Google search found several available instances, i.e. 1–3 pages of listings, of advice explicitly advising to “Turn off SSL peer verification via the false parameter in order to make curl SSL work,” with no additional advice on how to make it work with a secure setting of TRUE. This is troubling because SSL Verify Peer is a critical confirmation action. It is not a convenience. SSL is not about encryption and proper identity verification. It is unwise and not secure to turn off verifica-tion and confirmaverifica-tion of whom your applicaverifica-tion is talking to. As a result of this advice, the following line of code has been copied, reposted, and re-implemented many times:
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
The negative effects of this are several. First, it cancels peer verification altogether, which is a main purpose of SSL certificate verification, and second, it allows for man-in-the-middle interception attack. Because it is encrypted, these facts are easily for-gotten. One forgets that there is a TRUE setting that is needed. One stops seeing this as a problem. It becomes a bad habit to use this code and just switch it off without checking.
The Mantra Anti-Pattern
A common mantra these days is “Always use PDO prepared statements.” While there is some truth and some benefit to this saying, it isn’t completely helpful. Implementing PDO prepared statements wherever possible is undeniably a best practice. The real-ity is that they can’t be used in every situation. For example, PDO prepared state-ments cannot accommodate variable columns, and so cannot be implemented in this case. Legacy code cannot use PDO statements. Without understanding the problems solved by PDO prepared statements, or alternative defenses, security problems will persist when workarounds are required. It would be nice if security problems could be put in a box, and one “always use” solution worked, but this is not the case. Mantras simply promote complacency and work against understanding.
Another common mantra is “Always use a framework.” This borders on useless, equivalent to “Build all homes the same way, with the same architect, and the same blueprints.” The magnificent freedom of development does not rest on working with or depending on a third-party library. Frameworks cannot be leveraged in all cases.
Also, a current project cannot be retrofitted with a framework simply to overcome an immediate problem. Frameworks are not perfect and must be understood well and implemented properly to benefit from whatever protection they might provide. That task is not trivial. There is a significant learning curve involved. That said, frameworks
are indeed powerful and highly useful libraries of reusable tools. It is advisable to learn at least one framework well because they solve many common implementation issues, which makes them valuable and rewards the time invested in learning them.
However, this does not excuse a developer from understanding the issues solved, and knowing how to proceed correctly without one.
Another reason for learning a framework is that Zend Frameworks, Yii, Symphony, WordPress, and others have implemented very capable, context-aware security filters that are organized by name and context type, which allows them to be leveraged appro-priately in securing an application. These filters are powerful tools worth knowing about.
The last bad example of a mantra is “Always use function X” with either bad or insecure default parameters given. A common example is: “You only need to use htmlspecialchars("$data") instead of htmlentities($data) to avoid encoding all characters,” and this advice is often taken at face value without making adjustments to use the correct parameters for the actual data type used in the environ-ment. Both these functions, depending on environment, can be very insecure with default settings as shown. Depending on application, the default of ENT_COMPAT may not be sufficient since it will leave single quotes unencoded, and the default char-acter set of PHP may not match the actual supplied data passed in. The result is improper filtering leading to an incorrect result.
Critical Data Type Understanding and Analysis
One of the most widely perpetuated misunderstandings is the notion that there is a single method to clean data. There is not. Instead, consider the following case of a poor, single data type sanitization process, and an explicit, multidata type sanitization process applied to incoming data.
Single Data Type Anti-Pattern
There are many postings on the web advising how to clean data by treating all data as the same, both in type and in purpose. When all data is treated the same way, a devel-oper loses control over the process. Here is one example of a poor solution posted on the web. It treats data as all the same and applies improper filters for the task at hand.
A Poor Security Filter Application function cleanID(){
$i d = mysql_real_escape_string(intval(strip_tags ($_GET['id'])));
return $id;
}
$id = cleanID();
$result = mysql_query("SELECT name FROM users WHERE id = $id");
The cleanID() function applies three filters to a variable meant to be an integer in an effort to make it safe. Because $id is meant to be an integer, implied by the application of intval(), then only the intval() function is necessary in this case because actual number characters are not dangerous and do not need to be escaped.
If $id was by design comprised of alphanumeric characters, such as 456BBC, then $id would need to be treated as a string, and intval() could not be applied.
mysql_real_escape_string() works on strings, hence the name, and escapes potential SQL command characters according to the character set interpretation of the open MySQL connection. It makes sure ISO interpretation is applied to ISO data, or that UTF-8 interpretation is applied to UTF-8 data. mysql_real_escape_
string() has no effect whatsoever on integer values and is open to exploitation if the variable value is not additionally quoted inside the SQL string. See the example in the next section, “A Surprisingly Safe Implementation.”
strip_tags() is useful for removing tags from a variable when the variable, by design, is not supposed to contain HTML tags. It cannot be counted on to guarantee security, and has no effect on the integer ID value in this example. Developers have a responsibility to understand how and why a mechanism works, so let’s break it down.
First, because the variable, $id, is intended for use as an integer, the sanitization process is quite simple. The following code, using just one sanitization technique, is completely safe because the untrusted input is converted into a integer, therefore no interpretation anomalies exists, and the SQL is unaffected. This explicitly converted integer can be safely inserted into the query without escaping because it is now a com-pletely benign set of number characters (0–9).
A Surprisingly Safe Implementation
$id = intval($_GET['id']);
if ($id > 0)
$result = pdo->query ("SELECT name FROM users WHERE id = $id");
This statement is secure. No escaping or quoting is needed because of the explicitly converted integer. An actual integer does not need to be escaped for MySQL.
While the two statements above are secure if used together, as in this one case, this is not recommended, nor is it a best practice, because it is open to mistakes. Anytime intval() is forgotten, a large security hole is opened because the $id variable is not quoted, or escaped, inside the SQL statement.
It is important to understand why it is safe in this case. The value is made safe because it is explicitly converted to an integer, consisting of characters (0–9), and the underlying integer bits cannot harm the SQL by causing the statement to be misin-terpreted by the SQL engine compiler. Again, an actual integer does not need to be escaped for MySQL.
This knowledge can be useful when it needs to be wielded by necessity for per-formance reasons in high transaction environments because the above statement
is the fastest implementation of a query. A speedy legacy equivalent to the PDO implementation would be:
$id = intval($_GET['id']);
$result = mysql_query("SELECT name FROM users WHERE id = $id");
Explicitly casting to an integer type is also safe. A cast to an integer in PHP is done like this:
$id = (int)$_GET['id'];
$result = mysql_query("SELECT name FROM users WHERE id = $id");
The output is:
SELECT name FROM users WHERE id = 55
After the cast, $id is a numeric integer and no longer a string representation.
Any part that is not numeric is removed. Quoting and escaping are not needed as long as the parameter is indeed an actual integer. Again, not a best practice, but important to know and understand. The lack of quotes here is a security hole whenever the value is a string. Adding quotes here is defense in depth, which is a best practice.
The latest best practice, when applicable and possible, is to instead use prepared statements. The reason is not because it is a more secure escape method, but because they automate the process of escaping. It is the automation which helps prevent secu-rity holes by preventing accidents of forgetfulness. If all queries are implemented as prepared statements, and one is singled out and converted to a straight query for per-formance reasons, then that is the benefit of an applied best practice, an optimization easily made without compromising security elsewhere.
Strings Change Everything A developer should consider the functions as they were meant to be used in a given context. Consider the case where the parameter is an actual string value, and not an integer value. For example, a name most likely does not have a need for HTML tags, so remove them with strip_tags(). This is done not for security purposes, but because the design specification says HTML tags shouldn’t be part of the name.
A Legacy Best Practice
//The name string should not contain HTML tags, remove them per spec
$name = strip_tags($_GET['name']);
// This string now needs to be properly escaped for output into the database
$name = mysql_real_escape_string($name);
//make sure variable is quoted as well as escaped
$result = mysql_query("SELECT id FROM users WHERE name = '{$name}'");
The output is:
SELECT id FROM users WHERE name = 'BumbleBee'
A PDO Query Best Practice The name string is quoted and escaped via PDO quote(). Visual inspection is easier because of no embedded quotes in the SQL string. The quotes are produced in the output as part of what PDO::quote() does for you.
//The name string is quoted and escaped
$quotedName = "SELECT id FROM users
WHERE name = {$pdo->quote($name)}";
$result = $pdo->query($quotedName);
The output with a quoted parameter:
SELECT id FROM users WHERE name = 'OptimusPrime'
A Current Best Practice Approach
//ensure only safe ASCII characters if(ctype_alnum($_GET['name'])) {
$name = $_GET['name'];
pdo->prepare(SELECT id FROM members WHERE name = :name");
pdo->bindvalue(":name", $name, PDO_STR);
pdo->execute();
}
The above code has three levels of defense. First, ctype_alnum() ensures that in the untrusted string, $_GET parameter, name, only it contains the given characters a–z, A–Z, 0–9, which are safe. Second, a prepared statement is used, so that the user input cannot be combined with the SQL. The third level of defense is the explicit treatment of the data as a string type in the bound parameter.
Alternatively, the code below is not safe, even though the SQL similar.
$_GET['id'] = "46; DELETE FROM members";
$id = mysql_real_escape_string($_GET['id']);
$result = mysql_query("SELECT name FROM members WHERE id = $id");
First, note that the usage of $id remains a string, and not an explicit integer like the previous example. Here, the bad SQL is not escaped by mysql_real_escape_
string(), which properly, but ineffectively filters the id variable. The root problem
is the treatment of $id as a string and as an integer. Since $id is unquoted, and the DELETE keyword is allowed through, the resulting SQL statement is now turned into two SQL statements.
'SELECT name FROM users WHERE id = 0; DELETE FROM users';
Notice that mysql_real_escape_string(), by design, was unable to elimi-nate this threat. There was nothing to escape. The input submitted was valid SQL.
For direct comparison, if $_GET['id'] = "46; DELETE FROM members";
This is safe:
$id = (int)$_GET['id'];
$result = pdo->query("SELECT name FROM members WHERE id = $id");
This is not:
$id = mysql_real_escape_string($_GET['id']);
$result = mysql_query("SELECT name FROM members WHERE id = $id");
The difference lies in the explicit casting to an integer, the ineffectiveness of mysql_real_escape_string() for this attack, and the lack of quotes surround-ing the variable $id in the SQL statement.
The following defense in depth is safe because PDO::quote() escapes and quotes the variable, which in this case was converted to an actual integer via the cast.
$id = (int)$_GET['id'];
$result = pdo->query("SELECT name FROM members
WHERE id = pdo->quote($id)");
To prevent a security hole in this case, two things must always be remembered: explicitly convert the value to an integer, and quote the value inside the SQL statement to prevent statement alteration. The quoting is not necessary for explicit integers but is mandatory for strings, or the string representation of an integer. PDO prepared statements solve these con-stant implementation problems. This is why it is a best practice to use them wherever possible.
If the variable, $id, had been quoted, the resulting single SQL statement would have been:
'SELECT name FROM users WHERE id = "0; DELETE FROM users"';
which would not have matched an ID. Internally, MySQL would have converted the string "0; DELETE FROM users" to an integer for comparison.
Note: There is one area where prepared statements offer a higher degree of protection.
In true prepared statements, where two server calls are made, where the actual SQL
statement is compiled first without user-supplied variables, it is safer because untrusted input cannot ever then alter the SQL logic. In emulated prepared statements, where the untrusted input is automatically escaped first, then compiled with the SQL statement,
statement is compiled first without user-supplied variables, it is safer because untrusted input cannot ever then alter the SQL logic. In emulated prepared statements, where the untrusted input is automatically escaped first, then compiled with the SQL statement,