Chris Hope's LAMP Blog - The Electric Toolbox
Linux Apache MySQL and PHP articles by Chris Hope
This is Chris Hope's blog for Linux, Apache, MySQL and PHP (known as LAMP) and Javascript/jQuery. I started this website several years ago with articles about web programming, Linux and Windows tips and tricks, howtos etc.
The ten most recent articles can be found below in their entirety. Navigating the sections in the right navigation (under Categories) will bring up all the other posts, and you can also use the search box at the top of the page to find what you might be looking for.
Base64 decode from the command line
Posted December 17th, 2015 in Linux/Unix/BSD
I have some automated processes that receive updates via email attachments which are base64 encoded. The entire email is backed up into a text file, and very occasionally I need to check what was in an attachment. The command line base64 tool can help with this, either decoding a file or standard input.
Using base64 to decode a file
The -d or --decode flag tells base64 it's decoding data (on a Mac -d is a debugging flag, so it's -D and --decode instead). In the examples below, I've used --decode, but you can use -d or -D instead.
To decode the file at /tmp/encoded.txt do this:
base64 --decode /tmp/encoded.txt
And the decoded file will be written to standard output. If you wanted to write it to a file, you can redirect it like this, where it's written out to /tmp/decoded.txt
base64 --decode /tmp/encoded.txt > /tmp/decoded.txt
Using base64 to decode some text
If you run base64 --decode without a file, you can type text (or copy and paste it), hit return/enter, and then control+d / ctrl+d and it will be decoded.
So, for example, to decode VGhpcyBpcyBiYXNlNjQgZW5jb2RlZAo=, which is "This is base64 encoded" encoded as base64, do this:
base64 --decode VGhpcyBpcyBiYXNlNjQgZW5jb2RlZAo= [ctrl+d]
It will then be written to standard output. Again, you can redirect the output to a file:
base64 --decode > /tmp/decoded.txt VGhpcyBpcyBiYXNlNjQgZW5jb2RlZAo= [ctrl+d]
Using pbpaste on a Mac
If you have the encoded text in your clipboard, you can use pbpaste to pipe it through base64 instead of having to manually paste in the terminal. There are equivalent commands when using X (xclip) but I've never used these myself.
Using pbpaste:
pbpaste | base64 --decode
Install Java on the command line with Debian
Posted December 16th, 2015 in Linux/Unix/BSD
If it was nice and intuitive to install Java on Debian (and other Debian based distros such as Ubuntu), I wouldn't need to write this post because it would just be "apt-get install java". But no, I always forget what I need to install, hence this post...
Install openjdk-7-jre or openjdk-7-jre-headless
If you just want to install the runtime so you can run Java applications, install either openjdk-7-jre or openjdk-7-jre-headless. The first one will install a bunch of Gnome stuff if not already installed, so if you are doing this on a server without a GUI then install openjdk-7-jre-headless.
sudo apt-get install openjdk-7-jre
OR
sudo apt-get install openjdk-7-jre-headless
Right, now maybe I'll remember this next time :)
This is correct for (at least) Debian 7 & 8, possibly older versions, probably future versions, and possibly other Debian based distributions.
Discard mail with exim4 using :blackhole:
Posted December 15th, 2015 in Email Servers
The most obvious way to discard mail to a particular alias with the exim4 mail server is to send it to /dev/null, but unless file_transport is set in the exim configuration it won't work. You can use :blackhole: instead.
/etc/aliases with /dev/null
To route mail to a particular address (e.g. noreply@example.com) to /dev/null, you would add this to the /etc/alises file:
noreply: /dev/null
If file_transport isn't configured, you'll get error messages like this in the log:
2015-12-15 15:36:14 1a8fTK-0003ip-Lp DKIM: d=example-com.20150623.gappssmtp.com s=20150623 c=relaxed/relaxed a=rsa-sha256 [verification succeeded] 2015-12-15 15:36:14 1a8fTK-0003ip-Lp <= me@example.com H=mail-qk0-f179.google.com [209.85.220.179] P=esmtp S=3709 id=CAPP3RjhN1sy074zaqTHRD_P52fAN-=y4JCdavzSrQUF4bamz6A@mail.gmail.com 2015-12-15 15:36:14 1a8fTK-0003ip-Lp == /dev/null <noreply@example.com> R=system_aliases defer (-30): file_transport unset in system_aliases router
Although there's an error, the mail isn't bounced back to the sender immediately, and sits undelivered in the mail queue (and presumably will bounce back eventually).
Use :blackhole: instead
Without having to make any configuration changes to exim, you can instead route the mail to :blackhole: - change the line in /etc/alises to this:
noreply: :blackhole:
And then you'll get this in your log file:
2015-12-15 15:39:27 1a8fWR-0003jO-M3 DKIM: d=example-com.20150623.gappssmtp.com s=20150623 c=relaxed/relaxed a=rsa-sha256 [verification succeeded] 2015-12-15 15:39:27 1a8fWR-0003jO-M3 <= me@example.com H=mail-qk0-f179.google.com [209.85.220.179] P=esmtp S=3702 id=CAPP3RjiqYfwhkBZREBWSqwtjUxELsv4JTq2kBFDpT2yQqVgk7Q@mail.gmail.com 2015-12-15 15:39:27 1a8fWR-0003jO-M3 => :blackhole: <noreply@example.com> R=system_aliases 2015-12-15 15:39:27 1a8fWR-0003jO-M3 Completed
The mail is now "delivered" and there are no errors in the log file.
Work out PHP's error reporting from an integer value
Posted December 14th, 2015 in PHP
This post has some example PHP code which can be used to work out which error_reporting levels are reported and which are not from an integer value.
PHP code
Either use the $error = error_reporting() line to work out the error reporting levels for the current setting, or use a hard-coded value, replacing 22527 in the example code with the value you want to decode. Replace \n with <br> tags if it's being rendered on a webpage, or save the output to a variable if writing to file or emailing etc.
//$error = error_reporting();
$error = 22527;
$levels = array(
E_ERROR => "E_ERROR",
E_WARNING => "E_WARNING",
E_PARSE => "E_PARSE",
E_NOTICE => "E_NOTICE",
E_CORE_ERROR => "E_CORE_ERROR",
E_CORE_WARNING => "E_CORE_WARNING",
E_COMPILE_ERROR => "E_COMPILE_ERROR",
E_COMPILE_WARNING => "E_COMPILE_WARNING",
E_USER_ERROR => "E_USER_ERROR",
E_USER_WARNING => "E_USER_WARNING",
E_USER_NOTICE => "E_USER_NOTICE",
E_STRICT => "E_STRICT",
E_RECOVERABLE_ERROR => "E_RECOVERABLE_ERROR",
E_DEPRECATED => "E_DEPRECATED",
E_USER_DEPRECATED => "E_USER_DEPRECATED"
);
echo "Report on:\n";
foreach ($levels as $level => $description) {
if ($level & $error) {
echo "$description\n";
}
}
echo "\n";
echo "Do not report on:\n";
foreach ($levels as $level => $description) {
if ($level & ~$error) {
echo "$description\n";
}
}
Example output
Report on: E_ERROR E_WARNING E_PARSE E_NOTICE E_CORE_ERROR E_CORE_WARNING E_COMPILE_ERROR E_COMPILE_WARNING E_USER_ERROR E_USER_WARNING E_USER_NOTICE E_RECOVERABLE_ERROR E_USER_DEPRECATED Do not report on: E_STRICT E_DEPRECATED
Implement placeholders in IE8 & 9 (and other older browsers) with jQuery
Posted December 10th, 2015 in HTML and CSS and Javascript
HTML5 saw the introduction of the very useful placeholder attribute for input elements, but it's not supported in Internet Explorer before version 10. There are still a number of people out there with version 8 & 9 but fortunately there are some ways to support placeholders in these older browsers with Javascript.
The old way
Before placeholders, I used to toggle default text on and off using Javascript and used an alternative field for dealing with password fields (here and here). You could still use these older methods for browsers that don't support placeholders but I found a much simpler way to do it today.
Implementing placeholders in older browsers
I don't claim any credit for the code on this page, I've just assembled it together and tested it on IE8 & IE9 and made sure it doesn't run on modern browsers.
The testing for placeholders code comes from here.
The jQuery placeholder code comes from here.
You may want to change it to suit your own use-cases or other functions you might have for detecting whether placeholders are supported (e.g. with Modernizr) or use vanilla Javascript instead of jQuery.
function placeholderIsSupported() {
var test = document.createElement('input');
return ('placeholder' in test);
}
if(!placeholderIsSupported()) {
$('[placeholder]').focus(function () {
var input = $(this);
if (input.val() == input.attr('placeholder')) {
input.val('');
input.removeClass('placeholder');
}
}).blur(function () {
var input = $(this);
if (input.val() == '' || input.val() == input.attr('placeholder')) {
input.addClass('placeholder');
input.val(input.attr('placeholder'));
}
}).blur();
}
Note that the password placeholder will still be masked and will not show whatever text you have put in there as a placeholder (.e.g placeholder="password").
Google remarketing tag iframe height issue
Posted December 9th, 2015 in HTML and CSS
Google's remarketing code inserts an iframe (with no content) directly below the opening <body> tag, which is 300x13px and can throw out a website's layout. This post looks at how to effectively hide it with CSS in a couple of different ways that hopefully shouldn't mess with its functionality at all.
Using position: absolute
Add this to your CSS and it will use absolute positioning, pushing it off the left side of page so it cannot be seen and doesn't affect content flow:
iframe[name='google_conversion_frame'] {
position: absolute;
left: -1000px;
}
Setting the height
Another method is to set the height instead:
iframe[name='google_conversion_frame'] {
height: 0;
}
I've seen posts that suggest setting display: block as well, but just setting the height seems to work OK. If you find it's not setting the hide in a particular browser, then try adding display block too.
Setting display: none?
This will work to hide the frame, but it's probably not a good idea because it may mess with the functionality of the remarketing code. Obviously the above options may too, but something needs to be done to prevent it from pushing layouts out.
Send CSV data to the browser from MySQL & PHP with fputcsv
Posted December 8th, 2015 in MySql and PHP
I covered how to use PHP's fputcsv without writing to a file in yesterday's post and look at a practical example today: fetching data from MySQL using PDO and sending it to the browser as a CSV, using the fputcsv() function.
Example data
The data used in this post is from my example table for MySQL post.
Example code
Substitute the [database name], [username] and [password] fields with the appropriate credentials and name for your database, and modify the database query to suit your use case. If you are using a different library than PDO, then obviously you'll need to change the database function calls to use that library instead.
error_reporting(E_ALL);
ini_set('display_errors', 1);
$dsn = "mysql:host=localhost;dbname=[database name]";
$username = "[username]";
$password = "[password]";
try {
$pdo = new PDO($dsn, $username, $password);
} catch (PDOException $e) {
// error handler
}
header('Content-type: text/csv');
header('Content-Disposition: attachment; filename="filename.csv"');
try {
$stmt = $pdo->prepare("SELECT * FROM fruit ORDER BY name, variety");
$stmt->execute();
$output = fopen('php://output', 'w');
$header = true;
while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) {
if ($header) {
fputcsv($output, array_keys($row));
$header = false;
}
fputcsv($output, $row);
}
fclose($output);
} catch (PDOException $e) {
// error handler
}
The example code writes out a header line containing the column names, and then one line for each record.
If you prefer to write the data out to a file, remove the two header lines and change php://output to the filename.
I'll cover how to buffer it into memory instead (for e.g. attaching to an email without having to write it out to file) in a subsequent post.
Example output
Example output from the fruit table:
fruit_id,name,variety 6,Apple,"Cox's Orange Pippin" 7,Apple,"Granny Smith" 1,Apple,"Red Delicious" 11,Banana,Burro 12,Banana,Cavendish 10,Banana,Plantain 5,Orange,Blood 3,Orange,Navel 9,Orange,Valencia 8,Pear,Anjou 4,Pear,Bartlett 2,Pear,Comice
Use PHP's fputcsv without writing to a file
Posted December 7th, 2015 in PHP
PHP's fputcsv function is very useful for creating CSV files, but by default will write out to a file. There are a couple of tricks to be able to keep it all in memory or instead write the CSV data out directly to the web browser without having to write your own custom functions.
Array to CSV function using fputcsv
This function takes an array and returns the correctly formatted CSV data as a string using an in-memory buffer. It takes the same parameters as the fputcsv function, passing them through to it, so it can be used in the same way as fputcsv.
function array2csv($fields, $delimiter = ",", $enclosure = '"', $escape_char = "\\")
{
$buffer = fopen('php://temp', 'r+');
fputcsv($buffer, $fields, $delimiter, $enclosure, $escape_char);
rewind($buffer);
$csv = fgets($buffer);
fclose($buffer);
return $csv;
}
Note that you can substitute php://temp for php://memory. The only difference between the two is that php://memory will always store its data in memory, whereas php://temp will use a temporary file once the amount of data stored hits a predefined limit (the default is 2 MB).
Here's an example:
echo array2csv(array('apples', 'bananas', 'oranges', 'pears'));
And the output:
apples,bananas,oranges,pears
Writing the array straight out to the browser
Instead of buffering the CSV data in memory, you can write it directly out to the data by using php://output instead of php://temp or php://memory:
function array2csv($array)
{
$output = fopen('php://output', 'w');
fputcsv($output, $array);
fclose($output);
}
Note that if you are writing CSV data directly out to the browser, you'll want to set the appropriate headers too.
Count rows and columns with Excel
Posted December 5th, 2015 in Applications
Want to count rows and/or columns in a range with Excel no matter what content they contain? This post shows you how.
Count rows in Excel
Use this formula, where you want the number of rows selected from e.g. A1 to A10
=ROWS(A1:A10)
This example will show 10 in the cell where the formula is.
Count cells in Excel
Use this formula, where you want the number of columns selected from e.g. A1 to E1
=COLUMNS(A1:E1)
This example will show 5 in the cell where the formula is.
What happens when running the formulas against a block?
Note that the formulas will count the number of rows or columns, no matter what the selection. If we ran both rows() and columns() on the range A1 to E10, it would show 10 rows and 5 columns.
=ROWS(A1:E10)
ouputs 10
=COLUMNS(A1:E10)
outputs 5
Why?
Yes, selecting 5 cells from A to E is obviously 5 columns, but I personally manage a very basic ad booking system using a spreadsheet that is very very wide and I need to know numbers of days an ad is booked for. Using these functions makes it a lot easier, especially when it's running across cells FB17 to GS17. No, it's not the best ad booking system, but it works :)
And I'm sure you have your own reasons for needing to use these formulas, otherwise you wouldn't have been looking for this post.
Did you run git update-server-info on the server?
Posted December 4th, 2015 in Applications
This post looks at the error "fatal: https://domain/account/filename-plates.git/info/refs not found: did you run git update-server-info on the server?" when trying to access a git repository, doing e.g. a git pull.
Possible reasons for the error
The error is git telling you the repository could not be found. This could be because:
- the repository has been deleted
- the repository has been renamed
- it has been moved from one account to another at your repository provider
- the username has been changed at the repository provider
- you've been removed as a collaborator at the project (although you might see a different type of error in this case)
How to fix it if you've renamed the repository, username, etc
If you've renamed the repository, username or similar, then you need to update the URL in the configuration file. This is at .git/config in the project's root.
The config file will look something like this:
[core] repositoryformatversion = 0 filemode = true bare = false logallrefupdates = true [remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/* url = username@host:repo-account-name/project-name.git [branch "master"] remote = origin merge = refs/heads/master
Update the "url =" line to the correct URL and it should fix the problem. You can also use the "git remote set-url origin [url]" command, replacing [url] with the actual URL instead of editing the configuration file directly.