hmazter.com

Web and application development

2019-04-02

Speeding up PHP Docker with OpCache

After spending some time trying to figure out why my PHP Application was so much slower running in Docker locally then running directly with PHP-FPM on my machine. I finally realized that the base Docker image I was using did not have OpCache installed which I'm used to it is with local PHP. I was using the Official PHP image based on Alpine Linux with PHP-FPM

Installing OpCache

It's a single step required to install OpCache in the image. Using the following line in your Dockerfile

# Dockerfile
RUN docker-php-ext-install opcache

Tweaking OpCache settings

I wanted to tweak the settings for OpCache to both allow for more files in the cache, since Laravel and Symfony's vendor directories can grow quite large. And also disable the re-validation of files in production.

To do that I wrote a opcache.ini file that I included in the image. The file includes a environment variable to enable validation of files for local development.

# Dockerfile
ENV PHP_OPCACHE_VALIDATE_TIMESTAMPS="0"
ADD opcache.ini "$PHP_INI_DIR/conf.d/opcache.ini"
# opcache.ini
[opcache]

; maximum memory that OPcache can use to store compiled PHP files, Symfony recommends 256
opcache.memory_consumption=192

; maximum number of files that can be stored in the cache
opcache.max_accelerated_files=20000

; validate on every request
opcache.revalidate_freq=0

; re-validate of timestamps, is set to false (0) by default, is overridden in local docker-compose
opcache.validate_timestamps=${PHP_OPCACHE_VALIDATE_TIMESTAMPS}

opcache.interned_strings_buffer=16

opcache.fast_shutdown=1

When running docker-compose locally i include the environment variable PHP_OPCACHE_VALIDATE_TIMESTAMPS: 1 for my app container.

# docker-compose.yml
services:
  app:
    environment:
      PHP_OPCACHE_VALIDATE_TIMESTAMPS: 1

2018-09-25

Using database views as tables for Laravel Eloquent

I found myself in a situation where I had a legacy database which I was going to build a new Laravel application against. The database did not have a naming convention that played nice with Laravel and it did include a lot of tables and columns that was not in use anymore.

After some Googling and reading about how to alias column names in Laravel, I got the idea "How about putting a database view on the base table and use that as table for the Eloquent model". I started with some basic benchmarking to see if it was even a viable solution.

Benchmarking

I did some very simple benchmarking by creating a Laravel app that had 2 model, one directly against the database table and one against a database view containing a subset of the columns. Filled the database with 10 000 rows and fetch with the model based on the table, the model based on the view, using the DB facade to fetch from the table and the view.

Here is the repository if you want to check out what i did.

Benchmark result

php artisan fetch-posts-from-view  0.50s user 0.07s system 97% cpu 0.583 total
php artisan fetch-posts-from-table  0.50s user 0.07s system 98% cpu 0.575 total
php artisan fetch-posts-from-table-query  0.16s user 0.06s system 95% cpu 0.227 total
php artisan fetch-posts-from-view-query  0.17s user 0.07s system 93% cpu 0.251 total

I ran those a couple of time and they gave it pretty much the same result every time.

Performance Conclusion

Using those numbers I think my idea of using a view for the models in Laravel is okay from a performance perspective. There are other things that has a lot of more impact then just that, for example how the query is written, the hydration of objects, etc.

Migrations

Since database views only live "in memory" and is not permanently created (but they do live trough server restarts), I needed a easy way to create the views and change them over time without having to put the whole definition in a migration file every time.

I landed in a solution where I wrote a basic artisan command that takes all files from a folder, in this case database/views that I created, and creates a view with the name of the file and the select statement described in the file.

// app/Console/Commands/DbCreateViews.php

foreach ($this->filesystem->files() as $filename) {
    $viewName = str_replace('.sql', '', $filename);
    $viewContent = $this->filesystem->get($filename);

    $this->line("Creating database view $viewName");
    DB::statement("DROP VIEW IF EXISTS $viewName");
    DB::statement("CREATE VIEW $viewName AS $viewContent");
}

Example view file:

# posts.sql

SELECT postid          AS id,
       posttitle       AS title,
       postcategoryid  AS category_id,
       creationdate    AS created_at,
       lastupdated     AS updated_at,
       removeddate     AS deleted_at
FROM tbl_post

Summary

I pretty happy with the outcome from this. It seems that its no performance overhead for the views. Looks like Laravel has no problem using a database view for a model. Ad it simplifies the table and column naming a lot.


2018-08-11

Move from wordpress to static site with Jigsaw

I have read about the static site generator Jigsaw the last couple of years. And after seeing a lot of hype around Netlify lately, I was thinking it could be a good idea the convert my blog running on Wordpress to a static site with Jigsaw. Jigsaw builds on Laravel components and uses the templating engine from Laravel; Blade. Since I'm mostly do my web development in with Laravel, this was a good choice for me. Jigsaw also supports Markdown for pages, which is good since I wanted to write the actual blog posts in Markdown. Seeing Michael Dyrynda's post and video about getting Jigsaw up and running on Netlify I took that as a sign that now was the time to do it.

Getting started

I started with the Jigsaw installation and installed and initialized a new folder

mkdir hmazter.com
cd hmazter.com
composer init
composer require tightenco/jigsaw
./vendor/bin/jigsaw init

Export existing posts and convert to markdown

I don't have that many posts on this site, but those I have I wanted to keep on the new site. After some searching I found the post about converting the exported xml from a wordpress site to markdown: https://prefetch.net/blog/2017/11/24/exporting-wordpress-posts-to-markdown/

I exported the xml from my site, used pip to install Pelican to use pelican-import and tried the command from the post. Problem arouse! pelican-import wanted pandoc installed on the system to handle the markdown conversion. I installed it, but the version of pandoc that was installed with brew was not compatible with the version of pelican-import. I found a workaround in a Github issue, to modify the source and install it from there.

Cloned the Pelican github repo and updated pelican/tools/pelican_import.py as follow:

-        parse_raw = '--parse-raw' if not strip_raw else ''
-        cmd = ('pandoc --normalize {0} --from=html'
-               ' --to={1} -o "{2}" "{3}"')
+        parse_raw = '-raw_html' if not strip_raw else ''
+        cmd = ('pandoc  --from=html'
+               ' --to={1}{0} -o "{2}" "{3}"')

And then installed with python3 setup.py install. After that, the command to convert the posts to markdown worked!

In the posts there is also parts to automatically fix the front matter in the markdown. But since i only had 9 posts I thought it would be easier to just add and edit the parts needed by hand

Layout, post template and post collection

I created a very basic master layout, source/_layouts/master.blade.php and a template for the posts to extend source/_layouts/post.blade.php. Then created a collection in the config to hold all blog posts.

// config.php

return [
    'collections' => [
        'posts' => [
            'path' => '{date|Y/m}/{slug}',
            'sort' => '-date'
        ],
    ],
];

What this does is takes all files from the folder source/_posts, sorts them on the variable date in the front matter and outputs them with a path in the format year/month/slug. I used this path format to match the format I used on my Wordpress site.

The posts is written in markdown, extends the post layout, that in turn extends the master layout.

After I had a working version of the blog with post I added a simple, and according to me, elegant design with Bulma.io.

I did all this with yarn watch running to get all assets auto updated, but also the static site rebuilt and reloaded in the browser.

Pagination of posts

On my previous wordpress blog the index page was a pagination of all the posts and I wanted similar behaviour here. That was pretty straight forward, Jigsaw includes pagination functionality. Just specify which collection and pagination count on a page and the loop over the result.

---
pagination:
    collection: posts
    perPage: 3
---

// ...

@foreach ($pagination->items as $post)
    <h1 class="title">
        <a href="{{ $post->getUrl() }}">{{ $post->title }}</a>
    </h1>
    // ...
@endforeach

I also added a widget to the sidebar listing the latest posts.

Deploy to Netlify

Now I hade enought of a site to deploy it for testing. Signed up at Netlify with Github. Pushed the code to my (then private) Github repository. And created a "New site from Git" in Netlify.

During set up of the new site a specified the build command to yarn production which works similiar to the previous mentioned yarn watch that it builds the assets (css and js) in production mode this time and also generated the static site. The output from the build is placed in the folder build_production, that is our "Publish directory" in the set up of the site.

Environment

From what I read, Netlify defaults to PHP version 5.6 for builds. Jigsaw uses Laravel components that requires at least PHP 7.1.3. We can tell Netlify which PHP version we want with the environment variable PHP_VERSION. This variable can be set in the settings for a site or using the configuration file netlify.toml. For me it is always a good idea to have "infrastructure" configuration in the same place as the code, which means I go with the toml-file:

[build.environment]
  PHP_VERSION = "7.2"

Custom domain name

Time to point hmazter.com at the netlify-site.

This took some fiddling around. I use AWS Route53 for DNS and ended up with this:

Name Type Value
hmazter.com. A 104.198.14.52
www.hmazter.com. CNAME hmazter.netlify.com

When this was set and propagated trough DNS and enabled and forced HTTPS on the site with the one-click in Netlify.

Now this blog is served as static files, build with Jigsaw and hostad at Netlify.