Blog's revival. Angular 7 and Server-Side Rendering

Angular blog - bad indexing

Finally, after a bit of intense work, the Web Mystery blog has been updated to the latest version of Angular with Server-Side Rendering support. This will allow search engines to index pages more effectively and hopefully will bring the website back to Google.

2024 Update

The blog has been migrated to Astro. The article and the blog itself are kept for historical reasons.

How it began

Back it 2016, when all the buzz around Angular 2 reached its peak and my main work project was in Angular JS, I was looking for an opportunity to learn the latest version of the framework. Besides, I had this thrilling idea about giving my blog smooth single page experience, and Angular 2 seemed quite a right tool for this. Google assured that it can index single page application equally well to traditional multi-page Drupal website. So, I thought I would be great to do two things at once: make the blog more visually and functionally attractive and learn the latest Angular framework–which, since it was built by Google, was supposed to be Google friendly and have no impact on indexation.

It was a horrible mistake.

Upgrading to Angular 2 and Headless Drupal

At the beginning, everything went well. I added Services and Services Views modules to turn existing Drupal views into REST endpoints, found another module–Services Views Totals–to add pagination metadata, and everything seemed to work fine.

When it came to Angular, it was a time when first beta was just released. Even though the documentation was in place, bootstrapping the application seemed to contain a lot of hacks–can’t recall now why exactly, but it did not feel right at all. I managed to get through all the issues and made a working prototype, however I was not inspired enough for good design, so I postponed the further works.

Next time I returned to the project, it was when the second or third release candidate came out. Surprisingly, the Angular Router was completely rewritten, so I had to go through all my code and correct everything according to the new documentation. Changes happened in unreleased software, but should they introduce them in a RC? People were not amused.

When works on the new blog was finished, the stable Angular 2 version was out. There was already Angular Universal package around (which should take care of server-side rendering), but after struggling with it for a day and having almost no progress, I decided to postpone adding it to the next update. The blog already seemed pretty, and Google assured that it’d handle indexing, so I just rolled it out to the production.

The Post-upgrade Period

Right after update, Google seemed to hardly notice it. Old pages remained in index, including the junky ones–like taxonomy term pages, and that was my mistake–I should have excluded them in robots.txt. New pages did not want to show up–even though they long unique and human-oriented content. It was a sad evidence that Google crawler’s abilities are not that all-mighty when it comes to indexing single page applications.

As months passed, old pages were slowly dropping out of index, and eventually it ended up with only front page remaining, which–according to Google’s cache–froze in loading state. The crawler did not wait until the actual contents would be loaded via XHR Request and skipped pages as empty.

It seemed that all my struggling with new technologies was in vain, and the typical, time-proven Drupal website served much better than over-advertised modern Angular–with its TypeScript, Observables and other hipster stuff.

After a while, I upgraded Angular to 4, but it did not change a thing.

The Most Recent Upgrade

During the last couple of months, I had a luck to work on a React project with server-side rendering based on the Electrode platform. It brought me a lot of new knowledge and gave me confidence that I can finally bring SSR to my blog. So, I wiped the dust from the old repository and started working.

Before diving into SSR, I decided to upgrade to the latest Angular version–at the moment of writing it is 7.2.0. It was a pleasant surprise to find update.angular.io, which clearly instructed what steps I should take to move from Angular 4 to Angular 7. Indeed, third-party modules also had some API changes, but, luckily, I managed to fix all errors within a couple of days.

Next, the Angular Universal. The universal module, which is now called platform-server, has become a part of Angular based project. The documentation is pretty detailed, but it still took me a couple of days before figuring out why things did not work on my project. Turned out that I should remove intervals when processing on the server–otherwise it wouldn’t return any response, and replace all JSONP calls as they are not available on the server-side http-client implementation. There is a simple way to pass Express’s Request and Response objects as optional dependencies in the Angular application, as well as checking the platform during runtime, so I could easily determine whether I can use intervals or access DOM API.

One more pleasant thing is that a lot of common things can be done with Angular CLI–such as adding server-side rendering functionality to existing project or turning it into PWA. It installs required NPM packages and adds some boilerplate code that can be modified and tuned to fit your needs.

At the moment of writing, a few pages already went back to Google’s index, which is definitely a good sign. Cached versions of them say that the crawler now sees the full HTML content, and hopefully, all of the articles will return to index with time.

Issues

Even though, the Angular platform seems now mature, there were a few issues on server platform that should be mentioned:

  • Bad documentation on asynchrony and server-side HTTP-requests. Gathering the pieces of knowledge from the web and doing some experiments, I managed to get initial HTTP-requests work on server and not repeat them on the client (thanks, `ServerTransferHttpStateModule`).
  • No build-in way of caching requests. The content of blog articles does not change often, and I would rather not make an API request each time someone accesses the site. I solved this by adding Redis caching using `express-redis-cache` module–after all, this is still an express application that can use any third-party middleware. It improved the response time quite a lot.

As a Conclusion

It was an interesting experience–sometimes it was great, sometimes not so. Still, Angular has become much better since initial release, earned new features, and feels much more useful, but there is still space to improve.

I have some interesting stuff in mind to write about, and now–when my upgraded blog is finally ready–feeling quite motivated. See you soon!