SEO

Satan, SEO and Subdomains – VOL IV. – One To Rule MFI

Zdeněk Nešpor

SEO

One subdomain. However, absolutely essential for the continued existence of Heureka. Mobile subdomain m.heureka.cz. More than four years of work on the responsive design of Heureka with MFI at our backs.

Introduction

One To Rule MFI is the fourth part of the series of texts Satan, SEO and Subdomains. In the previous part, subtitled Controlled SEOcide, we described the war with 60,000 branded subdomains. However, in all of our previous work, we were working on something much bigger, even if it meant throwing away just one subdomain. 

Let's quickly review the four main types of Heureka subdomains:

  • Basic and System (www, blog, info) ~20 subdomains.
  • Mobile subdomain (m) 1 subdomain.
  • Category subdomains (laptops, mobile-phones, electronics) ~2500 subdomains.
  • Brand subdomains (sony, nikon, apple) ~60,000 subdomains.
  • Parametric subdomains (herni-notebooky, android-telefony, xbox-360) ~1000 subdomains.

This time we'll talk about only one single subdomain. However, it is absolutely crucial for the continued existence of Heureka. Mobile subdomain m.heureka.cz. More than four years of work on the responsive design of Heureka with MFI at our back.

Doom Is Coming

To start, let's recap the timeline of information and updates around mobile-first indexing, page speed, and page experience. 

There was quite a lot of confusion and slight panic around MFI. This was mostly fueled by Google's April 2015 update, which earned the unflattering nickname "Mobilegeddon.“1 MFI was sort of expected to be the second Mobilegeddon. It wasn't. 

In retrospect, it is actually amusing to look at some of the information. In 2016, we learn about MFI. The following years are followed by lots of resolute reports of a definitive move. And suddenly in 2021, when MFI has been somewhat forgotten, Google informs the public that actually all mobile-first sites are not ready and it will take some time. Something similar is currently happening with the move from GA UA to GA4.2 Most of these big things take longer than planned, and Google simply isn't out to sink your site.3 There's rarely any real reason to panic prematurely.

MFI Rules

Regardless of the current state of the MFI rollout, it's important to note that a mobile-first approach really matters. The base of users with mobile devices continues to grow, and it's important to reflect this. If you still have a separate mobile version, below are such MFI SEO minimums that we have addressed the most. 

  • The desktop and mobile versions should be plus or minus the same in terms of internal linking.
  • They should also be the same in terms of content.
  • Have identical structured data and metadata.
  • The mobile version should have the correct canonical set to the desktop version.
  • The mobile and desktop versions should have rel="alternate" set correctly.
  • Both versions of the site must be accessible to Googlebot.

However, nowadays it is a good idea to get rid of the mobile version and move to a fully responsive design. However, it is still not a good idea to rush responsive. It's better to stick with separate versions or just the desktop version for a bit longer. And do the responsive design properly and in good quality. It doesn't pay to rush it. 

The Road to Responsive

Making Heureka responsive has been a pretty solid nut to crack and a longer run than anyone originally expected. Nerves were running high for everyone involved. Almost half of the time, the bogeyman of MFI was knocking on our door. And the other half was on our backs as we waited to see when things would go wrong and how badly.

We threw the progress into a simplified timeline, listing most of the major milestones on the road to responsiveness. 

  • 2017 – start of responsive development.
  • March 2018 – new responsive detail shop launched.
  • April 2019 – new responsive categories deployed on m.heureka.cz.
  • June 2019 – deployed new responsive search on m.heureka.cz.
  • September 2019 – new product detail on m.heureka.cz running at 50% traffic.
  • September 2019 – we cancel parametric sections on subdomains.
  • October 2019 – MFI active.
  • January 2020 – device detection and automatic redirection between mobile and desktop.
  • June 2020 – we are removing brand corners and categories on subdomains.
  • February 2021 – desktop Heureka displays responsive design for 80% of traffic.
  • May 2021 – redirect mobile search to responsive desktop.
  • June 2021 – redirect product details to responsive design desktop.
  • June 2021 – redirect mobile homepage and categories to responsive desktop.
  • June 2021 – end of the mobile version of Heureka.
  • July 2021 – present -> continuous debugging. 

Development and full deployment took approximately four years and succeeded less than two years after the arrival of MFI. Four fairly significant things and countless other major or minor events interfered with the process.

Into the Clouds

Various technological changes stretched the length of the responsive launch. The gradual move to a cloud-based solution. And also the rewriting of all services and parts of Heureka to microservices. We're talking about major and complex changes that can't be done in a few days.

You can read more in the articles below. Most are in CZ only, please, use DeepL or some other translator if you are interested in the topic. 

For the People

A significant evolution of the company was the shift from traditional KPIs toward the OKR methodology. Furthermore, the structure was also changed by dividing it into tribes and dividing the product areas more logically. Such changes bring many positives in the long run. In the short term, however, there may be a period of slight confusion. In other words, it takes a while for things to "settle down".

All of this is documented in the articles below. Most are in CZ only, please, use DeepL or some other translator if you are interested in the topic. 

Beyond the Borders

During the same time period, the development of OnePlatform began to be addressed. That is a single backend and frontend for all sites in the nine countries of the Heureka Group. So two giant development projects were running side by side.

More about OnePlatform is in the articles below. All are in CZ only, please, use DeepL or some other translator if you are interested in the topic. 

To the Bats

As much as we hate to, we have to mention COVID-19. The arrival of the coronavirus and the pandemic changed things on many levels. Moving people to the home office and a bunch of calls is probably not news. People have moved more into the online world. The constant lockdowns, releases, and various measures meant a total "breaking" of analytics, which wasn't so easy anymore. It was fairly common to see government measures hitting our testing and evaluation. Often it was not at all clear whether the decline or growth was due to our work or to a wild covid situation.

Below is an article on how we handled the home office. All are in CZ only, please, use DeepL or some other translator if you are interested in the topic. 

Web Wilderness

You may have guessed from previous articles that even Heureka itself was not very clear from an SEO perspective. Besides, there were often multiple versions of Heureka in parallel. Some ran on production, some in (one of many) A/B tests. There were several staging environments. As a result, there were usually five or more versions to test side by side. 

  • The original desktop Heureka with the original parametric sections and brand corners, seen by bots.
  • A partially reimplemented desktop Heureka with new parametric sections and original brand corners, which we serve only to users in the A/B test, but not to search engine bots.
  • A partially responsive mobile Heureka, which we serve only to users in the A/B test and only show the responsive homepage and category to search engine bots.
  • A partially responsive mobile and desktop Heureka, where we automatically redirect users to a responsive desktop, but which the bots cannot see and may not be redirected.
  • Plus new stuff on staging. More tests etc. 

You get the idea. Some things were tested by adding special parameters to the URL. Like https://www.heureka.cz/?display=old. Then in later stages by using cookies to solve the (non)display of responsiveness or some A/B test. Later on, automatic redirection of users to the responsive desktop version was added, but still not seen by bots. 

Testing with conventional SEO tools made most of this impossible. So we were sitting at the PC with a pull, with various "hacks" on what and how to display. Over and over again, manually testing scenarios and verifying that the bot was actually seeing only what we needed. And often nothing could be evaluated anyway. After running the test, the weather completely changes in the spring, people head outdoors and we're looking at a 15% drop. Or a lockdown is declared and a 15% increase follows. Did the test work? Or did it fail? Is there some external factor to blame? If we repeat the test, then the two months of work that we're two months overdue is stretched out for another two months that we don't have. What follows is exhaustion, resignation, and within expert estimation and experience, you hope for a good result.

Before MFI

Before the MFI came along, we tried to spread enough awareness, train colleagues, analyze potential impacts, and fix bottlenecks on the site. The biggest enemy is stress and fear of the unknown. Realistically, no one had any idea what might happen after the site moved to MFI. And when exactly that would happen. 

MFI

We received the notification about mobile-first indexing for Heureka on October 23, 2019. The information was confusing right from the start, because the GSC had reported on the same day that the move had actually happened on October 1. For us, this meant that we had probably picked up over 3 weeks of delay.

At GSC, we noticed right after that that there was a huge drop in indexed URLs since the first of October, from 230 million to 78 million URLs. 

We sat up until 3 am at analytics. Nothing. The impact of MFI was nowhere to be seen. There's been no drop in indexed URLs either. 

One of the main signals of MFI was reported to be bot activity. Specifically, traffic from mobile bots was supposed to be (significantly) higher than desktop googlebots. Which didn't happen. During the period in question, the mobile Googlebot (turquoise on the graph) made up only a third of the traffic. On the other hand, we had been getting signals over the previous twelve months, where there was a gradual increase from one-tenth of the traffic to the aforementioned one-third.

In the final analysis, nothing happened. Only the mobile Googlebot activity slowly increased. No impact on traffic. But we wouldn't wish that stress on anyone.

After MFI

We weren't out of it. Responziv was still not finished and work was going on intensely. While we hadn't seen any impact up to that point, it was unclear how long we had before any problems would manifest themselves. The stress continued and any number of problems could have occurred on any given day. The worst part of the move to responsive followed after the MFI. A large amount of A/B testing and debugging took place in the time period from January 2020 to June 2021. All were influenced by the aforementioned onset of covid, which also impacted the e‑commerce world.

The New Direction

Fortunately, the mobile version only featured categories, product details, and internal search. The previously described parametric sections and brand corners were not there at all. 

In addition, at the time of 301 redirects, there was virtually no traffic on the mobile version, as all users had long been automatically redirected to the responsive desktop version, which we knew worked reliably. We, therefore, focused most of our attention on access logs and bot behavior.

The redirection of the mobile subdomain itself was relatively clean and seamless. The URL patterns were more or less the same on the mobile and desktop versions, which made the process relatively easy. 

We redirected the internal search first. There was no problem with this. 

m.heureka.cz/search -> www.heureka.cz/?h[phrase]=

Second, in order was the product detail. What we hadn't completely counted on was that Google would interpret this as a reason to stop crawling the rest of mobile Heureka. The graph below shows the Googlebot activity after the product detail redirect. The left part shows the normal activity across mobile Heureka. Increase in activity after redirection. And the subsequent decrease in crawling of all URLs except redirected product details.

So we quickly redirected the rest – categories and homepage. Interestingly, Googlebot then stopped visiting the redirected mobile URLs quite quickly. We were expecting a bigger onslaught of requests and a longer period of gradual decline, but it didn't happen. We mentioned something similar in a previous article.4 

It's very likely that Google is quite good at working with different patterns, and using some sort of "interpolation" mechanism, it can simply figure some things out without having to visit all URLs.

Besides, it's possible that in the case of the mobile version of the site, the correct canonical and alternate tags played in our favor. So by the time, the redirect pattern was clear, the bot already had enough information to not have to test all known URLs.

Finally, we add a couple of articles that describe redirection from a different perspective. All are in CZ only, please, use DeepL or some other translator if you are interested in the topic.

Lessons Learned from Travels

Lesson 1: Keep a cool head. This is easier said (or written) than implemented. There are some things that can't be influenced or accelerated. Throw in some cool stuff, otherwise you're going to go crazy with big website edits. And, of course, have plenty of documentation and arguments on hand to guide the company. 

Lesson 2: Google is not the enemy. Google has been fighting spam quite intensely for a long time and tends to be uncompromising on a lot of things. But in the case of useful services and well-known brands, it has a somewhat higher tolerance. 

Lesson 3: The indexing and data slump at GSC. Google has had and continues to have problems displaying the right data in the GSC. Especially for very large sites. Don't believe everything you see and don't panic.

Lesson 4: Custom tags. If you need to tag something, it is possible to create a custom tag or attribute. For example, <meta rel="my-test" value="yxz">.

Lesson 5: Test properly. It pays off. Better to get a little stuck in SEO and not have everything perfect. It's more important to have a quality usable website. Because without that, you'll lose more users than you will because of some SEO deficiency. 

Lesson 6: Move traffic upfront. By shifting traffic gradually, you can partially eliminate the potential negative impact of sudden SEO changes. 

Travel Gear

You may be wondering what tools we used in the course of this work. Here is a brief list.

  • Screaming Frog
  • Excel
  • Google Sheets
  • Gephi
  • Google Analytics
  • Kibana
  • Google Search Console
  • Marketing Miner
  • Collabim
  • Valentina Studio
  • Redirect Tool – internal custom-made URL redirection tool

Conclusion

One subdomain gave us more trouble than anything else. Not so much in terms of SEO. There wasn't much to deal with there. We just needed to keep an eye on a few tags and redirects. But mentally, those years were very challenging. And it's 100% true that you can't do this kind of work in one person. You need a smaller team that can somehow hold each other up and support each other.

Series on SEO and subdomains

Disclaimer

Approach the text with caution. This article and the entire series are not intended as a guide. The texts do not contain any "universal" truths. Each site represents a unique system with different starting conditions. An individual approach and perfect knowledge of the specific site and the subject matter are required.

The article describes our website. We do not evaluate the general effectiveness of subdomains or directories. Nor do we recommend any specific solution. Again, this is a highly individual matter, influenced by many factors.

Strategies and detailed plans for some of the activities described here have been in the works for over a year. Everything has been discussed, tested, and validated many times. Please keep this in mind when you do similar activities yourself.

Some of the data presented may be inaccurate and purposefully distorted. Specific numbers such as organic traffic stats, revenue, conversions, and the like, we don't plan to leak out for obvious reasons. However, key information such as subdomain counts, URLs, and our practices are presented truthfully without embellishment.

The text may contain advanced concepts and models that are not entirely standard in SEO. The articles are therefore supplemented with footnotes with sources where everything is explained in detail.

Footnotes:

  1. Mobilegeddon https://en.wikipedia.org/wiki/Mobilegeddon. ↩

  2. Delay end of measurement to GA UA for Analytics 360 users https://blog.google/products/marketingplatform/analytics/turn-insights-roi-google-analytics/. ↩

  3. Except perhaps YMYL (Your Money or Your Life) and EEAT (Experience, Expertise, Authority, Trustworthiness) https://static.googleusercontent.com/media/guidelines.raterhub.com/en//searchqualityevaluatorguidelines.pdf (chapters 2.3 a 3.4). ↩

  4. See note 12 for the article https://www.heurekadevs.com/satan-seo-and-subdomains-iii-seocide. ↩

Author

Zdeněk Nešpor

SEO

Technically focused SEO specialist and webmaster. 

<We are social too/>

Interested in our work, technology, team, or anything else?
Contact our CTO Lukáš Putna.

lukas.putna@heureka.cz