The latest version of the RGESN (General policy framework for the
ecodesign of digital services) was released over a year ago : https://www.arcep.fr/uploads/tx_gspublication/general_policy_framework_for_the_ecodesign_of_digital_services_version_2024.pdf [PDF, 2.8 MB]
It contributes greatly to the practice of ecodesign for digital services as we know it. However, the reference framework is not perfect. We have already discussed this in another article [FR]. Among other things, it is often criticized for being largely declarative (focusing on means rather than results).
Since then, I have started to list RGESN declarations on my website: https://ldevernay.github.io/green/2024/09/02/rgesn.html [FR]
The aim is to collect examples for those who would like to undertake an RGESN audit or who have doubts about the interpretation of certain criteria (which often happens). There are currently around 70 declarations, including around 50 based on the 2024 version of the standard.
The time has therefore come to embark on an experiment:
- Start with the list of RGESN declarations based on the latest version of the standard (50 in total)
- Launch measurements on the corresponding home pages
- Look for a correlation between the RGESN progress scores and the measurement results
It should be noted that almost all of the declarations today focus on websites.
We will see later that some scores are overestimated and the correlation with the metrics is difficult to establish. While it remains a formidable tool, the RGESN is not enough. But let’s look at this in detail.
It is important to note here that there is currently no legal obligation to implement the RGESN. It should also be noted that a large proportion of the declarations come from public bodies (regions, cities, etc.).
Methodology
For the sake of simplicity, we decided to get an initial idea of the eco-design of websites that have published RGESN declarations. To do this, we measured their home pages using the Greenspector Studio Benchmark tool. Websites are evaluated using a standard measurement (loading, scrolling, and pausing) on a real device. The results are evaluated using two scores:
- The Client Resources eco-score is based on the metrics collected (data transferred, battery drain, CPU, and memory).
- The Network eco-score is based on the automatic verification of around thirty technical best practices.
The average of these two scores gives the score for each page.
More details here: https://greenspector.com/comment-est-calcule-lecoscore-dans-le-cas-dun-benchmark-web-ou-mobile/ [FR]
For environmental projections, the methodology can be viewed here: https://greenspector.com/publications/methodologie-greenspector-2024-25/
Once these results have been obtained, we will be able to compare the sites with each other and see if there is a correlation between the measurement results and the progress scores announced in their RGESN declarations.
Ranking obtained
Already, we can see a surprising trend: most of the progress scores are very high. Just over half have an RGESN progress score of 90% or higher. This is even clearer when we look at the distribution of scores on a graph:
As a reminder, the RGESN is a standard for evaluating and structuring the eco-design process for digital services. Half of the sites evaluated are already at or close to the maximum score. Since eco-design is a process of continuous improvement, what happens to those who have already achieved the maximum score? In theory, we can imagine that eco-design is perfectly integrated into the processes for these sites, so congratulations to them! But we’ll come back to that later.
Moreover, we note that for these same sites, the Ecoscore is very disparate, as are the metrics and environmental impacts evaluated. Strange…
Conversely, if we look at the lowest progress scores (25% and 40%), the Ecoscores are good and the metrics very reasonable. Something surprising is definitely going on here.
For now, let’s try a new approach by synthesizing the data obtained in the form of graphs.
These two visualizations are fairly representative of what we get when we try to correlate the progress score with the measurement results. We also know that the same approach can lead to different results and that the RGESN is based on declarative statements. In the absence of more conclusive results, let’s take a different approach by analyzing a sample of sites.
Analysis
It is not surprising that some websites perform well despite having lower advancement scores than others. After all, even if the results are already convincing, it is possible that the eco-design process is still being implemented. Indeed, a lightweight website does not necessarily embrace sustainability “on purpose”. For example, good results can be achieved by seeking to improve the accessibility or performance of a site without necessarily having in mind the reduction of its environmental impact. Similarly, such a site may well deteriorate over time if it is not part of a continuous improvement process. For example, if all stakeholders, particularly contributors, are not aware of eco-design. On this subject, olease check our article on editorial restraint [FR].
But let’s focus instead on the websites that achieve the highest RGESN progress scores here. 100% progress is something to marvel at, isn’t it?
Spoiler : nope.
Analysis of a sample of websites with a 100% progress score
Lannion-Trégor
Among the sites with a 100% RGESN progress score, we start with the one with the best Ecoscore.
We will briefly analyze the home page to see what stands out from an eco-design perspective.
- From the moment it loads, the site is lightweight (less than 500 KB).
- The images are fairly light.
- We note (using the Chrome browser’s Coverage tool) that some CSS and JS files are only used to a very limited extent. However, this is often the case on the web, and the files remain fairly lightweight.
- Using Greenspector Studio and standard performance analysis tools (Webpagetest, Lighthouse), we confirm that most technical best practices are being applied: minification, compression, etc.
However, we also notice a high number of font files :
There are six font files. Even though each file is small, their number is sufficient to invalidate criterion 4.8 of the RGESN (Does the digital service limit the number of downloaded fonts?). This criterion limits the total number of variants to four.
Similarly, the search function offers autocompletion after the second character is entered, whereas according to the RGESN, it should wait until the third character is entered. Furthermore, the user interface does not allow autocomplete to be disabled (which, in fact, almost no website allows). This invalidates criterion 4.9 (Does the digital service limit server requests during user input?).
More generally, it is unfortunate that the eco-design declaration is not easily accessible, that it provides very little information, and that it does not appear to have been reviewed by a third party. Even though the site appears to be fairly lightweight, the announced progress score (100%) seems to have been overestimated.
Eau cœur d’Essonne
Among the sites reporting a 100% RGESN progress score, this site has the lowest Greenspector Ecoscore (52/100).
A quick analysis reveals one main problem: ECE_22-mars-toute_l_annee_actu-1.gif
This is a very large animated image (836 KB). Its integration invalidates the criteria:
- 5.1 – Does the digital service use a file format adapted to the content and
- the viewing context of each image? The GIF format is not optimal here.
- 4.1 – Does the digital service only have animations, videos and sounds
- whose automatic playback is disabled? Due to the format chosen, automatic playback is enabled by default.
- 6.5 – Does the digital service avoid triggering uploads of unused resources
- and content for each feature? : this image, included in the carousel at the top of the page, is loaded even if the user does not scroll through the carousel to display it.
As with the previous site, it is unfortunate that the declaration is not directly accessible, that justifications are missing, and that it has not been reviewed by a third party. Here too, the announced progress score (100%) appears to have been overestimated.
Conclusion : he who gives up striving to become better ceases to be good
For this article, we measured the home pages of websites with an RGESN declaration based on the latest version of the standard. No clear correlation emerged between the measurement results and the reported progress scores. This is not surprising, given that RGESN audits as such do not function in the same way or serve the same purpose as measurements. As the RGESN is largely based on declarative statements, it seems all the more important to carry out measurements in parallel. The latter have the advantage of being (relatively) objective and automatable.
We also note that the progress scores are mostly very (too?) high. It is surprising to find no fewer than 9 RGESN declarations with a 100% progress score. During the analysis presented above, we were surprised by this result. In itself, this misses the point of the RGESN: to be able to communicate about the eco-design approach and share it with as many people as possible. In the absence of a third-party review, such documents undermine the credibility of the RGESN and the approach. Whether this is due to a lack of expertise or clumsiness, the result is akin to greenwashing.
This also raises the question: what action should be taken once a 100% progress score has been achieved? I refer you to one of our previous articles on this subject: https://greenspector.com/feedback-rgesn/ [FR]
With a little digging, we quickly discover that the sites showing 100% progress on the RGESN were produced by the same company (which, incidentally, offers its expertise on the standard).
Bonus
We have identified a few areas for caution, particularly regarding the highest progress scores. So what should be used as a basis for an RGESN audit that is as reliable as possible?
Firstly, I would strongly encourage you to share the reasons that led to the criteria being validated. Even if some information cannot be disclosed, do your best to explain your approach.
Next, if you can, have a third party proofread your work.
Finally, refer to existing statements. In particular, I recommend the following:
- https://www.loire-atlantique.fr/44/declaration-d-ecoconception/declaration-d-ecoconception/c_1482025
Keep in mind that the RGESN is a guide and that the audit also serves to guide you on the path to follow. As is often the case, it is the journey rather than the destination that matters.
On the RGESN Consortium side, every effort is currently being made to improve the framework, but also to better structure (and thus facilitate) audits and declarations.
At Greenspector, we are happy to supplement your RGESN declaration of commitment and process maturity with a highly complementary resource consumption score via the digital sobriety certificate. In any case, measurements are strongly encouraged by the framework, which also helps to define a measurement and audit strategy.
Thanks to Anne-Sophie Tranchet and Dareth Nhang for their feedback!
