PERFORMANCE AND EFFICIENCY OF HIGH

PERFORMANCE AND EFFICIENCY OF HIGH-SPEED RAIL SYSTEMS
Jack E. Doomernik Lloyd´s Register and University of Antwerp 1. INTRODUCTION AND POLICY CONTEXT
Since the introduction in Japan in 1964 and in France in 1981, high-speed rail
systems have been developed in various countries in Asia and Europe. Governments
try to create new dynamics in railway transport to cater for the rising need for highspeed travel demand and railways are revitalized to be able to compete better with
other modes of transport. An important focus is on the development of new highspeed networks in order to facilitate growth in mobility and to limit air travel. The
building of high-speed rail systems requires substantial investment in infrastructure,
railway stations and rolling stock. Efficient use of these capital-intensive assets is
needed to justify the investments made. In addition, identification of areas of
improvement in production and marketing is important to optimize operational
performance and productivity. National governments decide on the development of
high-speed rail systems based on the expected future demand for high-speed travel
and the social benefits for the country. Long-term performance forecasts for highspeed rail are a basic input for the decision-making process. Ex post, in the
operational stage, the assumptions need to be validated based on the actual system
performance.
The goal of this paper is to identify the best high-speed rail practices in the world and
to clarify the efficiency of the world’s major high-speed rail systems currently in
operation. This study compares the high-speed rail performance of the world’s major
high-speed rail systems regarding travel performance, ridership, train fleet and
network. Based on the actual performance data and system characteristics, the
efficiency of the selected high-speed railway systems is benchmarked using Data
Envelopment Analysis (DEA) techniques. Four Asian and four European high-speed
rail systems are benchmarked against their peers using the actual system
characteristics and performance between 2007 and 2012. This study identifies the
most efficient high-speed rail systems and the contributing factors in achieving high
performance in production and marketing. High-speed rail system operators can use
the results to adjust their strategy in order to improve their performance and process
efficiency. Policy makers that are planning for a high-speed rail future may benefit
from the experiences in other countries to make better decisions on the investments
in infrastructure and rolling stock needed.
The paper is structured as follows. Section 2 gives a review of the development on
benchmarking in the railway industry and the applied methods. In section 3 the
methodology and the DEA-model, variables and data used in the study are presented
to benchmark eight high-speed rail systems across Europe and Asia. Section 4
provides the results for the Malmquist Productivity Index and the efficiency and
1
© AET 2014 and contributors
effectiveness scores. Finally, section 5 presents the conclusion from the benchmark
and discusses the results.
2. BENCHMARKING METHODS AND APPLICATION TO RAILWAYS
Benchmarking is intended to compare products or services with the competition or
with organisations that are recognized as leaders in their sector to find best practices
and ways to grow. This implies that it doesn’t give an answer to how industry leaders
themselves can improve. The best practices can be found by comparing individual
performances within a selected peer group. The main objective of benchmarking is to
measure and compare the realized output of a product or service with the amount of
inputs (Hansen et al, 2013). Besides the uni-dimensional Ratio Analysis (RA) or
Partial Productivity Measures (PPM) analysis for productivity and efficiency
measurement, four multi-dimensional approaches can be identified (see figure 1 for
an overview): Total Factor Productivity (TFP), Data Envelopment Analysis (DEA),
Least Squares Regression (LSR) and Stochastic Frontier Analysis (SFA) (Coelli et al,
2005, Ozcan, 2008, Merkert et al, 2010). The PPM analysis, where an output
variable is viewed in relation to a single input variable is a practical, easy and fast
way to of comparing performance. The challenge here is to find meaningful efficiency
indicators. In the more practical and technical-managerial studies like the
CoMET/Nova metro railway benchmark, the European IMPROVERAIL project and
the INFRACOST and LICB studies performed by the UIC to benchmark rail
infrastructure companies, Key Performance Indicators were developed for the
comparison (Anderson et al, 2003). An application for comparing the performance of
eight high-speed railways in Europe and Asia was presented by Doomernik (2013).
The main disadvantage is that only one indicator at the time can be evaluated. A
multi-PPM analysis, where more ratios are assessed at the same time can easily
lead to misinterpretation. The benchmarks established using old analytical schemes
based on various multiple ratios created more dilemmas than solutions (Ozcan,
2008). The TFP analysis enables to evaluate multiple inputs and outputs
simultaneously resulting in a single index for efficiency that makes it possible to rank
the entities under study. DEA, LSR and SFA are more sophisticated tools that can
also handle multiple inputs and outputs. All five benchmark methods can be
recognized in international (mostly European) railway efficiency and productivity
studies. As PPM is the most widely used measure in railways, DEA and SFA have
become the most commonly applied methods in rail efficiency analysis in recent
years (Merkert, 2010). A selection of recent benchmark studies presented by Hansen
et al. (2013) also shows that DEA and SFA have become frequently used since
about 2008. The utilization of either DEA or SFA is now one of the most defining
elements of the studies, while LSR and TFP have lost importance (Laird et al, 2011).
The same report states that no single benchmark can be applied to all railways and
2
© AET 2014 and contributors
several benchmarking methods should be used concurrently, since particular insight
can be gained from each of them.
Figure 1: Overview of Productivity and Efficiency Measurement (adopted from Laird et al, 2011)
There have been many studies in the rail sector where DEA is used as a comparison
technique. For an overview, see for example Merkert et al. (2010) and Hansen et al.
(2013). To our knowledge, there are no benchmark studies for high-speed railway
systems using DEA. DEA is however very suitable for the use in the rail sector, due
to the highly regulated and quasi-monopolistic industry structure (Coelli & Perelman
et al, 2000) and where the formal link between input and output is not clear in the first
instance. An important advantage of DEA is that the results are based on a relative
comparison and that DEA can work with index numbers, ensuring that no sensitive
information is provided to others as often desired by companies (Caldas, 2013). SFA
requires a-priori a production function specification and assumptions regarding the
distribution of input variables and technical inefficiencies (Karlaftis, 2012). In practice
this information is often missing. To by-pass these complications DEA is chosen as
the preferred technique for this study.
Data Envelopment Analysis (DEA) is a non-parametric technique to compare
performance between entities, normally indicated as Decision Making Units (DMU’s),
that allows multiple inputs and outputs. It is possible to increase the efficiency by
lowering the input or increasing the output, keeping the inputs unchanged (Caldas,
2013). Besides this distinction between input and output orientation, in DEA a
difference can be made regarding the returns to scale. The Constant Returns to
Scale (CRS) assumption implies that all DMU’s operate at an optimal scale, while
3
© AET 2014 and contributors
Variable Returns to Scale (VRS) divide the CRS-efficiency score into Technical
Efficiency (TE) and Scale Efficiency (SE). Figure 2 shows that the inefficiency of
DMU F is partly due to technical inefficiency and partly due to scale inefficiency
(Struyf, 2013).
Figure 2: Technical and Scale (in)efficiency (Struyf, 2013)
The difference between CRS and VRS is that in the VRS model an additional
condition on the weights is introduced. This is because the CRS model does not work
properly if there is more than one optimal solution. When economies-of-scale are not
changed by an increase in efficiency, CRS can be applied. If this is not the case, a
VRS model is needed.
An interesting feature of DEA is that, by using the Malmquist Productivity Index
(MPI), it can also capture the dynamics in efficiency. This index tells us how much
the ratio of aggregate output to aggregate input has changed between any two time
periods (Färe and Grosskopf, 2000). This is a commonly applied approach to
assessing dynamic efficiency in a DEA environment, assuming constant-return-toscale (CRS) technology. An important feature of the DEA Malmquist Index is that it
can decompose the overall efficiency into two mutually exclusive components, one
measuring Efficiency Change (EC) and the other measuring Technical Change (TC).
Traditional DEA (TDEA) models are based on a “black box” approach with multiple
inputs and outputs. The actual transformation process is generally not modelled
explicitly. TDEA reveals rather than imposes the structure of the transformation
process. Network DEA (NDEA) models allow to identify components inside the box
and to evaluate organizational performance and its component performance (Färe
and Grosskopf 2000). This is done by splitting the model into two or more stages
where an output feeds a subsequent stage. This approach can be applied to railways
4
© AET 2014 and contributors
to assess besides the overall technical effectiveness, the technical efficiency and
service effectiveness separately (Lan and Lin, 2006, Yu, 2008). By plotting these
results for all DMU’s in a performance matrix as illustrated in figure 3, best practices
can be found and strategies can be proposed to improve the position of
underperformers (Ozcan, 2008, Lan and Lin, 2006).
Figure 3: Performance matrix (Ozcan, 2008)
Performance can be defined as an appropriate combination of efficiency and
effectiveness. An organization can be efficient, but not effective; it can also be
effective and not efficient (Ozcan, 2008). Efficiency, the ratio between output and
input, is a key performance parameter indicating if assets are properly used.
Effectiveness indicates if the inputs are properly used to produce the best possible
outcome. Besides the evaluation of performance on both dimensions, the correlation
between efficiency and effectiveness can be studied as well to answer the question
whether efficient organisations are also effective or not. Karlaftis and Tsamboulas
found that efficiency is generally negatively related to effectiveness in their research
on 15 European transit systems for a ten year time period (1990-2000). This implies
that increasing efficiency may result in decreased effectiveness (2012).
3. METHODOLOGY AND NETWORK DEA MODEL
A railway system can be modelled as a Multiple-Input Multiple Output (MIMO) system
for efficiency, productivity and costs analyses (Cantos et al 2010, Mizutani and
Uranishi 2012). A system approach with N inputs and M outputs is the basis for our
DEA study. In the current study, NDEA is chosen for the high-speed rail systems’
performance and efficiency assessment. As DEA can be considered a “black box”
approach, we introduce a two-stage Network DEA (NDEA) model to evaluate the
overall technical effectiveness, the technical efficiency of the production process and
service effectiveness of the consumption process simultaneously in a single model
5
© AET 2014 and contributors
as proposed by Lin and Lan (2006) and Yu (2008). PPM gives the possibility to clarify
in more detail the differences in ranking and efficiency results. As the results of the
PPM analysis have already been reported earlier (Doomernik, 2013), we will focus in
this paper on the DEA approach to compare efficiency.
3.1 Input and output variables
To provide high-speed train services in a country, two major physical assets are
needed: i) a high-speed rail network and ii) a fleet of high-speed trains. For this
study, we only consider the network and the rolling stock assets, being the two major
production factors for railway performance. Railway stations for access and egress of
passengers are left out of the equation as in most cases they are not a performance
limiting factor. Difficulties with defining meaningful parameters is another reason not
to take stations into account as not only the number, but also the size, location and
accessibility by other modes of transport are normative parameters.
Besides physical assets, an operational model and timetable to run the trains on the
network is required to deliver the rail services. Operational expenditures and staff on
board and at the railway stations are also production factors, but we do not take
these into account in this study, mainly due to the fact that only limited data is
available on operational costs and staffing levels in high-speed rail.
Appropriate infrastructure and rolling stock are needed for supplying high-speed train
services. The total length of high-speed lines in the network and the number of
available high-speed trains and their seating capacity are key parameters for the
high-speed rail system performance. The final output performance can be expressed
in terms of travel volume and is defined as the product of yearly number of
passengers and the average travel distance per passenger. Ridership and train or
seat kilometres produced by the fleet are additional output variables indicating the
railway’s performance.
The high-speed rail MIMO system is detailed in figure 4 with two asset-related input
parameters (N=2) for the infrastructure and rolling stock and two output parameters
(M=2) for the transport and travel performance.
The overall process is split into two subsequent stages to assess the efficiency of the
production and consumption process separately. These stages are linked by the fleet
performance being an output of the production process and an input for the
consumption process. The production process uses network length and fleet capacity
to produce train-km’s, who are in turn input for the consumption process delivering
ridership and travel performance as outputs. By plotting the results for all DMU’s
regarding production efficiency, service effectiveness and system efficiency in a
6
© AET 2014 and contributors
performance matrix (figure 3), best practices can be found and strategies can be
proposed to improve the position of underperformers.
Figure 4: A two-stage multiple-input multiple-output NDEA model of a high-speed railway system
The efficiency and effectiveness scores and Malmquist productivity indices are
calculated for the overall process and the two separate stages. Merkert et al use an
input orientation because ”it assumes that rail firms have higher influence on the
inputs, since output volumes are substantially influenced by macro-economic factors
and often pre-determined by long-term contracts and exogenously controlled public
transport service level requirements” (Merkert et al., 2010). For this NDEA analysis
model the output orientation is applied for the overall model and the individual
stages. Regarding stage 1, improving the fleet performance has a preference over
decreasing the infrastructure or fleet capacity. In practice taking out of operation and
disinvestments in high-speed lines and rolling stock are very unusual to improve
technical efficiency. For the effectiveness (stage 2) it is easier on the short term to
influence ridership and travel performance by proper marketing and sales activities
than to change the timetable. The calculated VRS efficiency is split into Technical
Efficiency and Scale Efficiency scores. The Malmquist Productivity Index is
decomposed to identify the Efficiency Change and Technical Change factor from the
CRS results over the 2007 to 2012 time period. All efficiency and effectiveness
scores and Malmquist Productivity Indices are calculated by using DEAP (Data
Envelopment Analysis Program) software written by Tim Coelli (2001).
3.2 Selected high-speed rail networks
To find best practices in production and marketing in the worlds’ largest high-speed
rail systems, eight networks are identified, four of which can be found in Asia (Japan,
Taiwan, China, Korea) and four in Europe (France, Germany, Spain, Italy). The
selection was made on the basis of actual travel volume over the selected time
frame. The study not only compares the individual peers, but also explores the
7
© AET 2014 and contributors
differences between two regions, Europe and Asia. From the resulting performance
matrices, strategies are proposed to improve the overall efficiency.
3.3 System characteristics and performance data
Table 1 shows the descriptive statistics of the input and output variables used in the
study with their associated values from the data collected for the eight high-speed
railway systems for 2007 till 2012 (in total 48 observations). The complete dataset
can be found in appendix 1. For all countries, the figures are derived from UIC data,
statistical handbooks and annual reports. To fill in information gaps, additional data is
used from several other sources. For China, data from the World Bank (Bullock et al
2012) and the CRH timetable is used with estimations on travel performance made
by the author and input from the universities of Beijing and Shanghai, as data on
China’s high-speed rail programme is not made publicly available.
Table 1: Descriptive statistics of inputs and outputs (N = 48)
NL
Variable
Indicator
Unit
Europe
(N=4)
Asia
(N=4)
mean
SD
min
max
mean
SD
min
max
FS
AS
FPT
FPS
RS
TV
Network
Length
Fleet Size
Available
Seats
Fleet
Performance
Fleet
Performance
Ridership
Travel
Volume
Total routekm
Number of
trains
Number of
available
seats
(thousands)
Yearly trainkm of fleet
(millions)
Yearly seatkm of fleet
(billions)
Yearly
number of
passengers
(millions)
Yearly
passengerkm (billions)
km
-
-
km
km
-
km
1391
91
562
2056
2885
433
330
6405
243
26
97
475
449
63
30
632
105.9
12.1
37.5
216.4
278.4
38.4
29.7
455.4
95.1
10.3
45.4
182.6
177.5
25.4
7.9
300.0
42.1
4.8
13.4
83.2
111.3
15.7
7.8
216.2
60.0
7.5
11.4
115.5
222.6
32.4
15.6
485.5
24.1
3.5
8.5
54.0
65.3
9.5
3.5
144.6
Historically, the largest high-speed rail systems can be found in Japan, France,
Germany and Spain. These countries have mature networks built gradually over
decades. Heavy investments in high-speed rail over the last decade gave China the
position of operating the largest high-speed rail network and fleet in the world since
2010. Train densities on the European high-speed network (ratio of FPT and NL from
table 1) are about 10% higher than in Asia. In operational models where high-speed
trains run on conventional tracks as well, train densities will be higher than in
services where only high-speed tracks are used (Doomernik, 2013). From 2011 on,
China is leader in ridership and travel volume and shows the fastest growth. Smaller
networks can be found in Taiwan and Korea. Although France and China had a
comparable fleet size in 2010, China’s fleet capacity (number of available seats) is
larger as their train sets can carry more passengers. This is typical for the Asian train
sets (Doomernik, 2013). In Asia the average number of seats per train (ratio AS/FS
from table 1) is 620 compared to 436 for Europe. Due to such high-capacity trains,
Asia produces 170% more travel volume and 164% more seat kilometres than
8
© AET 2014 and contributors
Europe with only 86% more train kilometres. Large differences can also be seen
regarding the average travel distance (ratio TV/RS from table 1). In Asia travellers
take shorter trips (293 km) than in Europe (402 km). Seat occupancy (ratio of TV and
FPS from table 1) is comparable for Europe (57%) and Asia (59%).
4. EMPERICAL RESULTS
4.1 Malmquist Productivity Index
The results from the Malmquist Productivity Index and its decomposition in Efficiency
Change and Technical Change are listed in Table 2 and graphically displayed in
figure 2 for the eight high-speed rail systems. When the values of the Malmquist
index and its components are more than 1 in an output-oriented evaluation, they
indicate progress (Ozcan and Ozgen, 2004).
Table 2: Malmquist Productivity Index
2008‐2007 2009‐2008 2010‐2009 2011‐2010 2012‐2011 mean 2012‐2007 Malmquist Productivity Index (MPI) France Germany Italy Spain Mean Europe Japan Korea China Taiwan Mean Asia Mean Europe + Asia 1.011 1.063 0.931 0.965 0.991 0.965 1.012 1.252 1.917 1.237 1.108 0.967 0.972 1.250 1.028 1.048 0.922 0.953 0.663 1.052 0.885 0.963 0.984 1.058 1.003 0.963 1.001 1.015 1.075 0.832 1.116 1.003 1.002 0.998 0.987 1.001 0.786 0.938 1.019 1.056 1.201 1.103 1.093 1.012 0.974 1.064 1.070 1.036 1.035 0.998 1.054 1.200 1.066 1.077 1.056 0.986 1.028 1.046 0.951 1.002 0.983 1.029 0.999 1.215 1.053 1.027 0.940 1.141 1.188 0.778 0.998 0.969 1.184 0.878 2.573 1.269 1.125 1.000 1.063 0.912 0.928 0.974 0.998 1.062 1.000 1.920 1.194 1.079 1.000 0.957 1.255 1.070 1.065 0.945 0.904 1.000 1.000 0.961 1.012 1.000 1.049 1.003 0.984 1.009 1.023 1.107 0.840 1.000 0.988 0.998 0.954 0.928 0.942 0.746 0.888 1.037 1.000 1.115 1.000 1.037 0.960 0.916 0.961 0.920 0.891 0.922 0.949 1.000 1.067 1.000 1.003 0.961 0.973 0.990 0.999 0.917 0.969 0.990 1.012 1.000 1.139 1.034 1.001 0.874 0.951 0.995 0.649 0.856 0.949 1.062 1.000 1.920 1.179 1.005 1.011 1.000 1.020 1.040 1.018 0.967 0.953 1.252 0.998 1.036 1.027 0.967 1.016 0.996 0.960 0.984 0.976 1.055 0.663 1.052 0.921 0.952 0.984 1.008 1.000 0.979 0.993 0.993 0.971 0.990 1.116 1.016 1.004 1.045 1.063 1.063 1.054 1.056 0.983 1.056 1.077 1.103 1.054 1.055 1.063 1.107 1.163 1.163 1.123 1.052 1.054 1.125 1.066 1.074 1.098 1.013 1.038 1.047 1.037 1.034 0.994 1.017 0.999 1.066 1.019 1.026 1.075 1.199 1.195 1.199 1.166 1.021 1.114 0.878 1.340 1.076 1.120 Efficiency Change (EC) France Germany Italy Spain Mean Europe Japan Korea China Taiwan Mean Asia Mean Europe + Asia Technical Change (TC) France Germany Italy Spain Mean Europe Japan Korea China Taiwan Mean Asia Mean Europe + Asia 9
© AET 2014 and contributors
Figure 5: Malmquist Productivity Index and decomposition in Efficiency and Technical Change
10
© AET 2014 and contributors
The MPI reflects a productivity improvement for the whole peer group of 12.5% over
the five-year period from 2007 till 2012. This is caused by technical change rather
than improvement of efficiency.
In contrast with Europe, where the MPI was stable and close to 1 from 2007 to 2012,
Asia achieved a productivity growth of 26.9% over the same time period. Europe
didn´t show any productivity improvement because, despite the 16.6% technical
change, efficiency dropped with 14.4%. In Asia both technical efficiency
improvements (+17.9%) and technology change (+7.6%) contributed to the overall
productivity growth.
Looking at the individual HSR networks in Europe and Asia, the evolution of the
Malmquist index is fairly stable over the years for Germany, Japan, Korea and
France. Germany, Italy, Korea and Taiwan show an above-average MPI-value
between 2007 and 2012. The high productivity improvement in Taiwan is remarkable
(+157%). Taiwan is the only DMU that has achieved a productivity index above unity
in every successive year. This is in fact from the start, as the Taiwan high-speed rail
services were inaugurated in January 2007 and services were gradually increased.
This also explains the high 2008-2007 MPI. Underperformers are the networks in
Spain and China, but for different reasons. Efficiency in of the Spanish HSR-network
dropped with 34.1% in five years’ time, but this is partly compensated with a technical
improvement of 19.9%. China achieved to keep up efficiency, but shows a
decreasing technical change of 12.2%. A lot of variation can be seen in the China
technical change index, making progress over the last couple of years. In this case
we have to realise that China only started their high-speed operations in 2008 and is
still growing fast. The network is not fully mature yet and CRH (the Chinese national
high-speed railway operator) is still optimising their operations.
In general, productivity improvement for the peer group comes from technical
change, rather than from efficiency change, which is declining year-on-year. Only
Taiwan was able to maintain efficiency in five successive years.
4.2 Production efficiency and service effectiveness
The results from the DEA analysis are presented in Appendix 2. From the descriptive
statistics (table 3) can be seen that Asian high-speed rail systems are fully efficient in
the VRS-model. Scale efficiency is comparable for both Asian and European
systems. The CRS and VRS models show that Asia outperforms Europe regarding
production efficiency and service effectiveness.
11
© AET 2014 and contributors
Table 3: Descriptive statistics of efficiency of eight high-speed railways in Europe and Asia 2007 - 2012
Region Europe 2007 ‐2012 mean Production efficiency Service effectiveness System Efficiency CRS TE VRS TE SE CRS TE VRS TE SE CRS TE VRS TE SE 0.795 0.896 0.889 0.792 0.842 0.944 0.791 0.821 0.963 (N=4) SD 0.020 0.024 0.009 0.032 0.035 0.011 0.028 0.028 0.007 min 0.542 0.591 0.796 0.504 0.522 0.783 0.555 0.579 0.856 Asia mean 0.936 0.985 0.949 0.877 0.977 0.897 0.958 1.000 0.958 (N=4) SD 0.025 0.011 0.021 0.027 0.012 0.025 0.020 0.000 0.020 min 0.545 0.773 0.545 0.608 0.776 0.608 0.521 1.000 0.521 CRS TE = Technical Efficiency from CRS DEA VRS TE = Technical Efficiency from VRS DEA SE = Scale Efficiency The efficiency scores as displayed in Appendix 2 for the production and marketing
process are reflected in performance matrices in figure 6 (comparison between
networks) and 7 (development in time) for the years 2007-2012. Overall efficient
DMU’s are coloured green and inefficient ones orange (overall efficiency between
0.75 and 1.00) or red (overall efficiency between 0.50 and 0.75).
The Asian DMU’s and France are the best performers in the peer group. In all years
Italy appears to be the worst performer and Germany and Spain are in the middle of
the spectrum. Except for 2007, when the high-speed rail service was started, Taiwan
was overall efficient and efficient in production. Year-on-year Taiwan has improved
their marketing efficiency compared to others. This is in line with the MPI results
shown earlier. Although the efficiency of the production process varies over the years
China was able to be fully efficient in their marketing process. The results from the
Malmquist index shows that technical change has been lagging behind. This
indicates that improvements could be achieved in optimising the technical production
process. For Korea the opposite is the case: an efficient production process, but
variation in the marketing efficiency. In Japan we see a dip in 2008 and 2009 in their
marketing performance. The last three years they have an efficient production and
are improving their marketing performance, but are outperformed by Korea and
Taiwan. This ranking can also be recognised by the MPI results. The evolution of the
production and marketing efficiency in Italy and Germany shows the same pattern: a
steady reduction in production efficiency and improving marketing performance after
a light shortfall. Italy is performing a bit better in production, but this cannot
compensate for their marketing inefficiency. Spain and France show fluctuating
results. Their marketing is better than their production performance. France is
improving and Spain is losing on production efficiency over the last years.
12
© AET 2014 and contributors
Figure 6: Performance of four European and four Asian high-speed rail networks (2007 – 2012).
Efficient systems are not necessarily effective and vice versa (Ozcan, 2008). In this
context, an important question is raised by Karlaftis (2010): “How are efficiency and
effectiveness ratings related?” Karlaftis states that “For all systems and years, the
ratings on one performance attribute (efficiency) are – generally- negatively related to
the ratings on the other attribute (effectiveness).” (Karlaftis, 2010). The correlation
coefficients between production efficiency and service effectiveness for all highspeed rail systems in the peer group over the years 2007 to 2012 are presented in
table 4 for the CRS and VRS model (in total 48 observations from 8 countries and
over 6 years). The results indeed show a negative correlation which implies that
increased production efficiency tends to come with decreased service effectiveness.
For Europe this effect is much stronger than for Asia where a 10% increase in
production efficiency comes with a 7% loss in service effectiveness.
Table 4: Correlation coefficients between Production Efficiency and Service Effectiveness
Europe + Asia Europe Asia 13
CRS Model Production Efficiency Region VRS Model Service Effectiveness ‐0,091 ‐0,179 ‐0,657 ‐0,661 ‐0,170 ‐0,128 © AET 2014 and contributors
14
© AET 2014 and contributors
© AET 2014 and contributors
Figure 7: Efficiency development of four European and four Asian high-speed rail networks between 2007 and 2012.
15
© AET 2014 and contributors
© AET 2014 and contributors
5. SUMMARY AND CONCLUSIONS
In their search to optimise the utilisation of high-speed rail systems
governments and railway companies may benefit from good practices in the
rest of the world. Benchmarking of these systems in operation may give
guidance to best practices in this sector to learn from. This study was initiated
as to our knowledge no objective comparison of high-speed rail systems is
available. Based on the current knowledge and experience in benchmarking,
Network Data Envelopment Analysis (NDEA) in combination with the
Malmquist Productivity Index was chosen for the benchmark. For this
purpose, a performance matrix is presented to investigate the production
efficiency and service effectiveness of the high-speed railways under study.
The peer group consisted of the eight largest networks in the world (four in
Europe and four in Asia) and revealed significant differences between Asia
and Europe. Also within these regions remarkable differences can be found.
5.1 Results
Between 2007 and 2012, Asia achieved a productivity growth of 26.9%.
Europe didn´t show any productivity improvement because, despite the 16.6%
technical change, efficiency dropped with 14.4%. In Asia both technical
efficiency improvements (+17.9%) and technology change (+7.6%)
contributed to the overall productivity growth.
Germany, Italy, Korea and Japan show an above-average MPI-value between
2007 and 2012. The high productivity improvement in Taiwan is remarkable
(+157%). Taiwan is the only DMU that has achieved a productivity index
above unity in every successive year.
Underperformers are the networks in Spain and China, but for different
reasons. Efficiency in of the Spanish HSR-network dropped with 34.1% in five
years’ time, but this is partly compensated with a technical improvement of
19.9%. China achieved to keep up efficiency, but shows a decreasing
technical change of 12.2%.
The DEA model shows that Asian high-speed rail systems are fully efficient in
the VRS model and Asia outperforms Europe regarding production efficiency
and service effectiveness. The Asian DMU’s and France are the best
performers in the peer group. In all years Italy appears to be the worst
performer and Germany and Spain are in the middle of the spectrum.
The results show a negative correlation between production efficiency and
service effectiveness. For Europe, this effect is much stronger than for Asia
where a 10% increase in production efficiency comes with a 7% loss in
service effectiveness.
© AET 2014 and contributors
17
5.2 Methodology
The study shows that high-speed railways can be represented as a MIMOsystem with two input and two output variables for benchmark purposes.
Meaningful comparisons can be made on the basis of the overall efficiency
and the efficiency of the production and marketing process.
A Network DEA-model has proven to be very useful to analyse the differences
in performance among the peer group. It gives a better view if performance
differences come from the production or marketing and sales process. The
performance matrices reveal typical patterns regarding production efficiency
and service effectiveness. The conclusions from the Malmquist index are in
line with the resulting performance matrices from the DEA-model.
In the NDEA model the number of variables is rather limited. Including extra
variables will lead to a better representation and better understanding of the
actual situation. For the input one could include for example labour (number of
train drivers and train assistants) and operational costs. Besides trainkm’s to
describe the fleet performance, punctuality could also be an important
intermediate variable. Client satisfaction could be added as an extra output
variable. To what extent extra variables can be included depends on data
availability. The model could be refined by considering shared inputs and
adding environmental variables as suggested by Yu (Yu, 2008).
In the analysis coherent national networks are assumed. In Japan, the
national network consists of four sub-networks though, operated by JR East,
JR Central, JR West and JR Kyushu. The analysis could be detailed to take
not the country, but the individual operators as DMU’s.
The peer group consists of eight networks which show considerable
differences in operational models. The four basic operational models that can
be recognised in various countries are the exclusive model where only HStrains run on HS-track (Japan, Taiwan), the mixed high speed model where
HS-trains run on conventional track as well (France, China), the mixed
conventional model where also conventional trains can access the HSnetwork (Spain) and the fully mixed model where both high-speed and
conventional trains can run on high-speed and conventional tracks (Germany)
(Rus et al. 2009). Although theoretically it would be better to take different
operational models into account, the peer group is too limited to split it into
sub-groups.
ACKNOWLEDGEMENTS
I would like to thank Tom Duffhues from the Technical University Twente in
the Netherlands and Els Struyf from Antwerp University in Belgium for
identifying the proper DEA software tools, Geraint Roberts for being so kind to
share his results from his Master Thesis at the Starthclyde University in the
UK, Adrian Ng from Lloyd’s Register Rail in Hong Kong, Mr. Wu from Beijing
University, Liehui Wang from East China Normal University and Rang Zong
© AET 2014 and contributors
18
from Shanghai University for helping to find the right data on the CRH
operation in China.
BIBLIOGRAPHY
Anderson, Richard J.; Hirsch, Robin C; Trompet, Mark; Adeney, William E.
(2003) Developing Benchmark Methodologies for Railway Infrastructure
Management Companies, European Transport Conference 2003, Strasbourg,
Association for European Transport, 2003.
Amos, Bullock and Sondhi (2010) High-Speed Rail: The Fast Track to
Economic Development?, The World Bank, July 2010.
Bullock, Salzberg and Jin (2012) High-speed rail – the first three years: taking
the pulse of China’s emerging program, World Bank Office Beijing, China
Transport topics.
Caldas, M. A. F., Carvahal, R. L., Gabriele P. D.; Ramos, T. G. (2013) The
Efficiency of Freight Rail Transport; An Analysis from Brazil and United
States, 13th World Conference on Transportation Research, Rio de Janeiro,
Brazil, July 2013.
Cantos, Pastor and Serrano (2010) Vertical and horizontal separation in the
European railway sector and its effect on productivity, Journal of Transport
Economics and Policy, Volume 44, Part 2, May 2010, pp. 139-160.
Coelli, T.J. (2001) A guide to DEAP Version 2.1: A Data Envelopment
Analysis (Computer) Program, CEPA Working Paper No. 96/08,
http://www.owlnet.rice.edu/econ380/DEAP.PDF.
Coelli, T., Perelman, S. (2000) Technical efficiency of European railways: a
distance function approach, Applied Economics, 2000, 32, 1967-1976.
Coelli, T.J., Prasada Rao, D.S., O’Donnell, C.J., Battese, G.E. (2005), An
Introduction to Efficiency and Productivity Analysis, second edition, Springer
2005.
Doomernik, J.E. (2013) Performance of high speed rail in Europe and Asia,
13th World Conference on Transportation Research, Rio de Janeiro, Brazil,
July 2013.
Färe R. and Grosskopf, S. (2000) Network DEA, Socio-economic planning
sciences, 34, 35-50.
© AET 2014 and contributors
19
Growitsch, C., Wetzel, H. (2009) Testing for Economies of Scope in European
Railways, Journal of Transport Economics and Policy, 43(1), January 2009, 124.
Hansen, I. A., Wiggenraad, P. B. L., Wolff, Jeroen W. (2013) Benchmark
Analysis of Railway Networks and Undertakings, 5th International Conference
on Railway Operations Modeling and Analysis, Copenhagen, 13-15 May
2013.
Karlaftis, M.G., Tsambloulas, D. (2012) Efficiency measurement in public
transport: Are findings specification sensitive?, Transportation Research Part
A 46 (2012) 392-402.
Laird, K., Zamba, J., Antoniazzi, F., Focsaneanu, E., Israel, S., Ionnidou, A-M
(2011) Report On Technical Benchmarking of European Railways, European
Railway Agency.
Lan, L.W., Lin, E.T.J. (2006) Performance Measurement for Railway
Transport: Stochastic Distance Functions with Inefficiency and Ineffectiveness
Effects, Journal of Transport Economics and Policy, Volume 40, Part 3,
September 2006, pp. 383-408.
McCoullough, G. J. (2007) US Railroad Efficiency: A Brief Economic
Overview, Department of Applied Economics, University of Minnesota.
Merkert, R., Smith, A.S.J., Nash, C.A. (2010) Benchmarking of Train
Operating Firms – a Transaction Cost Efficiency Analysis, Transportation
Planning and Technology, 33(1), 35-53.
Mizutani and Uranishi (2012) Does vertical separation reduce cost? An
empirical analysis of the rail industry in the European and East Asian OECD
countries, Journal of Regulatory Economics, 13 April 2012.
OSEC (2011) China railway market study, SwissRail Industry Association,
final report, January 2011.
Ozgen, H. and Ozcan, Y.A. (2004) Longitudinal Analysis of Efficiency in
Multiple Output Dialysis Markets, Health Care Management Science, 7, 253261, 2004.
Ozcan, Y. A. (2008) Health Care Benchmarking and Performance Evaluation,
Springer-Science+Business Media, LCC.
Roberts, Geraint. (2012) GB Rail Efficiency & Benchmarking, Master Thesis,
University of Strathclyde.
© AET 2014 and contributors
20
Rus et al. (2009) Economic Analysis of High Speed Rail in Europe, BBVA
Foundation, May 2009.
Smith, A.S.J. (2012) The application of stochastic frontier panel models in
economic regulation, Experience from the European rail sector,
Transportation Research Part E 48 (2012) 503-515.
Struyf, Els (2013) Efficiency of airports in relation to their ownership, Doctoral
Day University of Antwerp 2013, Belgium.
Yu, M.-M. (2008) Assessing the technical efficiency, service effectiveness,
and technical effectiveness of the world’s railways through NDEA analysis,
Transportation Research Part A 42 (2008) 1283-1294.
© AET 2014 and contributors
21
APPENDIX 1: INPUT AND OUTPUT VARIABLES AND VALUES
Variable
Indicator
Unit
2007
2007
2007
2007
2007
2007
2007
2007
2008
2008
2008
2008
2008
2008
2008
2008
2009
2009
2009
2009
2009
2009
2009
2009
2010
2010
2010
2010
2010
2010
2010
2010
2011
2011
2011
2011
2011
2011
2011
2011
2012
2012
2012
2012
2012
2012
2012
2012
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
France
Germany
Italy
Spain
Japan
Korea
China
Taiwan
NL
FS
AS
FPT
FPS
RS
TV
Network
Length
Fleet Size
Available
Seats
Fleet
Performance
Fleet
Performance
Ridership
Travel
Volume
Total routekm
Number of
trains
Yearly trainkm of fleet
(millions)
Yearly seatkm of fleet
(billions)
km
-
Number of
available
seats
(thousands)
-
1540
1285
562
1272
2452
330
405
345
1872
1285
562
1511
2452
330
405
345
1872
1285
744
1599
2452
330
1409
345
1872
1285
923
1604
2452
330
3408
345
1896
1285
923
2056
2534
412
4584
345
1896
1285
923
2056
2664
412
6405
345
406
254
97
136
354
46
113
30
422
254
109
162
363
46
191
30
435
254
108
162
367
52
279
30
450
254
110
162
375
61
466
30
463
246
110
203
386
65
551
30
475
246
110
193
365
65
632
30
181.2
113.0
56.6
37.5
340.5
43.0
70.8
29.7
189.4
113.0
61.8
48.0
357.0
43.0
128.0
29.7
196.1
113.0
61.3
48.0
363.7
45.2
186.3
29.7
203.7
113.0
62.2
48.0
368.6
48.5
306.9
29.7
209.9
111.6
62.2
61.7
370.4
49.9
383.4
29.7
216.4
111.6
62.2
59.0
352.1
49.9
455.4
29.7
Note: Data for China based on estimations by the author.
© AET 2014 and contributors
22
km
km
Yearly
number of
passengers
(millions)
-
162.9
109.2
46.8
48.7
135.2
21.4
40.0
7.9
171.0
109.8
45.4
55.3
138.0
21.8
53.0
15.3
173.8
102.1
47.0
58.4
138.3
22.0
87.0
15.0
172.2
102.3
47.7
57.6
134.3
23.3
182.0
15.5
179.3
99.9
48.7
58.6
142.7
27.7
250.0
16.0
182.6
97.9
48.1
57.6
145.7
29.6
300.0
16.0
72.7
48.6
27.3
13.4
134.5
20.0
25.0
7.8
76.8
48.8
25.7
16.4
141.2
20.4
35.5
15.1
78.3
45.4
26.7
17.3
143.5
19.1
58.1
14.8
78.0
45.5
27.0
17.1
139.2
18.5
100.1
15.3
81.3
45.3
27.6
17.8
142.8
21.3
173.9
15.8
83.2
44.4
27.2
17.6
145.1
22.7
216.2
15.8
105.4
70.5
23.4
11.4
315.8
36.7
86.5
15.6
115.5
74.7
23.9
22.1
310.3
37.4
127.4
30.6
114.4
73.2
33.4
23.1
288.9
36.8
179.6
32.3
112.6
77.8
34.0
22.2
292.1
40.8
290.5
36.9
114.2
76.1
37.4
22.8
307.0
49.6
440.0
41.6
114.2
76.6
39.8
22.3
321.6
52.4
485.5
44.5
Yearly
passengerkm (billions)
km
48.0
21.9
9.2
8.5
82.8
9.9
13.0
3.5
52.2
23.3
9.3
10.5
81.7
10.0
25.5
6.6
51.9
22.6
11.3
10.8
76.0
9.8
46.3
6.9
52.8
23.9
11.5
10.4
77.4
10.8
65.2
7.5
54.0
23.3
11.5
10.5
81.4
13.4
98.1
8.1
54.0
24.8
12.3
10.4
86.0
14.1
144.6
8.6
APPENDIX 2: DEA EFFICIENCY AND EFFECTIVENESS RESULTS
Production Efficiency CRS TE VRS TE 2007 France 0.851 0.978 2007 Germany 0.891 1.000 2007 Italy 1.000 1.000 2007 Spain 0.741 0.896 2007 Japan 0.902 1.000 2007 Korea 1.000 1.000 2007 China 1.000 1.000 2007 Taiwan 0.545 1.000 2007 Mean 0.866 0.984 2008 France 0.821 0.988 2008 Germany 0.852 1.000 2008 Italy 0.854 0.920 2008 Spain 0.672 0.729 2008 Japan 0.901 1.000 2008 Korea 1.000 1.000 2008 China 1.000 1.000 2008 Taiwan 1.000 1.000 2008 Mean 0.888 0.955 2009 France 0.864 0.992 2009 Germany 0.813 0.968 2009 Italy 0.873 0.989 2009 Spain 0.722 0.791 2009 Japan 1.000 1.000 2009 Korea 1.000 1.000 2009 China 0.730 0.773 2009 Taiwan 1.000 1.000 2009 Mean 0.875 0.939 2010 France 0.855 0.988 2010 Germany 0.792 0.994 2010 Italy 0.841 0.993 2010 Spain 0.691 0.777 2010 Japan 1.000 1.000 2010 Korea 0.999 1.000 2010 China 0.651 0.858 2010 Taiwan 1.000 1.000 2010 Mean 0.854 0.951 2011 France 0.855 0.921 2011 Germany 0.767 0.873 2011 Italy 0.833 0.909 2011 Spain 0.542 0.591 2011 Japan 1.000 1.000 2011 Korea 0.995 1.000 2011 China 0.852 1.000 2011 Taiwan 1.000 1.000 2011 Mean 0.855 0.912 2012 France 0.824 0.897 2012 Germany 0.751 0.852 2012 Italy 0.821 0.874 2012 Spain 0.560 0.595 2012 Japan 0.989 1.000 2012 Korea 1.000 1.000 2012 China 0.891 1.000 2012 Taiwan 1.000 1.000 2012 Mean 0.854 0.902 CRS TE = Technical Efficiency from CRS DEA VRS TE = Technical Efficiency from VRS DEA SE = Scale Efficiency = CRS TE/VRS TE Year Network SE 0.870 0.891 1.000 0.827 0.902 1.000 1.000 0.545 0.879 0.831 0.852 0.929 0.922 0.901 1.000 1.000 1.000 0.929 0.871 0.840 0.883 0.913 1.000 1.000 0.944 1.000 0.931 0.865 0.796 0.848 0.889 1.000 0.999 0.758 1.000 0.894 0.928 0.878 0.916 0.917 1.000 0.995 0.852 1.000 0.936 0.919 0.881 0.939 0.942 0.989 1.000 0.891 1.000 0.945 © AET 2014 and contributors
23
Service Effectiveness CRS TE VRS TE 1.000 1.000 0.714 0.724 0.518 0.536 0.961 1.000 1.000 1.000 0.800 0.848 1.000 1.000 0.759 1.000 0.844 0.888 0.946 1.000 0.665 0.683 0.504 0.522 0.891 1.000 0.806 1.000 0.682 0.776 1.000 1.000 0.608 1.000 0.763 0.873 0.832 0.973 0.625 0.641 0.531 0.595 0.783 1.000 0.665 1.000 0.644 0.872 1.000 1.000 0.706 1.000 0.723 0.885 1.000 1.000 0.791 0.810 0.638 0.676 0.898 1.000 0.844 1.000 0.886 1.000 1.000 1.000 0.831 1.000 0.861 0.936 1.000 1.000 0.806 0.825 0.652 0.654 0.890 1.000 0.915 0.977 1.000 1.000 1.000 1.000 1.000 1.000 0.908 0.932 0.970 0.984 0.835 0.863 0.676 0.718 0.884 1.000 0.929 0.977 0.971 1.000 1.000 1.000 1.000 1.000 0.908 0.943 SE 1.000 0.986 0.968 0.961 1.000 0.944 1.000 0.759 0.952 0.946 0.973 0.964 0.891 0.806 0.880 1.000 0.608 0.884 0.855 0.974 0.892 0.783 0.665 0.739 1.000 0.706 0.827 1.000 0.977 0.944 0.898 0.844 0.886 1.000 0.831 0.923 1.000 0.977 0.998 0.890 0.937 1.000 1.000 1.000 0.975 0.987 0.968 0.942 0.884 0.950 0.971 1.000 1.000 0.963 System Efficiency CRS TE VRS TE 1.000 1.000 0.773 0.784 0.626 0.674 0.856 1.000 1.000 1.000 0.942 1.000 1.000 1.000 0.521 1.000 0.840 0.932 1.000 1.000 0.821 0.836 0.571 0.599 0.794 0.888 0.998 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.898 0.915 1.000 1.000 0.786 0.790 0.716 0.731 0.850 0.910 0.943 1.000 0.904 1.000 1.000 1.000 1.000 1.000 0.900 0.929 0.954 1.000 0.765 0.801 0.677 0.696 0.623 0.640 1.000 1.000 1.000 1.000 0.937 1.000 1.000 1.000 0.898 0.930 1.000 1.000 0.824 0.871 0.718 0.720 0.836 0.847 0.965 1.000 1.000 1.000 0.840 1.000 1.000 1.000 0.870 0.892 0.874 0.956 0.735 0.740 0.622 0.647 0.555 0.579 0.949 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.842 0.865 SE 1.000 0.985 0.929 0.856 1.000 0.942 1.000 0.521 0.904 1.000 0.983 0.952 0.894 0.998 1.000 1.000 1.000 0.978 1.000 0.995 0.980 0.933 0.943 0.904 1.000 1.000 0.969 0.954 0.955 0.973 0.973 1.000 1.000 0.937 1.000 0.967 1.000 0.946 0.998 0.986 0.965 1.000 0.840 1.000 0.974 0.914 0.994 0.962 0.959 0.949 1.000 1.000 1.000 0.972