The total number of examinees of the PROGOS test, AI English speaking exam,  provided by RareJob, Inc. (hereafter “RareJob”), which utilizes data related to human resource to develop global talents has exceeded 70,000 in June 2021. It has been a year since its launch.
We conducted a comparative analysis of speaking levels of examinees of the PROGOS test and the scores of TOEIC® Listening & Reading (hereafter “TOEIC® L&R”). This research showed the discrepancy among each level of English skills such as reading, listening and speaking.

The  Large-Scale Comparative Analysis of The PROGOS Test and TOEIC® L&R Results

Although a certain TOEIC® L&R score is required for promotion in a company, those who have high scores of TOEIC® L&R are likely to be unable to speak English fluently and deal with international business. This time, the comparative study was conducted to clarify the correlation between TOEIC® L&R test results and English speaking skill. Our study targeted 5,190 people who took the PROGOS test and provided their TOEIC® L&R scores. In the analysis, we compared their listening, reading and speaking skills by using the CEFR level as the common index. CFER is used as the common standard in various language tests.

Comparison of results of the PROGOS tests and TOEIC® L&R with CEFR

The study showed many people with CEFR B2 level cannot work adequately in English even if they got B2 level (the level that allows you to meet business needs in English) in TOEIC® L&R.
It is said that CFER B2 or higher is needed to work sufficiently in English and many global companies require B2 or higher for their employees expected to work internationally.
According to the analyses, examinees with B2 or higher (with 785 points or higher in TOEIC ® L&R) accounted for 31.3% of the total. Those with B2 or higher in the PROGOS test account for only 3.3%. In addition, those with B2 or higher in both TOEIC ® L&R and the PROGOS test accounted for 3.3%. These results revealed that a level of speaking skill is not related well to levels of reading and listening skills.

An English speaking skill is essential for international business. It proves that companies need to assess an English speaking skill as an indicator to understand how their employees are able to manage international business.

<Comparative analysis>

・Period: November 2020 to June 2021
・Targets: 5,190 people who took the PROGOS test (automated scoring) and provided TOEIC® L&R score during the above period
・About score sorting
-TOEIC® L&R scores are classified from A1 to C1 based on the following page.
https://www.iibc-global.org/toeic/official_data/toeic_cefr.html
・The overall result of the PROGOS test clarifies from A1 to B2. B2 includes B2 and higher levels.

The PROGOS test became the most taken English speaking test in Japan after its launch a year ago

Currently, many companies and universities use only TOEIC® L&R scores measuring English listening and reading skills as the indicator to evaluate how people can manage international business in English. However, the need to visualize English speaking skill has begun emerging since the PROGOS test launched and the number of examinees for the last 1 year exceeded 70,000 in June 2021.

At the start of its launch, only B2C and B2B members of “RareJob Eikaiwa” were eligible to take the PROGOS test. However, the test became available to B2B customers even if they do not subscribe to “RareJob Eikaiwa” in December 2020, therefore, the number of examinees has been increasing.
It became the English speaking test which has the largest number of examinees per year in Japan in the first year since its launch, compared to other English speaking tests. (*1)

The number of examinees of the PROGOS test has increased faster than TOEIC® L&R and other English tests for a year since its launch, which is overwhelming. We assume that the need for visualizing a practical English speaking skill has expanded, by introducing CEFR measured by PROGOS with TOEIC® L&R score which has been widely used to assess an English proficiency.

Challenges for companies and universities to promote to measure an English speaking skill

In order to visualize a speaking skill, more and more companies and universities have been introducing the PROGOS test for their employees and students to solve the following issues.

①TOEIC® L&R score is not a proof that their employees or students have enough speaking skills for work
・TOEIC® L&R score has been one of requirements for promotion to managerial and overseas posts, but some employees who have the required scores and contributed well in a domestic business could not make his/her ability the most in an international business due to lack of a English skill. Therefore, we decided to measure English speaking ability by the PROGOS test using CEFR as its index (Company A).

・Although we have English training based on TOEIC® L&R scores, there is a gap between scores and practical English speaking skills. We introduced the CEFR index that clarifies what can be done in practice for each level to evaluate the speaking skills of all their employees. We also carry out training to improve an English speaking skill according to each weakness of each employee analyzed by the test result (Company B).

②The cost of speaking tests is expensive, although introducing the tests for all employees is desirable.
・When English speaking tests used to be conducted in our company building, human resources and costs for tests were burdened a lot. However, employees can take tests anytime and anywhere, which greatly reduces these costs since the PROGOS test can be taken online only. In addition, results are given in minutes, which helps examinees stay motivated (Company C).

・Speaking tests which cost over 5,000 yen on average make it difficult to provide opportunities to take tests, though we would like to provide all employees with the opportunity to take examinations to form an international culture throughout the company. However, the PROGOS test, which can be taken at one tenth of the average price, enables us to measure the speaking skills of our employees. (Company D)

About the PROGOS test

The PROGOS test is the business English speaking test that uses AI to measure a “practical” English speaking skill based on the global index, CEFR. It gives results in a minimum 2-3 minutes. Fees of most English speaking tests has been over 5000 yen on average, due to costs for human resources to score tests and venues to conduct tests. The penetration of English speaking tests has been limited in Japan so far. The PROGOS test developed its AI-based system to automatically assess free answers in English to resolve the issue. This provides a new opportunity to take tests quickly and inexpensively regardless of the capacities of examiners. Because these advantages were highly regarded, the PROGOS test won the first silver prize in Japan in one of the world’s largest educational contests in December 2020. (*2)

Comments by President, COO of PROGOS Inc., Masuyo Ando

One Year has passed since PROGOS was released, I am sure that companies, universities and English learners have potential needs to “measure speaking skills easily” and “compare speaking skills based on the globally-renowned index.” More and more companies have been using the test to evaluate the speaking skills of their employees since December 2020. The PROGOS test can be taken without using our service such as “RareJob Eikaiwa” since that time. As DX (digital transformation) and other major changes have been seen in the current business environment, visualising skills and acquiring the skills faster should be realised as soon as possible. In this context, the PROGOS test goes beyond the conventional concept of “tests” and has contributed to the acceleration of human development by “Measure & Develop.”

Comments by Professor Yukio Tono, the leading person in the Japanese CEFR research

It has been one year since the PROGOS test launched. The presence of the test is outstanding as the English speaking test for business people. The test is based on CEFR-J which　was optimized for Japanese learners studying English and is compatible with CFER. In particular, the PROGOS test does not allow others to follow its capability which helps an examinee assess and improve his/her English skill from beginner to intermediate levels. The step-up to intermediate learners from beginners is the most difficult part for Japanese people. We are confident that the PROGOS test lets you acquire the necessary English skill to lead the globalization in Japan.

Professor, Graduate School of Global Studies
Director, the World Language Center
Tokyo University of Foreign Studies

*1 Based on our research
*2 Received the Silver Prize in Reimagine Education Award 2020, Learning Assessment category. The Japanese organization won this award for the first time.

About PROGOS Inc.

Address: 6th Floor, Cranes Park Building, 18-1, Jingumae 6-chome, Shibuya-ku, Tokyo
Representative: Masuyo Ando, President
URL:https://www.progos.co.jp/
Business: Evaluation, training, and recruitment of global leaders