You are here: Home > FAQ

FAQ Codility

 

If I subscribe, how do I unsubscribe/cancel?

How is Codility different from Brainbench, Prometric, etc.?

We have a fine-tuned recruitment process, should we replace it with Codility?

How do you know Codility does not reject good candidates?

I am looking for C++ tests (Java tests, Python tests, Ruby tests),
but I don't see them mentioned explicitly at your website?...

How do I know which task is difficult and which one is easy?

Candidate scored 70 out of 100. How do I know
whether this is a good or bad result?

Can I use one subscription to screen candidates in multiple branches of my multinational corporation?

Can I create my own, customized tasks?

Do you provide task descriptions in languages other than English?

I want to use Chinese versions of task descriptions, how do I access them?

Can I get my money back?

I am hiring a senior software engineer, don't you think that top-shelf candidates will get offended if I ask them to solve a simple programming test?

Does Codility assess coding style?

It takes much more than programming skills to be a good software engineer (communication skills, social skills, design skills). How does Codility deal with it?

How will Codility assess solutions that are correct, but unusual?

You purport that a performant and scalable solution is a good quality solution. Maintainability and well-formedness are even more important today, why do you ignore them?

What versions of programming languages are used to assess solutions?

What libraries are available to the candidate's programs?

Are solutions executed on 32- or 64-bit hardware?

I received an e-mail saying "Candidate requires special arrangements to be made due to a disability". What is this?

 

If I subscribe, how do I unsubscribe/cancel?

You can unsubscribe at any time from the web interface (log in, go to Dashboard|MyAccount|Unsubscribe). If for any reasons you cannot do it, but you wish to unsubscribe, let us know at support(at)codility.com and we will cancel the subscription for you.

 

How is Codility different from Brainbench, Prometric, etc.?

Several test vendors offer multiple-choice question tests to verify the candidate's knowledge. At Codility we request candidates to write actual code in a programming language to verify the practical programming skills. Using a metaphore of driving license examination, Codility is like a road test -- you have to jump in a car and demonstrate your skills in practice.

 

We have a fine-tuned recruitment process, should we replace it with Codility?

No, put Codility in front of your recrutiment pipeline to reduce workload. Codility is best applied as a first-line filter. It is designed to filter out the underqualified candidates early and thus reduce the number of candidates you have to run through your regular process. We have seen 10x reduction of workload in the recruitment pipeline (read the story).

 

How do you know Codility does not reject good candidates?

For every solution Codility demonstrates specific testcases exposing flaws in correctness or performance of the solution. Given that candidates are given a hefty amount of time (30 minutes per task) and that solutions are generally short (around 10 lines of code), inability to deliver a correct solution is a strong indication of problems with programming.

 

I am looking for C++ tests (Java tests, Python tests, Ruby tests),
but I don't see them mentioned explicitly at your website?...

Each Codility test is available in 13 programming language versions (C, C++, C#, Java, JavaScript, Pascal, Perl, PHP, Ruby, Python, VB.NET, SQL and Scala). Recruiter can require candidates to use specific programming language (e.g. C++) or may leave the choice of the language to candidates.

 

How do I know which task is difficult and which one is easy? What are the stars next to the problem name?

We assign star ratings (one, two or three stars) to indicate the difficulty level. One star means easy, three stars means difficult. Scientific textbooks frequently apply the same convention.

 

Candidate scored 70 out of 100. How do I know
whether this is a good or bad result?

If you are looking for high-end programmers this is rather poor result, high-end programmer should get scores above 90%. Whether you should reject a candidate who got 70% depends on how badly you need to hire and whether you have other candidates. 70% scored on 3-star (difficult) problem is better than 70% on 1-star (easy) problem. We suggest that you simply run your candidates through Codility, sort by score and talk to candidates with higest scores first.

 

Can I use one subscription to screen candidates
in multiple branches of my multinational corporation?

One subscription is good for one country. If you need to use Codility in multiple countries, you need to purchase multiple subscriptions (come on, it's only a few bucks!...).

 

Can I create my own, customized tasks?

If you have your favorite tasks that you would like to have imported into Codility, we can provide a service of importing them to Codility, contact us at support(at)codility.com for details. Your tasks will have to go through our process so that we make sure that the evaluation machinery and the nature of the tasks yield meaningful results. (Some programming tasks will not yield meaningful and fair scores, for example Traveling Salesman or other NP-complete problems).

 

Do you provide task descriptions in languages other than English?

Part of the task pool is available in Chinese. Adding translations is costly, because we have to be absolutely sure that there are no logic errors or discrepancies between translations. If you would like to see Codility problems in your language and your demand for Codility service justifies the cost of translation, please let us know at support(at)codility.com and we will be more than happy to work on it.

 

I want to use Chinese versions of task descriptions, how do I access them?

Conctact us at support(at)codility.com and we will enable them for you (it's free).

 

Can I get my money back?

We cannot refund the first payment, because it would be too easy for candidates to subscribe, review all problems, then get the money back. If you had problems canceling your subscription and you were re-billed unintentionally, please let us know at support(at)codility.com and we will refund the unintended payment.

 

I am hiring a senior software engineer, don't you think that top-shelf candidates will get offended if I ask them to solve a simple programming test?

Codility tests are fundamental, not necessarily simple. People who get offended too easily are probably those who you don't want to hire. On the other hand serious candidates like to be treated seriously, avoid simple tasks (aim at 3-star and 2-star tasks only). It helps to invite people on-site and ask them to solve a challenge (not a test), tell them it is a starting point for the interviews (good engineers enjoy interesting problems and are excited to discuss them, which is great canvas for a technical interview). WARNING: impressive CV does not mean that one can do programming right (we have seen PhDs in Computer Science from excellent schools who could not).

 

Does Codility assess coding style?

Codility assesses fundamental programming skills. Coding style assessment is very much an aesthetic judgement and it is best if it is performed by a human, preferably a prospective peer or manager of the candidate. We see ways to add some support for coding style assessment to Codility, but we doubt that Codility will ever judge whether a piece of code is nice or ugly.

 

It takes much more than programming skills to be a good software engineer (communication skills, social skills, design skills). How does Codility deal with it?

Codility filters out candidates who cannot get programming right. Period. Codility is not a substitute for an interview, Codility merely makes sure that you won't lose time to interview candidates who lack programming skills (we have seen cases where non-programming programmers account for 90% of applicants).

 

How will Codility assess solutions that are correct, but unusual?

Every solution that satisfies specification given in task description receives a high score, even if it is not similar to solutions that we had in mind when preparing task description. We specifically design tasks and evaluation machinery in such a way that unorthodox or ingenious solutions are scored fairly.

 

You purport that a performant and scalable solution is a good quality solution. Maintainability and well-formedness are even more important today, why do you ignore them?

We do not pretend to assess all the skills. Codility is a minimum competency test in programming: whoever fails, should not be hired as a programmer, whoeven passes, needs to be tested for remaining skills. When you are hiring a chaffeaur or chauffeuse you want the person to pass a road driving test, but this won't tell you whether the candidate will keep the car tidy.

 

What versions of programming languages are used to assess solutions?

The problems are simple enough to be solved with core constructs present in all versions of any given programming language; if you really need to convince yourself whether a particular feature works as expected, use "Run" button in the demo test or in the test proper.

 

What libraries are available to the candidate's programs?

The execution environment consists of the programming language's standard library and nothing more. If you ever need to convince yourself that a particular component is available, use "Run" button in the demo test or in the test proper.

 

Are solutions executed on 32- or 64-bit hardware?

Although in some programming languages the underlying hardware characteristics are exposed, the solutions generally can be written independently from them. If you ever need to verify whether you can rely on particular feature of the execution environment, use "Run" button in the demo test or in the test proper.

 

I received an e-mail saying "Candidate requires special arrangements to be made due to a disability". What is this?

This e-mail indicates that a candidate (to whom you have administered a test) has clicked a button indicating that his/her physical disability prevents him/her from completing the test on-line. You should contact the candidate directly to find the appropriate way of testing (e.g. phone screen, in-person interview). This function is critical when Codility is used as a part of purely automated flow that might otherwise silently discriminate against candidates with physical disabilities.