the company provides a test task. which is good, but the test must be adequate the current test provides 2 huge dlls (2 different versions of the same dll) and 15 points inside one dll. it is proposed to find the corresponding 15 points in another 3 hours are given for this (that is, on average 12 minutes per point) 1. This test only checks ownership of the static disassembler. in real work we also use a debugger for such tasks. this test does not provide an opportunity to use and test these skills 2. a large number of points (15) that is, a large number of absolutely similar tasks. For what ? It would be reasonable to reduce the number of points, say to 5, and instead of the others, give a different type of problems. Is all that you need to do and be able to do in this position to find such “points”? There is nothing more to check / ask? 3. limited and insufficient time to complete the task. If 1-2 days were given to complete such a task, I don’t think that this would reduce the information content of the test (on the contrary) or would give the candidate time to find a solution on the Internet or ask for help (very unlikely ). I doubt that in real work you need to be able to find this type of match in 10-15 minutes. 4. Some dots indicate the address inside the instruction. why so ? check whether the candidate understands the difference between the address of an instruction and the address inside it? a meaningless - those who do not understand such a basic thing will not be able to find a single point. on the other hand, this creates a real possibility of making a random error in the answer. and this error will not be associated with a lack of knowledge/understanding, but will be precisely typo 5. since there are many points and some cannot be found quickly (within the allotted 12 minutes), we are forced to skip some (and return to them later, if there is time) and move on to new ones. This creates the possibility of another typo - entering the address in the wrong cell (out of 15) is needed. Are this type of typo taken into account? I think no. 6. in the description of the task there is a warning that an incorrect answer will result in a severe fine (which is not specified). that is, 1 incorrect answer (even typo) “deletes” 2-3 (or even more?) correct answers. the question is - why do this? if you had questions with 4-5 answer options and you had to choose one correct option - in such a situation the penalty for the wrong answer is justified. After all, even without knowing (not being able to calculate) the correct answer, you can accidentally guess from 4-5 options. and the point of the penalty is to prevent guessing. with this test, each dll contains more than 1000 functions. sizes - 18-30mb. the probability of accidentally guessing (choosing a point from thousands of possible ones) is negligible. One of the points (functions) in the test approximately corresponds to 2 points (functions) from another dll. all 3 functions are very close to each other. and all 3 are different (there are slight differences). It is almost impossible to determine which of the 2 functions corresponds more accurately in the allotted time. then how to answer such a question/point? when you have 2 answer options and approximately 50%/50%. Considering that the penalty for an incorrect answer is significantly higher than the points for a correct answer, the test encourages the candidate not to give an answer to such a point at all. and all his work to find her and find 2 possible matches will be lost. maybe we found something. maybe they didn’t find anything - it won’t be visible. and there is no option to write honestly - I believe that this is one of 2 options, but I cannot determine which one specifically. 50/50. and it is completely inappropriate to penalize such an error in this test (on the contrary, it would be more reasonable to count this as 1/2 - 1/3 of the answer) 7. refusal form. as it was said that you failed the test. a template letter that is sent to everyone the same (changing only the name of the candidate). with obvious elements of untruth in the letter (“compared to other solutions”, etc.). result - how many correct/incorrect answers were given. and/or points scored are not given. It doesn’t say how many correct answers you had to give (or score points), how many you “fell short.” “You understand, we cannot give such information” - no, I don’t understand why you can’t. I don’t see any information leakage here; generally speaking, you have I correct answers and J incorrect ones (without specifying which are correct and which are not). and to pass the test you need at least N answers (without penalty). I'm not even talking about the possibility of typo For example, I gave solutions for 13 points out of 15. At the same time, one point is 50%/50% - I deliberately did just that, although I understood that it was not profitable - the minus for not guessing would be significantly higher than the plus. out of
