Pregunta de entrevista de HungerStation

How would design a crawler