6 Commity (1cbd5df5aea8f8755a2a0b75998abaaf0813b05d)

Autor	SHA1	Wiadomość	Data
Štěpán Škorpil	1cbd5df5ae	Added config for seed domain timeout	2022-11-29 14:11:27 +01:00
Štěpán Škorpil	704c7c066e	Added robots.txt handling	2022-11-22 16:37:11 +01:00
Štěpán Škorpil	29acce3906	Properly applied code quality tools	2022-09-18 13:32:25 +02:00
Štěpán Škorpil	b2725e5acf	Added nodeinfo content length limit (5k) solving issues with global redirects to endless streams	2022-04-18 22:18:43 +02:00
Štěpán Škorpil	0c2ade0b52	Added more env configs	2022-01-03 13:26:17 +01:00
Štěpán Škorpil	74e7dca8bd	Initial crawler commit	2021-12-23 15:14:06 +01:00