mirror
/
moonstream
kopia lustrzana https://github.com/bugout-dev/moonstream
1
0
Forkuj 0
Kod Zgłoszenia Projekty Wydania Wiki Aktywność
3 463 Commity
109 Gałęzie
42 Tagi
21 MiB
Wykres commitów

872 Commity (main)

Autor SHA1 Wiadomość Data
Andrey f68ab988c9 Add jobs folder. 2022-12-15 16:18:44 +02:00
Andrey a4dd79a663 Add read tasks from json file and custom web3 provider. 2022-12-15 15:45:41 +02:00
Andrey 6a379c1b55 Merge branch 'main' into fix-state-crawler 2022-12-15 14:26:00 +02:00
kompotkot 36e58c7275 Fix deprecated keys for gnosis 2022-12-14 13:47:57 +00:00
Andrey 36888404ef Add rollback. 2022-12-05 13:56:22 +02:00
Andrey f0857b4d73 Move hash generation to actions. 2022-11-30 17:31:19 +02:00
Andrey 0aeae49c97 Merge branch 'main' into add-parameters-in-s3-path 2022-11-30 17:14:37 +02:00
Andrey Dolgolev e917d9361d
Merge pull request #721 from bugout-dev/fix-passing-list-as-sql-parameter 
Fix passing list as sql parameter
 2022-11-29 17:15:31 +02:00
Andrey 5b4ddc6feb Fix pass params. 2022-11-29 17:05:22 +02:00
Andrey e34f544099 Add converting from json types. 2022-11-29 15:06:27 +02:00
kompotkot f3347224c9 Moonworm crawlers for ethereum 2022-11-28 17:35:18 +00:00
Andrey c7e8402785 remove comment. 2022-11-28 17:18:27 +02:00
Andrey 6c8046908d Black formating. 2022-11-28 17:16:53 +02:00
Andrey f60fe8693c Fix route. 2022-11-28 16:52:30 +02:00
Andrey b44881a28d Refactor code.. 2022-11-24 15:40:26 +02:00
Andrey a66039ad38 Move db state information. 2022-11-24 15:13:15 +02:00
Andrey 7bf1ef00af Move hash generation in api handler. 2022-11-24 15:02:32 +02:00
Andrey 3800c558b8 Move push to bucket in actions.py 2022-11-24 14:42:52 +02:00
Andrey e8a1187889 Add hash of parameter to s3 route. 
Add fix for list support.
 2022-11-24 14:40:39 +02:00
Andrey b8b72ea35d Remove refetch add later after apear hash of paramaters in s3 route. 2022-11-17 18:20:33 +02:00
Andrey f54c113e65 Bigger await for results. 2022-11-17 17:42:10 +02:00
Andrey 1a3509d554 Replace prints via logger. 2022-11-16 15:50:44 +02:00
kompotkot 8f98d5feba Updated service file for cu queries crawler 2022-11-16 13:36:36 +00:00
Andrey aae3b9bd3e Add MOONSTREAM_PUBLIC_QUERIES_DATA_ACCESS_TOKEN for run queries for public dashboard. 2022-11-16 15:20:05 +02:00
Andrey f5fc52e8be Add public queries access token. 2022-11-16 15:10:08 +02:00
Andrey Dolgolev ca8aef467f
Merge pull request #693 from bugout-dev/add-cu-reports-crawler 
Add cu reports crawler
 2022-11-16 14:48:20 +02:00
Andrey 1dff0431ed Fix mypy. 2022-11-16 14:31:53 +02:00
Andrey 9d89b6231e Try again. 2022-11-16 14:21:25 +02:00
Andrey d23e83e15a Ignore mypy not recognize of Any type in params. 2022-11-16 14:17:09 +02:00
Andrey 9c2b3d42a1 Fix mypy. 
Ignore for now note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
 2022-11-16 14:13:02 +02:00
Andrey 16a3dc24f0 Add requested fixes: 
1) CUSTOM_CRAWLER_S3_BUCKET -> MOONSTREAM_S3_PUBLIC_DATA_BUCKET_PREFIX
2) Add overwrite for init queries handlers
3) Add sleep
 2022-11-16 14:05:01 +02:00
Andrey d00c9d0790 Modify sample env. 2022-11-15 19:01:41 +02:00
Andrey 4808d600de Add creat query command. 
Add additional re request update logic.
Add volume query.
 2022-11-15 18:57:35 +02:00
Neeraj Kashyap 7172666a90
Merge pull request #685 from gnawali/patch-1 
Update README.md
 2022-11-14 16:50:04 -08:00
Andrey 5e100228cd Fix description and change version. 2022-11-14 16:03:51 +02:00
Andrey 69835d09c8 Black formating. 2022-11-09 22:09:17 +02:00
Andrey 2e65a2b889 Fix mypy. 2022-11-09 21:36:30 +02:00
Andrey c992475765 Merge branch 'main' into add-cu-reports-crawler 2022-11-09 21:20:12 +02:00
Andrey ef423b9739 Merge branch 'add-cu-reports-crawler' of github.com:bugout-dev/moonstream into add-cu-reports-crawler 2022-11-09 21:10:41 +02:00
Andrey 0c38a3d90e Add fixes in create queries logic. 2022-11-09 21:09:10 +02:00
Andrey 75ae88d2c3 Add fixes for parameters. Remove recrawling from different level already existing data. 
Current state have issues from node. Timeouts and unsinchronize.
 2022-11-09 19:47:04 +02:00
kompotkot 5fa3c07497 Create dir for env vars 2022-11-09 14:23:37 +00:00
kompotkot e8e2481428 Migrated deployment to checkenv 2022-11-09 11:47:55 +00:00
Andrey db77374149 Merge branch 'main' into fix-state-crawler 2022-11-08 23:16:51 +02:00
kompotkot 897d2d7321 Tokenonomics systemd services and changed on boot start timers 2022-11-03 16:56:18 +00:00
Andrey 0c6656fad5 cu dashboard generator. 2022-11-03 16:42:54 +02:00
kompotkot 592d2cf667 Updated moonworm version 2022-11-02 14:42:42 +00:00
kompotkot 2d1a4c4c9c Mumbai moonworm crawler 2022-11-02 13:37:23 +00:00
Andrey e23fd52453 Add terminus. 2022-10-27 18:46:16 +03:00
Andrey 920c8941ef add reports. 2022-10-27 16:41:08 +03:00
Andrey 0436ab784a init state. 2022-10-27 13:00:08 +03:00
Andrey 15126c5086 Add autoscale batch size. 2022-10-26 14:48:30 +03:00
Andrey c580a7d3c2 Add jobs. 2022-10-20 21:20:01 +03:00
gnawali 777414c7db
Update README.md 
The site is now available only with www prefix
 2022-10-20 11:36:17 -05:00
Andrey 94e98a3059 Add logging and thread poolexecutor. 2022-10-20 17:40:06 +03:00
kompotkot accb5caf03 Mumbai sync and missing crawlers 2022-10-19 09:22:06 +00:00
Sergei Sumarokov a8470441cb
Merge pull request #680 from zomglings/fix-crawler-tx-rollback 
Added transaction rollback when there is a violation of unique constraint when crawling blocks
 2022-10-18 17:13:44 +03:00
Andrey 9f441373fe Add covert args address to checksum address. 2022-10-18 16:39:44 +03:00
Andrey b8ce45470b Add logging. 2022-10-18 16:28:03 +03:00
Neeraj Kashyap 3d7bbb03ce Added a rollback when unique constraint is violated 
when attempting to add blocks to database.
 2022-10-14 21:05:49 -07:00
Andrey 2f5147a84f Add timeout for HTTPProvider. 2022-10-05 18:18:12 +03:00
Andrey 20cda62a1b Fix missing rollback. 2022-10-05 14:26:56 +03:00
Andrey e5f2c4a550 Blocks cache was increased. 2022-10-03 18:43:40 +03:00
Andrey 03af327f8e Move log under if. 2022-10-03 17:26:58 +03:00
Andrey dd9b462c4e Request not full block db raw. Move log under if. 2022-10-03 17:25:39 +03:00
Andrey cccb42156e Remove unnecessary log. 2022-09-21 19:09:47 +03:00
Andrey 8c6d22c88b Replace moonworm==0.5.1 -> moonworm[moonstream]==0.5.1. 
Change version.
 2022-09-21 19:01:04 +03:00
Andrey 3e3288a55f Add remove comments. 2022-09-21 18:41:42 +03:00
Andrey 3f35b47a91 Add autoscale to historical crawl. 2022-09-21 18:38:05 +03:00
Sergei Sumarokov 01d0b8dd03
Merge pull request #670 from bugout-dev/fix-timer-on-boot 
Different timer on boot start time
 2022-09-20 15:35:03 +03:00
Andrey bffe318cd9 Remove address requirements for historical_crawler. 2022-09-20 14:14:37 +03:00
kompotkot a4c04a472a Different timer on boot start time 2022-09-15 13:22:10 +00:00
kompotkot 66ac8c9d5e Clean state service and timer 2022-09-14 14:24:37 +00:00
Andrey 2b8e1f4b7b Add commit in that commit. 2022-09-14 15:51:38 +03:00
Andrey e8f787dcb0 Fix. 2022-09-14 13:29:06 +03:00
Andrey 7074eb9923 Add orm version of query. 2022-09-14 13:23:35 +03:00
Andrey d31ef8b909 Add initial version. 2022-09-13 19:03:48 +03:00
kompotkot 7dc045c3bf Metadata services and deploy 2022-09-08 19:17:30 +00:00
Andrey d200e82460 Fix user MOONSTREAM_DB_URI_READ_ONLY -> MOONSTREAM_DB_URI. 2022-09-08 18:34:50 +03:00
Andrey ec718f0e4c Move table name from parameters to format. 2022-09-08 17:51:01 +03:00
Andrey 14dedab479 Merge branch 'main' into metadata-crawler 2022-09-08 17:42:40 +03:00
Andrey 31aa042dfa Fix mypy. Add logger. Query use table name. 2022-09-08 17:39:28 +03:00
Andrey b0a605cf90 Add logging. 2022-09-08 13:26:14 +03:00
Andrey Dolgolev 7252d06cad
Merge pull request #659 from bugout-dev/cu-state-crawler 
Cu state crawler
 2022-09-08 13:04:50 +03:00
Andrey 4a22a593e0 Fix moonstream db user MOONSTREAM_DB_URI_READ_ONLY -> MOONSTREAM_DB_URI. 2022-09-08 13:01:44 +03:00
Andrey 808c604299 Change veesion. 2022-09-08 12:53:00 +03:00
Andrey 95ab437a22 Merge branch 'cu-state-crawler' of github.com:bugout-dev/moonstream into cu-state-crawler 2022-09-08 12:52:16 +03:00
Andrey 88d1c5a77b Add fixes. 2022-09-08 12:51:41 +03:00
kompotkot 6582d12759 Deployment for state crawler 2022-09-08 09:39:38 +00:00
Andrey def1dc1527 Add sanityze for label data. 2022-09-07 15:15:15 +03:00
Andrey 7d073c3e63 Fix lint issues. 2022-09-07 14:48:33 +03:00
Andrey 8855097728 Switch using web3py instead brownie. 2022-09-06 19:05:02 +03:00
Andrey 12cbe3e797 Add replace of r"\u0000". 2022-09-05 17:05:11 +03:00
Andrey d9c80e566c Add general interface. 2022-09-05 13:35:48 +03:00
Andrey 59f8f22b2e Clean up code. 2022-09-01 17:26:57 +03:00
Andrey 42ebb66e4d Switch to urlopen lib. 2022-09-01 16:04:07 +03:00
Andrey aaa6c95953 Add init metadata parser. 2022-08-31 19:26:30 +03:00
Andrey c2d790d08d Add settings. 2022-08-31 16:50:36 +03:00
Andrey 7fe587fdbc Merge branch 'main' into cu-state-crawler 2022-08-30 17:38:47 +03:00
Andrey 54870630c4 Add inital version. 2022-08-30 17:37:38 +03:00
Andrey b3eba087f0 fix try. 2022-08-25 20:28:57 +03:00
Andrey af8e114515 Update version in setup.py add pool_pre_ping. 2022-08-25 20:25:26 +03:00
Andrey 5646dbf442 Add fixes. 2022-08-25 19:58:02 +03:00
Andrey 77c3e7e2f8 Add worked tasks resolver without write to database. 2022-08-25 19:09:55 +03:00
Andrey 9d8566cb59 Black formating. 2022-08-25 15:22:01 +03:00
Andrey 935ebe11a2 remove print. black formating. 2022-08-25 15:21:09 +03:00
Andrey e6a730d760 Add more complex typecheck. 
Tested new created session with custom timeout..
Add reporter for stats execution.
 2022-08-25 15:13:46 +03:00
kompotkot 1cfd22adb4 Mypy fix 2022-08-17 14:56:35 +00:00
kompotkot 6a140e7cac Naming and try catch for moonstream bugout client api call 2022-08-17 14:50:47 +00:00
Andrey b14eae475c Black formating. 2022-08-17 17:42:37 +03:00
Andrey 1339de073b Fix name. 2022-08-17 17:38:54 +03:00
Andrey 870faf15d7 Add timeout for bugout in crawlers. 2022-08-17 17:25:00 +03:00
kompotkot 70f96ab457 Merge branch 'main' into mumbai-support-crawlers 2022-08-11 14:28:30 +00:00
Andrey 3534456ede Black formating. 2022-08-10 23:25:30 +03:00
Andrey a0b3418bf6 All not string and int columns transform to string. 2022-08-10 21:06:48 +03:00
kompotkot 5b912ff0d9 Bumped version 2022-08-10 17:03:04 +00:00
kompotkot de879a7adf Mumbai chain support 2022-08-10 16:55:49 +00:00
kompotkot 9496214d58 Modifications of crawlers for mumbai 2022-08-09 16:29:19 +00:00
Yhtyyar Sahatov df4f11ef48
Merge pull request #634 from bugout-dev/moonworm-historical-crawler 
Moonworm historical crawler
 2022-07-25 15:49:47 +03:00
kompotkot ab0ce50cf9 There is no need of data key at tx 2022-07-16 15:04:24 +00:00
kompotkot 3b965522d4 Better with exception 2022-06-22 19:01:06 +00:00
kompotkot c2b8a55778 Pointing to correct network at moonworm_crawler 2022-06-22 18:45:53 +00:00
Yhtyyar Sahatov da78bffbd0 removed select 1 2022-06-22 18:19:23 +03:00
Yhtyyar Sahatov 0f110bf16a removed unneeded log 2022-06-22 18:18:02 +03:00
Yhtyyar Sahatov 2336dba7c9 fixed some issues 2022-06-22 15:34:02 +03:00
Yhtyyar Sahatov 4166d71d41 added help message for `crawl` command 2022-06-21 14:33:45 +03:00
Yhtyyar Sahatov a5a9df8695 added 2022-06-21 14:31:26 +03:00
Andrey Dolgolev 648931d2b1 remove comments. 2022-06-21 13:35:11 +03:00
Andrey Dolgolev 96e570039c Black formating. 2022-06-21 13:24:41 +03:00
Andrey Dolgolev ab9622b2c3 Add fix. 2022-06-21 13:21:01 +03:00
Andrey Dolgolev 1b106a08e0 Merge branch 'main' into dashboard-stats-use-replica 2022-06-21 13:12:05 +03:00
Andrey Dolgolev d135a0710c Remove moonworm condition. 2022-06-21 13:00:05 +03:00
Andrey Dolgolev b32a993bd7 Fix. 2022-06-21 12:58:05 +03:00
Andrey Dolgolev 257bcc7728 Remove sqlalchemy logger. 2022-06-21 12:51:31 +03:00
Yhtyyar Sahatov a3f2418820 isort 2022-06-16 15:54:19 +03:00
Yhtyyar Sahatov 14ef9372a2 working hisotic crawler 2022-06-16 15:53:19 +03:00
kompotkot b3fe75ae34 Removed start block for moonworm xdai crawler 2022-06-11 12:07:20 +00:00
kompotkot 2cfdc2dc99 Applied CPU Weight 2022-06-11 12:06:01 +00:00
kompotkot 5cf74404f9 Merge branch 'main' into fix-systemd-assignment-out 2022-06-11 09:51:51 +00:00
Andrey Dolgolev cdfc970f37 Add try select 1; 2022-06-02 23:39:11 +03:00
kompotkot 4f2ccb9830 XDai moonworm crawler service 2022-05-26 14:32:27 +00:00
kompotkot 60e3c078fb Merge branch 'moonworm-prod-xdai-support' into xdai-subscription 2022-05-26 14:23:59 +00:00
Yhtyyar Sahatov 7c80be419b moved commit back to interval 2022-05-26 17:22:45 +03:00
Yhtyyar Sahatov d25b4d6866 moved commit_session out of heart beat interval 2022-05-26 17:19:48 +03:00
kompotkot 3e7b566c63 Black synt fix 2022-05-26 13:43:49 +00:00
kompotkot 9776db79d6 Update moonstreamdb version 2022-05-26 13:31:00 +00:00
kompotkot 5fd98a3b8a Added access id to work with nb 2022-05-26 13:10:55 +00:00
kompotkot ea62342b07 Merge branch 'main' into xdai-subscription 2022-05-26 13:00:47 +00:00
Yhtyyar Sahatov 5dc7716343 added xdai support 2022-05-26 16:00:35 +03:00
Yhtyyar Sahatov 23e4ec367c isort 2022-05-26 15:41:58 +03:00
Yhtyyar Sahatov 5641fe7119 linting fixes 2022-05-26 15:41:27 +03:00
Yhtyyar Sahatov b085bb9049 Merge branch 'main' into nft-dataset-v2 2022-05-26 15:32:26 +03:00
Yhtyyar Sahatov 181d1769a0 fixed cli choices, added help messages 2022-05-26 15:22:09 +03:00
Yhtyyar Sahatov 5d31b8d58a fixed choices in cli 2022-05-26 15:15:12 +03:00
Andrey Dolgolev d04b877621 Add cte condition. 2022-05-26 13:35:19 +03:00
kompotkot d09570b5dc Moonworm version updated in setup dependencies 2022-05-25 14:23:28 +00:00
kompotkot f72a99295e XDai statistics dashboard deployment 2022-05-25 14:10:52 +00:00
kompotkot a2e4565c45 Updated version and moonstreamdb dependence for mooncrawl package 2022-05-25 13:48:42 +00:00
kompotkot b38455826c XDai deployment services and scripts 2022-05-25 10:08:25 +00:00
kompotkot 942eca1330 XDai blockchain support 2022-05-24 08:21:53 +00:00
kompotkot 2bfec90e87 With backslash systemd throwing error with assignment outside 2022-05-17 20:31:01 +00:00
kompotkot 3d97c4cb25 Moonworm crawler update to work with nodebalancer access id 2022-05-17 14:49:11 +00:00
kompotkot 4e358326af Updated version of mooncrawl 2022-05-16 13:46:11 +00:00
kompotkot 0d6f6203b8 Fixed mypy 2022-05-16 13:42:24 +00:00
kompotkot 6fba3957c8 Services with access id for node balancer 2022-05-16 13:31:34 +00:00
kompotkot 668c9b6484 Merge branch 'main' into crawlers-access-id 2022-05-16 12:45:21 +00:00
kompotkot 97567e9de9 Auto deploy of moonworm cu mainnet crawler 2022-05-16 12:35:24 +00:00
kompotkot 7a3b7c15d7 Moonworm crawler optimized to work with access id 2022-05-16 11:53:35 +00:00
kompotkot a25e149e1d Merge branch 'main' into crawlers-access-id 2022-05-16 11:20:09 +00:00
Neeraj Kashyap 78da81790c
Merge pull request #585 from bugout-dev/moonworm-job-creation-fix 
fix v2.
 2022-05-05 09:16:55 -07:00
Andrey Dolgolev a9b0831740 Add black formating. 2022-05-05 13:24:00 +03:00
Andrey Dolgolev 5df12db95c Add fix. 2022-05-05 13:12:33 +03:00
Andrey Dolgolev e7f3a8717b Add fixes for mypy. 2022-05-05 12:52:54 +03:00
Andrey Dolgolev b623ff96aa Merge branch 'main' into dashboard-stats-use-replica 2022-05-05 11:42:42 +03:00
Andrey Dolgolev 49f19e5dd7 Clean up code from prints and traceback. 2022-05-05 11:29:48 +03:00
Andrey Dolgolev a1ce774b99 Add worked merged subscription. 2022-05-03 16:53:34 +03:00
kompotkot a0e3165cf2 Sys log identifiers 2022-05-03 12:39:33 +00:00
Andrey Dolgolev 6a7217a70a Init version of merging in stats generation. 2022-05-02 17:49:55 +03:00
Andrey Dolgolev b58cfd34b8 Switch stats generation to use replica. 2022-04-19 17:02:44 +03:00
Yhtyyar Sahatov 330fcceb51 mypy fixes 2022-04-18 23:36:19 +03:00
Yhtyyar Sahatov a76721fdde recuded min block batch and max block batch 2022-04-18 17:54:35 +03:00
Yhtyyar Sahatov dbbe24fb21 revert settings.py 2022-04-18 17:28:12 +03:00
Yhtyyar Sahatov 440fc0fbef fix v2. 2022-04-18 17:25:13 +03:00
Neeraj Kashyap 84cbfcc38a
Revert "checking function abi before creating job" 2022-04-14 16:00:06 -07:00
Yhtyyar Sahatov 22228a7627 checking function abi before creating job 2022-04-14 19:22:49 +03:00
Yhtyyar Sahatov e82e255709 slightly time optimization 2022-03-22 17:26:59 +03:00
Yhtyyar Sahatov ee0f22cff2 more correct logic for even poppulation 2022-03-21 22:16:24 +03:00
Yhtyyar Sahatov ba541b80af generic crawler fix 2022-03-21 22:05:02 +03:00
kompotkot 6c1143de8e updated txpool services and sample env 2022-03-18 19:19:03 +00:00
kompotkot 631555ab94 specified header for rpc dial if access id is set 2022-03-18 19:12:07 +00:00
Yhtyyar Sahatov 87ccc82049 update erc20 populate 2022-03-18 20:41:28 +03:00
kompotkot d76098718b content type header for rpc connect 2022-03-18 15:23:26 +00:00
kompotkot 3a1de527a4 access id default from env variable for all mooncrawl crawlers 2022-03-17 20:07:28 +00:00
kompotkot 9435568c18 crawler work with access id 2022-03-17 19:19:48 +00:00
Sergei Sumarokov 665a857b29
Merge pull request #557 from bugout-dev/ldb-crawler 2022-03-11 19:37:07 +03:00
Yhtyyar Sahatov 3a8d1bbd70 added continuous crawling 2022-03-10 18:25:21 +03:00
Yhtyyar Sahatov c1729dd836 added generic crawler 2022-03-09 22:35:27 +03:00
kompotkot d375438668 Query validator with tests 2022-03-09 15:51:49 +00:00
kompotkot b408c20b0b Added missed env variables in sample.env 2022-03-09 12:44:54 +00:00
kompotkot 29e84431a6 Bumped db version, extended sample.end, docker config and mypy fix 2022-03-09 12:08:31 +00:00
kompotkot cf14df0333 Merge branch 'main' into add_sql_execution_endpoint 2022-03-09 11:47:34 +00:00
Andrey Dolgolev 2077ea2c81 Add black formating. 2022-03-08 17:10:06 +02:00
Andrey Dolgolev 542f411449 Add using read only user. 2022-03-08 17:06:50 +02:00
Andrey Dolgolev 3c59048459 Fix issue with items. 2022-03-08 16:00:39 +02:00
Andrey Dolgolev f5b2550505 Add fixes. 2022-03-08 15:18:27 +02:00
kompotkot 1473ca3b40 Fixes 2022-03-06 23:33:28 +00:00
kompotkot d55209df42 Code structure organization 2022-03-06 23:21:24 +00:00
kompotkot 838c75ae29 Verification command in goroutines 2022-03-06 22:59:47 +00:00
kompotkot 39c391dfa0 Final 1 thread version 2022-03-06 19:04:01 +00:00
kompotkot c33d033ad9 Smart verification humbug report counter 2022-03-06 18:51:36 +00:00
kompotkot cb8bb515aa Blocks range generator 2022-03-06 18:10:55 +00:00
kompotkot 392bdf801d Insert block 2022-03-06 00:11:13 +00:00
kompotkot e8ae125dd1 Blockchain flag 2022-03-05 19:49:53 +00:00
kompotkot 4085d11226 Respect geth structure due work, optimised sql query to db 2022-03-05 19:13:42 +00:00
kompotkot b7578838c8 Parse uint fix 2022-03-03 21:32:34 +00:00
kompotkot 1642ed9e0a Functionality to report at humbug journal with corrupt blocks 2022-03-03 21:05:33 +00:00
kompotkot 45cc528582 ldb blockchain database crawler 2022-03-03 20:11:40 +00:00
Yhtyyar Sahatov 301f3bb745 added value when saving 2022-02-24 18:32:16 +03:00
Yhtyyar Sahatov 07efbd44fd fucking breakpoint 2022-02-24 18:30:44 +03:00
Yhtyyar Sahatov 14ddc78bac added values to tx 2022-02-24 18:27:57 +03:00
Yhtyyar Sahatov 3cba4d3fb3 fixed sleep time bug 2022-02-22 18:46:17 +03:00
Yhtyyar Sahatov 0bab8d30dd tx value 2022-02-22 18:37:39 +03:00
Andrey Dolgolev d031a71013 Add temp solution for show block and timestamp. 2022-02-17 17:26:48 +02:00
Andrey Dolgolev 1498d1bac6 remove testing. 2022-02-17 12:24:14 +02:00
Andrey Dolgolev 1c741d5084 Add create update get endpoints. 2022-02-17 12:22:20 +02:00
Yhtyyar Sahatov 169dc78563 working crawler 2022-02-17 00:58:37 +03:00
Andrey Dolgolev 4dd101eb43 Add lint fixes. 2022-02-16 18:23:29 +02:00
Andrey Dolgolev 025baeb1c4 Add test workable version. 2022-02-16 18:12:42 +02:00
Andrey Dolgolev 5485a73b98 Add black formating. 2022-02-16 03:00:49 +02:00
Andrey Dolgolev 373f768a3f Quries worker. 2022-02-16 02:57:39 +02:00
Andrey Dolgolev 0cbd177442 Add fixes. 2022-02-16 02:55:14 +02:00
Andrey Dolgolev c41f8d1616 black formating crawlers. 2022-02-16 02:49:23 +02:00
Andrey Dolgolev 3d1108e569 Add json response from drones. 2022-02-16 02:39:49 +02:00
Andrey Dolgolev bc45e5cc76 Add init queries drone api. 2022-02-16 02:29:12 +02:00
kompotkot 3798f39946 In deploy scripts point to go root dir $HOME/ -> /root/ 2022-02-08 08:41:13 +00:00
kompotkot 1d54c8758b Dasboard updated modified from 15m -> 6h 2022-01-31 13:05:33 +00:00
Sergei Sumarokov 7e0c7b52ba
Revert "Client id to work with node balancer" 2022-01-24 20:35:46 +03:00
kompotkot 89caa99213 Services for crawlers with client id 2022-01-24 17:17:44 +00:00
kompotkot deabed9837 Token->client_id 2022-01-24 17:16:58 +00:00
kompotkot c8b458bf8b CLI client token arg for mooncrawl 2022-01-24 14:00:49 +00:00
kompotkot 4830d28a55 Sync crawlers restart automaticly with systemd 2022-01-21 13:13:48 +00:00
kompotkot 30c5f6da9b Ethereum sync service in multi threads 2022-01-20 11:38:58 +00:00
Andrey Dolgolev 4206012192 Add fix of keys. 2022-01-19 13:25:32 +02:00
Andrey Dolgolev 8f2d494039 Add fix. 2022-01-19 01:21:24 +02:00
Andrey Dolgolev 9510cecc17 Remove address from special rule. 2022-01-18 18:36:46 +02:00
kompotkot bb30fe44c1 Work with env var port for mooncrawl service 2022-01-18 09:22:18 +00:00
Andrey Dolgolev 7947d6f3f7 Increase service timer. 2022-01-18 11:18:40 +02:00
Andrey Dolgolev 261065f91a Add function for extention data which go to cards componener. 2022-01-17 14:44:58 +02:00
Andrey Dolgolev 6dfbb986a0 Fix. 2022-01-17 10:46:29 +02:00
Andrey Dolgolev 2ddb6a49c7 Romove unused variables. 2022-01-17 10:32:32 +02:00
Andrey Dolgolev 04f7e10cd0 Add fix. 2022-01-14 12:01:57 +02:00
Andrey Dolgolev d221ffd6be Merge branch 'main' into refresh-dashboard-stats 2022-01-14 12:00:27 +02:00
kompotkot 606b965cd4 --no-block for statistic timer crawler 2022-01-13 18:01:18 +00:00
Andrey Dolgolev abb5415c8e Add fix. 2022-01-13 20:01:05 +02:00
Andrey Dolgolev d3c8b85cc8 Add mypy ignore. 2022-01-13 19:26:39 +02:00
Andrey Dolgolev e0b5b00eab Remove and test generation. 2022-01-13 17:31:25 +02:00
Andrey Dolgolev a1678aa08c Add logger instead prints. 2022-01-13 12:29:36 +02:00
Andrey Dolgolev 40e6e6c99b Add rollback. 2022-01-13 00:36:21 +02:00
Andrey Dolgolev 098c4de6e8 Add fixes. 2022-01-12 10:50:53 +02:00
Andrey Dolgolev 15f19583fc Merge branch 'main' into refresh-dashboard-stats 2022-01-10 12:52:30 +02:00
Andrey Dolgolev ef064827fd Add correct refreshing status. 2022-01-10 12:46:46 +02:00
Sergei Sumarokov 2419d95f5b
Merge pull request #512 from bugout-dev/mooncrawl-service-api 
Moonstream crawlers service for HTTP API server
 2021-12-23 17:41:32 +03:00
Yhtyyar Sahatov 2e297aa299
Merge pull request #510 from bugout-dev/continuous-crawler-batch-saving 
Continuous crawler batch saving
 2021-12-23 15:31:53 +03:00
Andrey Dolgolev 95b6ccdb4e All updates is working but require extend presignUrl responce. 2021-12-22 18:55:42 +02:00
kompotkot 6fd9319fd3 Moonstream crawlers service for HTTP API server 2021-12-22 15:14:50 +00:00
kompotkot 0c1e67a45b Mypy fix 2021-12-22 12:36:06 +00:00
kompotkot 6529e19bc6 Retrieve local ip for crawlers 2021-12-22 10:57:11 +00:00
kompotkot 64e0bf2645 Mooncrawl ready to go with WEB3_URI instread of ips and ports 2021-12-22 09:34:18 +00:00
kompotkot 4989e3afc1 Txpool now works with WEB3_URI env variable 2021-12-22 09:23:23 +00:00
kompotkot 0016fbf8ed Modified deploy for crawlers to work with checkenv 2021-12-22 09:15:18 +00:00
Andrey Dolgolev 968a1ddf45 Merge branch 'main' into refresh-dashboard-stats 2021-12-21 17:37:57 +02:00
yhtiyar 874b330846 bump of version 2021-12-21 18:35:04 +03:00
yhtiyar 6be687b1ed update of continuous crawler to batch save session 2021-12-21 18:33:31 +03:00
yhtiyar 3228e5e740 update of moonworm 2021-12-21 18:32:38 +03:00
Andrey Dolgolev d5b997e935 pre init 2021-12-21 17:23:26 +02:00
Neeraj Kashyap 6f2813710a
Merge pull request #502 from bugout-dev/job-pickup-fix 
fixed deduplication of abi
 2021-12-16 15:39:53 -08:00
yhtiyar aa112952c0 fixed deduplication of abi 2021-12-17 02:33:00 +03:00
Neeraj Kashyap 954f6014f7
Merge pull request #501 from bugout-dev/job-pickup-fix 
fixed job pickup
 2021-12-16 15:09:32 -08:00
yhtiyar 3a22e60226 fixed job pickup 2021-12-17 02:04:25 +03:00
kompotkot ddf33b9c92 Polygon syncronize crawler works in multi threads 2021-12-16 22:33:08 +00:00
kompotkot b1e828cf8e Merge branch 'moonworm-crawler' of github.com:bugout-dev/moonstream into moonworm-crawler 2021-12-16 21:49:20 +00:00
kompotkot 471a81277a Fixed moonworm service entrypoint 2021-12-16 21:48:47 +00:00
yhtiyar b8b8e01787 crawler begins from 1000 behind 2021-12-17 00:48:36 +03:00
yhtiyar 16a272ab71 black fix 2021-12-17 00:44:31 +03:00
yhtiyar 8e8c1c1569 Merge branch 'moonworm-crawler' of github.com:bugout-dev/moonstream into moonworm-crawler 2021-12-17 00:39:58 +03:00
yhtiyar c9ea6b85ff working crawler 2021-12-17 00:39:35 +03:00
kompotkot 364775dc94 Version, deployment service, setup requirements update for moonworm 2021-12-16 21:32:17 +00:00
yhtiyar 8a7413500c Merge branch 'moonworm-crawler' of github.com:bugout-dev/moonstream into moonworm-crawler 2021-12-16 23:18:14 +03:00
yhtiyar 034c20c913 added --force arg 2021-12-16 23:17:37 +03:00
yhtiyar ee91fadf40 fixed cli 2021-12-16 23:13:00 +03:00
yhtiyar 5df739b00e fixed import 2021-12-16 23:12:39 +03:00
Andrey Dolgolev ccc5cf42c9 fix label 2021-12-16 21:51:23 +02:00
yhtiyar 49454a87cc isort and mypy fixes 2021-12-16 22:46:44 +03:00
yhtiyar 83eaeebe1a [WIP] crawler 2021-12-16 22:43:32 +03:00
Andrey Dolgolev 489bb66e57 Merge branch 'main' into moonworm-crawler 2021-12-16 21:34:05 +02:00
Andrey Dolgolev ee476c5b46 Add fixes. 2021-12-16 21:25:24 +02:00
Andrey Dolgolev 3c79aaf23a Working state. 2021-12-16 19:17:13 +02:00
Tim Pechersky fd8ee961f1 Merge branch 'main' into customized-dashboard 2021-12-16 16:37:57 +00:00
Andrey Dolgolev 4d4032e274 Revert changes abi processing. 2021-12-16 17:15:23 +02:00
Andrey Dolgolev 9783690f8b isort and init state of api tasks. 2021-12-16 15:26:04 +02:00