Projects/your-project/topics/crypto_bitcoin.blocks and projects/your-project/topics/crypto_ansactions Blocks and transactions will be pushed to projects/your-project/topics/crypto_bitcoin. Use -output option to specify the Google Pub/Sub topic where to publish blockchain data,Į.g.This command outputs blocks and transactions to the console by default.stream > bitcoinetl stream -provider-uri -start-block 500000 You can tune -export-batch-size, -max-workers for performance. p "(item).astimezone().strftime('%Y-%m-%d') = ''" export_all > bitcoinetl export_all -provider-uri -start -end However the returned block range may also contain blocks outside the specified date, because block times are notīlocks.json/ transactions.json with the below command: > bitcoinetl filter_items -i blocks.json -o blocks_filtered.json \ This command is guaranteed to return the block range that covers all blocks with block.time on the specifiedĭate. get_block_range_for_date > bitcoinetl get_block_range_for_date -provider-uri -date = You can tune -batch-size, -max-workers for performance. transactions-input transactions.json -transactions-output enriched_transactions.json You need to run bitcoin daemon with option txindex=1 for this command to work. Use enrich_transactions to populate those fields. Note that required_signatures, type, addresses, and value fields will be empty in transactions inputs. Omit -blocks-output or -transactions-output options if you want to export only transactions/blocks. export_blocks_and_transactions > bitcoinetl export_blocks_and_transactions -start-block 0 -end-block 500000 \ The format type is inferred from the output file name. Useįor the -output parameters the supported type is json. Provided transactions will not be exported. transactions-output TEXT The output file for transactions. If not providedīlocks will not be exported. blocks-output TEXT The output file for blocks. w, -max-workers INTEGER The maximum number of workers. p, -provider-uri TEXT The URI of the remote Bitcoin node ![]() b, -batch-size INTEGER The number of blocks to export at a time. Usage: bitcoinetl.py export_blocks_and_transactions Refer to for deploying the streaming app toĪll the commands accept -h parameter for help, e.g.: > bitcoinetl export_blocks_and_transactions -help > docker run -v /path_to_credentials_file/:/bitcoin-etl/ -env GOOGLE_APPLICATION_CREDENTIALS =/bitcoin-etl/credentials_file.json bitcoin-etl:latest-streaming stream -p -start-block 500000 -output projects/your-project/topics/crypto_bitcoin > docker run bitcoin-etl:latest-streaming stream -p -start-block 500000 Run streaming to console or Pub/Sub > docker build -t bitcoin-etl:latest-streaming -f Dockerfile_with_streaming. provider-uri -blocks-output output/blocks.json -transactions-output output/transactions.json Run a container out of the image > docker run -v $HOME/output:/bitcoin-etl/output bitcoin-etl:latest export_blocks_and_transactions -start-block 0 -end-block 500000 \ In case bitcoinetl command is not available in PATH, use python -m bitcoinetl instead.īuild a docker image > docker build -t bitcoin-etl:latest. The result will be in the output subdirectory, partitioned in Hive style: output/blocks/start_block = 00000000/end_block = 00000099/blocks_00000000_00000099.csv Install Bitcoin ETL: > pip install bitcoin-etlĮxport blocks
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |