水滴

# 批量发单

通过 droplet-client 批量发布离线订单。

批量发单命令：droplet-clientstorage deals batch [command options] [miners price duration]，具体使用和发送单个订单类似，新增了一些 Option。

部分 Option:

  • --miner 接收订单的矿工，可以是多个 --miner=t010001 --miner=t010002，多个会平均分配订单。
  • --manifest go-graphsplit 生成的 manifest.csv 文件，包含 payload cid、piece cid 和 piece size 等信息
  • --start-index 从 manifest.csv 第 N 条订单开始
  • --end-index 在 manifest.csv 第 N 条订单结束，start-index - end-index 就是本次预期发布的订单总数
  • --filter 是否启动过滤，默认是 false，目前有两条过滤规则
    • 存储提供者重复的数据不能超过20%
    • 一个发单者对同一个存储提供者发的datacap订单不能超过25%

# 例子

# 准备订单数据

使用 go-graphsplit (opens new window) 生成订单数据，go-graphsplit 是一种用于将大型数据集分割成固定大小的 car 文件的工具。

# --car-dir 指定文件目录存储生成的 car 文件和 manifest.csv 文件
# --slice-size 指定car文件大小
# --graph-name 图的名字，可随意填
# --calc-commp 是否计算 commp，必须要计算，离线订单需要piece cid 和 piece size
# --rename 生成的car文件默认是以 payloadcid.car 作为文件名，加上本 flag 后以 piececid 作为文件名，建议加上本flag，方便后面数据导入
# path-to-source-file 指定用于生成car文件的原始数据

./graphsplit chunk --car-dir <car-dir> --slice-size <piece-size> --graph-name <name> --calc-commp --rename <path-to-source-file>

# 结果
会在指定的目录生成 car 文件和 manifest.csv 文件

manifest.csv 文件内容，发单的时候会使用到 playload_cid、piece_cid、payload_size 和 payload_size。

playload_cid,filename,piece_cid,payload_size,payload_size,detail
bafybeib6kembdoggfwssslpcrxgpbhxmeyretwcnaspx5ff2r4rybt73gq,test-total-19-part-1.car,baga6ea4seaqevygou3u5mlxl6prwifaw2n4e6eanrbdl7wm2guzfm2r5tool4fi,14241759,16646144,"{""Name"":"""",""Hash"":""bafybeib6kembdoggfwssslpcrxgpbhxmeyretwcnaspx5ff2r4rybt73gq"",""Size"":0,""Link"":[{""Name"":""filecoin-ffi"",""Hash"":""bafybeibum5nzwz54733lioyzwhqrod2zauax2jaa4qiu5hdqw7fidzl47m"",""Size"":14240979,""Link"":[{""Name"":""libfilcrypto.a.00000001"",""Hash"":""bafybeigptobclymvmmlxn4n764dey2p6hgot6h34vtte5fzm67l2znwz4q"",""Size"":14240905,""Link"":null}]}]}"
bafybeihh6sq3w5qmzrxakde5wwjkw7wnbwwxm5yvfo3nd5zjw25vtgcqom,test-total-19-part-2.car,baga6ea4seaqh4vblymafqc3tskmy5jilgs462gsbvuu6mxnsql364bhbj5omcji,14241759,16646144,"{""Name"":"""",""Hash"":""bafybeihh6sq3w5qmzrxakde5wwjkw7wnbwwxm5yvfo3nd5zjw25vtgcqom"",""Size"":0,""Link"":[{""Name"":""filecoin-ffi"",""Hash"":""bafybeigiprvgstre4ctcwleocm2gg52hzc32ccnjr66phud5qrnbmjyqli"",""Size"":14240979,""Link"":[{""Name"":""libfilcrypto.a.00000002"",""Hash"":""bafybeia4fxflzrihwbjkjayasrtvmnocvq5xn3pesendcjnlog47fchd6q"",""Size"":14240905,""Link"":null}]}]}"
bafybeihw7afqagfvg2uhmigyunsgvxeidaqfbg6odi4jwbus6557kqhsd4,test-total-19-part-3.car,baga6ea4seaqij5nrrmypz22lchdrjcaowjdjk67agjntbcfbcbbrcrnrdbn7kfy,14241759,16646144,"{""Name"":"""",""Hash"":""bafybeihw7afqagfvg2uhmigyunsgvxeidaqfbg6odi4jwbus6557kqhsd4"",""Size"":0,""Link"":[{""Name"":""filecoin-ffi"",""Hash"":""bafybeicn4bdznskvsjt3on2633mm3uh5gungf37hmfhsizraugq4uleosu"",""Size"":14240979,""Link"":[{""Name"":""libfilcrypto.a.00000003"",""Hash"":""bafybeidi5kz5hhfwcqechj4czbyat3tlyyd4d6rigigkks66ysjy5agfke"",""Size"":14240905,""Link"":null}]}]}"
bafybeigq66gjr6c6junro7t4fwcpkhl7vpowi3soadksjvleusq6nlzu6a,test-total-19-part-4.car,baga6ea4seaqaamzryns5cphp5bynuq5nrxdltxzcj6hoybboduhh4377rw6lqgy,14241759,16646144,"{""Name"":"""",""Hash"":""bafybeigq66gjr6c6junro7t4fwcpkhl7vpowi3soadksjvleusq6nlzu6a"",""Size"":0,""Link"":[{""Name"":""filecoin-ffi"",""Hash"":""bafybeih5lopymib4lbnoonqatu43r2avm2rw2nz4eeva6zmnyyegjsoyma"",""Size"":14240979,""Link"":[{""Name"":""libfilcrypto.a.00000004"",""Hash"":""bafybeieo2dahkmiak4rlyfehpgu4gyjq2v6wulbmcobha3f2nk6ftysaae"",""Size"":14240905,""Link"":null}]}]}"

# 批量发单

  1. 发布普通离线订单
./droplet-client storage deals batch --from <address> --manifest <path-to-manifest.csv> --end-index 10 --start-index 5 --miner=t019150 --miner=t018682 0 518400

# 结果
has 5 deals need to publish, t018682: 3, t019150: 2
create deal success, proposal cid: bafyreihwvsr3vfsdbrxagtdjzsemngtc3r3xra2gaunbs6pjb63lyodl6a
create deal success, proposal cid: bafyreid2oakcs2di6lq5mv3e4h5sybkezl3zn7lhvf656zeqentcsqetem
create deal success, proposal cid: bafyreidmzdh7zee7inm65cdxmuwyxyv7uwtoo2hiphsjfseyymoir2z4nm
create deal success, proposal cid: bafyreigt663jymtilnxocfybs27w6b564yxfczdpsx47fbsqwzrj54zxz4
create deal success, proposal cid: bafyreigtemnxftqg65gtwsw3rwfvqaqzbb47d4r75ipksjlypcs56y7qei
  1. 批量发布 datacap 订单

请加上 --verified-deal flag

./droplet-client storage deals batch --from <address> --manifest <path-to-manifest.csv> --end-index 15 --start-index 10 --verified-deal --miner=t019150 --miner=t018682 0 518400

# 查询订单

  1. 查询单个订单
./droplet-client storage deals get bafyreify4mm46vfdqsogcfhki44vi3ll5n2h35p5fflcclew53fgm3yieu
  1. 列出所有离线订单
./droplet-client storage deals list --offline

# 批量导入订单数据

前提：使用 go-graphsplit 生成的 car 文件是以 piececid 作为文件名。

  1. 导出待导入的离线订单

可以导出所有订单状态在 StorageDealWaitingForData 的订单，导出数据包括 proposal cid 和 piece cid。

./droplet-client storage deals export --output proposal_piece.txt

cat proposal_piece.txt

proposalCID,pieceCID
bafyreihwvsr3vfsdbrxagtdjzsemngtc3r3xra2gaunbs6pjb63lyodl6a,baga6ea4seaqbj3yywnq3yisdxy4zlf4if2whlm5sdjcz7ricm2wrow2b7rc2uja
bafyreid2oakcs2di6lq5mv3e4h5sybkezl3zn7lhvf656zeqentcsqetem,baga6ea4seaqcdstiui27aajpz2dcpx2f6brimxhfvepgxljwsweicul32pkeofq
bafyreidmzdh7zee7inm65cdxmuwyxyv7uwtoo2hiphsjfseyymoir2z4nm,baga6ea4seaqlrwtnhj322vczuuiy2ekb4kjftbf3ho6f4bgy6k5rnzh67eia4lq
bafyreigt663jymtilnxocfybs27w6b564yxfczdpsx47fbsqwzrj54zxz4,baga6ea4seaqdgfsfsdtpnsgwlwhtj4ecvk7432gaqheltfrzun3vju3yc3d7cnq
bafyreigtemnxftqg65gtwsw3rwfvqaqzbb47d4r75ipksjlypcs56y7qei,
baga6ea4seaqknn4cstmtmscdhebaxmv5dopnxqthwbdmvuslbuv5dcupzvw46ni
  1. 批量导入订单数据
# --cardir car 文件的目录，若提前把 car 文件放到 piecestore，则本 flag 可以省略，如果确认piece数据没有问题，可以使用 `--skip-commp` 参数来加速数据导入过程。
./droplet storage deal batch-import-data --manifest <proposal_piece.txt> --car-dir <path-to-cardir>

# 结果
import data success: bafyreihwvsr3vfsdbrxagtdjzsemngtc3r3xra2gaunbs6pjb63lyodl6a
import data success: bafyreid2oakcs2di6lq5mv3e4h5sybkezl3zn7lhvf656zeqentcsqetem
import data success: bafyreidmzdh7zee7inm65cdxmuwyxyv7uwtoo2hiphsjfseyymoir2z4nm
import data success: bafyreigt663jymtilnxocfybs27w6b564yxfczdpsx47fbsqwzrj54zxz4
import data success: bafyreigtemnxftqg65gtwsw3rwfvqaqzbb47d4r75ipksjlypcs56y7qei

# 查询占比情况

  1. 查看存储提供者订单重复情况
./droplet-client storage deals verified-deal-stat --provider t019150

# 结果
Provider  Total      Uniq       DuplicationPercentage  
t019150   981467136  612368384  37.61%
  1. 查看样本分布情况
./droplet-client storage deals verified-deal-stat --client t3wivhkdivcxj5zp2l4wjkzon232s52smnd5m3na66ujl5nel75jggguhgaa3zbhjo3as4epf5ytxl6ly3qoha

# 结果
Client: t3wivhkdivcxj5zp2l4wjkzon232s52smnd5m3na66ujl5nel75jggguhgaa3zbhjo3as4epf5ytxl6ly3qoha
Total:  16.17 GiB / 17362337792 B
Uniq:   14.77 GiB / 15860764672 B
DuplicationPercentage: 8.65%
Provider  Total                      Percentage  Uniq                       DuplicationPercentage  
t019150   620 MiB / 650117120 B      3.74%       620 MiB / 650117120 B      0.00%                  
t018682   15.56 GiB / 16712220672 B  96.26%      14.42 GiB / 15487471616 B  7.33%

← DC 订单续期 →