Quantcast

Prepare vs Write

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Prepare vs Write

eppo
Can someone tell me the difference between prepare and normal write?

When I'm doing a normal write run while using the r selector and # of operations I would expect that this would create unique objects for the number of operations?

However when I'm reading the objects that I have written previously not all objects seem to have been created and I'm unable to find the put requests for any of the "missing" objects on the storage system.

When doing a read run after a prepare does result in a successful run and all objects are accounted for.

Is there a performance difference with prepare vs normal writes? I would like to avoid writing the objects twice to get performance results I'm looking for.

Please see my workload file below, first I run prepare and then I read this is successful. Then I write a new range of objects and read that same range with the r selector but the read fails.

        ```<workflow>
          <workstage name="create-buckets">
              <work type="init" workers="1" config="cprefix=bucket; containers=r(1,1)"/>
          </workstage>
         
        <workstage name="prepare">
            <work type="prepare" workers="1" config="cprefix=bucket;containers=r(1,1);oprefix=1-10-1M; objects=r(1,100000);sizes=c(1)MB" />
        </workstage>
         
          <workstage name="Read 1MB" closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="30" rampdown="30" runtime="180" workers="10" totalOps="0">
                  <operation type="read" config="cprefix=bucket; containers=r(1,1);oprefix=1-10-1M; objects=r(1,100000)"/>
              </work>
          </workstage>
       
          <workstage name="Write 1MB" closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="0" rampdown="0" runtime="0" workers="10" totalOps="100000">
                  <operation type="write" config="cprefix=bucket; containers=r(1,1); oprefix=1-10-1M; objects=r(100001,200000); sizes=c(1)MB"/>
              </work>
          </workstage>
         
          <workstage name="Read 1MB"  closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="30" rampdown="30" runtime="180" workers="10" totalOps="0">
                  <operation type="read" config="cprefix=bucket; containers=r(1,1);oprefix=1-10-1M; objects=r(100001,200000)"/>
              </work>
          </workstage>
         
      </workflow>
     
  </workload>```
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Prepare vs Write

ywang19
Administrator
"Prepare" is actually a special write stage, it will write all the objects once sequentially. A normal write stage will potentially have write conflictions, which means an object might be written a few times, so you may see some missing objects when read back.

在 2016年11月18日,上午8:24,eddojansen [via COSBench] <[hidden email]> 写道:

Can someone tell me the difference between prepare and normal write?

When I'm doing a normal write run while using the r selector and # of operations I would expect that this would create unique objects for the number of operations?

However when I'm reading the objects that I have written previously not all objects seem to have been created and I'm unable to find the put requests for any of the "missing" objects on the storage system.

When doing a read run after a prepare does result in a successful run and all objects are accounted for.

Is there a performance difference with prepare vs normal writes? I would like to avoid writing the objects twice to get performance results I'm looking for.

Please see my workload file below, first I run prepare and then I read this is successful. Then I write a new range of objects and read that same range with the r selector but the read fails.

        ```<workflow>
          <workstage name="create-buckets">
              <work type="init" workers="1" config="cprefix=bucket; containers=r(1,1)"/>
          </workstage>
         
        <workstage name="prepare">
            <work type="prepare" workers="1" config="cprefix=bucket;containers=r(1,1);oprefix=1-10-1M; objects=r(1,100000);sizes=c(1)MB" />
        </workstage>
         
          <workstage name="Read 1MB" closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="30" rampdown="30" runtime="180" workers="10" totalOps="0">
                  <operation type="read" config="cprefix=bucket; containers=r(1,1);oprefix=1-10-1M; objects=r(1,100000)"/>
              </work>
          </workstage>
       
          <workstage name="Write 1MB" closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="0" rampdown="0" runtime="0" workers="10" totalOps="100000">
                  <operation type="write" config="cprefix=bucket; containers=r(1,1); oprefix=1-10-1M; objects=r(100001,200000); sizes=c(1)MB"/>
              </work>
          </workstage>
         
          <workstage name="Read 1MB"  closuredelay="30">
              <work name="main" type="normal" interval="5" division="none" rampup="30" rampdown="30" runtime="180" workers="10" totalOps="0">
                  <operation type="read" config="cprefix=bucket; containers=r(1,1);oprefix=1-10-1M; objects=r(100001,200000)"/>
              </work>
          </workstage>
         
      </workflow>
     
  </workload>```
 
 


If you reply to this email, your message will be added to the discussion below:
http://cosbench.1094679.n5.nabble.com/Prepare-vs-Write-tp535.html
To start a new topic under cosbench-user, email [hidden email]
To unsubscribe from COSBench, click here.
NAML
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Prepare vs Write

eppo
Thank you for the prompt response!

When reading the userguide it was my understanding that using the range selector would mitigate that issue. But from your response I understand that even when using the r selector it might skip numbers or use some of them double?

Select from [min,max] incrementally
For example, r(1,100) means the element numbering incrementally increases from min to max, and each number is selected only once; this is generally used in special stages (init, prepare, cleanup, dispose), if it’s used in normal stage, please make sure you understand how to use it correctly (see FAQ 6.1.17 for details).

Thanks,
Loading...