Quantcast

createContainer S3 put bucket

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

createContainer S3 put bucket

Cos_help
Reading the COSBench documentation I'm led to believe that the tool offers the capability to create an s3 bucket better known as a container. The corresponding flag is createContainer=true.

From Documentation ODT:
createContainer Boolean - Default(False) - Create related container if it does not exist

My Configuration File:
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="COSBench Concurrency Example" description="Example COSBench Concurrency workflow">
  <storage type="s3" config="accesskey=l2NRETfFCWa8DeBSsn4g;secretkey= 8REmZ8VT84zxBNe9uOhqnl7eSo91ONhUNctqm2ch;endpoint=http://localhost" />
  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="createContainer=true;cprefix=TestVault;containers=r(1,2)" />
    </workstage>
    <workstage name="prepare">
      <work type="prepare" workers="1000" config="createContainer=true;cprefix=TestVault;containers=r(1,2);objects=r(1,6000); sizes=c(4)MB " />
    </workstage>
    <workstage name="main">
      <work name="main" workers="1000" runtime="1000">
        <operation type="read" ratio="80" config="createContainer=true;cprefix=TestVault;containers=u(1,2);objects=u(1,6000);sizes=c(4)MB" />
        <operation type="write" ratio="20" config="createContainer=true;cprefix=TestVault;containers=u(1,2);objects=u(6001,12000);sizes=c(4)MB" />
      </work>
    </workstage>
    <workstage name="cleanup">
      <work type="cleanup" workers="1000" config="createContainer=true;cprefix=TestVault;containers=r(1,2);objects=r(1,12000)" />
    </workstage>
    <workstage name="dispose">
      <work type="dispose" workers="1" config="createContainer=true;cprefix=TestVault;containers=r(1,2)" />
    </workstage>
  </workflow>
</workload>


What the tool appears to be doing is a max-keys=0 listing request against the container endpoint. This is obviously failing with a 404 since the container isnt present on the storage system. The tool bails out after this point and the test terminates. At no point does it try to create the container. Is this mode currently supported for S3 or swift and if so what do I need to do to my configuration provided to get it working?

Thanks much.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: createContainer S3 put bucket

ywang19
Administrator

“CreateContainer” parameter is only in action in “prepare” work, by default, it’s disabled.

 

In your workload, you already created those containers in “init” stage, so no need to enabled “createContainer” in “prepare” stage.

 

 

From: Cos_help [via COSBench] [mailto:ml-node+[hidden email]]
Sent: Friday, August 28, 2015 3:04 AM
To: Wang, Yaguang
Subject: createContainer S3 put bucket

 

Reading the COSBench documentation I'm led to believe that the tool offers the capability to create an s3 bucket better known as a container. The corresponding flag is createContainer=true.

From Documentation ODT:
createContainer Boolean - Default(False) - Create related container if it does not exist

My Configuration File:
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="COSBench Concurrency Example" description="Example COSBench Concurrency workflow">
  <storage type="s3" config="accesskey=l2NRETfFCWa8DeBSsn4g;secretkey= 8REmZ8VT84zxBNe9uOhqnl7eSo91ONhUNctqm2ch;endpoint=http://localhost" />
  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="createContainer=true;cprefix=TestVault;containers=r(1,2)" />
    </workstage>
    <workstage name="prepare">
      <work type="prepare" workers="1000" config="createContainer=true;cprefix=TestVault;containers=r(1,2);objects=r(1,6000); sizes=c(4)MB " />
    </workstage>
    <workstage name="main">
      <work name="main" workers="1000" runtime="1000">
        <operation type="read" ratio="80" config="createContainer=true;cprefix=TestVault;containers=u(1,2);objects=u(1,6000);sizes=c(4)MB" />
        <operation type="write" ratio="20" config="createContainer=true;cprefix=TestVault;containers=u(1,2);objects=u(6001,12000);sizes=c(4)MB" />
      </work>
    </workstage>
    <workstage name="cleanup">
      <work type="cleanup" workers="1000" config="createContainer=true;cprefix=TestVault;containers=r(1,2);objects=r(1,12000)" />
    </workstage>
    <workstage name="dispose">
      <work type="dispose" workers="1" config="createContainer=true;cprefix=TestVault;containers=r(1,2)" />
    </workstage>
  </workflow>
</workload>


What the tool appears to be doing is a max-keys=0 listing request against the container endpoint. This is obviously failing with a 404 since the container isnt present on the storage system. The tool bails out after this point and the test terminates. At no point does it try to create the container. Is this mode currently supported for S3 or swift and if so what do I need to do to my configuration provided to get it working?

Thanks much.


If you reply to this email, your message will be added to the discussion below:

http://cosbench.1094679.n5.nabble.com/createContainer-S3-put-bucket-tp411.html

To start a new topic under cosbench-user, email [hidden email]
To unsubscribe from COSBench, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: createContainer S3 put bucket

Cos_help
In reply to this post by Cos_help
That flag doesn't appear to be doing anything interesting even when set to true. I removed it from the configuration entirely. As it turned out I actually had a configuration issue on the default container profile on the storage system which was the root cause of the problem. The default behavior of COSBench appears to be to; issue a HEAD request to the container endpoint if that fails with a 404 (not found) a put container operation is issued to the storage system.

127.0.0.1 - "" swift [28/Aug/2015:16:20:44 +0000] "HEAD /openstack/v1/swift/zestvault1 HTTP/1.1" 404 1286 "-" "cosbench/2.0" 1
127.0.0.1 - "" swift [28/Aug/2015:16:20:44 +0000] "PUT /openstack/v1/swift/zestvault1 HTTP/1.1" 201 0 "-" "cosbench/2.0" 3774

Will close this out. In short the answer is I was looking for is "Yes" COSBench does support container creation through both S3 and Swift.
Loading...