Unable to test S3 using COSBench

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Unable to test S3 using COSBench

deba.daz
Hi,

I was trying to test S3 using the sample xml file which looks like this below but unfortunately can test it successfully. Can you please let me know what might be the issue, i am sure something is wrong with xml configuration.
Note -

1. Changed the accesskey & secretkey in the below xml for security purpose.
2. While sending the load init, cleanup & dispose completes successfully while prepare & main (read, write) fails everytime. Error message is pasted below for reference -

AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: 11D6734D0FB0E494, AWS Error Code: AccessDenied, AWS Error Message: Access Denied, S3 Extended Request ID: RfUdUlKHVil0MVc0nhmPV6qFbHGIyn1ACIGpCk2nJ4sAy4Hqc1o1Sgrk0PEIpWD4N0vCfxLk1NM=

############################################

XML file -

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="s3" config="accesskey=AKIAIUGO7PRTHP5QGXXX;secretkey=hhhbYQ/8sWWdmEadTiiBF8F9TPOxMVsCSDnmuXXX;endpoint=http://s3.amazonaws.com" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="30">
        <operation type="read" ratio="80" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />
        <operation type="write" ratio="20" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>

  </workflow>

</workload>

Regards
Debashish Das
Reply | Threaded
Open this post in threaded view
|

Re: Unable to test S3 using COSBench

ywang19
Administrator
the "s3testqwer1" and "s3testqwer2" are already used by us, and your aws account should have no permissions to access. You need modify the cprefix  in xml to some other unique name.
Reply | Threaded
Open this post in threaded view
|

Re: Unable to test S3 using COSBench

deba.daz
Thanks a lot. it worked after changing the cprefix to some unique name.

Now my result looks like (attached in the image) but i am not sure how to read & make some sense out of it, can you please explain me the result.

Also I am not sure what is function of different tasks in brief (like init, prepare, read, write, dispose etc).

I will really appreciate you help on this.

Thanks a ton once again !!!Result

Reply | Threaded
Open this post in threaded view
|

RE: Unable to test S3 using COSBench

ywang19
Administrator

Glad to hear it’s working. There are 5 types of work stages:

          Init              à to create containers

          Prepare        à to fill objects in advance

          Normal (main) à to run specified workload, like 80% GET 20% PUT mix in your case

          Cleanup       à to delete objects

          Dispose       à to delete containers

For a workload run, the “main” stage is the major one to demonstrate performance.

 

-yaguang

 

From: deba.daz [via COSBench] [mailto:ml-node+[hidden email]]
Sent: Tuesday, April 14, 2015 6:24 PM
To: Wang, Yaguang
Subject: Re: Unable to test S3 using COSBench

 

Thanks a lot. it worked after changing the cprefix to some unique name.

Now my result looks like (attached in the image) but i am not sure how to read & make some sense out of it, can you please explain me the result.

Also I am not sure what is function of different tasks in brief (like init, prepare, read, write, dispose etc).

I will really appreciate you help on this.

Thanks a ton once again !!!Result



If you reply to this email, your message will be added to the discussion below:

http://cosbench.1094679.n5.nabble.com/Unable-to-test-S3-using-COSBench-tp374p377.html

To start a new topic under cosbench-user, email [hidden email]
To unsubscribe from COSBench, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Unable to test S3 using COSBench

kanchana
In reply to this post by deba.daz
Hi,

I used the below xml and reads failed. Please let me know where I am going wrong. Client has internet access and rgw is running.

General Report
Op-Type Op-Count Byte-Count Avg-ResTime Avg-ProcTime Throughput Bandwidth Succ-Ratio
read 0 ops 0 B N/A N/A 0 op/s 0 B/S 0%
write 7 ops 0 B 7977.57 ms 7977.57 ms 0.24 op/s 0 B/S 100%


<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="s3" config="accesskey=E6QRAJ14DSDDOY432E2M;secretkey=tZ8H3N20yNaZGHmdEeL3aCCuwSEvxELd3FD0km4x;endpoint=http://s3.amazonaws.com" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=vedams;containers=r(1,2)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=vedams;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="30">
        <operation type="read" ratio="80" config="cprefix=vedams;containers=u(1,2);objects=u(1,10)" />
        <operation type="write" ratio="20" config="cprefix=vedams;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=vedams;containers=r(1,2);objects=r(1,20)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=vedams;containers=r(1,2)" />
    </workstage>

  </workflow>

</workload>
Reply | Threaded
Open this post in threaded view
|

Re: Unable to test S3 using COSBench

light
i have the same problem, have you solved this problem ? and how?