Quantcast

Cosbench workload on s3 fails

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Cosbench workload on s3 fails

kanchana
Hi,

Below is the s3 workload xml i am using to run cosbench using s3, it is failing in s3-read/write stage with the below error.

Here, accesskey and secret key are the keys taken from s3 user creation. 10.10.100.4 is the node on which radosgw installed.  


<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="librados" config="accesskey=M04EEO36AI1FI4TYR1Z3;secretkey=reOKHTvmb1g1E5c15brqmzIhwe2vUlyKKVjszIi3;proxyhost=10.10.100.4;proxyport=7480;endpoint=<a href="http://10.10.100.4:7480&quot;">http://10.10.100.4:7480" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200);sizes=c(32)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="900">
        <operation type="read" ratio="20" config="cprefix=s3testqwer;containers=u(1,10);objects=u(1,100)" />
        <operation type="write" ratio="80" config="cprefix=s3testqwer;containers=u(1,10);objects=u(101,200);sizes=c(32)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

  </workflow>

</workload>

Error log shows:
2016-06-15 12:45:10,611 [INFO] [NoneStorage] - performing PUT at /s3testqwer1
2016-06-15 12:45:10,615 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)
2016-06-15 12:45:18,543 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects1
2016-06-15 12:45:18,546 [WARN] [S3Storage] - below exception encountered when creating object myobjects1 at s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)

Can some one guide me on how to run cosbench for s3.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Cosbench workload on s3 fails

ywang19
Administrator

Pls make sure no existence of bucket “s3testqwer1” before creation.

 

 

 

From: kanchana [via COSBench] [mailto:ml-node+[hidden email]]
Sent: Thursday, June 16, 2016 1:36 PM
To: Wang, Yaguang <[hidden email]>
Subject: Cosbench workload on s3 fails

 

Hi,

Below is the s3 workload xml i am using to run cosbench using s3, it is failing in s3-read/write stage with the below error.

Here, accesskey and secret key are the keys taken from s3 user creation. 10.10.100.4 is the node on which radosgw installed.  


<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="librados" config="accesskey=M04EEO36AI1FI4TYR1Z3;secretkey=reOKHTvmb1g1E5c15brqmzIhwe2vUlyKKVjszIi3;proxyhost=10.10.100.4;proxyport=7480;endpoint=<a href="<a href="http://10.10.100.4:7480&amp;quot;">http://10.10.100.4:7480&quot;">http://10.10.100.4:7480" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200);sizes=c(32)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="900">
        <operation type="read" ratio="20" config="cprefix=s3testqwer;containers=u(1,10);objects=u(1,100)" />
        <operation type="write" ratio="80" config="cprefix=s3testqwer;containers=u(1,10);objects=u(101,200);sizes=c(32)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

  </workflow>

</workload>

Error log shows:
2016-06-15 12:45:10,611 [INFO] [NoneStorage] - performing PUT at /s3testqwer1
2016-06-15 12:45:10,615 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)
2016-06-15 12:45:18,543 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects1
2016-06-15 12:45:18,546 [WARN] [S3Storage] - below exception encountered when creating object myobjects1 at s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)

Can some one guide me on how to run cosbench for s3.


If you reply to this email, your message will be added to the discussion below:

http://cosbench.1094679.n5.nabble.com/Cosbench-workload-on-s3-fails-tp472.html

To start a new topic under cosbench-user, email [hidden email]
To unsubscribe from COSBench, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Cosbench workload on s3 fails

kanchana
Hi,

Modified bucket name "s3testqwer1" to "s3tester" which doesn't exist before. But still it fails with below error. Attached is the workload sample and steps I performed. Let me know if I am missing some steps.

ERROR MESSAGE:

FreeMarker template error: The following has evaluated to null or missing: ==> info.errorStatistics.stackTraceAndMessage[trace] [in template "mission.ftl" at line 238, column 48] Tip: If the failing expression is known to be legally null/missing, either specify a default value with myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing. (These only cover the last step of the expression; to cover the whole expression, use parenthessis: (myOptionVar.foo)!myDefault, (myOptionVar.foo)?? The failing instruction (FTL stack trace): ---------- ==> ${info.errorStatistics.stackTraceAndM... [in template "mission.ftl" at line 238, column 46] #list info.errorStatistics.stackTrace... [in template "mission.ftl" at line 232, column 9] #if showErrorStatistics [in template "mission.ftl" at line 223, column 5] ---------- Java stack trace (for programmers): ---------- freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...] at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:98) at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:382) at freemarker.core.Expression.evalAndCoerceToString(Expression.java:115) at freemarker.core.DollarVariable.accept(DollarVariable.java:76) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.MixedContent.accept(MixedContent.java:93) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.IteratorBlock$Context.runLoop(IteratorBlock.java:181) at freemarker.core.Environment.visitIteratorBlock(Environment.java:509) at freemarker.core.IteratorBlock.accept(IteratorBlock.java:103) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.MixedContent.accept(MixedContent.java:93) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.IfBlock.accept(IfBlock.java:84) at freemarker.core.Environment.visitByHiddingParent(Environment.java:286) at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:86) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.MixedContent.accept(MixedContent.java:93) at freemarker.core.Environment.visit(Environment.java:265) at freemarker.core.Environment.process(Environment.java:243) at freemarker.template.Template.process(Template.java:277) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.processTemplate(FreeMarkerView.java:366) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.doRender(FreeMarkerView.java:283) at org.springframework.web.servlet.view.freemarker.FreeMarkerView.renderMergedTemplateModel(FreeMarkerView.java:233) at org.springframework.web.servlet.view.AbstractTemplateView.renderMergedOutputModel(AbstractTemplateView.java:167) at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:250) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1047) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:817) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549) at javax.servlet.http.HttpServlet.service(HttpServlet.java:690) at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:745)

On 16 June 2016 at 11:32, ywang19 [via COSBench] <[hidden email]> wrote:

Pls make sure no existence of bucket “s3testqwer1” before creation.

 

 

 

From: kanchana [via COSBench] [mailto:[hidden email][hidden email]]
Sent: Thursday, June 16, 2016 1:36 PM
To: Wang, Yaguang <[hidden email]>
Subject: Cosbench workload on s3 fails

 

Hi,

Below is the s3 workload xml i am using to run cosbench using s3, it is failing in s3-read/write stage with the below error.

Here, accesskey and secret key are the keys taken from s3 user creation. 10.10.100.4 is the node on which radosgw installed.  


<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="librados" config="accesskey=M04EEO36AI1FI4TYR1Z3;secretkey=reOKHTvmb1g1E5c15brqmzIhwe2vUlyKKVjszIi3;proxyhost=10.10.100.4;proxyport=7480;endpoint=<a href="<a href="http://10.10.100.4:7480&amp;quot;">http://10.10.100.4:7480&quot;">http://10.10.100.4:7480" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200);sizes=c(32)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="900">
        <operation type="read" ratio="20" config="cprefix=s3testqwer;containers=u(1,10);objects=u(1,100)" />
        <operation type="write" ratio="80" config="cprefix=s3testqwer;containers=u(1,10);objects=u(101,200);sizes=c(32)KB" />
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,10)" />
    </workstage>

  </workflow>

</workload>

Error log shows:
2016-06-15 12:45:10,611 [INFO] [NoneStorage] - performing PUT at /s3testqwer1
2016-06-15 12:45:10,615 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)
2016-06-15 12:45:18,543 [INFO] [NoneStorage] - performing PUT at /s3testqwer1/myobjects1
2016-06-15 12:45:18,546 [WARN] [S3Storage] - below exception encountered when creating object myobjects1 at s3testqwer1: Unable to unmarshall error response (Content is not allowed in prolog.)

Can some one guide me on how to run cosbench for s3.


If you reply to this email, your message will be added to the discussion below:

http://cosbench.1094679.n5.nabble.com/Cosbench-workload-on-s3-fails-tp472.html

To start a new topic under cosbench-user, email [hidden email]
To unsubscribe from COSBench, click here.
NAML




If you reply to this email, your message will be added to the discussion below:
http://cosbench.1094679.n5.nabble.com/Cosbench-workload-on-s3-fails-tp472p474.html
To unsubscribe from Cosbench workload on s3 fails, click here.
NAML



--
--Kanchana

workload-log(2).txt (400K) Download Attachment
workload-config(2).xml (7K) Download Attachment
steps.txt (684 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to add a pool to swift user and run cosbench

kanchana
In reply to this post by kanchana
Hi,

When a rgw service is started, by default below pools are created.
.rgw.root
default.rgw.control
default.rgw.data.root
default.rgw.gc
default.rgw.log

When  a swift user is created, some default pools are created. But I would like to use "Pool_A" for the swift user.
From client when I run Cosbench the data should be placed in "Pool_A" instead of placing it in default pools. How can i achieve that.

Also need help on how to run Cosbench with swift user. Your help is very much appreciated.

​Thanks,
kanchana.​
Loading...