2019年10月21日月曜日

[Nginx 1.17.3] Create Certificate and Private Key files for Nginx to proxy Orchestrator


 Nginx needs Client Certificate, Private Key files, and Root Certificate in order to proxy UiPath Orchestrator ssl. Both Server Certificate and Root Certificate files are created by extracting the .pfx file of SSL certificate downloaded from IIS where Orchestrator is running.

○ Environment:

[Nginx]
    Version: 1.17.3
  [OS]
     Windows Server 2016 (for Nginx)
     Windows 10 (for Openssl)

○ Create a Certificate and Private Key files for MuleSoft.

1. Export .pfx file from IIS where Orchestrator is running.
  1. Open IIS Manager on the server where Orchestrator is running. Then Open "Server Certificate".
  2. Select the certificate file to be exported.
     
  3. Select the "Detail" tab and press "Copy to File..." button.
  4. Press "Next" button.
  5. Select "Yes, export the private key" and press "Next" button.
  6. Check on "Personal Information in the certification path if possible" and press "Next" button.
  7. Input "Password" and "Confirm password" and press "Next" button.
  8. Input "File name" and press "Next" button.
  9. Press "Finish" button. The .pfx file is created.

         
  10. Export Root Certificate as .pfx file as like the above.

2. Convert .pfx files.

  1. Openssl must be installed. You can download MSI file for Openssl on Windows from here.
  2. The exported .pfx file must be copied in any directory in the PC where openssl is installed.
  3. Type the below command in the directory that .pfx file is copied in order to extract Certificate from .pfx file.
  4. C:\any directory> openssl pkcs12 -in {server_pfx_file_name} -clcerts -nokeys -out domain.cer
    Enter Import Password: {input_password_for_server_pfx_file}
    MAC verified OK
    Warning unsupported bag type: secretBag
  5. Type the below command in order to extract a Private Key file from .pfx file.
  6. C:\any directory> openssl pkcs12 -export -in {server_pfx_file_name} -nocerts -nodes -out domain.rsa
    Enter pass phrase for orchestrator.pem: {input_password_for_pem_file_above}
    unable to write random state'
  7. Type the below command in order to convert root .pfx file to pem file.
  8. C:\any directory> openssl pkcs12 -export -in {root_pfx_file_name} -out root.pem
    Enter pass phrase for orchestrator.pem: {input_password_for_pem_file_above}
    Enter PEM pas prase: {input_new_password_for_PEM_file}
    Verifying - Enter PEM pass phrase: {retype_new_password}
    unable to write random state'

  3. Set the configuration for converted file on Nginx.

  1. Set the 3 files created aboce (domain.cer, domain.rsa and root.pem) in Nginx onf file as the below
        ...
        location / {
     proxy_ssl_server_name on;
            proxy_ssl_name orchestrator.nbdlab.local;
            proxy_pass https://orchestrator;
            proxy_ssl_session_reuse off;
            proxy_redirect https://{orchestrator_server_name} https://{proxy_server_name};

            proxy_http_version 1.1;  
            proxy_ssl_password_file PATH_TO_PASSWORD_FILE/password.txt;
            proxy_ssl_certificate PATH_TO_DOMAIN_CER//domain.crt;
            proxy_ssl_certificate_key PATH_TO_DOMAIN_RSA/domain.rsa;
            proxy_ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
            proxy_ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
            proxy_ssl_trusted_certificate PASS_TO_ROOT_PEM/root.pem;
            proxy_ssl_verify on;
            proxy_ssl_verify_depth 2;

            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host orchestrator.nbdlab.local;
            proxy_set_header X_FORWARDED_PROTO https;
        }
        location /MixedAuth {
     proxy_ssl_server_name on;
            proxy_ssl_name orchestrator.nbdlab.local;
            proxy_pass https://orchestrator;
            proxy_ssl_session_reuse off;
            proxy_redirect https://{orchestrator_server_name} https://{proxy_server_name};

            proxy_http_version 1.1;  
            proxy_ssl_password_file PATH_TO_PASSWORD_FILE/password.txt;
            proxy_ssl_certificate PATH_TO_DOMAIN_CER//domain.crt;
            proxy_ssl_certificate_key PATH_TO_DOMAIN_RSA/domain.rsa;
            proxy_ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
            proxy_ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
            proxy_ssl_trusted_certificate PASS_TO_ROOT_PEM/root.pem;
            proxy_ssl_verify on;
            proxy_ssl_verify_depth 2;

            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host orchestrator.nbdlab.local;
            proxy_set_header X_FORWARDED_PROTO https;
            proxy_set_header Connection "";
        }

        location /signalr {

     proxy_ssl_server_name on;
            proxy_ssl_name orchestrator.nbdlab.local;
            proxy_pass https://orchestrator;
            proxy_ssl_session_reuse off;
            proxy_redirect https://{orchestrator_server_name} https://{proxy_server_name};

            proxy_http_version 1.1;  
            proxy_ssl_password_file PATH_TO_PASSWORD_FILE/password.txt;
            proxy_ssl_certificate PATH_TO_DOMAIN_CER//domain.crt;
            proxy_ssl_certificate_key PATH_TO_DOMAIN_RSA/domain.rsa;
            proxy_ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
            proxy_ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
            proxy_ssl_trusted_certificate PASS_TO_ROOT_PEM/root.pem;
            proxy_ssl_verify on;
            proxy_ssl_verify_depth 2;

            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host orchestrator.nbdlab.local;
            proxy_set_header X_FORWARDED_PROTO https;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
        ...

○ Reference:

1. https://nginx.org/en/





[MuleSoft Standalone 3.9] Create pkcs12 files for MuleSoft


 MuleSoft Standalone needs pkcs12 file in order to proxy UiPath Orchestrator REST API. Both Certificate and Private Key files are created by extracting the .pfx file of SSL certificate downloaded from IIS where Orchestrator is running.

○ Environment:

[MuleSoft Standalone]
    Version: 3.9.0
  [MuleSoft Anypoint Studio]
     Version: 6.4
  [OS]
     Windows Server 2016 (for MuleSoft Standalone)
     Windows 10 (for MuleSoft Anypoint Studio)

○ Create a Certificate and Private Key files for MuleSoft.

1. Export .pfx file from IIS where Orchestrator is running.
  1. Open IIS Manager on the server where Orchestrator is running. Then Open "Server Certificate".
  2. Select the certificate file to be exported.
     
  3. Select the "Detail" tab and press "Copy to File..." button.
  4. Press "Next" button.
  5. Select "Yes, export the private key" and press "Next" button.
  6. Check on "Personal Information in the certification path if possible" and press "Next" button.
  7. Input "Password" and "Confirm password" and press "Next" button.
  8. Input "File name" and press "Next" button.
  9. Press "Finish" button. The .pfx file is created.

           

2. Create pkcs12 files.

  1. Openssl must be installed. You can download MSI file for Openssl on Windows from here.
  2. The exported .pfx file must be copied in any directory in the PC where openssl is installed.
  3. Type the below command in the directory that .pfx file is copied in order to convert to pem file.
  4. C:\any directory> openssl pkcs12 -in {.pfx_file_name} -out orchestrator.pem
    Enter Import Password: {input_password_for_pfx_file}
    MAC verified OK
    Enter PEM pas prase: {input_new_password_for_PEM_file}
    Verifying - Enter PEM pass phrase: {retype_new_password}
    Warnug unsupported bag type: secretBag
  5. Type the below command in order to convert pem file to p12 file.
  6. C:\any directory> openssl pkcs12 -export -in {.pfx_file_name} -out orchestrator.p12 -name "orchestratotr"
    Enter pass phrase for orchestrator.pem: {input_password_for_pem_file_above}
    Enter Export Password: {input_new_password_for_pkcs12_file}
    Verifying - Enter Export Password: {retype_new_password}
    unable to write random state'
  7. Do the same for Root Certificate from IIS as well.

  2. Set the configuration for pkcs12 file on Mule Application.

  1. Open Mule App on MuleSoft Anypoint Studio.
  2. Cope pkcs12 file to "resources" directory.
  3. Open "HTTP activity" to set pkcs12 file as certificate.
  4. Set pkcs12 files in configuration for Certificate.

○ Reference:

1. https://slproweb.com/products/Win32OpenSSL.html





2017年3月23日木曜日

[iDempiere 2.1] WebService Schema Update


 iDempiere 2.1 provide Web Service (WS) interfaces over SOAP and REST. The user can use the WS in order to create, update and view the Records in DB Table.

WebService Request XML is managed by XML Schema strictly. I needed to modify this XML Schema, and I researched how to generate Data-binding Java classes to parse Request XML depending on XML Schema.

○ Environment:

[iDempiere]
    Version: 2.1
 [Eclipse]
     Pleiades 4.5 Mars
  [OS]
     Windows 7 SP1

 iDempiere 2.1 uses xmlbeans-2.5.0 to parse Request XML. The Data-binding Java classes are stored in idempiere-xmlbeans-1.0.jar at @WORKSPACE@\org.idempiere.webservices\WEB-INF\lib. I had to re-create idempiere-xmlbeans-1.0.jar after updating XML schema in order to reflect schema change to iDempiere WebService actually. I could use xmlbeans Ant Task to create Data-binding Java classed from XML Schema.

○ Create a Project for idempiere-xmlbeans

    First of all, I created a new eclipse project for idempiere-xmlbeans like the below.


      1. build.xml - Ant build file to create Xmlbeans Data-binding Java classes. It is mentioned later.
      2. idempiere-schema.xsd - Request XML Schema. I found its original file at @WORKSPACE@\org.idempiere.webservices\WEB-INF\xsd and modified and copied it here.
      3. xmlbeans-2.5.0.jar - xmlbeans jar iDempiere 2.1 is using. I found it at @WORKSPACE@\org.idempiere.webservices\WEB-INF\lib and copied it here.

○ Execution of Ant Task to generate Xmlbeans Data-binding Java classes

   I wrote  in build.xml like the below.

<?xml version="1.0" encoding="UTF-8"?>
<project default="xmlbeans_build" name="build">
<taskdef name="xmlbean" classname="org.apache.xmlbeans.impl.tool.XMLBean" classpath="lib/xmlbeans-2.5.0.jar" />
 <target name="xmlbeans_build">
                <xmlbean schema="idempiere-schema.xsd" destfile="idempiere-xmlbeans-1.0.jar"/>
 </target>
</project>
    Then  I execute Ant build by eclipse Ant Runner. I add xmlbean-2.5.0.jar into classpath for Ant like the below.



   Ant showed console messages like the below. I was successful to create the new  idempiere-xmlbeans-1.0.jar file.

  Buildfile: @WORKSPACE@\idempiere-xmlbeans\build.xml
xmlbeans_build:
  [xmlbean] Time to build schema type system: 2.53 seconds
  [xmlbean] Time to generate code: 0.853 seconds
  [xmlbean] Compiling 90 source files to @
HOMEPATH@\AppData\Local\Temp\xbean1594992734447386704.d\classes
  [xmlbean] warning: [options] bootstrap class path not set in conjunction with -source 1.4
  [xmlbean] 1 warning
  [xmlbean] Time to compile code: 5.379 seconds
  [xmlbean] Building jar:
@WORKSPACE@\idempiere-xmlbeans\idempiere-xmlbeans-1.0.jar
BUILD SUCCESSFUL
Total time: 14 seconds

○ Reference:

1. https://xmlbeans.apache.org/





2017年3月10日金曜日

[iDedempiere 2.1] Web Service


 iDempiere 2.1 provide Web Service (WS) interfaces over SOAP and REST. The user can use the WS in order to create, update and view the Records in DB Table.

ModelADService - It can be used for view Records and  Simple Record edition.
CompositeInterface - It can be used for Multiple Record edition.

○ Environment:

[iDempiere]
    Version: 2.1
 [Eclipse]
     Pleiades 4.5 Mars
  [OS]
     Windows 7 SP1

○ Setting Web Service on iDempiere

 The WS is working on iDempiere if iDempiere is installed already. But, "Web Service Security" Record is need to be created to allow the user to access WS on iDempiere. Web Service Security is managed depending on Client as well as other Record in iDempiere. So, Client Admistrator must create Web Service Security Record with the Client in order to allow members in the Client to access WS.
 If you test WS by Garden Client, you don't have to prepare Web Service Security Record by yourself because these exist already. The examples following use
"CreateBPartner".

○SoapUI

 Adempire and iDempiere have test scripts to test WS in org.adempiere.webservice package. Soap UI is the application to test WS over SOAP and REST, which is introduced in Adempiere book in order to execute these test scripts in Adempiere.
 The latest version of Soap UI can be downloaded here. Test scripts in iDempiere are created by former version, but these can be executed by the latest version as well.

○Test WS over SOAP

 Test WS over SOAP can be done by test scripts. 

1. Import SOAP project from the script in iDempiere (%iDempiere_Home%\org.idempiere.webservices\testScripts\iDempiereWebServices-soapui-project.xml)

      


2. Edit SOAP XML.
 Open XML for createData

And edit the following fields under "_0:ADLoginRequest" to fit your environment if it is needed.
  1. _0:user
  2. _0:pass
  3. _0:ClientID
  4. _0:OrgID
  5. _0:WarehouseID
  6. _0:stage

2. Edit SOAP XML.
3.  Click Submit button to execute the test script.


○Test WS over REST

 iDempiere provides REST interfaces also. the REST provided by iDempiere recieves XML as request, not JSON. The request XML is as same as XML request in SOAP request.
 REST request also can be sent by Soap UI. But, there is no test scripts for REST in iDempiere now. The user need to create project and request record in Soap UI by themselves to execute WS over REST test.

1. Create REST project on Soap UI.
   URL: http://localhost:8080/ADInterface/services/rest/model_adservice/create_data




2.  Create new POST method under new REST project.

3.  Create new Request under POST method.


Set Media Type: application/xml
Set XML following as POST data. edit tags under "_0:ADLoginRequest" tag to fit your environment if it is needed as SOAP test.

<_0:ModelCRUDRequest xmlns:_0="http://idempiere.org/ADInterface/1_0">
 <_0:ModelCRUD>
  <_0:serviceType>CreateBPartner</_0:serviceType>
  <_0:TableName>C_BPartner</_0:TableName>
  <!--<_0:RecordID>0</_0:RecordID>-->
  <!--<_0:Action>Create</_0:Action>-->
  <_0:DataRow>
   <_0:field column="Value">
    <_0:val>GlobalQSS</_0:val>
   </_0:field>
   <_0:field column="Name">
    <_0:val>Quality Systems &amp; Solutions</_0:val>
   </_0:field>
   <_0:field column="TaxID">
    <_0:val>830.085.359-4</_0:val>
   </_0:field>
   <_0:field column="IsVendor">
    <_0:val>Y</_0:val>
   </_0:field>
   <_0:field column="IsCustomer">
    <_0:val>N</_0:val>
   </_0:field>
   <_0:field column="IsTaxExempt">
    <_0:val>N</_0:val>
   </_0:field>
   <_0:field column="Name2">
    <_0:val>QSS Ltda. - http://www.globalqss.com</_0:val>
   </_0:field>
   <_0:field column="C_BP_Group_ID">
    <_0:val>104</_0:val>
   </_0:field>
  </_0:DataRow>
 </_0:ModelCRUD>
 <_0:ADLoginRequest>
  <_0:user>WebService</_0:user>
  <_0:pass>WebService</_0:pass>
  <_0:lang>en_US</_0:lang>
  <_0:ClientID>11</_0:ClientID>
  <_0:RoleID>50004</_0:RoleID>
  <_0:OrgID>11</_0:OrgID>
  <_0:WarehouseID>103</_0:WarehouseID>
  <_0:stage>9</_0:stage>
 </_0:ADLoginRequest>
</_0:ModelCRUDRequest>

4.  Click Submit button to execute the test script.



     Response XML is displayed in the right pane.
   

○ Reference:

1. http://ultra00.hatenablog.com/entry/2013/11/18/114542
2. https://groups.google.com/forum/#!topic/idempiere/82cXxgq_Qh4





2017年1月18日水曜日

[QNAP] Upload to Dropbox by shell script.


 I could upload the files on QNAP NAS to Dropbox by shell script. It is useful to make replica of local backup file in NAS.

○ Environment:

NAS: QNAP TS-531P

○Install Ubuntu LXC on NAS.

 The shell script is executed on Ubuntu LXC. Before creating the shell scrip, Ubuntu LXC should be installed on NAS. 

1. After setting up "Container Station" on NAS, open it.Select Create Container menu.

 2. Click Create button for "Ubuntu LXC".

  3. Set Shared Folder in Advanced Setting.
 

○Get Access Token of Dropbox.

 The Access Token is needed to upload file by shell script. Access Token can be got as the followings. 

  1. Access to https://www.dropbox.com/developers/apps.

  2. Click Create app button.

  3. Choose an API -> Select "Dropbox API" 

      Choose the type of access you need -> Select "Full Dropbox"

      Name your app -> input your App name (ex. QNAP_Uploader)

  4. Click Create app button.

  5. Open App you created, and click Generate button to create a new access token.

      Copy the new access token, which is needed to set up Dropbox-Uploader.


○Install Dropbox-Uploader into Ubuntu LXC on NAS.

 Dropbox-Uploader is  the free shell application to operate Dropbox from shell command line including uploading files. It can work on any Distribution of Linux installed bash and curl.

 Download and execute Dropbox-Uploader.

[root@servername~]# git clone https://github.com/andreafabrizi/Dropbox-Uploader.git
[root@servername~]# cd Dropbox-Uploader/
[root@servername~]# ./dropbox_uploader.sh

 In the first time you execute Dropbox-Uploader, Dropbox-Uploader asks Dropbox access token. Input the access token you created. Next time you execute Dropbox-Uploader, you can operate any command (upload, delete and so on) of it.

 

○Preparing the shell file to upload files into Dropbox.

This shell script works as the followings.

1. The func_upload_files function searches directories and files under {the local root path}/{first arg} recursively.

2. If a files is found, then the files are upload the files into {Dropbox root path}.

  uploadDropbox.sh

#!/bin/sh
DROPBOX_ROOT="Write Dropbox root path to upload files."
ROOT="Write local root path to search the files to upload."
func_upload_files()
{
    BASE_DIR=$ROOT$1/
    DEST_DIR=$DROPBOX_ROOT/$1
    SRC_DIR=`find $BASE_DIR -maxdepth 1 -type d | sort -nr | head -n 1`"/"
    if [ $SRC_DIR != $BASE_DIR ]; then
        /usr/local/Dropbox-Uploader/dropbox_uploader.sh delete $DROPBOX_ROOT$1
        func_search_file $SRC_DIR $DEST_DIR
    fi
}

func_search_file()
{
    for d in `find $1 -maxdepth 1 -type d | sort -nr`; do
        if [ $d != $1 ]; then
            func_search_file $d $2
        fi
    done
    for f in `find $1 -maxdepth 1 -type f | sort -nr`; do
        FILENAME=${f##*$SRC_DIR}
        #echo $DROPBOX_ROOT/$FILENAME
        /usr/local/Dropbox-Uploader/dropbox_uploader.sh upload $f $2/$FILENAME
    done
}
func_upload_files XXXX
func_upload_files YYYY
func_upload_files ZZZZ
...
...
...

○Reference 

1. https://daisukeblog.com/?p=2067

2016年12月15日木曜日

[iDempiere2.1] Build by Jenkins


 I was successful to build iDempiere 2.1 in SVN repository by Jenkins.

○ Environment:

CPU : AMD FX-9590 (4.7 GHz)
Hypervisor : Citrix  XenServer 6.5
OS : Jessie (Debian)
Jenkins : 2.19.3 (at 2016/12/15)

○install Jenkins.

 Jenkins is installed on Linux by the folloings.

[root@servername ~]# -q -O - http://pkg.jenkins-ci.org/debian-stable/jenkins-ci.org.key | sudo apt-key add -
[root@servername ~]# echo "deb http://pkg.jenkins-ci.org/debian-stable binary/" | sudo tee -a /etc/apt/sources.list
[root@servername ~]# apt-get update
[root@servername ~]# apt-get install jenkins -y
[root@servername ~]# service jenkins restart

○install Eclipse Mars2.

 In order to build the plugin jars of iDempiere, Eclipse must be installed. Mars2 is installed by the folloings.

[root@servername ~]# wget -q http://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/mars/2/eclipse-jee-mars-2-linux-gtk-x86_64.tar.gz
[root@servername ~]# tar zxvf ./eclipse-jee-mars-2-linux-gtk-x86_64.tar.gz -C {install dir}

○Preparing the shell files for Jenkins Build.

This shell file kicks Ant Runner in Eclipse to compile all project of iDempiere.

  eclipse_build.sh

export JAVA_HOME={Path to Java home dir to be used for build}
export ECLIPSE_HOME={Elipse instaled dir}
export BUILD_FILE={eclipse_build.xml path}
export WORKSPACE={Path to workspace Jenkins creates}

$JAVA_HOME/bin/java -jar $ECLIPSE_HOME/plugins/org.eclipse.equinox.launcher_1.3.100.v20150511-1540.jar 
-application org.eclipse.ant.core.antRunner -DuseJARFormat=true -data $WORKSPACE -buildfile $BUILD_FILE

  eclipse_build.xml


<?xml version="1.0" encoding="UTF-8"?>
<project default="eclipse_build" name="build">
 <target name="eclipse_build">
                <eclipse.incrementalBuild kind="full" />
 </target>
</project>

  This shell file kicks Ant Runner in Eclipse to export plugin jar files of iDempiere.

  plugin_build.sh

export JAVA_HOME={Path to Java home dir to be used for build}
export ECLIPSE_HOME={Elipse instaled dir}
export BUILD_FILE={plugin_build.xml path}
export WORKSPACE={Path to workspace Jenkins creates}

$JAVA_HOME/bin/java -jar $ECLIPSE_HOME/plugins/org.eclipse.equinox.launcher_1.3.100.v20150511-1540.jar 
-application org.eclipse.ant.core.antRunner -Dplugins=$1 -DuseJARFormat=true -data $WORKSPACE -buildfile $BUILD_FILE

  plugin_build.xml


<?xml version="1.0" encoding="UTF-8"?>
<project default="plugin_export" name="build">
 <target name="plugin_export">
  <eclipse.incrementalBuild project="${plugins}" kind="full" />
  <pde.exportPlugins destination="/usr/local/idempiere/jenkins/build" exportSource="false" exportType="directory" plugins="${plugins}" allowBinaryCycles="true" useJARFormat="${useJARFormat}" />
 </target>
</project>

○Setting Jenkins item & check out

 Access to jenkins (http://{jenkins server}:8080) from browser. And create Project with the following build configuration.In sample configuration, This configuration will update SVN chckouts, execute full-build of eclipse workspace and 2 plugin projects in workspace, org.adempiere.base and org.adempiere.base.process are export as plugin jar files.



  Execute Build at once to check out the resources into Jenkins workspace. After that, Jenkins workspace dir is created at "${JENKINS_HOME}/jobs/${ProjectName}/workspace" and SVN resources are checked out all in it. Default $JENKINS_HOME is "/var/lib/jenkins". If you defined Local module directory in Source Code Management, adittional directory is created under default Jenkins workspace path, then SVN resources is checked out into this.

○Configure Eclipse workspace.

Start Eclipse installed on linux.

[root@servername ~]# cd {installed dir}
[root@servername ~]# ./eclipse

 Open the workspace Jenkins created from Eclipse. Before build by Jenkins, all file should be close and eclipse should be shutdowned.


○Jenkins Build

  Execute Jenkins build again. The latest SVN resources checked out and compiled end exporsed into plugin jar files ant then these are copied into deply dir.


○ Reference:

1. http://blog.programster.org/debian-8-install-jenkins/
2. http://qiita.com/yoshi389111/items/b7c7c1dfb5411c410798





2016年9月27日火曜日

[XenServer] UnixBench


 I was successful to install end execute UnixBench on  XenServer 6.5.\


○ Environment:

CPU : AMD FX-9590 (4.7 GHz)
Hypervisor : Citrix  XenServer 6.5
OS : Stretch (Debian)

Stretch is running with 2 CPUs And 2GB memory on XenServer.

○install UnixBench.

[root@servername ~]# wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/byte-unixbench/UnixBench5.1.3.tgz
--2016-09-27 12:01:43--  https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/byte-unixbench/UnixBench5.1.3.tgz
Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.23.128, 2404:6800:4008:c01::80
Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.23.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 143259 (140K) [application/octet-stream]
Saving to: UnixBench5.1.3.tgz 2019

UnixBench5.1.3.tgz  100%[===================>] 139.90K   586KB/s    in 0.2s   

2016-09-27 12:01:45 (586 KB/s) - UnixBench5.1.3.tgz saved [143259/143259] [root@servername ~]#tar zxvf ./UnixBench5.1.3.tgz -C /usr/local/

○Execute UnixBench

[root@servername ~]# cd /usr/local/UnixBench
[root@servername ~]# ./Run
gcc -o ./pgms/arithoh -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Darithoh ./src/arith.c
gcc -o ./pgms/register -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum='register int' ./src/arith.c
gcc -o ./pgms/short -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum=short ./src/arith.c
gcc -o ./pgms/int -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum=int ./src/arith.c
gcc -o ./pgms/long -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum=long ./src/arith.c
gcc -o ./pgms/float -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum=float ./src/arith.c
gcc -o ./pgms/double -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -Ddatum=double ./src/arith.c
gcc -o ./pgms/hanoi -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/hanoi.c
gcc -o ./pgms/syscall -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/syscall.c
./src/syscall.c: In function \u2018main\u2019:
./src/syscall.c:93:21: warning: null argument where non-null required (argument 2) [-Wnonnull]
                     execl("/bin/true", (char *) 0);
                     ^~~~~
gcc -o ./pgms/context1 -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/context1.c
gcc -o ./pgms/pipe -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/pipe.c
gcc -o ./pgms/spawn -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/spawn.c
gcc -o ./pgms/execl -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/execl.c
In file included from ./src/execl.c:34:0:
./src/big.c: In function \u2018getwork\u2019:
./src/big.c:452:11: warning: variable \u2018c\u2019 set but not used [-Wunused-but-set-variable]
     char  c;
           ^
cd ./src; gcc -c -DTIME -Wall -pedantic -ansi -DHZ= -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall dhry_1.c
cd ./src; gcc -c -DTIME -Wall -pedantic -ansi -DHZ= -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall dhry_2.c
gcc -o ./pgms/dhry2 -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/dhry_1.o ./src/dhry_2.o
cd ./src; rm -f dhry_1.o dhry_2.o
cd ./src; gcc -c -DTIME -Wall -pedantic -ansi -DREG=register -DHZ= -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall dhry_1.c -o dhry_1_reg.o
cd ./src; gcc -c -DTIME -Wall -pedantic -ansi -DREG=register -DHZ= -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall dhry_2.c -o dhry_2_reg.o
gcc -o ./pgms/dhry2reg -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/dhry_1_reg.o ./src/dhry_2_reg.o
cd ./src; rm -f dhry_1_reg.o dhry_2_reg.o
gcc -o ./pgms/looper -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/looper.c
gcc -o ./pgms/fstime -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall ./src/fstime.c
gcc -o ./pgms/whetstone-double -DTIME -Wall -pedantic -ansi -O2 -fomit-frame-pointer -fforce-addr -ffast-math -Wall -DDP -DUNIX -DUNIXBENCH ./src/whets.c -lm
make all
make[1]: Entering directory '/usr/local/UnixBench'
Checking distribution of files
./pgms  exists
./src  exists
./testdir  exists
./results  exists
make[1]: Leaving directory '/usr/local/UnixBench'
sh: 1: 3dinfo: not found

   #    #  #    #  #  #    #          #####   ######  #    #   ####   #    #
   #    #  ##   #  #   #  #           #    #  #       ##   #  #    #  #    #
   #    #  # #  #  #    ##            #####   #####   # #  #  #       ######
   #    #  #  # #  #    ##            #    #  #       #  # #  #       #    #
   #    #  #   ##  #   #  #           #    #  #       #   ##  #    #  #    #
    ####   #    #  #  #    #          #####   ######  #    #   ####   #    #

   Version 5.1.3                      Based on the Byte Magazine Unix Benchmark

   Multi-CPU version                  Version 5 revisions by Ian Smith,
                                      Sunnyvale, CA, USA
   January 13, 2011                   johantheghost at yahoo period com


1 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10

1 x Double-Precision Whetstone  1 2 3 4 5 6 7 8 9 10

1 x Execl Throughput  1 2 3

1 x File Copy 1024 bufsize 2000 maxblocks  1
 2 3

1 x File Copy 256 bufsize 500 maxblocks  1 2 3

1 x File Copy 4096 bufsize 8000 maxblocks  1 2 3

1 x Pipe Throughput  1 2 3 4 5 6 7 8 9 10

1 x Pipe-based Context Switching  1 2 3 4 5 6 7 8 9 10

1 x Process Creation  1 2 3

1 x System Call Overhead  1 2 3 4 5 6 7 8 9 10

1 x Shell Scripts (1 concurrent)  1 2 3

1 x Shell Scripts (8 concurrent)  1 2 3

2 x Dhrystone 2 using register variables  1 2 3 4 5 6 7 8 9 10

2 x Double-Precision Whetstone  1 2 3 4 5 6 7 8 9 10

2 x Execl Throughput  1 2 3

2 x File Copy 1024 bufsize 2000 maxblocks  1 2 3

2 x File Copy 256 bufsize 500 maxblocks  1 2 3

2 x File Copy 4096 bufsize 8000 maxblocks  1 2 3

2 x Pipe Throughput  1 2 3 4 5 6 7 8 9 10

2 x Pipe-based Context Switching  1 2 3 4 5 6 7 8 9 10

2 x Process Creation  1 2 3

2 x System Call Overhead  1 2 3 4 5 6 7 8 9 10

2 x Shell Scripts (1 concurrent)  1 2 3

2 x Shell Scripts (8 concurrent)  1 2 3

========================================================================
   BYTE UNIX Benchmarks (Version 5.1.3)

   System: BandaAceh: GNU/Linux
   OS: GNU/Linux -- 4.3.0-1-amd64 -- #1 SMP Debian 4.3.5-1 (2016-02-06)
   Machine: x86_64 (unknown)
   Language: en_US.utf8 (charmap="UTF-8", collate="UTF-8")
   CPU 0: AMD FX(tm)-9590 Eight-Core Processor (9380.0 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSCALL/SYSRET
   CPU 1: AMD FX(tm)-9590 Eight-Core Processor (9380.0 bogomips)
          Hyper-Threading, x86-64, MMX, AMD MMX, Physical Address Ext, SYSCALL/SYSRET
   12:46:36 up  1:07,  1 user,  load average: 3.27, 1.22, 0.60; runlevel 5

------------------------------------------------------------------------
Benchmark Run: Tue Sep 27 2016 12:46:36 - 13:14:44
2 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       37698819.4 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4648.1 MWIPS (9.7 s, 7 samples)
Execl Throughput                                946.5 lps   (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        316189.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           86468.0 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        828011.4 KBps  (30.0 s, 2 samples)
Pipe Throughput                              512535.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  63324.0 lps   (10.0 s, 7 samples)
Process Creation                               1725.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   5501.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1055.3 lpm   (60.0 s, 2 samples)
System Call Overhead                         509390.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   37698819.4   3230.4
Double-Precision Whetstone                       55.0       4648.1    845.1
Execl Throughput                                 43.0        946.5    220.1
File Copy 1024 bufsize 2000 maxblocks          3960.0     316189.3    798.5
File Copy 256 bufsize 500 maxblocks            1655.0      86468.0    522.5
File Copy 4096 bufsize 8000 maxblocks          5800.0     828011.4   1427.6
Pipe Throughput                               12440.0     512535.0    412.0
Pipe-based Context Switching                   4000.0      63324.0    158.3
Process Creation                                126.0       1725.9    137.0
Shell Scripts (1 concurrent)                     42.4       5501.0   1297.4
Shell Scripts (8 concurrent)                      6.0       1055.3   1758.8
System Call Overhead                          15000.0     509390.6    339.6
                                                                   ========
System Benchmarks Index Score                                         606.5

------------------------------------------------------------------------
Benchmark Run: Tue Sep 27 2016 13:14:44 - 13:42:53
2 CPUs in system; running 2 parallel copies of tests

Dhrystone 2 using register variables       77572329.0 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     9325.4 MWIPS (9.8 s, 7 samples)
Execl Throughput                               3223.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        481966.9 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          130799.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1351926.1 KBps  (30.0 s, 2 samples)
Pipe Throughput                              734678.3 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 134536.7 lps   (10.0 s, 7 samples)
Process Creation                               5801.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   8146.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1084.0 lpm   (60.1 s, 2 samples)
System Call Overhead                         901477.2 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   77572329.0   6647.2
Double-Precision Whetstone                       55.0       9325.4   1695.5
Execl Throughput                                 43.0       3223.2    749.6
File Copy 1024 bufsize 2000 maxblocks          3960.0     481966.9   1217.1
File Copy 256 bufsize 500 maxblocks            1655.0     130799.5    790.3
File Copy 4096 bufsize 8000 maxblocks          5800.0    1351926.1   2330.9
Pipe Throughput                               12440.0     734678.3    590.6
Pipe-based Context Switching                   4000.0     134536.7    336.3
Process Creation                                126.0       5801.5    460.4
Shell Scripts (1 concurrent)                     42.4       8146.8   1921.4
Shell Scripts (8 concurrent)                      6.0       1084.0   1806.7
System Call Overhead                          15000.0     901477.2    601.0
                                                                   ========
System Benchmarks Index Score                                        1113.0

------------------------------------------------------------------------
Benchmark Run: Mon Oct 03 2016 09:19:18 - 09:47:38
8 CPUs in system; running 1 parallel copy of tests
The result of the case of allocation 8 CPU and 15 GB Memory to Stretch.
Dhrystone 2 using register variables       40947340.1 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     4407.1 MWIPS (10.9 s, 7 samples)
Execl Throughput                                542.9 lps   (29.8 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        426851.1 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          113232.8 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1042961.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                              671837.6 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  43131.9 lps   (10.0 s, 7 samples)
Process Creation                               1074.5 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   4607.3 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1830.9 lpm   (60.0 s, 2 samples)
System Call Overhead                         546894.8 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   40947340.1   3508.8
Double-Precision Whetstone                       55.0       4407.1    801.3
Execl Throughput                                 43.0        542.9    126.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     426851.1   1077.9
File Copy 256 bufsize 500 maxblocks            1655.0     113232.8    684.2
File Copy 4096 bufsize 8000 maxblocks          5800.0    1042961.7   1798.2
Pipe Throughput                               12440.0     671837.6    540.1
Pipe-based Context Switching                   4000.0      43131.9    107.8
Process Creation                                126.0       1074.5     85.3
Shell Scripts (1 concurrent)                     42.4       4607.3   1086.6
Shell Scripts (8 concurrent)                      6.0       1830.9   3051.6
System Call Overhead                          15000.0     546894.8    364.6
                                                                   ========
System Benchmarks Index Score                                         613.2

------------------------------------------------------------------------
Benchmark Run: Mon Oct 03 2016 09:47:38 - 10:15:53
8 CPUs in system; running 8 parallel copies of tests

Dhrystone 2 using register variables      218684221.3 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                    30972.0 MWIPS (9.7 s, 7 samples)
Execl Throughput                               6237.2 lps   (29.9 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        505477.7 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks          135568.9 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks       1383821.7 KBps  (30.0 s, 2 samples)
Pipe Throughput                             4042237.9 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                 564378.2 lps   (10.0 s, 7 samples)
Process Creation                               9520.7 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                  16182.0 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   2216.4 lpm   (60.1 s, 2 samples)
System Call Overhead                        1980170.7 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0  218684221.3  18739.0
Double-Precision Whetstone                       55.0      30972.0   5631.3
Execl Throughput                                 43.0       6237.2   1450.5
File Copy 1024 bufsize 2000 maxblocks          3960.0     505477.7   1276.5
File Copy 256 bufsize 500 maxblocks            1655.0     135568.9    819.1
File Copy 4096 bufsize 8000 maxblocks          5800.0    1383821.7   2385.9
Pipe Throughput                               12440.0    4042237.9   3249.4
Pipe-based Context Switching                   4000.0     564378.2   1410.9
Process Creation                                126.0       9520.7    755.6
Shell Scripts (1 concurrent)                     42.4      16182.0   3816.5
Shell Scripts (8 concurrent)                      6.0       2216.4   3693.9
System Call Overhead                          15000.0    1980170.7   1320.1
                                                                   ========
System Benchmarks Index Score                                        2322.4

○ Reference:

1. http://blog.idcf.jp/entry/cloud/unixbench/
2. http://www.bravotouring.com/%7Eyano/diary/it/20130314unixbench.html