hide's memo
9 Apr, 2023

How to get a refresh token from servicenow

[Japanese(日本語)]

How to get a refresh token from your servicenow instance with using OAuth2.0 Authorization code flow.

(1)Create endpoint on your servicenow.
(1.1)Login your servicenow instance.
(1.2)System OAuth -> Application Registory
(1.3)Push New button.
(1.4)Select “Create an OAuth API endopint for external clients”
(1.5)Apply your endpoint as below.

(2)Get Access code
(2.1)Open web browser
(2.2)Acccess the url below.


https://${YOUR_INSTANCE}.service-now.com/oauth_auth.do?response_type=code&redirect_uri=https%3A%2F%2Flocalhost%2Ftest&client_id=${YOUR_CLIENT_ID}&grant_type=authorization_code&state=123

* redirect_url ... https://localhost/test (you can apply the url that you specified on your endpoint.)
* state ... 123(you can apply your favorit word. this is just a sample)

 

(2.3)you can get login page.(if you don’t login your servicenow instance)


(2.4)Put “Allow” Button.


(2.5)Your redirect must be fail, but you can get the code on your address bar.


(2.6)save the code.

(3)Get Access token and Refresh token.
(3.1)prepare shell script as below.

#!/bin/sh

MY_INSTANCE="**** YOUR INSTANCE ****"
CLIENT_ID="**** YOUR CLIENT ID ****"
CLIENT_SECRET="**** YOUR CLIENT SECRET ****"
REDIRECT_URI="https%3A%2F%2Flocalhost%2Ftest" # you can apply the url that you specified on your endpoint.)
STATE="123" # specify the same word of (2.2)
CODE=$1

RESULT=`curl -X POST https://${MY_INSTANCE}.service-now.com/oauth_token.do -d "grant_type=authorization_code" -d "code=${CODE}" -d "redirect_uri=${REDIRECT_URI}" -d "client_id=${CLIENT_ID}"\
-d "client_secret=${CLIENT_SECRET}" -d "state=${STATE}"`

echo $RESULT

 

(3.2)exec the script above with argument of the code(sequence No.2.6)
(3.3)you can get the refresh token.

{“access_token”:”***********”,“refresh_token”:”*******”,”scope”:”useraccount”,”token_type”:”Bearer”,”expires_in”:1799}

 

■how to access servicenow REST-API with using Refresh token

seek the link below.

Get data from ServiceNow using curl

6 Apr, 2023

Fiddler

[Japanese(日本語)]

HTTP Packet capture tool.
you can see HTTP communications between your web browser and web server.
Additionally, you can change the HTTP request and response.

 

■How to see HTTP communications

(1)select the communication you want to see.
(2)select “Inspectors” tab.
(3)Select “Raw” tab.
(4)you can see HTTP request in the right upper window.
(5)you can see HTTP resonse in the right upper window.

 

 

■How to change HTTP request.

(1)Select Rules -> Automatic Breakpoints -> Before Request.
then Fiddler holds every HTTP request before sending.
(2)Choose holded HTTP communication.
(3)You can change HTTP request.
(4)Push Run to Completion Button.

As Fiddler holds every HTTP request,
You have to process every HTTP request .

■How to use Fiddler’s record to JMeter.

(1)Export Fiddler’s record as “cURL Script”.

(2)Import data to JMeter.

When you import the “cURL Script”. JMeter kindly prepare Cookie Manager which is basically required by Most Web Application.
Additionaly , you will have “View Result Tree”, So you can check the JMeter’s test result.

But, Thread Group is set with more than 10 thread.
I think you shuld better change Thred properties so that you can simply see how the test senario run.

25 Mar, 2023

Align images from right to left

[Japanese(日本語)]

How to align images from right to left.

 

 

 

 

 

 

 

 

 

 

 

 

 

you can set “rtl” to “p” tag’s “dir” attribute.

<html>
<body>
<p dir="rtl">
<img src="001.png">
<img src="002.png">
<img src="003.png">
<img src="004.png">
<img src="005.png">
<img src="006.png">
<img src="007.png">
</p>
</body>
</html>
18 Mar, 2023

adidas ADIZERO BOSTON 11

[Japanese(日本語)]

I bought an adidas ADIZERO BOSTON 11 on march 2023.
But, when I run with these shoes, my instep of my right foot hurts.

Even loosening the shoelace did not help.
the shoe tongue of this shoe is connected with the sole and it is very tight.
finally I cut the tongue of this shoe, and now, the shoes are very confortable for me.

16 Mar, 2023

convert csv to json

Sample src of How to convert csv to json.
You can change csv file to json.
You can also add keys, convert string to integer, convert string to boolean.

[Japanese(日本語)]

 

import csv
import json
import sys

#---------------------------------------
# change string(0/1) to Boolean(False/True)
#---------------------------------------
def to_boolean(s):
    if(s == ""):
        return None
    if(s == "0"):
        return False
    elif(s=="1"):
        return True
    else:
        raise ValueError("not boolean")


#---------------------------------------
# main
#--------------------------------------
args = sys.argv
jdata = {"field01":"hello", "filed02":"world", "country_records":[]}
r = open(args[1],"r")

d_reader = csv.DictReader(r)
d_list = [row for row in d_reader]

for row in d_list:
    row["population"] = int(row["population"])
    row["capital"] = to_boolean(row["capital"])

jdata["country_records"] = d_list
print(json.dumps(jdata, ensure_ascii=False, indent=2))
25 Feb, 2023

Print out excel file by VBScript

[Japanese(日本語)]

This is a sample of how to print out excel file’s cells by VBScript.
https://github.com/hidemiubiz/public/blob/main/VBS/excelparse.vbs

'-------------------------------------------------------------------
' Excelファイルの中で、文字列が入ったセルから文字列を抜き出すサンプル
' this is a sample extracting string from excel files.
'
' Usage: csript excelparse.vbs input.xls left-top-cell right-bottom-cell
' Example:
' If you want to print sample.xls's B5 to E7 celss, put the followig command.
' cscript excelparse.vbs sample.xls B5 E7
'-------------------------------------------------------------------

set objXls = CreateObject("Excel.Application")
objXls.Visible=False
objXls.ScreenUpdating=False

Dim filename
Dim left_top_cell
Dim right_bottom_cel
filename = Wscript.Arguments(0)
left_top_cell = Wscript.Arguments(1)
right_bottom_cell = Wscript.Arguments(2)

call ParseExcel(filename, left_top_cell, right_bottom_cell)

'-----------------------------------------------------
' 標準出力(Print stdout)
'-----------------------------------------------------
Function Print(m)
WScript.Stdout.Write m
End Function

'-----------------------------------------------------
' Get Full path
'-----------------------------------------------------
Function GetFullPath(ByRef filename)
GetFullPath = CreateObject("Scripting.FileSystemObject").GetAbsolutePathName(filename)
End Function

'-----------------------------------------------------
' Excelファイルをパースして出力
' print out excel file
' filename .............. excel filename パースするExcelファイル
' left_top_cell ......... left_top_cell string(example ... "B5")
' right_bottom_cell ..... right_bottom_cell string (such as "D10")
'-----------------------------------------------------
Sub ParseExcel(ByRef filename, ByRef left_top_cell, ByRef right_bottom_cell)
Dim wb,s
Dim lef_top_row
Dim lef_top_col
Dim right_bottom_row
Dim right_bottom_col

Set wb = objXls.Workbooks.Open(GetFullPath(filename))
Dim i
For i=1 To wb.Sheets.Count
Set s = wb.Sheets(i)
Set lt = s.Range("A1:" & left_top_cell)
Set rb = s.Range("A1:" & right_bottom_cell)
left_top_row = lt.Rows.Count
left_top_col = lt.Columns.Count
right_bottom_row = rb.Rows.Count
right_bottom_col = rb.Columns.Count
Call PrintRange(s, left_top_row, right_bottom_row, left_top_col, right_bottom_col)
Next
wb.Close
End Sub

'-----------------------------------------------------
' 指定された範囲のセルの文字列を表示する。
' print out specific range cells
' sheet .......... シートObject
' startRow ....... 行の開始位置(left_top_row)
' endRow ......... 行の終了位置(right_bottom_row)
' startCol ....... 列の開始位置(left_top_col)
' endCol ......... 列の終了位置(right_bottom_col)
'-----------------------------------------------------
Sub PrintRange(ByRef sheet, ByVal startRow, ByVal endRow, ByVal startCol, ByVal endCol)
Dim y
Dim x
For y=startRow To endRow
For x=startCol To endCol
Set c = sheet.Cells(y,x)
Print(c.Text & ",")
Next
Print(vbCrLf)
Next
End Sub

 

■How to use

if you want to print out from B5 cell to E7 cell like below image.
cscript  thissample.cbs  input.xlsx  B5  E7

 

31 Jul, 2022

How to UPSERT using Spark Dataframe

[Japanese(日本語)]

I am not sure if this article has any problem. If you found something wrong in this article. please let me know.

you have 2 csv data files and want to upsert as follows.

■Sampe Code No.1

df1 = spark.read.option("header","true").csv("data01.csv")
df2 = spark.read.option("header","true").csv("data02.csv")

df = df1.join(df2, ['Country','City'], 'left_anti')
df = df.unionAll(df2)

df.write.option("header","true").format("csv").save("output")

 

■Sample Code No.2
Another example(this code is sometimes work intensionally, But sometimes doesn’t work intensionally.
It depends on how many partitions the program execute)

df1 = spark.read.option("header","true").csv("data01.csv")
df2 = spark.read.option("header","true").csv("data02.csv")

df = df2.unionAll(df1)
df = df.dropDuplicates(['Country','City'])

df.write.option("header","true").format("csv").save("output")

 

 

■Sample Code No.3
this code will probably work intensionally(sorry, no guarantee).

df1 = spark.read.option("header","true").csv("data01.csv")
df2 = spark.read.option("header","true").csv("data02.csv")

df = df2.unionAll(df1)
df = df.dropDuplicates(['Country','City'])

df=df.coalesce(1) 
df.write.option("header","true").format("csv").save("output")

 

■Difference between No2. and No.3 code is the following place.

I think Spark is “Lazy Evaluation”. So, When you call the folowing code.

df = df2.unionAll(df1)
df = df.dropDuplicates(['Country','City'])

Spark Dataframe doesn’t work actually yet.It works when the following code was executed.

df.write.option("header","true").format("csv").save("output")

So, I guess the Sample Code No.3  forced Dataframe work with 1 partition,and then, dropDuplicaets() work intentionally.

28 May, 2022

Upload files to S3 Bucket using curl.

[日本語]

I followed the information below.
tuxfight3r/s3_upload.sh

#!/bin/bash

S3KEY="Your Acess Key"
S3SECRET="Your Access Secret"
S3BUCKET="Your Bucket Name"
S3STORAGETYPE="STANDARD"
AWSREGION="Your S3 Region"

# If you use mac OS. enable following code. 
#OS="mac"

function putS3
{
  file_path=$1
  aws_path=$2
  bucket="${S3BUCKET}"
  date=$(date -R -u)

  acl="x-amz-acl:private"
  content_type="text/plain"
  storage_type="x-amz-storage-class:${S3STORAGETYPE}"
  string="PUTnn$content_typen$daten$acln$storage_typen/$bucket$aws_path${file_path##/*/}"

  if [ $OS = "mac" ]; then
    signature=$(printf "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)
  else
    signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64)
  fi

  curl -s --retry 3 --retry-delay 10 -X PUT -T "$file_path" 
  -H "Host: $bucket.${AWSREGION}.amazonaws.com" 
  -H "Date: $date" 
  -H "Content-Type: $content_type" 
  -H "$storage_type" 
  -H "$acl" 
  -H "Authorization: AWS ${S3KEY}:$signature" 
  "https://$bucket.${AWSREGION}.amazonaws.com$aws_path${file_path##/*/}"

}

# Upload test.txt file to /test/ folder.
putS3 test.txt /test/
27 May, 2022

Extend HttpWebRequest Timeout

[日本語]

The default Timeout of HttpWebRequest is 100 seconds.
So, It will timeout if the server response take more than 100 seconds.

If you want to extend the Timeout. You can set The Timeout as your preferrd time.

But If you set the Timeout, You shoud call the timeout before calling “GetRequestStream()”

 

 

using System;
using System.Web;
using System.Net;
using System.Text;
using System.IO;

namespace ConsoleNetFramework
{
class Sample
{
static void Main(string[] args)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(“http://localhost/slow”);

request.Method=”POST”;
byte[] postData = System.Text.Encoding.ASCII.GetBytes(“message=hello”);

 request.Timeout = 3*60*1000; // <– You need to set Timeout here!!
                                                           //  before calling “GetRequestStream()”.
Stream writeStream = request.GetRequestStream();
 request.Timeout = 3*60*1000; // <– If you set Timeout here after calling “GetRequestStream()”.
// it doesn’t work.

writeStream.Write(postData, 0, postData.Length);

HttpWebResponse response;
// Get Response… and happen the TIMEOUT ERROR.
 response = (HttpWebResponse)request.GetResponse(); 

Stream receiveStream = response.GetResponseStream();
StreamReader sr = new System.IO.StreamReader(receiveStream, System.Text.Encoding.GetEncoding(“utf-8”));
Console.WriteLine(sr.ReadToEnd());
}
}
}