Align images from right to left
How to align images from right to left.
you can set “rtl” to “p” tag’s “dir” attribute.
<html> <body> <p dir="rtl"> <img src="001.png"> <img src="002.png"> <img src="003.png"> <img src="004.png"> <img src="005.png"> <img src="006.png"> <img src="007.png"> </p> </body> </html>
adidas ADIZERO BOSTON 11
I bought an adidas ADIZERO BOSTON 11 on march 2023.
But, when I run with these shoes, my instep of my right foot hurts.
Even loosening the shoelace did not help.
the shoe tongue of this shoe is connected with the sole and it is very tight.
finally I cut the tongue of this shoe, and now, the shoes are very confortable for me.
convert csv to json
Sample src of How to convert csv to json.
You can change csv file to json.
You can also add keys, convert string to integer, convert string to boolean.
import csv import json import sys #--------------------------------------- # change string(0/1) to Boolean(False/True) #--------------------------------------- def to_boolean(s): if(s == ""): return None if(s == "0"): return False elif(s=="1"): return True else: raise ValueError("not boolean") #--------------------------------------- # main #-------------------------------------- args = sys.argv jdata = {"field01":"hello", "filed02":"world", "country_records":[]} r = open(args[1],"r") d_reader = csv.DictReader(r) d_list = [row for row in d_reader] for row in d_list: row["population"] = int(row["population"]) row["capital"] = to_boolean(row["capital"]) jdata["country_records"] = d_list print(json.dumps(jdata, ensure_ascii=False, indent=2))
Print out excel file by VBScript
This is a sample of how to print out excel file’s cells by VBScript.
https://github.com/hidemiubiz/public/blob/main/VBS/excelparse.vbs
'------------------------------------------------------------------- ' Excelファイルの中で、文字列が入ったセルから文字列を抜き出すサンプル ' this is a sample extracting string from excel files. ' ' Usage: csript excelparse.vbs input.xls left-top-cell right-bottom-cell ' Example: ' If you want to print sample.xls's B5 to E7 celss, put the followig command. ' cscript excelparse.vbs sample.xls B5 E7 '------------------------------------------------------------------- set objXls = CreateObject("Excel.Application") objXls.Visible=False objXls.ScreenUpdating=False Dim filename Dim left_top_cell Dim right_bottom_cel filename = Wscript.Arguments(0) left_top_cell = Wscript.Arguments(1) right_bottom_cell = Wscript.Arguments(2) call ParseExcel(filename, left_top_cell, right_bottom_cell) '----------------------------------------------------- ' 標準出力(Print stdout) '----------------------------------------------------- Function Print(m) WScript.Stdout.Write m End Function '----------------------------------------------------- ' Get Full path '----------------------------------------------------- Function GetFullPath(ByRef filename) GetFullPath = CreateObject("Scripting.FileSystemObject").GetAbsolutePathName(filename) End Function '----------------------------------------------------- ' Excelファイルをパースして出力 ' print out excel file ' filename .............. excel filename パースするExcelファイル ' left_top_cell ......... left_top_cell string(example ... "B5") ' right_bottom_cell ..... right_bottom_cell string (such as "D10") '----------------------------------------------------- Sub ParseExcel(ByRef filename, ByRef left_top_cell, ByRef right_bottom_cell) Dim wb,s Dim lef_top_row Dim lef_top_col Dim right_bottom_row Dim right_bottom_col Set wb = objXls.Workbooks.Open(GetFullPath(filename)) Dim i For i=1 To wb.Sheets.Count Set s = wb.Sheets(i) Set lt = s.Range("A1:" & left_top_cell) Set rb = s.Range("A1:" & right_bottom_cell) left_top_row = lt.Rows.Count left_top_col = lt.Columns.Count right_bottom_row = rb.Rows.Count right_bottom_col = rb.Columns.Count Call PrintRange(s, left_top_row, right_bottom_row, left_top_col, right_bottom_col) Next wb.Close End Sub '----------------------------------------------------- ' 指定された範囲のセルの文字列を表示する。 ' print out specific range cells ' sheet .......... シートObject ' startRow ....... 行の開始位置(left_top_row) ' endRow ......... 行の終了位置(right_bottom_row) ' startCol ....... 列の開始位置(left_top_col) ' endCol ......... 列の終了位置(right_bottom_col) '----------------------------------------------------- Sub PrintRange(ByRef sheet, ByVal startRow, ByVal endRow, ByVal startCol, ByVal endCol) Dim y Dim x For y=startRow To endRow For x=startCol To endCol Set c = sheet.Cells(y,x) Print(c.Text & ",") Next Print(vbCrLf) Next End Sub
■How to use
if you want to print out from B5 cell to E7 cell like below image. cscript thissample.cbs input.xlsx B5 E7
How to UPSERT using Spark Dataframe
I am not sure if this article has any problem. If you found something wrong in this article. please let me know.
you have 2 csv data files and want to upsert as follows.
■Sampe Code No.1
df1 = spark.read.option("header","true").csv("data01.csv") df2 = spark.read.option("header","true").csv("data02.csv") df = df1.join(df2, ['Country','City'], 'left_anti') df = df.unionAll(df2) df.write.option("header","true").format("csv").save("output")
■Sample Code No.2
Another example(this code is sometimes work intensionally, But sometimes doesn’t work intensionally.
It depends on how many partitions the program execute)
df1 = spark.read.option("header","true").csv("data01.csv") df2 = spark.read.option("header","true").csv("data02.csv") df = df2.unionAll(df1) df = df.dropDuplicates(['Country','City']) df.write.option("header","true").format("csv").save("output")
■Sample Code No.3
this code will probably work intensionally(sorry, no guarantee).
df1 = spark.read.option("header","true").csv("data01.csv") df2 = spark.read.option("header","true").csv("data02.csv") df = df2.unionAll(df1) df = df.dropDuplicates(['Country','City']) df=df.coalesce(1) df.write.option("header","true").format("csv").save("output")
■Difference between No2. and No.3 code is the following place.
I think Spark is “Lazy Evaluation”. So, When you call the folowing code.
df = df2.unionAll(df1) df = df.dropDuplicates(['Country','City'])
Spark Dataframe doesn’t work actually yet.It works when the following code was executed.
df.write.option("header","true").format("csv").save("output")
So, I guess the Sample Code No.3 forced Dataframe work with 1 partition,and then, dropDuplicaets() work intentionally.
Upload files to S3 Bucket using curl.
I followed the information below.
tuxfight3r/s3_upload.sh
#!/bin/bash S3KEY="Your Acess Key" S3SECRET="Your Access Secret" S3BUCKET="Your Bucket Name" S3STORAGETYPE="STANDARD" AWSREGION="Your S3 Region" # If you use mac OS. enable following code. #OS="mac" function putS3 { file_path=$1 aws_path=$2 bucket="${S3BUCKET}" date=$(date -R -u) acl="x-amz-acl:private" content_type="text/plain" storage_type="x-amz-storage-class:${S3STORAGETYPE}" string="PUTnn$content_typen$daten$acln$storage_typen/$bucket$aws_path${file_path##/*/}" if [ $OS = "mac" ]; then signature=$(printf "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64) else signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64) fi curl -s --retry 3 --retry-delay 10 -X PUT -T "$file_path" -H "Host: $bucket.${AWSREGION}.amazonaws.com" -H "Date: $date" -H "Content-Type: $content_type" -H "$storage_type" -H "$acl" -H "Authorization: AWS ${S3KEY}:$signature" "https://$bucket.${AWSREGION}.amazonaws.com$aws_path${file_path##/*/}" } # Upload test.txt file to /test/ folder. putS3 test.txt /test/
Extend HttpWebRequest Timeout
The default Timeout of HttpWebRequest is 100 seconds.
So, It will timeout if the server response take more than 100 seconds.
If you want to extend the Timeout. You can set The Timeout as your preferrd time.
But If you set the Timeout, You shoud call the timeout before calling “GetRequestStream()”
using System;
using System.Web;
using System.Net;
using System.Text;
using System.IO;namespace ConsoleNetFramework
{
class Sample
{
static void Main(string[] args)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(“http://localhost/slow”);request.Method=”POST”;
byte[] postData = System.Text.Encoding.ASCII.GetBytes(“message=hello”);request.Timeout = 3*60*1000; // <– You need to set Timeout here!!
// before calling “GetRequestStream()”.
Stream writeStream = request.GetRequestStream();
request.Timeout = 3*60*1000; // <– If you set Timeout here after calling “GetRequestStream()”.
// it doesn’t work.writeStream.Write(postData, 0, postData.Length);
HttpWebResponse response;
// Get Response… and happen the TIMEOUT ERROR.
response = (HttpWebResponse)request.GetResponse();Stream receiveStream = response.GetResponseStream();
StreamReader sr = new System.IO.StreamReader(receiveStream, System.Text.Encoding.GetEncoding(“utf-8”));
Console.WriteLine(sr.ReadToEnd());
}
}
}
How to use PostgreSQL as a sink (Azure Data Factory)
This is sample of how to use PostgreSQL as a sink.
ADF can’t use PostgreSQL as a sink without using Self-hosted integration runtime.
So, at first, you need to set up a self-hosted runtime.
(See What is the Self Hosted Runtime(Azure Data Factory))
There is no PostgreSQL link service which can be used as a sink.
so you need to set “ODBC” link service.
You need to create the environment as below.
■Step1
Install PostgreSQL ODBC Driver as follows.
(1)Download PostgreSQL ODBC Driver
(1.1)access https://www.postgresql.org/
(1.2)”Download” -> “File Browser” -> “odbc” -> “versions” -> “msi”
(1.3)Download psqlodbc_13_02_0000-x64.zip (at first, I tried Version 9.2. but it didn’t work)
(2)Install ODBC Driver
■Step2
(1)Set sink’s dataset “ODBC”
(2)Set ODBC’s linked service as below.
DBD::mysql on MacBook Air(M1 macOS Monterey 12.0.1)
Actually, I did a lot of try and error. so I am not sure the following instruction is sufficient.
You can also get information on the link below.
・apache + perl + cgi + mysql (DBD::mysql, DBI) on MacBook M1 using Docker
(1) brew install openssl
(2) add following setting on ~/.zshrc
export LIBRARY_PATH=/opt/homebrew/opt/openssl/lib export C_INCLUDE_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/Extras/5.30/darwin-thread-multi-2level/auto/DBI/ export CPLUS_INCLUDE_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/Extras/5.30/darwin-thread-multi-2level/auto/DBI/
(3)source ~/.zshrc
(4)download and install DBI-1.643.tar.gz
(4.1)GET https://www.cpan.org/modules/by-module/DBI/DBI-1.643.tar.gz
(4.2)extract it on any directory you like.
(4.3)perl Makefile.PL
(4.4)make
(4.5)make install
(5) cpan install DBD::mysql
■Additional information
(1)Why you need to add some directory in ~/.zshrc
export LIBRARY_PATH=/opt/homebrew/opt/openssl/lib
-> If you don’t add the path above. you will encounter the following error, when “cpan install DBD::mysql”
“Can’t link/include C library ‘ssl’, ‘crypto’, aborting.”
export C_INCLUDE_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/Extras/5.30/darwin-thread-multi-2level/auto/DBI/ export CPLUS_INCLUDE_PATH=/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/System/Library/Perl/Extras/5.30/darwin-thread-multi-2level/auto/DBI/
-> If you don’t add the path above. you will encounter the following error.
‘DBIXS.h’ file not found
Why you need to install DBI-1.643.tar.gz
->I followed the information below.
https://stackoverflow.com/questions/68774807/dbdmysql-installation-on-catalina-big-sur-fail-with-extern-h-file-not-found
If you want to create the environment using Docker.
See the following link.
apache + perl + cgi + mysql (DBD::mysql, DBI) on MacBook M1 using Docker