Shell Script 12 : Regular Expression (b)

How to search a string located at the beginning or the end of a line?
jtony@genoa:~/learn/shell$ cat test
specialize
aaaaaaaaaa
tomcat
tom
atom
tom333
jyjtom

specialise

(a) Using a caret (outside of brackets) allows you to designate the “beginning” of a line.
jtony@genoa:~/learn/shell$ grep '^tom' test
tomcat
tom
tom333

(b) To search for the end of a line, use the dollar sign.
jtony@genoa:~/learn/shell$ grep 'tom$' test
tom
atom

jyjtom

(c) To search for both the beginning and the end of a line, i.e., searach lines only contains 'tom'
jtony@genoa:~/learn/shell$ grep '^tom$' test
tom

Shell Script 11 : Regular Expression (a)

How to grep both specialize and specialise in one file?

jtony@genoa:~/learn/shell$ cat test
specialize
aaaaaaaaaa
specialise
jtony@genoa:~/learn/shell$ egrep "speciali(s|z)e" test
specialize
specialise






Note: grep won't work.

 

A PowerPC instruction a day series: 0

Plan:
Here are some PowerPC instructions that are useful to beginners:

li REG, VALUE

    loads register REG with the number VALUE
add REGA, REGB, REGC

    adds REGB with REGC and stores the result in REGA
addi REGA, REGB, VALUE

    add the number VALUE to REGB and stores the result in REGA
mr REGA, REGB

    copies the value in REGB into REGA
or REGA, REGB, REGC

    performs a logical "or" between REGB and REGC, and stores the result in REGA
ori REGA, REGB, VALUE

    performs a logical "or" between REGB and VALUE, and stores the result in REGA
and, andi, xor, xori, nand, nand, and nor

    all of these follow the same pattern as "or" and "ori" for the other logical operations
ld REGA, 0(REGB)

    use the contents of REGB as the memory address of the value to load into REGA
lbz, lhz, and lwz

    all of these follow the same format, but operate on bytes, halfwords, and words, respectively (the "z" indicates that they also zero-out the rest of the register)
b ADDRESS

    jump (or branch) to the instruction at address ADDRESS
bl ADDRESS

    subroutine call to address ADDRESS
cmpd REGA, REGB

    compare the contents of REGA and REGB, and set the bits of the status register appropriately
beq ADDRESS

    branch to ADDRESS if the previously compared register contents were equal
bne, blt, bgt, ble, and bge

    all of these follow the same form, but check for inequality, less than, greater than, less than or equal to, and greater than or equal to, respectively.
std REGA, 0(REGB)

    use the contents of REGB as the memory address to save the value of REGA into
stb, sth, and stw

    all of these follow the same format, but operate on bytes, halfwords, and words, respectively
sc

    makes a system call to the kernel

How to calculate an instruction encoding?

How to calculate an instruction encoding? like calculate the encoding for fctidu 2, 3  (Here we are using Power PC hardware instruction as an example)

(a)First find the Instruction description in the ISA document.  In this example, On page 141 of Power ISA V2.06
(b) Second,  replace the operand field according to the encoding format. e.g., fctidu 2, 3 belong to
    the first format (Record Form, It “records” the result (i.e. saturation in this case I guess) in CR1), so Rc is 0,
        FRT is the first operand, its value is 2, and FRB is the second value, its value is 3, therefore, the encoding value
        will be:
              bit:        0-5       6-10      11-15    16-20     21-30    31
        original: |     63    |   FRT   |      ///     |   FRB   |   942   |   Rc   |
        Decimal replace:   |63 | 2 |  0 | 3 | 942 | 0 |    // note: here "///" in the ISA meaning all 0 (probably because it's padding)
        in binary:
        | 11 1111 | 00010 | 00000 | 00011 |  11 1010 1110 | 0 |
        re-organized by 8 bits (by byte)
        | 1111 1100 | 0100 0000 | 0001 1111 | 0101 1100 |
        in Hexadecimal:
        | FC | 40 | 1F | 5C |
       
(c) Therefore, the eventually encoding is 0XFC 0X40 0X1F 0X5C

file reference:
test/MC/PowerPC/ppc64-encoding-fp.s  -- tests assembly
test/MC/Disassembler/PowerPC/ppc64-encoding-fp.txt  -- tests disassembly
       

LLVM assembly example on PPC (.s file)

jtony@genoa:~/learn/shell$ cat jyj.c
int glb1 = 1;
int glb2 = 2;
int glb3 = 3;
int foo(int a, int b, int c, int test) {
  int tmp1 = test ? a : glb1;
  int tmp2 = test ? b : glb2;
  int tmp3 = test ? c : glb3;

  return (tmp1 + tmp2 + tmp3);
}

# BB#0:                                 # %entry
        addis r7, r2, glb1@toc@ha  ---> (what does "addis r7, r2, .glb1@toc@ha" here mean? That just                                                                puts the high 16-bits of the TOC offset for glb1 into r7)
        addis r8, r2, glb2@toc@ha
        addis r9, r2, glb3@toc@ha
        cmplwi cr0, r6, 0
        lwz r7, glb1@toc@l(r7)
        lwz r8, glb2@toc@l(r8)
        lwz r12, glb3@toc@l(r9)
        bc 12, 2, .LBB0_1
        b .LBB0_2
.LBB0_1:                                # %entry
        addi r3, r7, 0
        addi r4, r8, 0   (eqivalent to ld r4, 0(r8)) r8 has an address, this instr loads 8 bytes from that                                          address into r4,  i.e,  r4 <--- *(&glb2 + 0))
        addi r5, r12, 0
.LBB0_2:                                # %entry
        add r3, r4, r3
        add r3, r3, r5
        extsw r3, r3
        blr
        .long   0
        .quad   0
.Lfunc_end0:
        .size   foo, .Lfunc_end0-.Lfunc_begin0

---------------------------------------------------------------------------------------------------

In the above example assembly, the addis and addi are used several times.  These two instructions are often used together to load a 32-bit address to a register. Here are their syntax and usages. Here r2 is the TOC pointer (TOC == Table Of Contents, r2 has the address of the TOC base)

[10:47]  

it is a table that stores the locations of globals and constants

2. ADDI - Add Immediate
Syntax: addi rD,rA,SIMM
This command adds a 16-bit signed integer (SIMM) to register rA and puts the result into the register rD (destination).
Example: addi 3,6,4
In this example GPR6 and the value 4 are added and the result is put into GPR3.
As you can see the registers are specified as a number and the integer is specified as a number. To improve the readability of the code you can define constants for the registers, e.g.:
.set r0,0; .set r1,1; .set r2,2; .set r3,3; .set r4,4; .set r5,5; .set r6,6; .....
Then the above command can be written as: addi r3,r6,4
You can also use the addi command as a move instruction:
addi 3,0,4
This sets GPR3 to the value 4. If the second parameter is a zero here this does not mean a GPR0 but the value zero. 

3. ADDIS - Add Immediate Shifted
Syntax: addis rD,rA,SIMM
This command is used to add a 16-bit immediate value to the upper 16 bits of a 32bit register. It adds a 16-bit signed integer (SIMM) to register rA, then shifts left register rA by 16 bits and then puts the result into the register rD (destination). The lower 16 bits are cleared by this command. So to fill a 32 bit register with an immediate 32-bit value you first have to use addis to fill the upper 16 bits and then addi to fill the lower 16 bits. Otherwise the lower 16 bits would be cleared again by the addis command.
Example: addis 3,3,4
In this example GPR3 and the value 4 are added, then GPR3 is shifted 16 bits to the left and the result is put into GPR3. This will then contain 0x00040000 in hexadecimal. If you then execute an addi 3,3,4 command GPR3 will contain 0x00040004 in hexadecimal.
To move a pointer to an address of a variable or function into a register there are the @ha/@h and @l modifiers available. If you append these to the variable name you get the lower (@l) 16 bit of the absolute 32-bit address of the variable and with @ha you get the higher 16 bit of the absolute 32-bit address.
Example:

addis 3,0,hello@ha

addi 3,0,hello@l

With these two instructions the absolute 32-bit address of the string variable hello is moved into the GPR 3 register. 

How to change your shell?

You can easily install and remove different shells using the Software Center on Ubuntu or your preferred package manager. Shells are located in the /bin/ directory, and as long as you’re running a modern Linux distribution, it’s easy to change what your default is. Just enter the following command:

chsh

You’ll be prompted for your password, then you can change to a different shell by entering its path.

-bash-4.2$ chsh
Changing shell for jtony.
New shell [/bin/bash]: /bin/ksh
Password:
Shell changed.
-bash-4.2$
 

In square brackets, you’ll see your current default, and if you want to leave it as-is, just hit Enter.

Shell Script 10: Conditions & If-Then-else Statements (c)

if CONDITION
then
command1
command2
…
commandn
else
command1
command2
…
commandn
fi

jtony@genoa:~/learn/shell$ cat ifthenelse.sh
#!/bin/bash

if [ $1 = $2 ]
  then
  echo "$1" is same than "$2"
else
  echo "$1" is NOT same with "$2"
fi

echo "This comes after if-then unconditionally."

Shell Script 10: Conditions & If-Then Statements （ b)

jtony@genoa:~/learn/shell$ cat testString.sh
#!/bin/bash

#if test $1 = $2
if test $1 != $2
  then
  #echo "$1" is same with "$2"
  echo "$1" is NOT same with "$2"
fi

echo "This comes after if-then unconditionally."

Shell Script 9: Conditions & If-Then Statements （a)

jtony@genoa:~/learn/shell$ cat ifthenelse.sh
#!/bin/bash

if test $1 -gt $2
  then
  echo "$1" is greater than "$2"
fi

echo "This comes after if-then unconditionally."
 

Here are some other numerical operators you may want to try out:

• -eq: equal to
• -ne: not equal to
• -lt: less than
• -le: less than or equal to
• -gt: greater than
• -ge: greater than or equal to

Shell Script 8: redirect symbol > and >> < and <<

'>' overwrites the file after it while '>>' append content to the file after it.

jtony@genoa:~/learn/shell$ cat test.txt
Red Pepper

jtony@genoa:~/learn/shell$ echo "Green Peper" > test.txt
jtony@genoa:~/learn/shell$ cat test.txt
Green Peper

jtony@genoa:~/learn/shell$ echo "Yellow Peper" >> test.txt
jtony@genoa:~/learn/shell$ cat test.txt
Green Peper
Yellow Peper

You can also take inputs from files for commands by using the less-than symbol (<).
jtony@genoa:~/learn/shell$ cat < test
Green Peper
Yellow Peper

Shell Script 7： for-loop

The basic outline of a for-loop is as follows:

for VARIABLE in LIST; do
command1
command2
…
commandn
done

VARIABLE can be any variable, though most often the lowercase “i” is used by convention. LIST is a list of items; you can specify multiple items (separating them by a space), point to an external text file, or use an asterisk (*) to denote any file in the current directory. The commands listed are indented by convention, so it’s easier to see nesting – putting loops in loops (so you can loop while you loop).

jtony@genoa:~/learn/shell$ cat forloop.sh
#!/bin/bash

# A simply display for loop
#for i in item1 item2 item3 item4 item5 item6; do
for i in *; do
  echo "$i"
done
jtony@genoa:~/learn/shell$ cat zip.sh
#!/bin/bash

# Zip all the arguments with for loop
for i in $@; do
  zip archive "$i"
done


example 3:
remove all the killed key words in all the expand-isel-*.mir files:
for f in expand-isel-*.mir; do sed -e 's/killed//g' $f  > $f.cpp; done

Note all these series blogs are originated from, I did some modification to it, many thanks to the author.

http://www.howtogeek.com/68184/the-beginners-guide-to-shell-scripting-2-for-loops/








 

Machine IR (MIR)

quoted from: Testing Individual Code Generation Passes

The run-pass option in llc allows you to create MIR tests that invoke just a single code generation pass. When this option is used, llc will parse an input MIR file, run the specified code generation pass, and print the resulting MIR to the standard output stream.

You can generate an input MIR file for the test by using the stop-after option in llc. For example, if you would like to write a test for the post register allocation pseudo instruction expansion pass, you can specify the machine copy propagation pass in the stop-after option, as it runs just before the pass that we are trying to test:

llc -stop-after machine-cp bug-trigger.ll > test.mir

After generating the input MIR file, you’ll have to add a run line that uses the -run-pass option to it. In order to test the post register allocation pseudo instruction expansion pass on X86-64, a run line like the one shown below can be used:

# RUN: llc -run-pass postrapseudos -march=x86-64 %s -o /dev/null | FileCheck %s

The MIR files are target dependent, so they have to be placed in the target specific test directories. They also need to specify a target triple or a target architecture either in the run line or in the embedded LLVM IR module

return function call inside a void function

C++ have some special feature that you may never used or heard of. Here is one example, you could return a void function call inside a void function.

jtony@genoa:~/learn$ xlC void.cpp
jtony@genoa:~/learn$ ./a.out
I am in FOO!
I am in BAR !
jtony@genoa:~/learn$ cat void.cpp
#include<iostream>
using namespace std;

void bar() {
  cout << "I am in BAR !" << endl;
}

void foo() {
  cout << "I am in FOO!" << endl;
  return bar();
}

int main(void) {
 foo();
 return 0;
}

According to link: http://stackoverflow.com/questions/2249108/can-i-return-in-void-function

Interestingly, you can also return void from a void function. For example:

void foo()
{
  return void();
}

How to toggle between highlight search and non highlight search?

To highlight all search matches, set the following option:
:set hlsearch
With the defaults, setting this option causes all text matching the current search to be highlighted using the  Search highlight group, which adds a yellow background to the current highlighting. See :help hl-Search, or type :hi Search to see what color you have it set to. You can easily change the default highlighting with, for example, :hi Search guibg=LightBlue.

To disable the highlighting temporarily, enter (this is a command, not an option):
:nohlsearch
This command (which can be abbreviated to :noh) removes the highlighting for the current search. The highlighting returns for the next search.

Write them into vimrc, if you don't want to set the option each time when you open vim.


More about highlight search, see link:
http://vim.wikia.com/wiki/Highlight_all_search_pattern_matches

How to recursively compress and tranfer files using rsync ?

rsync -avz MyBackups user@server:/path/to/backup/

This is usually better/faster than the scp -r command.

How to show the size of a given directory?

jtony@genoa:~$ du -hs test/
480M    test/

How to show all the process of a particular user?

Use -u option of ps:
ps -u jtony

How to change colors for vim syntax highlighting when use clang-format style

Note I copied the ./utils/vim/vimrc in llvm source to the default .vimrc in my home directory. If you just want to change the ./utils/vim/vimrc  file, you can do it there.

(1) Open  ~/.vimrc file    (or Open  ./utils/vim/vimrc)
(2) Find and change the following line:
From:
  highlight LongLine ctermbg=DarkYellow  guibg=DarkYellow
To:
  highlight LongLine ctermbg=LightBlue  guibg=DarkYellow

How to calculate the summation of all the numbers in first column of a file

Calculate the summation of all the number in the first (1) column in file test.stats and print out the result.
cat test.stats |  awk '{s+=$1} END {printf "%.0f\n", s}'

clang build VERBOSE and debug llvm

44. How to show clang build verbose message? Like the detailed build command and options?
add VERBOSE=1, like:
make VERBOSE=1

45. How to debug llvm without gdb?
add option:
 -mllvm -debug -mllvm -print-after-all -mllvm -print-before-all
This is give you enough information to start with.

Adding a block to a function

A function consists of basic blocks. A basic block has an entry point. A basic block consists of a number of IR instructions, the last instruction being a terminator instruction. It has single exit point. LLVM provides the BasicBlock class to create and handle basic blocks. A basic block might have an entry point as its label, which indicates where to insert the next instructions. We can use the IRBuilder object to hold these new basic block IR.

Check Compilation Pass Info

41. How to check the compilation pass info for clang for PPC?
Check file PPCTargetMachine.cpp and
./include/llvm/CodeGen/Passes.h
Top-level implementation for the PowerPC target.

Shell Script 6

Final version cpdate
#!/bin/bash

# This is copy a file and appending the date and  time to the end of the file.
# Not the following format is wrong, because you can't add the space between the
# varible name and = symbol.
#date_formatted =$(date +%m_%d_%y-%H.%M.%S)
echo "This is the date and Time: " $date_formatted

date_formatted=$(date +%Y-%m-%d_%H.%M%S)
file_extension=$(echo "$1" | awk -F . '{print $NF}')
file_name=$(basename $1 .$file_extension)
cp -iv $1 $file_name-$date_formatted.$file_extension
~

Shell Basics 5

If you want to hand file names that contain space, you need to double quote the paramters.

#!/bin/bash

# This is copy a file and appending the date and  time to the end of the file.
# Not the following format is wrong, because you can't add the space between the
# varible name and = symbol.
#date_formatted =$(date +%m_%d_%y-%H.%M.%S)
date_formatted=$(date +%m_%d_%y-%H.%M.%S)
echo "This is the date and Time: " $date_formatted
#cp -iv $1 $2.$date_formatted      # This command does not handle the file name that
                                                      # contains space
cp -iv "$1" "$2".$date_formatted # This can handle file name contains space

Shell Script Basics 4: compare two strings

This post explains the if-then-else usage of shell script with an example. Please note that the square bracket is equivalent with the 'test' command.

~/bin$ cat testingifthenelse
#!/bin/bash
#if test $1 = $2
if [ $1 = $2 ] # this format is equivalent with the above 'test' format
  then
  echo "The arguments are equal."
else
  echo "The arguments are NOT equal."
fi

I don't know whether the readers have noticed that  we use '=' to compare the two the two string $1 and $2, this is a different syntax with C/C++, which use '==' to compare strings. DO NOT confuse between them.

Shell Script Basics 3

jtony@genoa:~/bin$ cat datecp
#!/bin/bash

# This is copy a file and appending the date and  time to the end of the file.
# Not the following format is wrong, because you can't add the space between the
# varible name and = symbol.
#date_formatted =$(date +%m_%d_%y-%H.%M.%S)
date_formatted=$(date +%m_%d_%y-%H.%M.%S)
echo "This is the date and Time: " $date_formatted
cp -iv $1 $2.$date_formatted

Shell Script Basics 2

jtony@genoa:~/bin$ cat hello.sh
#!/bin/bash

# This is copy a file and appending the date and  time to the end of the file.
# Not the following format is wrong, because you can't add the space between the
# varible name and = symbol.
#date_formatted =$(date +%m_%d_%y-%H.%M.%S)
date_formatted=$(date +%m_%d_%y-%H.%M.%S)
echo "This is the date and Time: " $date_formatted

How to use C++11 range based for loop correctly

Without deep understanding of C++11 range based for loop, it is easy to make stupid mistakes which is hard to understand reason by reading long and complex compiler error message. Here I list what is the right and wrong format to use C++11 range based for loop.

#include <iostream>
#include <map>
#include <string>
using namespace std;

int main()

{
  map<int, string> mapStudent;

  mapStudent.insert(pair<int, string>(1, "student_one"));

  mapStudent.insert(pair<int, string>(2, "student_two"));

  mapStudent.insert(pair<int, string>(3, "student_three"));

  map<int, string>::iterator iter;

  // Correct iterator usage before C++11
  for(iter = mapStudent.begin(); iter != mapStudent.end(); iter++)
  {
    cout << iter->first << " " << iter->second << endl;
  }
  
  // Wrong range based for loop usage, mix up iterator and range based for loop element
  //for (iter : mapStudent) {
  //  cout << iter->first << " " << iter->second << endl;
  //}






  
// Correct range based for loop usage, auto deduct element type
  for (auto &elem : mapStudent) {
    cout << elem.first << " " << elem.second << endl;
  }
// Correct range based for loop usage, explicitly tell the compiler the element type
  for (pair<int, string> elem : mapStudent) {
    cout << elem.first << " " << elem.second << endl;
  }
}



references:
http://www.kuqin.com/cpluspluslib/20071231/3265.html
http://en.cppreference.com/w/cpp/language/range-for

The Beginner’s Guide to Shell Scripting: The Basics 1

there are a few guidelines you need to know.

1. Every script should being with “#!/bin/bash”
2. Every new line is a new command
3. Comment lines start with a #
4. Commands are surrounded by ()