How to count the exectution time inside of a function?

Use __builtin_readcyclecounter
like :

int foo(void) {
  int start = __builtin_readcyclecounter();
  CODE to be TESTED
 int end = __builtin_readcyclecounter();
 int diff = end - start
 printf(diff);
}

Assembler and Dissassembler

LLVM provides two different representation after generating the hardware instructions. The .s file (assembly representation) and .o file ( the object file). They are just two different "IR". If we want to generate the .o file from .s file, we need to call the assembler (usually /usr/bin/as). If we want to do the opposite, we can use objdump -d (meaning disassemble)

The assembler and disassembler together can be used to modify the assembly and do some testing.

e.g. :

clang a.c -S -O2 -o a.s

do some modifications to the a.s

clang a.s -o a.out (here clang implicitly invoke the assembler to generate exe)

You can also do (equivalent to the above):

assemble:

/usr/bin/as  a.s -o a.o

clang a.o -o a.out 

To look at the assembly of .o or exe

disassemble:

objdump modified.o-d > modified.out.disassembly

objdump modified.out -d > modified.out.disassembly

llvm PPC Register classes [Implementation based]

98 Register classes?
class name    spill

VSSRC          32
VSFRC          64
VSRC           128
VRRC           128
F4RC           32
F8RC           64
VSHRC          128
we have 64 VSR’s (128-bit vector registers). The first 32 of them overlap with the FPR’s (floating point registers that occupy the most significant 64 bits of each).
The last 32 of them overlap with VR’s (128-bit Altivec/VMX registers).

- VSSRC(Vector-Scalar, Single-precision) is a scalar register class consisting of 64 registers that can each hold an f32 (edited)

- VSFRC (Vector Scalar Floating-points) class consisting of 64 registers that can each hold an f64

- VSRC is a register class consisting of 64 registers that can each hold a vector. This is currently restricted to `v4i32, v4f32, v2i64, v2f64` since those are the only types that we have meaningful operations on in the ISA.


And VSSRC and VSFRC are the upper 64-bits of VSR0-VSR63. I.e. they model higher 64 bit of VSR (all 64 of them)

- VRRC is a register class consisting of 32 registers that each hold an Altivec (VMX) vector type which is VSR32-VSR63

- F4RC is just the 32 floating point registers used for f32 values

So F4RC models the upper 64 bit of VSR0-VSR31? and for 32 floating point

- F8RC are the 32 floating point registers that each hold an f64

Then we have a bit of complexity when it comes to GPRC/G8RC. On the one hand, it’s pretty simple - 32/64 bit integer registers respectively.
R0-R31 are 32-bit ones. X0-X31 are the 64-bit ones. However, we have a number of instructions that treat R0/X0 as special - a zero means immediate zero, not contents of register zero.
For such instructions we use the `GPRC_NOR0` and `G8RC_NOX0`. They have a special register called `ZERO` and `ZERO8` respectively. And what we do is mark this register as reserved. Then we scan the instructions that use this register class to see if they’re fed by a register that contains a zero, we get rid of the instruction that defines that register and we allocate `ZERO/ZERO8`
because they instructions need constant 0 instead of R0

So we’ll catch stuff like this:
```li 4, 0
lfsx 2, 4, 3
```
we can freely get rid of the first instruction and put a `ZERO` instead of 4 in the `lfsx`


If you use the wrong register class, you could get werid SDAG selcetion error. (Can not map REGA to REGB, or something)
FPR(i) = upper VSR(i)
isCodeGenOnly means not suitable for disassembly and assembly etc.


Error for using wrong register class:
: error: In XXBRH: Type inference contradiction found, merging '{v4i32:v2i
64:v4f32:v2f64}' into 'v8i16'
  def XXBRH : XX2_XT6_XO5_XB6<60, 7, 475, "xxbrh", vsrc,
   
    The COPY_TO_REGCLASS must be used, if you want to copy a value  to a class that doesn't support that type.
    e.g. $A is v1i128, which is not supported by VSRC, thus we used COPY_TO_REGCLASS, we also copied the result of XXBRQ
    to VRRC  because after exectuting XXBRQ the result is still v1i128 even though you COPY_TO_REGCLASS $A to it,
    and VRRC the only register class that support v1i128.
    (v1i128 (COPY_TO_REGCLASS (XXBRQ (COPY_TO_REGCLASS $A, VSRC)), VRRC))>;

How to disable llvm opt optimization only? Leave the clang FE and llc optimization there

`-X -disable-llvm-passes`
This is the option to disable just llvm opt

91. llvm architecture (partially, to be extend)

IR--> SDAG Construction --> Type Legalization --> Operation Legalization --> Lowering --> ISel (Instruction Selection) --> MIR (Machine IR)

How to dump scheduling graph?

78. How to dump scheduling graph?
/home/jtony/git-llvm/build/team-llvm/bin/llc /home/jtony/scrum/s11/memcpy/jtony/ppc.ll -view-misched-dags 2>&1 | /home/sfertile/bin/capture_dot
test/CodeGen/PowerPC/reduced.ll

79. How to show hidden llc options?
llc -help-hidden
like:
 -view-misched-dags                                       - Pop up a window to show MISched dags after they are processed
 -view-sched-dags                                         - Pop up a window to show sched dags as they are processed
 -view-sunit-dags                                         - Pop up a window to show SUnit dags after they are processed

What is llvm-dis?

The llvm-dis command is the LLVM disassembler. It takes an LLVM bitcode file and converts it into human-readable LLVM assembly language.

If the input is being read from standard input, then llvm-dis will send its output to standard output by default. Otherwise, the output will be written to a file named after the input file, with a .ll suffix added (any existing .bc suffix will first be removed). You can override the choice of output file using the -o option.

How to use bugpoint reduce a IR test case?

(1) Find the temp files created by clang in the crash report. Usually, it is at the end of the crash report, which
points to a file in the /tmp/ directory. for example:
clang-5.0: note: diagnostic msg: /tmp/tsan_mutexset-bb0974.cpp
clang-5.0: note: diagnostic msg: /tmp/tsan_mutexset-bb0974.sh

(2) add '-S -emit-llvm -o tony.ll' to the script (tsan_mutexset-bb0974.sh in this example) to produce an .ll file.

(3) Then you write a simple script like this:
```#!/bin/bash
<broken-llc> $1 2>&1 | grep "<text of the assertion>" > /dev/null
if [[ $? -eq 0 ]]
then
  exit 1
else
  exit 0
fi

examples:
    $ cat reduceScript.sh
    ```#!/bin/bash
    /home/jtony/llvm/build/memcpyBootstrapSeanPatch/bin/llc $1 2>&1 | grep "Node emitted out of order - late" > /dev/null
    if [[ $? -eq 0 ]]
    then
       exit 1
    else
       exit 0
    fi
       
        jyj14ibm


(4)run the following to reduce the given *.ll file once you have both the IR file (tony.ll) and the script (reduceScript.sh):
`$ bugpoint -compile-custom -compile-command=reduceScript.sh tony.ll`

(5) Use llvm-dis to generate the eventual reduced IR (*.ll) file from the bitcode file (*.bc).
For example:
 llvm-dis bugpoint-reduced-simplified.bc

Feel free to update this initial bugpoint document.

How to NOT inline a function?

Put __attribute__ ((noinline)) for the function (this semantics is supported by both gcc and clang).

   char __attribute__ ((noinline)) ges(long long a, long long b) {                                                         
       return (a >= b);
    }  

In the following example, if you don't put  __attribute__ ((noinline)) for the function, the compiler would be very clever, it would generate the result for you during compile time, so you never got chance to run your llvm BE code.


$ cat functional.cpp
#include <iostream>
#include <cassert>
using namespace std;

 long long __attribute__ ((noinline)) ges(long long a, long long b) {
    return (a >= b);
}

int main(void) {
  cout << "ges(1, 2) = " << ges(1, 2) << endl;
  assert(ges(1,2) == 0);
  cout << "ges(2, 2) = " << ges(2, 2) << endl;
  assert(ges(2,2) == 1);
  cout << "ges(3, 2) = " << ges(3, 2) << endl;
  assert(ges(3,2) == 1);
}

llvm SDValue

00091 /// Unlike LLVM values, Selection DAG nodes may return multiple
00092 /// values as the result of a computation.  Many nodes return multiple values,
00093 /// from loads (which define a token and a return value) to ADDC (which returns
00094 /// a result and a carry value), to calls (which may return an arbitrary number
00095 /// of values).
00096 ///
00097 /// As such, each use of a SelectionDAG computation must indicate the node that
00098 /// computes it as well as which return value to use from that node.  This pair
00099 /// of information is represented with the SDValue value type.
00100 ///
00101 class SDValue {
00102   friend struct DenseMapInfo<SDValue>;
00103 
00104   SDNode *Node;       // The node defining the value we are using.
00105   unsigned ResNo;     // Which return value of the node we are using.
00106 public:
00107   SDValue() : Node(nullptr), ResNo(0) {}
00108   SDValue(SDNode *node, unsigned resno);
00109 
00110   /// get the index which selects a specific result in the SDNode
00111   unsigned getResNo() const { return ResNo; }

Note, by default the result value is in the the first one,  cause we have  the default

set "ResNo(0)", knowing this is  very important for handling hardware instruction that

has multiple outputs which cannot be handled in .td file (like ADDC, SUBFC ADDIC etc) 

Code Generation Options

-O0, -O1, -O2, -O3, -Ofast, -Os, -Oz, -Og, -O, -O4

Specify which optimization level to use:

-O0 Means “no optimization”: this level compiles the fastest and generates the most debuggable code.
-O1 Somewhere between -O0 and -O2.
-O2 Moderate level of optimization which enables most optimizations.
-O3 Like -O2, except that it enables optimizations that take longer to perform or that may generate larger code (in an attempt to make the program run faster).
-Ofast Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards.
-Os Like -O2 with extra optimizations to reduce code size.
-Oz Like -Os (and thus -O2), but reduces code size further.
-Og Like -O1. In future versions, this option might disable different optimizations in order to improve debuggability.
-O Equivalent to -O2.
-O4 and higher

Currently equivalent to -O3

How to calculate an instruction encoding?

How to calculate an instruction encoding? like calculate the encoding for fctidu 2, 3  (Here we are using Power PC hardware instruction as an example)

(a)First find the Instruction description in the ISA document.  In this example, On page 141 of Power ISA V2.06
(b) Second,  replace the operand field according to the encoding format. e.g., fctidu 2, 3 belong to
    the first format (Record Form, It “records” the result (i.e. saturation in this case I guess) in CR1), so Rc is 0,
        FRT is the first operand, its value is 2, and FRB is the second value, its value is 3, therefore, the encoding value
        will be:
              bit:        0-5       6-10      11-15    16-20     21-30    31
        original: |     63    |   FRT   |      ///     |   FRB   |   942   |   Rc   |
        Decimal replace:   |63 | 2 |  0 | 3 | 942 | 0 |    // note: here "///" in the ISA meaning all 0 (probably because it's padding)
        in binary:
        | 11 1111 | 00010 | 00000 | 00011 |  11 1010 1110 | 0 |
        re-organized by 8 bits (by byte)
        | 1111 1100 | 0100 0000 | 0001 1111 | 0101 1100 |
        in Hexadecimal:
        | FC | 40 | 1F | 5C |
       
(c) Therefore, the eventually encoding is 0XFC 0X40 0X1F 0X5C

file reference:
test/MC/PowerPC/ppc64-encoding-fp.s  -- tests assembly
test/MC/Disassembler/PowerPC/ppc64-encoding-fp.txt  -- tests disassembly
       

LLVM assembly example on PPC (.s file)

jtony@genoa:~/learn/shell$ cat jyj.c
int glb1 = 1;
int glb2 = 2;
int glb3 = 3;
int foo(int a, int b, int c, int test) {
  int tmp1 = test ? a : glb1;
  int tmp2 = test ? b : glb2;
  int tmp3 = test ? c : glb3;

  return (tmp1 + tmp2 + tmp3);
}

# BB#0:                                 # %entry
        addis r7, r2, glb1@toc@ha  ---> (what does "addis r7, r2, .glb1@toc@ha" here mean? That just                                                                puts the high 16-bits of the TOC offset for glb1 into r7)
        addis r8, r2, glb2@toc@ha
        addis r9, r2, glb3@toc@ha
        cmplwi cr0, r6, 0
        lwz r7, glb1@toc@l(r7)
        lwz r8, glb2@toc@l(r8)
        lwz r12, glb3@toc@l(r9)
        bc 12, 2, .LBB0_1
        b .LBB0_2
.LBB0_1:                                # %entry
        addi r3, r7, 0
        addi r4, r8, 0   (eqivalent to ld r4, 0(r8)) r8 has an address, this instr loads 8 bytes from that                                          address into r4,  i.e,  r4 <--- *(&glb2 + 0))
        addi r5, r12, 0
.LBB0_2:                                # %entry
        add r3, r4, r3
        add r3, r3, r5
        extsw r3, r3
        blr
        .long   0
        .quad   0
.Lfunc_end0:
        .size   foo, .Lfunc_end0-.Lfunc_begin0

---------------------------------------------------------------------------------------------------

In the above example assembly, the addis and addi are used several times.  These two instructions are often used together to load a 32-bit address to a register. Here are their syntax and usages. Here r2 is the TOC pointer (TOC == Table Of Contents, r2 has the address of the TOC base)

[10:47]  

it is a table that stores the locations of globals and constants

2. ADDI - Add Immediate
Syntax: addi rD,rA,SIMM
This command adds a 16-bit signed integer (SIMM) to register rA and puts the result into the register rD (destination).
Example: addi 3,6,4
In this example GPR6 and the value 4 are added and the result is put into GPR3.
As you can see the registers are specified as a number and the integer is specified as a number. To improve the readability of the code you can define constants for the registers, e.g.:
.set r0,0; .set r1,1; .set r2,2; .set r3,3; .set r4,4; .set r5,5; .set r6,6; .....
Then the above command can be written as: addi r3,r6,4
You can also use the addi command as a move instruction:
addi 3,0,4
This sets GPR3 to the value 4. If the second parameter is a zero here this does not mean a GPR0 but the value zero. 

3. ADDIS - Add Immediate Shifted
Syntax: addis rD,rA,SIMM
This command is used to add a 16-bit immediate value to the upper 16 bits of a 32bit register. It adds a 16-bit signed integer (SIMM) to register rA, then shifts left register rA by 16 bits and then puts the result into the register rD (destination). The lower 16 bits are cleared by this command. So to fill a 32 bit register with an immediate 32-bit value you first have to use addis to fill the upper 16 bits and then addi to fill the lower 16 bits. Otherwise the lower 16 bits would be cleared again by the addis command.
Example: addis 3,3,4
In this example GPR3 and the value 4 are added, then GPR3 is shifted 16 bits to the left and the result is put into GPR3. This will then contain 0x00040000 in hexadecimal. If you then execute an addi 3,3,4 command GPR3 will contain 0x00040004 in hexadecimal.
To move a pointer to an address of a variable or function into a register there are the @ha/@h and @l modifiers available. If you append these to the variable name you get the lower (@l) 16 bit of the absolute 32-bit address of the variable and with @ha you get the higher 16 bit of the absolute 32-bit address.
Example:

addis 3,0,hello@ha

addi 3,0,hello@l

With these two instructions the absolute 32-bit address of the string variable hello is moved into the GPR 3 register. 

Machine IR (MIR)

quoted from: Testing Individual Code Generation Passes

The run-pass option in llc allows you to create MIR tests that invoke just a single code generation pass. When this option is used, llc will parse an input MIR file, run the specified code generation pass, and print the resulting MIR to the standard output stream.

You can generate an input MIR file for the test by using the stop-after option in llc. For example, if you would like to write a test for the post register allocation pseudo instruction expansion pass, you can specify the machine copy propagation pass in the stop-after option, as it runs just before the pass that we are trying to test:

llc -stop-after machine-cp bug-trigger.ll > test.mir

After generating the input MIR file, you’ll have to add a run line that uses the -run-pass option to it. In order to test the post register allocation pseudo instruction expansion pass on X86-64, a run line like the one shown below can be used:

# RUN: llc -run-pass postrapseudos -march=x86-64 %s -o /dev/null | FileCheck %s

The MIR files are target dependent, so they have to be placed in the target specific test directories. They also need to specify a target triple or a target architecture either in the run line or in the embedded LLVM IR module

How to change colors for vim syntax highlighting when use clang-format style

Note I copied the ./utils/vim/vimrc in llvm source to the default .vimrc in my home directory. If you just want to change the ./utils/vim/vimrc  file, you can do it there.

(1) Open  ~/.vimrc file    (or Open  ./utils/vim/vimrc)
(2) Find and change the following line:
From:
  highlight LongLine ctermbg=DarkYellow  guibg=DarkYellow
To:
  highlight LongLine ctermbg=LightBlue  guibg=DarkYellow

clang build VERBOSE and debug llvm

44. How to show clang build verbose message? Like the detailed build command and options?
add VERBOSE=1, like:
make VERBOSE=1

45. How to debug llvm without gdb?
add option:
 -mllvm -debug -mllvm -print-after-all -mllvm -print-before-all
This is give you enough information to start with.

Adding a block to a function

A function consists of basic blocks. A basic block has an entry point. A basic block consists of a number of IR instructions, the last instruction being a terminator instruction. It has single exit point. LLVM provides the BasicBlock class to create and handle basic blocks. A basic block might have an entry point as its label, which indicates where to insert the next instructions. We can use the IRBuilder object to hold these new basic block IR.

Check Compilation Pass Info

41. How to check the compilation pass info for clang for PPC?
Check file PPCTargetMachine.cpp and
./include/llvm/CodeGen/Passes.h
Top-level implementation for the PowerPC target.

LTO & PGO (FDO/PDF)

1. LTO
Link Time optimization.
for example, we can use clang option -flto=thin  to specify the LTO type.

2. Profile Guided Optimization (PGO)

From Wikipedia, the free encyclopedia

Profile-guided optimization (PGO, sometimes pronounced as pogo<sup>[1]</sup>), also known as profile-directed feedback (PDF)<sup>[2]</sup> and feedback-directed optimization (FDO),<sup>[3]</sup> is a compiler optimization technique in computer programming that uses profiling to improve program runtime performance.

3.  IPO     Interprocedural Optimization (IPO)

4. CMO/IPO (cross module optimizations)

5. LIPO - Profile Feedback Based Lightweight IPO

IR simple example and meaning

Source code:
int a, b;
int fn1() {
  char c;
  if (b)
    return 1;
  c = fn1 == a;
  return c;
}
~

Corresponding IR:

; ModuleID = 'utf8.i'
source_filename = "utf8.i"
target datalayout = "e-m:e-i64:64-n32:64"
target triple = "powerpc64le-unknown-linux-gnu"

 ---> Every global variable begins with @
@b = common local_unnamed_addr global i32 0, align 4  
@a = common local_unnamed_addr global i32 0, align 4

---> clang IR use ';' (semi colon) as comments symbol.
; Function Attrs: norecurse nounwind readonly
define signext i32 @fn1() #0 {  ---> Global function fn1
entry:        ---> label entry, which denotes a start of a block.
 
  %0 = load i32, i32* @b, align 4, !tbaa !1   --> every local variable begins with %
 Here the first i32, means the type of load, it loads as i32, the second i32* @b, means
it loads the value from global variable b, with alignment 4 bytes.


  %tobool = icmp eq i32 %0, 0   ---> compare the result i32 %0 (temp variable) from 
last step with 0 and assign the compared bool result to a temp variable named %tobool.
 The above code correspond to the user code if (b)
 
  br i1 %tobool, label %if.end, label %cleanup
---> branch  according to one bit type (i1) variable %tobool value, if true goto %if.end, else goto label%cleanup.

---> The start of if.end block (label name is if.end)
if.end:                                           ; preds = %entry
  %1 = load i32, i32* @a, align 4, !tbaa !1 ---> Load global variable a to a temp variable %1, i32 type, usually this is put on a virtual register, you can consider %1 as r1.

  %conv = sext i32 %1 to i64   ---> sign extend %1 from i32 to i64 and save it to %conv
  %2 = inttoptr i64 %conv to i32 (...)*            ---> convert int to pointer from last step to i32(...) * 
 , i.e., to a function pointer, save the converted result    to %2
 
  %cmp = icmp eq i32 (...)* %2, bitcast (i32 ()* @fn1 to i32 (...)*)  ---> compare equal, function pointer converted from last step: i32 (...)* %2 with bitcast (i32 ()* @fn1 to i32 (...)*), this corresponds to the fn==a in the user source code

  %conv3 = zext i1 %cmp to i32 --> zero extend bit 1 variable %cmp to i32, i.e. convert bool to int
  br label %cleanup ---> branch to label %cleanup to do some clean up work before exit.  Here %conv3 corresponds to variable c

cleanup:                                          ; preds = %entry, %if.end  ---> some preceding labels
  %retval.0 = phi i32 [ %conv3, %if.end ], [ 1, %entry ] 
  ret i32 %retval.0
the return (ret) value is a phi node (a function of the temp variable %conv3 in block %if.end and the constant value 1 in block %entry. This corresponds to return 1; return c
}