💽

Information Management Concepts

TagsComputer scienceComputingDatabaseInformation Management
Created
Updated

Preface

Prerequisites

Learning ethics

ASCII

What are Information management concepts?

Why do Information management concepts matter to you?

Research

Ecosystem

Standards, jobs, industry, roles, …

Story

FAQ

Worked examples

Chapter Name

Subsection

Subsubsection

Exercises

  1. Logic. Mathematics. Code. Automatic Verification such as Lean Proven or Frama-C.
  1. Languages in Anki.

Projects

Summary

FAQ

Reference Notes

Information theory Overview

0.13 Number systems

0.130 Positional number systems

Assume (0)b=0(0)_b=0 and (1)b=1(1)_b=1

Convert from bb-base to decimal. If b2,b\geq 2,  then

(anan1...a1a0)bk=0nakbk=(m)10=m(a_na_{n-1}...a_1a_0)_b\equiv \sum_{k=0}^na_kb^k=(m)_{10}=m

Convert from decimal to bb-base.

m=bqk+ak,until q=0 and q0=nm=bq_k+a_k,\text{until } q=0 \text{ and } q_0=n
def a(number, k):
    return math.trunc(number / math.pow(10, k)) % 10


def length(number):
    return math.trunc(math.log10(number)) + 1


def from_base_to_decimal(number, b):
    acc = 0
    for k in range(length(number)):
        acc +=def from_decimal_to_base(number, base):
    return acc
def from_decimal_to_base(number, base):
    base_number = ""
    q = number
    while q > 0:
        qk = trunc(q/base)
        ak = q - base*qk
        base_number = str(ak)+base_number
        q = qk
    return base_number
function from_base_to_decimal(number, base) {
    return String(number).
			split('').
			map((n)=> parseInt(n, base)).
			reduce((acc, current, index, array) => {
			   console.log(`${current}+${base}*${acc}=${current+base*acc}`);
			   return current+base*acc;
			});
}
function from_decimal_to_base(number, base) {
    if (base <= 1) {
        return;
    }
    base_number = ""
    q = number
    while (q>0)
    {
        qk = Math.trunc(q/base)
        ak = q - base*qk
				console.log(`${base * qk + ak}=${base}*${qk}+${ak}`);
        base_number = ak.toString(base).toUpperCase()+base_number
        q = qk
    }
    return base_number
}
function from_any_base_to_any_base(number, start_base, end_base) {
	console.log(`from base ${start_base} to decimal`);
	const start_number = from_base_to_decimal(number, start_base);
	console.log(`from decimal to base ${end_base}`);
	return from_decimal_to_base(start_number, end_base);
}

Fast algorithm.

(1010)2(22)4=(10 10)2(12)8=(001 010)2(A)16=(1010)2(1010)_2\\ (22)_4=(10\text{ }10)_2\\ (12)_8=(001\text{ }010)_2\\ (A)_{16}=(1010)_2

http://www.opentextbookstore.com/mathinsociety/2.4/HistoricalCounting.pdf

https://www.gcu.ac.uk/media/gcalwebv2/gcuoutreach/NUMBERS & NUMBER SYSTEMS.pdf

https://www.radford.edu/~wacase/Number Systems Unit Math 116.pdf

https://www.cl.cam.ac.uk/teaching/1415/CompFund/NumberSystemsAnnotated.pdf

https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-01-unified-engineering-i-ii-iii-iv-fall-2005-spring-2006/comps-programming/mud5.pdf

https://en.wikipedia.org/wiki/Numeral_system

https://ocw.mit.edu/courses/aeronautics-and-astronautics/16-01-unified-engineering-i-ii-iii-iv-fall-2005-spring-2006/comps-programming/number_systems.pdf

https://ocw.mit.edu/resources/res-18-008-calculus-revisited-complex-variables-differential-equations-and-linear-algebra-fall-2011/study-materials/MITRES_18_008_supp_notes01.pdf

https://www.cs.princeton.edu/courses/archive/spr15/cos217/lectures/03_NumberSystems.pdf

http://www.unitconversion.org/unit_converter/numbers-ex.html

COMS W3827 Fundamentals of Computer Systems

0.1311 Binary and boolean function

Binary vs boolean and bits

Ecosystem

0.132 Nonpositional Number Systems

0.14 Data Storage

0.140 Data Types

0.141 Storing Numbers

⚠️
When you're saving numbers, so computer saves them as binary.

Method of complements

Cb1(m)=bnm>0,bn>mC_b1(m)=b^n-m>0,b^n>m

Nine's complement

C9(m)=(10nm1)C_9(m)=(10^n-m – 1)

Ten's complement

C10(n)=10nC_{10}(n)=10^n

One's complement

One's complement is an operation to invert bits.

C1(m)=2nm1C_1(m)=2^n-m-1

Worked examples.

m=56,C(56)=28561=199m=-56,C(56)=2^8-56-1=199

56=00111000199=1100011156=00111000\\ 199=11000111

Two's complement

There is only one zero in two’s complement notation.

One's complement + 1. Because (0)onescomplement=(1)2(0)_{one's complement}=(1)_2, but (0)twoscomplement=(0)2(0)_{two'scomplement}=(0)_2

C2(m)=2nmC_2(m)=2^n-m

Worked examples.

m=56,C8bits(56)=2856=200m=-56,C_{8 bits}(56)=2^8-56=200

056=00111000200=11001000056=00111000\\ 200=11001000

Trick.

https://www.csestack.org/how-to-find-2s-complement/

Example. 1110011011100110 two's complement format to integer.

 The sign is negative. 11100110 Apply two’s complement 00011010Integer changed to decimal 26Sign is added 26\text{ The sign is negative. } 11100110\\ \text{ Apply two's complement } 00011010\\ \text{Integer changed to decimal } 26\\ \text{Sign is added } -26

How to encode negative numbers in binary number systems?

Summary

Contents of memoryUnsignedSign-and-magnitudeTwo's complementOne's complement
000000+00
000111+11
001022+22
001133+33
010044+44
010155+55
011066+66
011177+77
10008-0-8-7
10019-1-7-6
101010-2-6-5
101111-3-5-4
110012-4-4-3
110113-5-3-2
111014-6-2-1
111115-7-1-0
NotesThe leftmost bit defines the sign. If is 0, the integer is positive else negative.The leftmost bit defines the sign. If is 0, the integer is positive else negative. The leftmost bit defines the sign. If is 0, the integer is positive else negative. This has two 0.

How to encode real numbers in binary number systems?

0.142 Storing Text

A character is an element of grammar (English, Spanish, ...) + accepted human-computer interface by convention (backspace, delete, escape, @, ...), i.e. a code. code={charactercharactergrammar or character human-computer interface}.code=\{character | character \in grammar \text{ or } character \in \text{ human-computer interface} \}.

For example, English grammar is {A,B,C,...,Z}{a,b,c,...,z}{.,;,,+,!,...,}{0,1,2,3,...,9}\{A,B,C,...,Z\}\cup \{a,b,c,...,z\} \cup \{.,;,-,+,!,...,*\} \cup \{0,1,2,3,...,9\} and human-computer interface in ASCCI is{NUL,SOH,Space,...CAN}\{NUL,SOH,Space,...CAN\}.

We can represent each character with a bit pattern of n bits (bit pattern length=n). If we have a code={A}code=\{A\}, his cardinality code=1|code|=1. Then computer understands computer code=bit patterns ={0}\text{computer code}=\text{bit patterns =}\{0\} and bit pattern length=1\text{bit pattern length}=1

code={A,B},code=2,bit pattern={0,1},bit pattern length=1code=\{A,B\},|code|=2,\text{bit pattern}=\{0,1\},\text{bit pattern length}=1 .

code={A,B,C,D},code=4,computer={00,01,10,11},bit pattern length=2code=\{A,B,C,D\},|code|=4,computer=\{00,01,10,11\},\text{bit pattern length}=2.

code=8,bit pattern length=3,lg(8)=3|code|=8,\text{bit pattern length}=3,lg(8)=3.

Therefore, bitPatternLength=lg(code)\text{bitPatternLength}=lg(|code|).

0.1421 ASCII Code and UNICODE

IEEE milestones.

ASCII.

UNICODE. Emojis.

printf "\r12345\n\r6\n"; printf "\r5\n"
printf "\r12345"; printf "\r5\n"

https://stackoverflow.com/questions/3091524/what-are-carriage-return-linefeed-and-form-feed

Hex Code.

How works internally?

https://www.w3schools.com/tags/ref_urlencode.ASP

https://onlineunicodetools.com/convert-unicode-to-hex

https://en.wikipedia.org/wiki/List_of_Unicode_characters

Mackenzie, Charles E. (1980). Coded Character Sets, History and Development (PDF). The Systems Programming Series (1 ed.). Addison-Wesley Publishing Company, Inc. pp. 6, 66, 211, 215, 217, 220, 223, 228, 236–238, 243–245, 247–253, 423, 425–428, 435–439. ISBN 978-0-201-14460-4. LCCN 77-90165. Archived (PDF) from the original on May 26, 2016. Retrieved August 25, 2019.

https://stackoverflow.com/questions/12747722/what-is-the-difference-between-a-line-feed-and-a-carriage-return

https://pjb.com.au/comp/diacritics.html

ASA standard X3.4-1963

https://stackoverflow.com/questions/1761051/difference-between-n-and-r

0.143 Storing Audio

0.144 Storing Images

0.145 Storing Videos

0.15 Operations on Data

0.150 Logic Operations

0.151 Shift Operations

0.152 Arithmetic Operations

Sum of naturals.

00010001000101100010011100010001\\ 00010110\\ 00100111

Rules. 0+0=0, 0+1=1, 1+0=1 and 1+1=10

Binary Addition Algorithm. (2015, July 06). Retrieved from https://chortle.ccsu.edu/assemblytutorial/zAppendixE/binaryAdd.html

Subtraction of naturals (No complements).

00010001000010100000011100010001\\ 00001010\\ 00000111

Rules. 0-0=0, 1-0=1, 1-1=0, 0-1=10-1=1

https://www.calculator.net/binary-calculator.html

Subtraction of naturals (Two's complement)

9146=91+(46)=4591-46=91+(-46)=45

91=0101101146=00101110+91=0101101146=11010010+45=0010110191=01011011\\ 46=00101110\\\\ +91=01011011\\ -46=11010010\\ +45=00101101

https://www.exploringbinary.com/twos-complement-converter/

Why does char - '0' successfully convert a char to int in C?

https://www.quora.com/Why-does-char-0-successfully-convert-a-char-to-int-in-C/answer/Greg-Kemnitz

Nine's complement

Pascaline.

0.153 Bitwise operations in C

Bitwise Operators in C and C++

x & 1 is equivalent to x % 2 (no sign).

if x&1 is true, then x is an odd number.

First of all, an example:

5(00000101)& 1(00000001)
00000101 &
00000001
00000001 (1 True)

Informal Proof. For non-complement binary number. Where nn is position left to right.

P(n)=2n+1:odd Always it needs +1Q(n)=2n:evenP(n)=2n+1:odd \text{ Always it needs +1}\\ Q(n)=2n:even
Odd number0...1..0...1&1000...0001000...00011T\text{Odd number} \equiv 0...1..0...1\&\\ 1\equiv000...0001\\ 000...0001\equiv1\equiv T
std::list<int> v = { 1, 2, 3, 4, 5, 6 };
    auto it = v.begin();
    while (it != v.end())
    {
        // remove odd numbers.
        if (*it & 1)
        {
            // `erase()` invalidates the iterator, use returned iterator
	            it = v.erase(it);
        }
            // Notice that the iterator is incremented only on the else part (why?)
        else {
            ++it;
        }
    }

x >> 1 is equivalent to x / 2

https://www.cprogramming.com/tutorial/bitwise_operators.html

0.154 Boolean Algebra and Digital Logic: Arithmetic Logic Unit

https://www2.southeastern.edu/Academics/Faculty/kyang/2018/Spring/CMPS375/ClassNotes/CMPS375ClassNotesChap03.pdf

Summary

Floating numbers

IEEE 754

File classification

In Unix, everything is a file, so what is a directory? a file, what is a process? a file, what is a driver? a file. If you don’t separate how looks and how does it, you have blobs.

Blobs and plain text.

Blobs tracking in Git, databases, spreadsheets

Blobs and byte arrays

https://security.stackexchange.com/questions/220560/rsa-key-string-vs-hex-vs-base64-vs-byte-array-represantational-form

Binary-to-text encoding

Split a file into chunks

Images

How can you see an image? and process it? OpenCV

PACS

PACS (picture archiving and communication system)

DICOM

Raster images

png, jpg, webp

Vector images

SVG and canvas

High-performance images and animations

WebGL, OpenGL, directx, vulkan

Computer graphics

QR Code

By field

Myers Automatic Booth

(LAS, Geology)

(G-code, Manufacturing)

(EDF, Neuroscience)

(FASTA, Genetics)

GIS (Geographic Information System)

Financial Information eXchange (FIX)

(XML, JSON and YAML, Information Technology)

https://www.npmjs.com/package/serialize-javascript

https://docs.python.org/3/library/pickle.html

(PDF, Standard)

https://www.youtube.com/watch?v=48tFB_sjHgY&ab_channel=Computerphile

SWIFT

DICOM Standard

https://pydicom.github.io/

DICOM networking protocol

File as Programming

CSS, JS, HTML, WASM

Internet media type (MIMES)

Video

The filmmaker’s handbook

Hex editing

We tell you that files are binary, right? But, how we can you show that if you don’t feel it? And yes, you could write your programs in hex — ”since C and assembly are too high-level” you’re thinking and yes you could do reverse engineering to software.

In Unix, you have different options to edit and show files as hex. Some choices are xxd, hexdump [2], and so on [4]. We’re going to use xxd and vim.

Write a plain text in Unix with “Hello World” and vim. Change to mode xxd.

But, how can show a “Hello World” in binary? xxd -b

Writing a binary file

Unicode. UTF-8. Emojis.

References

[1] hexdump(1) - Linux manual page. (2022, December 19). Retrieved from https://man7.org/linux/man-pages/man1/hexdump.1.html

[2] Explains, F. S. (2021, December 24). Writing binary files: a tutorial in C and Python (security@cambridge screencast). Youtube. Retrieved from https://www.youtube.com/watch?v=jAu0oyxsP20&list=PLbyW0t9gkXg2lcU_T4LgZryxrcJBdP_iF&index=6&ab_channel=FrankStajanoExplains

[3] Hex dump. (2023, January 14). Retrieved from https://vim.fandom.com/wiki/Hex_dump

[4] Top Hex Editors for Linux. (2021, May 17). Retrieved from https://www.tecmint.com/best-hex-editors-for-linux

Data type size

Extended ASCII

https://www.gnu.org/software/emacs/manual/html_node/emacs/User-Input.html

Threads

Overflow

Automation and insights

Drawing.

Ofimatics. Spreadsheet programming.

Spreadsheets Are An Awesome Functional Programming Tool | Lucas Reis' Blog. (2018, December 29). Retrieved from https://lucasmreis.github.io/blog/spreadsheets-are-an-awesome-functional-programming-tool

Conference, S. L. (2014, September 21). "Spreadsheets for developers" by Felienne Hermans. Youtube. Retrieved from https://www.youtube.com/watch?v=0CKru5d4GPk&ab_channel=StrangeLoopConference

Editor

Keyboard remapping

Exercises

  1. Logic. Mathematics. Code. Automatic Verification such as Lean Proven or Frama-C.
  1. Languages in Anki.

Projects

PDF

Is standard?

Is it capable to render HTML, JS?

Summary

FAQ

Reference Notes

References

http://neerc.ifmo.ru/wiki/index.php

https://tug.org/texshowcase/cheat.pdf

Explains, F. S. (2021, March 19). Touch typing: why you MUST learn it ASAP. Youtube. Retrieved from https://www.youtube.com/watch?v=0FUsiyiWWw8&list=PLbyW0t9gkXg2OjpTqGig1nWCqzxwrF8h5&ab_channel=FrankStajanoExplains

Next steps

Problems

From LeetCode, HackerRank, ...

Assume the environment does not allow you to store 64-bit integers (signed or unsigned).

  1. Multiply
  1. Divide
  1. Length