1
Q
What is Python?
A
Python is a general-purpose programming language created by Guido Van Rossum. Python is most praised for its elegant syntax and readable code, if you are just beginning your programming career Python suits you best.
With Python you can do everything from
GUI (graphical user interface ) development,
Web application,
System administration tasks,
Financial calculation,
Data Analysis,
Visualization
and list goes on…
2
Q
What doeas it mean that Python is interpreted language?
A
Język interpretowany – język programowania, w którym interpreter analizuje program linia po linii. Przeciwieństwem języków interpretowanych są języki implementowane w postaci kompilatora (kompilator najpierw kompiluje cały program, a następnie zaczyna działać). Języki interpretowane są nieco wolniejsze od języków kompilowanych, lecz prostsze do napisania (mniej rzeczy o które trzeba się martwić podczas pisaniaskodu)
Python is interpreted language, when you run python program an interpreter will parse python program line by line basis, as compared to compiled languages like C or C++, where compiler first compiles the program and then start running.
Now you may ask, so what’s the difference??
The difference is that interpreted languages are a little bit slow as compared to compiled languages. Yes, you will definitely get some performance benefits if you write your code in compiled languages like C or C++.
But writing codes in such languages is a daunting task for a beginner. Also in such languages, you need to write even most basic functions like calculate the length of the array, split the string etc. For more advanced tasks sometimes you need to create your own data structures to encapsulate data in the program. So in C/C++ before you actually start solving your business problem you need to take care of all minor details. This is where Python comes. In Python, you don’t need to define any data structure, no need to define small utility functions because Python has everything to get you started.
Moreover, Python has hundreds of libraries available at https://pypi.python.org/ which you can use in your project without reinventing the wheel.
3
Q
What does it mean that Python is Dynamically Typed ?
A
Python sam automatycznie rozpoznaje typ zmiennych na podstawie kodu.
Python doesn’t require you to define variable data type ahead of time. Python automatically infers the data type of the variable based on the type of value it contains.
For e.g:
myvar = “Hello Python”
The above line of code assigns string “Hello Python” to the variable myvar, so the type of myvar is string.
Note that unlike languages like C, C++ and Java, in Python you do not need to end a statement with a semicolon (;).
Suppose, a little bit later in the program we assign variable myvar a value of 1 i.e
myvar = 1
Now myvar variable is of type int.
4
Q
What does it mean that Python is strongly typed?
A
Nie dokonuje on automatycznej konwersji kodu.
If you have programmed in PHP or javascript. You may have noticed that they both convert data of one type to another automatically.
For e.g:
In JavaScript
1 + “2”
will be ‘12’
Here, before addition (+) is carried out, 1 will be converted to a string and concatenated to “2”, which results in ‘12’, which is a string. However, In Python, such automatic conversions are not allowed, so
1 + “2”
will produce an error.
5
Q
Which code will be longer, the one written in Python or Java?
A
Programs written in Python are usually 1/3 or 1/5 of the Java code. It means we can write less code in Python to achieve the same thing as in Java.
6
Q
Who uses Python?
A
Python is used by many large organizations like Google, NASA, Quora, HortonWorks and many others.
Okay, what I can start building in Python?
Pretty much anything you want. For e.g:
GUI applications.
Web apps.
Scrape data from websites.
Analyse Data.
System administration utilities.
Game Development.
Data Science
and many more …
7
Q
Do I have to install Python on my PC?
A
Mac also comes with python 2 and python 3 installed (if not see this link for instructions), but this is not the case with windows. Similarly, most Linux distribution for e.g Ubuntu 14.04 comes with python 2 and 3 installed, bu you have to install Python on Windows
8
Q
How to install Python on Windows/Ubuntu/Mac?
A
To install python you need to download python binary from https://www.python.org/downloads/, specifically we will be using python 3.4.3 which you can download from here. While installing remember to check “Add Python.exe to path”
If you are using Ubuntu 14.04 which already comes with python 2 and python 3, you need to enter python3 instead of just python to enter python 3 shell. Also you will need a text editor to write Python programs, you can use text editor like notepad. If you want to use full-fledged text editor then use notepad++ or sublime text. Download and install text editor of you choice.
On Mac is good to check Python version by typing in terminal: ~ % python3 –version
Then to install Python 3 with the Official Installer from: https://www.python.org/downloads/
9
Q
How to run python programs?
A
You can run python programs in two ways, first by typing commands directly in python shell:
Np.
print(“Hello World”)
> Hello World
or run program stored in a file. But most of the time you want to run programs stored in a file.
Np. do terminala wpisujemy:
python hello.py
W ten sposób wgrywamy plik z zapisanym tam uprzednio programem.
10
Q
How many arguments does print() function have?
A
as many arguments as you provide it with (as many as editor allow). When two or more arguments are passed, the print() function displays each argument separated by space.
Np.
print(“Hello World”)
> Hello World
print(“Hello”, “World”)
> Hello World
11
Q
Which command is used to change directory?
A
cd ‘od change directory’
Open terminal and change current working directory to C:\Users\YourUserName\Documents using cd command
12
Q
What to do when you want to know more about some method or functions?
A
Type help() into the terminal.
Sooner or later while using python you will come across a situation when you want to know more about some method or functions. To help you Python has help() function, here is how to use it.
Syntax:
To find information about class: help(class_name)
To find more about method belong to class: help(class_name.method_name)
Now suppose you want to know arguments required for index() method of str class, to find out you need to type the following command in the python shell: help(str.index)
13
Q
What are variables in Python and what are the rules of naming them?
A
Variables are named locations that are used to store references to the object stored in memory. The names we choose for variables and functions are commonly known as Identifiers. In Python, Identifiers must obey the following rules.
- All identifiers must start with a letter or underscore (_), you can’t use digits. For e.g: my_var is a valid identifier but 1digit is not.
- Identifiers can contain letters, digits and underscores (_). For e.g: error_404, _save are valid identifiers but $name$ ($ is not allowed) and #age (# is not allowed) are not.
- They can be of any length.
- Identifiers can’t be a keyword. Keywords are reserved words that Python uses for special purposes). The following are keywords in Python 3.
False class finally is return
None continue for lambda try
True def from nonlocal while
and del global not with
as elif if or yield
pass else import assert
break except in raise
14
Q
Values or literals?
A
Values = literals
They are basic things that programs work with. For e.g: 1, 11, 3.14, “hello” are all values.
They are also commonly known as literals.
They can be of different types for e.g 1, 11 are of type int, 3.14 is a float and “hello” is a string.
15
Q
What is a object in Python?
A
Remember that in Python everything is object even basic data types like int, float, string,
16
Q
How to assign value to a variable?
A
To assign value to a variable equal sign (=) is used. The = sign is also known as the assignment operator.
17
Q
Should I declare types of variables while using Python?
A
In Python, you don’t need to declare types of variables ahead of time. The interpreter automatically detects the type of the variable by the data it contains.
18
Q
Show some examples of variable declaration
A
x = 100 # x is integer
pi = 3.14 # pi is float
sentence = “python is great” # sentence is string
a = b = c = 100 # this statement assign 100 to c, b and a.
19
Q
What the variable stores?
A
When a value is assigned to a variable, the variable doesn’t store the value itself. Instead, the variable only stores a reference (address) of the object where it is stored in the memory.
Therefore, in the x = 100, the variable x stores a reference (or address) to the 100 ( an int object ). The variable x doesn’t store the object 100 itself.
20
Q
Comments
A
Comments are notes which describe the purpose of the program or how the program works. Comments are not programming statements that Python interpreter executes while running the program. Comments are also used to write program documentation. In Python, any line that begins with a pound sign (#) is considered a comment.
e.g:
#This program prints “hello world”
print(“hello world”)
We can also write comments at the end of a statement:
print(“hello world”) # display “hello world”
21
Q
What are end-line comments?
A
Comments that appear in this form:
print(“hello world”) # display “hello world”
22
Q
simultaneous assignment
A
The simultaneous assignment or multiple assignment allows us to assign values to multiple variables at once. The syntax of simultaneous assignment is as follows:
var1, var2, …, varn = exp1, exp2, …, expn
e.g:
a, b = 10, 20
print(a)
> 10
print(b)
>20
Simultaneous assignments is quite helpful when you want to swap the values of two variables.
e.g:
x = 1 # initial value of x is 1
y = 2 # initial value of y is 2
y, x = x, y # assign y value to x and x value to y
print(x) # final value of x is 2
print(y) # final value of y is 1
> 2
1
23
Q
Name Python data types
A
- Numbers
- String
- List
- Tuple
- Dictionary
- Boolean
24
Q
List values in Python, which are considered as false
A
Boolen False
0 - zero , 0.0
[] - empty list ,
() - empty tuple ,
{} - empty dictionary ,
‘ ‘ - empty string
None
25
How to receive input from the console?
The input() function is used to receive input from the console.
The input() function accepts an optional string argument called prompt and returns a string.
Note that the input() function always returns a string even if you entered a number. To convert it to an integer you can use int() or eval() functions.
26
eval()
Python's eval() allows you to evaluate (parse, compile, evaluate and return) arbitrary Python expressions from a string-based or compiled-code-based input.
e.g.
>>> eval("2 ** 8")
256
27
Python modules
Python organizes codes using modules. Python comes with many built-in modules ready to use for
e.g
there is a 'math' module for mathematical related functions,
're' module for regular expression,
'os' module for operating system related functions and so on.
28
How to import module?
To use a module we first import it using the import statement. Its syntax is as follows:
import module_name
We can also import multiple modules using the following syntax:
import module_name_1, module_name_2
29
Regular expression
A Regular Expression (RegEx) is a sequence of characters that defines a search pattern. For example:
^a...s$
The above defines a RegEx pattern. The pattern is: any five letter string starting with a and ending with s
Python has a module named re to work with RegEx. Here's an example:
e.g
import re
pattern = '^a...s$'
test_string = 'abyss'
result = re.match(pattern, test_string)
if result:
print("Search successful.")
else:
print("Search unsuccessful.")
30
How to access objects in a module?
e.g
import math, os
print(math.pi)
print(math.e)
print(os.getcwd())
In this listing, the first line imports all functions, classes, variables and constants defined in the math and os module. To access objects defined in a module we first write the module name followed by a dot (.) and then the name of the object itself. (i.e class or function or constant or variable). In the above example, we are accessing two common mathematical constants pi and e from the math math. In the next line, we are calling the getcwd() function of the os module which prints the current working directory.
31
What numerical types are supported in Python?
Python supports 3 different numerical types.
1) int - for integer values like 1, 100, 2255, -999999, 0, 12345678.
2) float - for floating-point values like 2.3, 3.14, 2.71, -11.0.
3) complex - for complex numbers like 3+2j, -2+2.3j, 10j, 4.5+3.14j.
32
Complex number
As you may know complex number consists of two parts real and imaginary, and is denoted by j. You can define complex number like this:
>>> x = 2 + 3j # where 2 is the real part and 3 is imaginary
33
How to determine variable type?
Python has type() inbuilt function which is use to determine the type of the variable.
e.g
x = 12
type(x)
>
34
Name Python operators
Name Meaning Example Rresult
+ Addition. 15+20. >35
- Subtraction. 24-3. >21.0
* Multiplication 15*4 >60
/ Float Division. 4/5. >0.8
// Integer Division. 4//5. >0
** Exponentiation. 4**2 >16
% Remainder. 27%4. >3
35
Integer division
Integer Division (//) : The // perform integer division i.e it will truncate the decimal part of the answer and return only integer.
36
Float division
Float Division (/) : The / operator divides and return result as floating point number means it will always return fractional part. e.g
3/2
>1.5
37
Exponentiation operator
Exponentiation Operator (**) : This operator helps to compute a^b (a raise to the power of b). Let's take an example:
e.g
2 ** 3 # is same as 2 * 2 * 2
> 8
38
Remainder operator
Remainder operator (%, reszta) : The % operator also known as remainder or modulus operator. This operator return remainder after division. For e.g:
> 7 % 2
1
39
Operator precedence
Pierszeństwo operatorów w Pythonie:
1.Parentheses (grouping) ()
2. Function call f(args...)
3.Slicing e.g list, string x[index:index]
4. Subscription x[index]
5. Atribute reference. x.attribute
6. ** Exponentiation (potęga)
7 ~ NOT, also known as bitwise not - inverts all the bits ~x
8. Positive, negative +x -x
9. Multiplication, division, remainder
* / %
10. Addition, subtraction + -
11. Bitwise shifts e.g
<< Zero fill left shift Shift left by pushing zeros in from the right and let the leftmost bits fall off
>> Signed right shift Shift right by pushing copies of the leftmost bit in from the left, and let the rightmost bits fall off
12. Bitwise AND &
Sets each bit to 1 if both bits are 1
13. Bitwise XOR ^
Sets each bit to 1 if only one of two bits is 1
14. Bitwise OR |
Sets each bit to 1 if one of two bits is 1
15. Comparision, membership identity
in, not in, is, is not, <, <=, >, >=, <>, !=, ==
16. Boolean NOT not x
17. Boolean AND and
18. Boolean OR or
19. Lambda expression lambda
If operators have the same precedence then they will be evaluated from left to right, i.e addition will be applied first then subtraction
e.g
3 + 4 - 2
> 5
The only exception to this rule is assignment operator (=) which occur from right to left.
e.g
a = b = c
40
Augmented Assignment Operators
+= , -=, *= , /*=, //*=, %*=, **=
These operators allows you write shortcut assignment statements. For e.g:
count = 1
count += 1
print(count)
> 2
Is the same as:
count = 1
count = count + 1
print(count)
> 2
41
+=
Addition assignment
Example:
x += 4
Same as:
x = x + 4
42
-=
Subtraction assignment
Example:
x -= 2
Same as:
x = x - 2
43
*=
Multiplication assignment
Example:
x *= 5
Same as:
x = 5 * x
44
/=
Division assignment
Example:
x /= 5
Same as:
x = x / 5
45
//=
Integer division assignment
Example:
x //= 5
Same as:
x = x // 5
46
//=
Integer division assignment
Example:
x //= 5
Same as:
x = x // 5
47
%=
Remainder assignment
Example:
x %= 5
Same as:
x = x % 5
48
**=
Exponent assignment
Example:
x **= 5
Same as:
x = x ** 5
49
Strings
Strings in python are contiguous series of characters delimited by single or double quotes. Python doesn't have any separate data type for characters so they are represented as a single character string.
50
How to create strings?
name = "tom" # a string
mychar = 'a' # a character
You can also use the following syntax to create strings.
name1 = str() # this will create empty string object
name2 = str("newstring") # string object containing 'newstring'
51
id()
Every object in python is stored somewhere in memory. We can use id() to get that memory address.
e.g
str1 = "welcome"
str2 = "welcome"
id(str1)
>78965411
id(str2)
>78965411
As both str1 and str2 points to the same memory location, hence they both point to the same object.
52
Are strings mutable?
Strings in Python are Immutable.
What this means to you is that once a string is created it can't be modified. Let's take an example to illustrate this point.
e.g
str1 = "welcome"
str2 = "welcome"
id(str1)
>78965411
id(str2)
>78965411
Let's try to modify str1 object by adding a new string to it.
str1 += " mike"
print(str1)
>welcome mike
id(str1)
>78965579
As you can see now str1 points to totally different memory location, this proves the point that concatenation doesn't modify the original string object instead it creates a new string object.
53
Concatanation
String concatenation is the operation of joining character strings end-to-end.
For example, the concatenation of "snow" and "ball" is "snowball"
54
Are int type mutable?
Number (i.e int type) is, like string, immutable.
e.g
int1 = 1
int2 = 1
print(id(int1), id(int2))
> 9316800, 9316800
int2 += 3
print(int2)
print(id(int1), id(int2))
> 4
9316800, 9316896
55
Could I access string characters?
Yes.
String index starts from 0, so to access the first character in the string type:
name = "tom"
print(name[0])
print(name[1])
>t
o
56
Operator for string contatanation
+
e.g
s = "tom and " + "jerry"
print(s)
> tom and jerry
57
Operator for string repetition
*
e.g
s = "spamming is bad " * 3
print(s)
> 'spamming is bad spamming is bad spamming is bad '
58
Operator for string slicing
[]
e.g
s = "Welcome"
s[1:3]
> el
Cięcie następuje przed [1] i przed [3]
Some more slicing examples.
s = "Welcome"
s[:6]
> 'Welcom'
s[4:]
>'ome'
s[1:-1]
>'elcom'
The start index and end index are optional. If omitted then the default value of start index is 0 and that of end is the last index of the string.
59
ord()
function returns the ASCII code of the character
print(ord('a'))
> 97
60
chr()
chr() - function returns character represented by a ASCII number.
print(chr(97))
> a
61
Name 3 string function in Python
len() - returns length of the string
max() - returns character having highest ASCII value
min() - returns character having lowest ASCII value
e.g
print(len("jagoda"))
print(max("jagoda"))
print(min("jagoda"))
<6
o
a
62
ASCII
ASCII (czyt. aski, skrót od ang. American Standard Code for Information Interchange) – siedmiobitowy system kodowania znaków.
Jest używany we współczesnych komputerach oraz sieciach komputerowych, a także innych urządzeniach wyposażonych w mikroprocesor.
Przyporządkowuje liczbom z zakresu 0−127:
- litery alfabetu łacińskiego języka angielskiego,
- cyfry,
- znaki przestankowe
i inne symbole oraz polecenia sterujące.
Na przykład litera „a” jest kodowana jako liczba 97, a znak spacji jest kodowany jako 32.
63
membership operator
in, not in operators
e.g
s1 = "Welcome"
"come" in s1
>True
"come" not in s1
>False
>>>
64
What is used to compare strings with each other?
Python compares string lexicographically i.e using ASCII value of the characters.
You can use ( > , < , <= , <= , == , != ) to compare two strings.
Suppose you have str1 as "Mary" and str2 as "Mac". The first two characters from str1 and str2 ( M and M ) are compared. As they are equal, the second two characters are compared. Because they are also equal, the third two characters (r and c ) are compared. And because r has greater ASCII value than c, str1 is greater than str2.
e.g
print("Mac" == "Mary")
print("Mac" < "Mary")
print(ord("c")), print(ord("r"))
>False
True
99, 114
65
By default, print() function prints string with a newline, we change this behavior by passing named keyword argument called ____
end
print("my string", end=" ")
> my string
print("my string", end="foo")
>my stringfoo
print("my string")
print("my string", end="foo")
> my string
my stringfoo
print("my string", end="")
print("my string", end="foo")
> my stringmy stringfoo
66
Can I use for loop on strings?
String is a sequence type and also iterable using for loop
e.g
word = "hello"
for letter in word:
print(letter)
>h
e
l
l
o
Print na końcu automatycznie dodaje /n. Możemy go jednak zastąpić pustym stringiem
e.g
word = "hello"
for letter in word:
print(letter, end="")
> hello
67
Testing strings: isalnum()
Returns True if string is alphanumeric
e.g
s = "welcome to python"
s.isalnum()
> False (bo jest spacja)
txt = "Company12"
x = txt.isalnum()
print(x)
>True
txt = "Company"
x = txt.isalnum()
print(x)
> True
txt = "12"
x = txt.isalnum()
print(x)
>True
68
Testing strings: isalpha()
Returns True if string contains only alphabets
e.g
s = "welcome"
s.isalpha()
> True
s = "welcome to python"
s.isalpha()
> False
69
Testing strings: isdigit()
Returns True if string contains only digits
e.g
"2012".isdigit()
>True
70
Testing strings: isidentifier()
Return True is string is valid identifier
A string is considered a valid identifier if it only contains alphanumeric letters (a-z) and (0-9), or underscores (_). A valid identifier cannot start with a number, or contain any spaces
71
Testing strings: islower()
Returns True if string is in lowercase
e.g
s = "welcome''
s.islower()
>True
72
Testing strings: isupper()
Returns True if string is in uppercase
e.g
"WELCOME".isupper()
>True
73
Testing strings: isspace()
Returns True if string contains only whitespace
e.g
" \t".isspace()
> True
74
Searching for Substrings: endswith(s1: str):
bool
Returns True if strings ends with substring s1
e.g
s = "welcome to python"
print(s.endswith("thon"))
> True
75
Searching for Substrings:
startswith(s1: str):
bool
Returns True if strings starts with substring s1
e.g
s = "welcome to python"
print(s.startswith("good"))
>False
76
Searching for Substrings:
count(substring):
int
Returns number of occurrences of substring in the string
e.g
s = "welcome to python"
print(s.count("o"))
>3
77
Searching for Substrings:
find(s1):
int
Returns lowest index from where s1 starts in the string, if string not found returns -1
(znajduje indeks od którego zaczyna się substring o któy pytamy, jeśli nic nie znajdzie drukuje -1)
e.g
s = "welcome to python"
print(s.find("come"))
> 3
print(s.find("become"))
> -1
78
Searching for Substrings:
rfind(s1):
int
Returns highest index from where s1 starts in the string, if string not found returns -1
(znajduje ostatni index w którym występuje litera lub pierwszy indeks w którym wystepuje dany substring)
e.g
s = "welcometopython"
print(s.rfind("o"))
print(s.rfind("top"))
> 13
7
79
Converting Strings:
capitalize():
str
Returns a copy of this string with only the first character capitalized.
e.g
s = "string in python"
s1 = s.capitalize()
print(s1)
> String in python
80
Converting Strings:
lower():
str
Return string by converting every character to lowercase
e.g
s = "This Is Test"
s1 = s.lower()
print(s1)
> this is test
81
Converting Strings:
upper():
str
Return string by converting every character to uppercase
e.g
s = "This Is Test"
s1 = s.upper()
print(s1)
> THIS IS TEST
82
Converting Strings:
title():
str
This function return string by capitalizing first letter of every word in the string
e.g
s = "this is test"
s1 = s.title()
print(s1)
> This Is Test
83
Converting Strings:
swapcase():
str
Return a string in which the lowercase letter is converted to uppercase and uppercase to lowercase
e.g
s = "jAgOdA"
s1 = s.swapcase()
print(s1)
> JaGoDa
84
Converting Strings:
replace(old\, new):
str
This function returns new string by replacing the occurrence of old string with new string
e.g
s = "Jagoda"
s1 = s.replace("a", "e")
print(s1)
> Jegode
85
How to create a list in Python?
list1 = [1, 2, 3]
other way list1 = list([1,2,3])
empty list:
list1 = []
list1 = list()
86
Are lists mutable?
Yes, lists are mutable.
87
How to acces list elements?
list[1]
access second element in the list
88
Common List Operations:
x in s
True if element x is in sequence s, False otherwise
89
Common List Operations:
x not in s
True if element x is not in sequence s, False otherwise
90
Common List Operations:
s1 + s2
Concatenates two sequences s1 and s2
91
Common List Operations:
n * s
s * n
n copies of sequence s concatenated
92
Common List Operations:
s [i]
access ith element in sequence s.
93
Common List Operations:
len(s)
Length (number of elements) of list
94
Common List Operations:
min(s)
Smallest element in sequence s.
95
Common List Operations:
max(s)
Largest element in sequence s.
96
Common List Operations:
sum(s)
Sum of all numbers in sequence s.
97
Common List Operations:
for loop
Przechodzi elementy od lewej do prawej w pętli for.
98
Slice operator ([start:end])
Slice operator ([start:end]) allows to fetch sublist from the list. It works similar to string.
e.g
list = [11,33,44,66,788,1]
list[0:5] # this will return list starting from index 0 to index 4
>[11,33,44,66,788]
list[:3]
>[11,33,44]
list[2:]
>[44,66,788,1]
99
The ___ operator joins the two list.
+
100
The ___ operator replicates the elements in the list.
*
101
The __ operator is used to determine whether the elements exists in the list. On success it returns True on failure it returns False.
in
102
List is a sequence and also iterable - you can use ___ to traverse through all the elements of the list.
for i in
e.g
list = [1,2,3,4,5]
for i in list:
print(i, end=" ")
1 2 3 4 5
103
Common list methods:
append
append(x:object):None
Adds an element x to the end of the list and returns None.
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.append(19)
print(list1)
>[2, 3, 4, 1, 32, 4, 19]
104
Common list methods:
count
count(x:object):int
Returns the number of times element x appears in the list.
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.count(4) # Return the count for number 4
>2
105
Common list methods:
extend
extend(l:list):None
Appends all the elements in l to the list and returns None.
e.g
list1 = [2, 3, 4, 1, 32, 4]
list2 = [99, 54]
list1.extend(list2)
print(list1)
> [2, 3, 4, 1, 32, 4, 19, 99, 54]
106
Common list methods:
index
index(x: object):int
Returns the index of the first occurrence of element x in the list
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.index(4) # Return the index of number 4
> 2
107
Common list methods:
insert
insert(index: int, x: object):None
Inserts an element x at a given index. Note that the first element in the list has index 0 and returns None.
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.insert(1, 25) # Insert 25 at position index 1
print(list1)
> [2, 25, 3, 4, 1, 32, 4, 19, 99, 54]
108
Common list methods:
remove
remove(x:object):None
Removes the first occurrence of element x from the list and returns None
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.remove(32) # Remove number 32
print(list1)
>[2, 25, 4, 1, 4, 19, 99]
109
What 'returns None' mean?
that function ends succesfully
110
Common list methods:
reverse
reverse():None
Reverse the list and returns None
e.g
list1 = [2, 3, 4, 1, 32, 4]
list1.reverse() # Reverse the list
print(list1)
>[4, 32, 1, 4, 3, 2]
111
Common list methods:
sort
sort(): None
Sorts the elements in the list in ascending order and returns None.
e.g
list1 = [2, 3, 4, 1, 32, 4]
>>> list1.sort() # Sort the list
print(list1)
>[1, 2, 4, 4, 19, 25, 99]
112
Common list methods:
pop
pop(i): object
Removes the element at the given position and returns it. The parameter i is optional. If it is not specified, pop() removes and returns the last element in the list.
e.g
list1 = [2, 25, 4, 1, 32, 4, 19, 99, 54]
list1.pop()
print(list1)
[2, 25, 4, 1, 32, 4, 19, 99]
113
List comprehension
List comprehension provides a concise way to create list. It consists of square brackets containing expression followed by for clause then zero or more for or if clauses.
>>> list1 = [ x for x in range(10) ]
>>> list1
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
>>>
>>> list2 = [ x + 1 for x in range(10) ]
>>> list2
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>>
>>>
>>> list3 = [ x for x in range(10) if x % 2 == 0 ]
>>> list3
[0, 2, 4, 6, 8]
>>>
>>>
>>> list4 = [ x *2 for x in range(10) if x % 2 == 0 ]
[0, 4, 8, 12, 16]
114
Dictionary
Dictionary is a python data type that is used to store key-value pairs. It enables you to quickly retrieve, add, remove, modify, values using a key. Dictionary is very similar to what we call associative array or hash on other languages.
e.g
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
here friends is a dictionary with two items. One point to note that key must be of a hashable type, but the value can be of any type. Each key in the dictionary must be unique.
115
How to create mepty dictionary?
>>> dict_emp = {} # this will create an empty dictionary
Alternatively:
dict_eml = dict()
116
How to get an item from the dictionary?
dictionary_name['key']
e.g
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
friends['tom']
> '111-222-333'
If the key exists in the dictionary, the value will be returned, otherwise a KeyError exception will be thrown.
117
How to delete items from the dictionary?
dictionary_name['newkey'] = 'newvalue'
e.g
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
del friends['bob']
friends
> {'tom': '111-222-333', 'jerry': '666-33-111'}
If the key is found the item will be deleted, otherwise a KeyError exception will be thrown.
118
How to use for loop to traverse elements in the dictionary?
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
for key in friends:
print(key, ":", friends[key])
> tom : 111-222-333
jerry : 666-33-111
119
How to find the length of the dictionary?
You can use the len() function to find the length of the dictionary.
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
len(friends)
>2
120
How to check whether key exists in the dictionary?
in and not in operators to check whether key exists in the dictionary.
friends = {
'tom' : '111-222-333',
'jerry' : '666-33-111'
}
'tom' in friends
> True
'tom' not in friends
> False
121
How to test whether two dictionaries contains the same items ?
The == and != operators tells whether dictionary contains the same items or not.
d1 = {"mike":41, "bob":3}
d2 = {"bob":3, "mike":41}
d1 == d2
> True
d1 != d2
> False
122
Dictionary methods:
popitem()
Returns randomly selected item from the dictionary and also remove the selected item.
e.g
friends = {
'tom': '111-222-333',
'bob': '888-999-666',
'jerry': '666-33-111'
}
friends.popitem()
>('tom', '111-222-333')
123
Dictionary methods:
clear()
Delete everything from a dictionary
e.g
friends = {
'tom': '111-222-333',
'bob': '888-999-666',
'jerry': '666-33-111'
}
friends.clear()
friends
> {}
124
Dictionary methods:
keys()
Return keys in the dictionary as tuples
e.g
friends = {
'tom': '111-222-333',
'bob': '888-999-666',
'jerry': '666-33-111'
}
friends.keys()
> dict_keys(['tom', 'bob', 'jerry'])
125
Dictionary methods:
get(key)
Return value of key, if key is not found it returns None, instead of throwing KeyError exception
e.g
friends = {
'tom': '111-222-333',
'bob': '888-999-666',
'jerry': '666-33-111'
}
friends.get('tom')
>'111-222-333'
126
Dictionary methods:
pop(key)
Remove the item from the dictionary, if the key is not found KeyError will be thrown
e.g
friends = {
'tom': '111-222-333',
'bob': '888-999-666',
'jerry': '666-33-111'
}
friends.pop('bob')
friends
> {'tom': '111-222-333', 'jerry': '666-33-111'}
127
Relational operators
Relational operators allows us to compare two objects.
128
Relational operators
<=
smaller than or equal to
129
Relational operators
<=
smaller than or equal to
130
Relational operators
<, >
smaller than
greater than
131
Relational operators
>=
greater than or equal to
132
Relational operators
==
equal to
133
Relational operators
!=
not equal to
134
Example of statement with boolean-expression
if boolean-expression:
#statements
else:
#statements
e.g
i = 10
if i % 2 == 0:
print("Number is even")
else:
print("Number is odd")
135
Example of nested if statement
today = "holiday"
bank_balance = 25000
if today == "holiday":
if bank_balance > 20000:
print("Go for shopping")
else:
print("Watch TV")
else:
print("normal working day)
136
Syntax for def
arg = argument/parameter
def function_name(arg1, arg2, arg3, .... argN):
#statement inside function
e.g
def myfunc():
pass
Example of a function that calculates the sum of all the numbers starting from start to end:
def sum(start, end)
result=0
for i in range(start, end+1)
result = result +1
s = sum(10, 50)
print(s)
>1230
137
In python, if you do not explicitly return value from a function, then a special value of _____ is always returned.
None
e.g
def test(): # test function with only one statement
i = 100
print(test()
> None
As you can see, the test() function doesn't explicitly return any value, so None is returned.
138
Global variables
Global variables: Variables that are not bound to any function , but can be accessed inside as well as outside the function are called global variables.
e.g
global_var = 12
def func():
local_var = 100
print(global_var)
you can access global variables inside function
139
Local variables
Local variables: Variables which are declared inside a function are called local variables.
e.g
xy = 100
def cool():
xy = 200
print(xy)
cool()
> 200
print(xy)
> 100
W przypadku gdy mamy globalną i lokalna zmienną o takiej samej nazwie i wywołujemy funkcję zawierającą tę lokalną zmienną, to lokalna zmienna bedzie miała pierwszeństwo i nie bedziemy mogli wywołać globalnej zmiennej o tej samej nazwie wewnątrz tej funkcji.
140
How to convert local variable into global variable?
You can bind local variable in the global scope by using the global keyword followed by the name of variable
e.g
t = 1
def increment():
global t
t = t + 1
print(t)
# now t inside the function is same as t outside the function
In fact there is no need to declare global variables outside the function. You can declare them global inside the function.
e.g
def foo():
global x
x = 100
141
How to specify default values of argument?
To specify default values of argument, you just need to assign a value using assignment operator.
e.g
def func(i, j = 100):
print(i, j)
Above function has two parameter i and j. The parameter j has a default value of 100, it means that we can omit value of j while calling the function.
i and j are positional argument
142
Keyword arguments
Keyword arguments allows you to pass each arguments using name value pairs like this name=value. Let's take an example:
e.g
def named_args(name, greeting):
print(greeting + " " + name )
named_args(name='jim', greeting='Hello')
> Hello jim
143
Example of mixing positional and keyword arguments
def my_func(a, b, c):
print(a, b, c)
my_func(12, 13, 14)
> 12, 13, 14
my_func(12, c=14, b=13)
> 12, 13, 14
144
How to return multiple values from the function?
We can return multiple values from function using the return statement by separating them with a comma (,). Multiple values are returned as tuples.
e.g
def bigger(a, b):
if a > b:
return a, b
else:
return b, a
145
How many loop types of loop Python has?
Python has only two loops:
for loop
while loop
All the statements inside for and while loop must be indented to the same number of spaces. Otherwise, SyntaxError will be thrown.
146
The for loop Syntax:
for i in iterable_object:
# do something
e.g.
my_list = [1,2,3,4]
for i in my_list:
print(i)
>1
2
3
4
147
range(a, b)
for i in range(1, 10):
print(i)
>1
2
3
4
5
6
7
8
9
We receive the same output, when we type:
for i in range(10):
... print(i)
>0
1
2
3
4
5
6
7
8
9
148
range(a, b, c)
The range(a, b) function has an optional third parameter to specify the step size.
e.g
for i in range(1, 20, 2):
print(i)
>1
3
5
7
9
11
13
15
17
19
149
The while loop Syntax:
while condition:
# do something
e.g
count = 0
while count < 10:
print(count)
count += 1
> 0
1
2
3
4
5
6
7
8
9
150
The while loop Syntax:
while condition:
# do something
e.g
count = 0
while count < 10:
print(count)
count += 1
> 0
1
2
3
4
5
6
7
8
9
151
break statement
The break statement allows to breakout out of the loop.
e.g
count = 0
while count < 10:
count += 1
if count == 5:
break
print("inside loop", count)
print("out of while loop")
>inside loop 1
inside loop 2
inside loop 3
inside loop 4
out of while loop
152
continue statement
When continue statement encountered in the loop, it ends the current iteration and programs control goes to the end of the loop body.
e.g
count = 0
while count < 10:
count += 1
if count % 2 == 0:
continue
print(count)
>1
3
5
7
9
153
Python Mathematical Function:
round(number[, ndigits])
rounds the number, you can also specify precision in the second argument
e.g
round(62.998, 1)
> 63.0
154
Python Mathematical Function:
pow(a, b)
Returns a raise to the power of b
e.g
pow(3, 2)
>9
155
Python Mathematical Function:
abs(x)
Return absolute value of x
e.g
abs(-22)
>22
156
Python Mathematical Function:
max(x1, x2, ..., xn)
Returns largest value among supplied arguments
e.g
max(9, 3, 12, 81)
>81
157
Python Mathematical Function:
min(x1, x2, ..., xn)
Returns smallest value among supplied arguments
e.g
min(78, 99, 12, 32)
>12
158
Python Mathematical Function:
import math
ceil(x)
This function rounds the number up and returns its nearest integer
math.ceil(3.4123)
> 4
159
Python Mathematical Function:
import math
floor(x)
This function rounds the down up and returns its nearest integer
math.floor(24.99231)
>24
160
Python Mathematical Function:
import math
sqrt(x)
Returns the square root of the number
e.g
math.sqrt(16)
> 4.0
161
Python Mathematical Function:
import math
sin(x)
Returns sin of x where x is in radian
e.g
math.sin(1)
> 8414709848078965
162
Python Mathematical Function:
import math
cos(x)
Returns cosine of x where x is in radian
e.g
math.cos(2))
> -0.4161468365471424
163
Python Mathematical Function:
import math
tan(x)
Returns tangens of x where x is in radian
e.g
math.tan(9)
> -0.4523156594418099
164
Python ________ module contains function to generate random numbers.
random
You need to import random module using the following line:
import random
165
import random
random()
returns random number between 0 and 1
e.g
for i in range(0, 3):
print(random.random())
> 0.5994361280943458
0.023307374459719976
0.7901690676332683
166
import random
randint()
randint(a, b)
generate random numbers between a and b (inclusively)
e.g
for i in range(0, 3):
print(random.randint(1, 20))
> 5
8
18
167
Syntax for opening the file in Python
We need to use open() function
open(filename, mode)
The open() function accepts two arguments filename and mode. The filename is a string argument which specify filename along with it's path.
Mode is a string argument which is used to specify how file will be used i.e for reading or writing.
We usually use open() function assigning it to f known as file pointer.
Syntax of opening a file is:
f = open(filename, mode)
168
Which method is used to close the file
close()
f.close() # where f is a file pointer
169
Modes of opening a file:
"r"
Open a file for read only
170
Modes of opening a file:
"w"
Open a file for writing. If file already exists its data will be cleared before opening. Otherwise new file will be created
# open file for writing
f = open('myfile.txt', 'w')
# write a line to the file
>>> f.write('this first line\n') *
# write one more line to the file
>>> f.write('this second line\n')
close the file
>>> f.close()
*The write() method will not insert new line ('\n') automatically like print() function, you need to explicitly add '\n' to write write() method.
171
Modes of opening a file:
"a"
Opens a file in append mode i.e to write a data to the end of the file
172
Modes of opening a file:
"wb"
Open a file to write in binary mode
173
Modes of opening a file:
"rb"
Open a file to read in binary mode
174
Reading data from the file in Python
read([number])
Return specified number of characters from the file. if omitted it will read the entire contents of the file.
175
Reading data from the file in Python
readline()
Return the specified line of the file.
176
Reading data from the file in Python
readlines()
Read all the lines as a list of strings in the file
177
Reading data from the file in Python
read()
Reading all the data at once.
178
Reading data from the file in Python
Reading all the data at once.
open("file.txt", "r") as f:
print (f.read())
179
To append the data to a file, you need to ________
open the file in 'a' mode.
e.g
f = open('myfile.txt', 'a')
f.write("this is third line\n")
>19
f.close()
180
How to use for loop on a file?
f = open('myfile.txt', 'r')
for line in f:
print(line)
>this first line
this second line
this is third line
f.close()
181
To perform binary reading and writing you need to use a module called _______
pickle
182
import pickle
What is the method to write binary data?
dump
e.g
import pickle
f = open('pick.dat', 'wb')
pickle.dump(11, f)
pickle.dump("this is a line", f)
pickle.dump([1, 2, 3, 4], f)
f.close()
183
import pickle
What is the method to read binary data?
load
import pickle
f = open('pick.dat', 'rb')
pickle.load(f)
> 11
pickle.load(f)
> "this is a line"
>>> pickle.load(f)
[1,2,3,4]
f.close()
If there is no more data to read from the file, pickle.load() throws EOFError or end of file error.
184
What is an object in Python?
In python everything is object i.e int, str, bool even modules, functions are also objects.
Python is an object-oriented language
185
How to define class in Python?
Class name in python is preceded with class keyword followed by a colon (:).
e.g
class Person:
def __init__(self, name):
self.name = name
def whoami(self):
return "You are " + self.name
186
What are classes for?
Classes commonly contains data field to store the data and methods for defining behaviors.
187
What is always the first def in class?
A special method called initializer (commonly known as constructor).
It is invoked automatically every time new object is created.
e.g
class Person:
# constructor or initializer
def __init__(self, name):
self.name = name
def whoami(self):
return "You are " + self.name
188
Each class has _____
instance variables
e.g
class Person:
def __init__(self, name):
self.name = name*
* name is data field also commonly known as instance variables
189
Create a class called Person which contains one data field called name and method whoami().
class Person:
def __init__ (self, name):
self.name = name
def whoami (self):
print(f"You are {self.name}.")
name = Person("Jagoda")
print(name.whoami())
> You are Jagoda.
190
What is self?
All methods in python including some special methods like initializer have first parameter self. This parameter refers to the object which invokes the method. When you create new object the self parameter in the __init__ method is automatically set to reference the object you have just created.
191
How to create private data fields in Python?
To create private (hide) datafields in Python in a way that they are not accessible outside the class, we need to use two underscores
e.g
class BankAccount:
# constructor or initializer
def __init__(self, name, money):
self.__name = name
self.__balance = money # __balance is private now, so it is only accessible inside the class
def deposit(self, money):
self.__balance += money
def withdraw(self, money):
if self.__balance > money :
self.__balance -= money
return money
else:
return "Insufficient funds"
def checkbalance(self):
return self.__balance
b1 = BankAccount('tim', 400)
print(b1.withdraw(500))
> Insufficient funds
b1.deposit(500)
print(b1.checkbalance())
> 900
print(b1.withdraw(800))
> 800
print(b1.checkbalance()
> 100
Let's try to access __balance data field outside of class.
print(b1.__balance)
> AttributeError: 'BankAccount' object has no attribute '__balance'
192
+ operator is overloaded by both __ class and __class.
int, str
193
What is operator overloading?
Defining methods for operators
e.g To use + operator with custom objects you need to define a method called __add__.
194
_ _add__ is used for ...
using + operators with custom objects
e.g
class Circle:
def __init__(self, radius):
self.__radius = radius
def setRadius(self, radius):
self.__radius = radius
def getRadius(self):
return self.__radius
def __add__(self, another_circle):
return Circle( self.__radius + another_circle.__radius )
c1 = Circle(4)
print(c1.getRadius())
> 4
c2 = Circle(5)
print(c2.getRadius())
> 5
c3 = c1 + c2 # This became possible because we have overloaded + operator by adding a method named __add__
print(c3.getRadius())
> 9
In the above example we have added __add__() method which allows use to use the + operator to add two circle objects. Inside the __add__() method we are creating a new object and returning it to the caller.
Python has many other special methods like __add__().
195
Operator Overloading
+
Addition:
__add__(self, other)
196
Operator Overloading
*
Multiplication:
__mul__(self, other)
197
Operator Overloading
-
Subtraction:
__sub__(self, other)
198
Operator Overloading
%
modulus (remainder):
__mod__(self, other)
bo % to modulo operator
modulo operator ( % ), returns the remainder of dividing two numbers.
199
Operator Overloading
/
Division:
__truediv__(self, other)
200
Operator Overloading
<
Less than:
__lt__(self, other)
201
Operator Overloading
<=
Less than or equal to:
__le__(self, other)
202
Operator Overloading
==
Equal to:
__eq__(self, other)
203
Operator Overloading
!=
Not equal to:
__ne__(self, other)
204
Operator Overloading
>
Greater than:
__gt__(self, other)
205
% in Python means:
remainder
206
Inheritance:
In object-oriented terminology when class B extend class A, then A is called _____ class and B is called ___ class.
A is called super class or base class
B is called subclass, derived class or child class.
207
Inheritance:
What kinds of methods and data fields are accessible by child class? Which are not?
Not private methods and data fields are accessible by child class.
private data fields and methods are accessible only inside the class.
208
Example of class inheritance
class Vehicle:
def __init__(self, name, color):
self.__name = _name
self.__color = color
# __name is private to Vehicle class
def getColor(self):
return self.__color
# getColor() function is accessible to class Car
def setColor(self, color):
self.__color = color
# setColor is accessible outside the class
def getName(self):
return self.__name
# getName() is accessible outside the class
class Car(Vehicle):
def __init__(self, name, color, model):
super().__init__(name, color)
self.__model = model
# call parent constructor to set name and color
def getDescription(self):
return self.getName() + self.__model + " in " + \n
self.getColor() + " color"
in method getDescrition we are able to call getName(), getColor() because they are
# accessible to child class through inheritance
209
_____ method is used to call method of the base class.
super()
Suppose you need to call a method called get_information() in the base class from child class , you can do so using the following code.
super().get_information()
Similarly, you can call base class constructor from child class constructor using the following code.
super().__init__()
210
What is multiple inheritance? Is it possible in Python?
Multiple inheritance means that you can create class, which inherit from multiple classes at the same time.
It is not possible in Java and C programming languages
class Subclass(SuperClass1, SuperClass2, ...):
# initializer
# methods
e.g
class MySuperClass1():
def method_super1(self):
print("method_super1 method called")
class MySuperClass2():
def method_super2(self):
print("method_super2 method called")
class ChildClass(MySuperClass1, MySuperClass2):
def child_method(self):
print("child method")
c = ChildClass()
c.method_super1()
c.method_super2()
211
Overriding methods
To override a method in the base class, sub class needs to define a method of same signature. (i.e same method name and same number of parameters as method in base class).
(metody child class nadpisują metody klasy bazowej)
212
isinstance() function
The isinstance() function is used to determine whether the object is an instance of the class or not.
213
What is polymorphism in python?
Polymorphism is the ability of an object to take on many forms.
The most common use of polymorphism in OOP occurs when a parent class reference is used to refer to a child class object.
214
What is used to handle exception?
Python handles exception using try, except block.
try:
# write some code
# that might throw exception
except :
# Exception handler, alert the user
e.g
try:
f = open('somefile.txt', 'r')
print(f.read())
f.close()
except IOError:
print('file not found')
A try statement can have more than once except clause, It can also have optional else and/or finally statement.
try:
except :
except :
except:
else:
finally:
Statements under the else clause run only when no exception is raised.
Statements in the finally clause will run every time no matter exception occurs or not.
215
eval()
The eval() function evaluates the specified expression, if the expression is a legal Python statement, it will be executed.
e.g Evaluate the expression 'print(55)':
x = 'print(55)'
eval(x)
216
raise
To raise your exceptions from your own methods you need to use raise keyword like this
def enterage(age):
if age < 0:
raise ValueError("Only positive integers are allowed")
217
How to assign exception object to a variable?
try:
# this code is expected to throw exception
except ExceptionType as ex:
# code to handle exception
e.g
try:
number = eval(input("Enter a number: "))
except NameError as ex:
print("Exception:", ex)
>Exception: name 'one' is not defined
218
As you can see from most of the exception classes in python extends from the _______ class.
BaseException
You can derive you own exception class from BaseException class or from some sublcass of BaseException like RuntimeError (depends on what you need)
e.g
class NegativeAgeException(RuntimeError):
def __init__(self, age):
super().__init__()
self.age = age
Above code creates a new exception class named NegativeAgeException, which consists of only constructor which call parent class constructor using super().__init__() and sets the age.
219
What is Python module?
Python module is a normal python file which can store function, variable, classes, constants etc. Module helps us to organize related codes . For e.g math module in python has mathematical related functions.
To create a new module, just save .py file in your project folder.
220
To use module in another file, we need to....
import mymodule
to use module in our programs we first need to import it using import statement
221
When you want to access only specific function or variable from another module you use ___________ instead of import
from mymodule import myfunction
222
dir()
# Comparing Python and Node.Js: Which Is Best for Your Project?
The dir() returns a list of string containing the names of the available attributes.
e.g
dir(mymodule)
> ['__builtins__', '__cached__', '__doc__', '__file__',
'__loader__', '__name__', '__package__', '__spec__', 'foo', 'hello']
223
Regular Expression
Regular expression is widely used for pattern matching.
Python has built-in support for regular function. To use regular expression you need to import re module.
224
import re
re.search()
The re.search() is used to find the first match for the pattern in the string.
Syntax:
re.search(pattern, string, flags[optional])
e.g
import re
example= "my is 123"
print(re.search(r"\d\d\d", example))
>
r to przedrostek, którym poprzedzamy pattern lub string, który wyszukujemy
/d/d/d to patern na wyszukanie 3 liczb (d od digits)
UWAGA:
re.search() find only first match for the pattern, if we want to find all matches in string, this is where findall() comes into the play.
225
Basic patterns used in regular expression
dot matches any character except newline
.
226
Basic patterns used in regular expression
\w
word character
matches any word character i.e letters, alphanumeric, digits and underscore (_)
227
Basic patterns used in regular expression
\W
non-word character
matches non word characters
228
Basic patterns used in regular expression
\d
matches a single digit
229
Basic patterns used in regular expression
\D
matches a single character that is not a digit
230
Basic patterns used in regular expression
\s
matches any white-spaces character like \n, \t, spaces
e.g
import re
example= "my is 123"
print(re.search(r"\w\w\s", example))
>
231
Basic patterns used in regular expression
\S
matches single non white space character
e.g.
import re
example= "my is 123"
print(re.search(r"\s\S\S\s", example))
>
232
Basic patterns used in regular expression
[abc]
matches single character in the set i.e either match a, b or c
e.g
import re
example= "my is 123"
print(re.search(r"[mjx]", example))
>
233
Basic patterns
[a-z]
match a single character in the range a to z.
234
Basic patterns used in regular expression
[^abc]
match a single character other than a, b and c
235
Basic patterns used in regular expression
[a-zA-Z]
match a single character in the range a-z or A-Z
236
Basic patterns used in regular expression
[0-9]
match a single character in the range 0-9
237
Basic patterns used in regular expression:
caret symbol ^
oznacza, że dopasowanie ma wystąpić na początku ciągu tekstowego
e.g
import re
example= "my is 123"
print(re.search(r"^", example))
>
238
Basic patterns used in regular expressions
$
Oznacza, że dopasowanie ma wystąpić na końcu ciągu tekstowego
e.g
import re
example= "my is 123"
print(re.search(r"$", example))
>
239
Basic patterns used in regular expression
+
matches one or more of the preceding character (greedy match).
240
Basic patterns used in regular expression
*
matches zero or more of the preceding character (greedy match).
241
Regular expressions: jak wyszukać adres e-mail używając patternów?
import re
s = "tim email is tim@somehost.com"
match = re.search(r'[\w.-]+@[\w.-]+', s)
the above regular expression will match a email address
if match:
print(match.group())
else:
print("match not found")
> Here we have used [\w.-]+@[\w.-]+ pattern to match an email address
242
Regular expressions: Jak wyciągnąć imię i rodzaj poczty ze string
group capturing
Group capturing allows to extract parts from the matching string. You can create groups using parentheses ().
e.g
import re
s = "tim email is tim@somehost.com"
match = re.search(r'([\w.-]+)@([\w.-]+)', s)
if match:
print(match.group()) ## tim@somehost.com (the whole match)
print(match.group(1)) ## tim (the username, group 1)
print(match.group(2)) ## somehost (the host, group 2)
> tim@somehost.com
tim
somehost.com
243
re.findall()
is used instead of re.search(), when, we want to findall matches in string (re.search() stops after it finds first)
e.g
import re
s = "Tim's phone numbers are 12345-41521 and 78963-85214"
match = re.findall(r'\d{5}', s)
if match:
print(match)
>['12345', '41521', '78963', '85214']
244
Can I use group capturing with re.findall()?
You can also use group capturing with findall(), when group capturing is applied then findall() returns a list of tuples where tuples will contain the matching groups
245
What are flags in the following syntax for re.search() and re.findall():
re.search(pattern, string, flags[optional])
findall(pattern, string, flags=0[optional])
re.IGNORECASE, re.DOTALL and re.MULTILINE
re.IGNORECASE - ignores uppercase and lowercase
re.DOTALL - allows (.) to match newline, be default (.) matches any character except newline
re.MULTILINE - this will allow ^ and $ to match start and end of each lin
246
Czym różni się re.search() od re.match()?
The re.match() is very similar to re.search() difference is that it will start looking for matches at the beginning of the string.
247
What is *args?
The *args allows us to pass variable number of arguments to the function.
e.g
def sum(a, b):
print("Result:", a+b)
print(sum(2, 2) )
> 4
def sum(a, b):
print("Result:", a+b)
print(sum(2, 2, 3))
> Traceback (most recent call last): File... TypeError: sum() takes 2 positional arguments but 3 were given
But when we use *args:
def sum(*args):
s = 0
for i in args:
s += i
print("Result:", s)
print(sum(2, 2, 3))
> 7
Now you can pass any number of arguments to the function like this
Please note that, The name of *args is just a convention you can use anything that is a valid identifier. For e.g *myargs is perfectly valid.
248
What is **kwargs?
**kwargs allows us to pass variable number of keyword argument like this (argumentów nazwanych):
func_name(name='tim', team='school')
e.g
def my_func(**kwargs):
for i, j in kwargs.items():
print(i, j)
print(my_func(name='tim', sport='football', roll=19))
Pozycyjne tu nie przechodzą
249
You can use *args to pass elements in an iterable variable to a function.
def my_three(a, b, c):
print(a, b, c)
a = [1,2,3]
print(my_three(*a))
> 1 2 3
Podaliśmy tylko a, a dopasowało do a, b i c, *a to było *args
args = [1,2,3]
print(my_three(*args))
> 1 2 3
This works only when number of argument is same as number of elements in the iterable variable.
Similarly you can use **kwargs to call a function like this:
def my_three(a, b, c):
print(a, b, c)
a = {'a': "one", 'b': "two", 'c': "three" }
print(my_three(**a))
> one two three
Note that for this to work 2 things are necessary:
Names of arguments in function must match with the name of keys in dictionary.
Number of arguments should be same as number of keys in the dictionary.
250
Python Generators
Generators are function used to create iterators, so that it can be used in the for loop.
251
What is pip
PIP is a package management system used to install packages from repository. You can use pip to install various software packages available on http://pypi.python.org/pypi. PIP is much similar to composer in php. PIP is a recursive acronym which stands for PIP installs packages.
252
How to install pip?
Python 3.4 and later (python 3 series) already comes with pip.
To check your python version you need to enter the following command :
python3 -V
253
How to install, upgrade and uninstall packages?
pip3 install package_name
pip3 install --upgrade package_name
pip uninstall package_name
254
How to search for packages?
pip search "your query"
but this command is no longer supported
254
How to list installed packages?
pip3 list
255
How to list outdated installed packages?
pip3 list --outdated
256
How to get information about an installed package?
pip3 show package_name
257
What is virtualenv?
virtualenv is a tool used to separate different dependencies required by the projects. While working on multiple projects it's a common issue that one project need a version of package that is completely different from the other one, virtualenv helps us to resolve such kind of issues. It also helps to prevent polluting global site package.
virtualenv is just a package available at pypi, you can use pip to install virtualenv.
258
How to install virtualenv?
pip3 install virtualenv
After installation you may need to add C:\Python34\Scripts to your PATH environment variable. This way commands like pip, virtualenv will become available in any directory level.
259
How to create a virtualenv in a folder with your project?
mkdir new_project
cd new_project
virtualenv my_env
(This will create a new folder my_env inside python_project. This folder will contain a copy of python executables and pip library used to install packages. Here we have used my_env as name, but you can use anything you want. Now your virtual environment is ready to use, you just need to activate it.)
source my_env/bin/activate
After issuing the above command your command prompt string will change and will look something like,
( my_env ) Path_to_the_project: $
ANOTHER WAY (my PC):
mkdir new_project
python3.10 -m venv .venv/app
source .venv/app/bin/activate
260
How to deactivate virtualenv?
To deactivate virtual environment you need to use the following command.
deactivate
261
What is recursion?
When a function call itself is knows as recursion. Recursion works like loop but sometimes it makes more sense to use recursion than loop. You can convert any loop to recursion.
262
What is base condition in recursive function?
It is a condition that stops recursion
Here is how recursion works. A recursive function calls itself. As you you'd imagine such a process would repeat indefinitely if not stopped by some condition. This condition is known as base condition. A
263
What this error message means:
'RuntimeError: maximum recursion depth exceeded in comparison'
This happens because python stop calling recursive function after 1000 calls by default. To change this behavior you need to amend the code as follows.
import sys
sys.setrecursionlimit(3000)
def fact(n):
if n == 0:
return 1
else:
return n * fact(n-1)
print(fact(2000))
264
Every module in Python has a special attribute called ____
__name__
265
The value of __name__ attribute is set to _____ when module run as main program
'__main__'
266
Python allows you to create anonymous function i.e function having no names using a facility called ________
lambda function.
267
Lambda function
Lambda functions are small functions usually not more than a line.
- lambda function has no name
- the body of lambda function consists of only one expression
e.g result = lambda x, y: x*y
expression after colon is the body of the lambda function
- there is no need for any return statement in lambda function.
268
Convert following expression into a lambda function:
def multiply(x, y):
return x * y
result = lambda x, y: x*y
result(6*8)
You don't need to assign lambda function to a variable, you can use parentheses instead:
(lambda x, y: x * y)(6, 8)
269
format()
The format() function formats a specified value into a specified format.
Syntax:
format(value, format)
270
format codes
If you want to specify precision in floating point number use following syntax:
{[argument_index_or_keyword]:[width][.precision][type]}
np.
"Floating point pi = {0:.3f}, with {1:d} digit precision".format(math.pi, 3)
0,1 to indeksy argumentów funkcji format()
.2f oznacza, że chcemy precyzje do dwóch miejsc po przecinku
"Floating point {0:10.3f}".format(math.pi)
0 to indeks argumentu funkcji format() (jest tylko 1)
10 width
.3f precyzja do trzech miejsc po przecinku
You need to specify precision only in case of floating point numbers if you specify precision for integer ValueError will be raised.
In old code you can find % instead of {}
np.
"%d pens cost = %.2f" % (12, 150.87612)
>12 pens cost = 150.88
271
abs()
abs() function returns the absolute value (just magnitude without sign) of the number
e.g
>>> abs(-45)
45
272
bin()
bin() function returns the binary representation of the integer as a string
e.g
bin(4) # convert decimal to binary
> '0b100'
bin(0xff) # convert hexadecimal to binary, 0xff is same decimal 255
'> 0b11111111'
273
id()
id() function returns a memory address of the object
We can use this identifier to determine whether two objects are same or not.
e.g
a = 10
b = 5
> (10919712, 10919552)
id(a), id(b)
274
map()
map() function executes a specified function for each item in an iterable. The item is sent to the function as a parameter.
Syntax: map(function, iterables)
function - Required. The function to execute for each item
iterable - Required. A sequence, collection or an iterator object. You can send as many iterables as you like, just make sure the function has one parameter for each iterable.
e.g
def myfunc(a, b):
return a + b
x = map(myfunc, ('apple', 'banana', 'cherry'), ('orange', 'lemon', 'pineapple'))
print(x)
#convert the map into a list, for readability:
print(list(x))
>
def myfunc(a, b):
return a + b
x = map(myfunc, ('apple', 'banana', 'cherry'), ('orange', 'lemon', 'pineapple'))
print(x)
#convert the map into a list, for readability:
print(list(x))
def myfunc(a, b):
return a + b
x = map(myfunc, ('apple', 'banana', 'cherry'), ('orange', 'lemon', 'pineapple'))
print(x)
#convert the map into a list, for readability:
print(list(x))
275
zip()
zip() function returns a zip object, which is an iterator of tuples where the first item in each passed iterator is paired together, and then the second item in each passed iterator are paired together etc.
a = ("John", "Charles", "Mike")
b = ("Jenny", "Christy", "Monica")
x = zip(a, b)
#use the tuple() function to display a readable version of the result:
print(tuple(x))
> (('John', 'Jenny'), ('Charles', 'Christy'), ('Mike', 'Monica'))
276
filter()
Funkcja filter() tworzy nową listę elementów na podstawie wejściowej listy elementów, wybierając tylko te wartości, dla których funkcja testując zwróci prawdę (True)
276
reduce()
reduce() function accepts a function and a sequence and returns a single value calculated as follows:
1. Initially, the function is called with the first two items from the sequence and the result is returned.
2. The function is then called again with the result obtained in step 1 and the next value in the sequence. This process keeps repeating until there are items in the sequence.
e.g
from functools import reduce
def do_sum(x1, x2):
return x1 + x2
print(reduce(do_sum, [1, 2, 3, 4]))
>10
277
sorted()
built-in function allows us to sort the data. It accepts an iterable and returns a sorted list containing the items from the iterable. By default, it sorts in ascending order.
The syntax of sorted() function is as follows:
Syntax: sorted(iterable, key=None, reverse=False)
iterable - (required) Iterable to sort like string, list, dictionary, tuple etc.
key (optional) - It refers to the single argument function to customize the sort order. The function is applied to each item on the iterable. By default, this argument is set to None.
reverse - (optional) A boolean flag to reverse the sorting order. It defaults to False.
e.g
fruits = ['lime', 'blueberry', 'plum', 'avocado']
> ['avocado', 'blueberry', 'lime', 'plum']
sorted(fruits, key=len) # sort by string length
['lime', 'plum', 'avocado', 'blueberry']
> ['lime', 'plum', 'avocado', 'blueberry']
Note that the sorted() returns a new list containing items from the iterable. It doesn't change the original iterable in the process.
Notice that in the result of the first sorted() call A come before a. This is because the ASCII value of A is 65 and that of a is 97. For the same reason space character (' '), ASCII value of 32 comes before A.
278
enumerate()
enumerate() function takes an iterable and returns an enumerate object (an iterator) that produces a tuple of the form (index, value), where index refers to the offset of the item and item refers to the corresponding item from the iterable.
Syntax:
enumerate(iterable[, start=0]
iterable - (required) Any iterable object like string, list, dictionary, etc.
start -initial value of the index. It defaults to 0.
e,g
list(enumerate("hello"))
> [(0, 'h'), (1, 'e'), (2, 'l'), (3, 'l'), (4, 'o')]
for index, value in enumerate("hello"):
print(index, value)
> 0 h
1 e
2 l
3 l
4 o
278
reversed()
reversed() function allows us to process the items in a sequence in reverse order. It accepts a sequence and returns an iterator.
Syntax:
reversed(sequence)
sequence -A sequence list string, list, tuple etc.
print( reversed([44, 11, -90, 55, 3]) )
>
print(list(reversed([44, 11, -90, 55, 3]))) # reversing a list
> [3, 55, -90, 11, 44]
To reverse user-defined objects the class must do one of the following:
1. Implement __len__() and __getitem__() methods
2. Implement __reversed__() method
279
range()
range() function is used to generate a sequence of numbers over time. At its simplest, it accepts an integer and returns a range object (a type of iterable).
Syntax:
range(start, stop, step)
start (optional) Starting point of the sequence. It defaults to 0.
stop (required) Endpoint of the sequence. This item will not be included in the sequence.
step (optional) Step size of the sequence. It defaults to 1.
Here range() is called with two arguments, 5 and 10. As a result, it will generate a sequence of numbers from 5 up to 10 (but not including 10).
e.g
print(list(range(5, 10)))
> [5, 6, 7, 8, 9]
print(list(range(1, 20, 3)))
> [1, 4, 7, 10, 13, 16, 19]
for i in range(5):
print(i)
> 0
1
2
3
4
280
sum()
function takes an iterable and returns the sum of items in it.
e.g.
sum([1, 2, 3, 4, 5]) # sum values in a list
> 15
It does sum elements of a list, tuple, set and dict. In dict the sum() adds the keys in the dictionary, ignoring its values.
281
max()
max() function returns the largest of the input values. It accepts iterable (list, tuple, dict). It can accept key argument.
Syntax:
max(iterable, default=obj, key=func)
iterable (required) An iterable object like string, list, tuple etc.
default (optional) The default value to return if the iterable is empty.
key (optional) It refers to the single argument function to customize the sort order. The function is applied to each item on the iterable.
e.g
print(max(("python", "lua", "ruby"), key=len))
> python
282
min()
function returns the smallest of the input values.
Syntax:
min(iterable[, default=obj, key=func]) -> value
iterable (required) An iterable object like string, list, tuple etc.
default (optional) - The default value to return if the iterable is empty.
key (optional) - It refers to the single argument function to customize the sort order. The function is applied to each item on the iterable.
min([2, -1, 4, 3])
> -1
min(("java", "python", "z++"), key=len)
'z++'
> 'z++'
283
eval()
eval() allows us to execute arbitrary strings as Python code. It accepts a source string and returns an object
Syntax:
eval(expr, globals=None, locals=None)
expr (required) can be any valid Python expression
globals (optional) Global namespace to use while executing the source. It must be a dictionary. If not provided then the current global namespace will be used.
locals (optional) Local namespace to use while executing the source. It can be any mapping. If omitted, it defaults to globals dictionary.
eval("5 == 5")
> True
eval("4 < 10")
> True
eval("'hello' + 'py'")
> 'hellopy'
The eval() is not just limited to simple expression. We can execute functions, call methods, reference variables and so on.
eval("abs(-11)")
> 11
globals = {
'a': 10,
'fruits': ['mangoes', 'peaches', ' bananas'],
}
locals = {}
eval("str(a) + ' ' + fruits[0]", globals, locals)
> '10 mangoes'
The key takeaway is that only use eval() with the trusted source.
284
len()
len() function counts the number of items in an object.
Syntax is as follows:
len(obj) -> length
obj can be a string, list, dictionary, tuple etc.
e.g
len({"spande", "club", "diamond", "heart"}) # length of set
> 4
def gen_func():
for i in range(5):
yield i
len(gen_func())
> Traceback (most recent call last):
File "main.py", line 5, in
len(gen_func())
TypeError: object of type 'generator' has no len()
As you can see above the len() function doesn't work with a generator. Trying to call len() on a generator object will result in TypeError exception
To use len() on user-defined objects you will have to implement the __len__() method.
e.g
class Stack:
def __init__(self):
self._stack = []
def push(self, item):
self._stack.append(item)
def pop(self):
self._stack.pop()
def __len__(self):
return len(self._stack)
285
ord()
ord() function (short of ordinal) returns an integer representing the character passed to it.
For ASCII characters, the returned value is 7-bit ASCII code, and for Unicode characters, it refers to the Unicode code point.
e.g
ord("A")
> 65
ord("😀") # Grinning Face
> 128512
To convert an integer returned by ord() back to its character equivalent we use chr() function.
286
chr()
chr() function returns a single character string represented by the integer ordinal value
chr(65)
> 'A
chr(128512) # Grinning Face
'😀'
287
any()
any() function tests whether any item in the iterable evaluates to True to not. It accepts an iterable and returns True, if at least one item in the iterable is true, otherwise, it returns False.
Syntax:
any(iterable) -> boolean
e.g
any([10, "", "one"])
> True
any(("", {}))
> False
any([])
> False
288
globals()
globals() function returns a dictionary containing the variables defined in the global namespace. When globals() is called from a function or method, it returns the dictionary representing the global namespace of the module where the function or method is defined, not from where it is called
zwraca słownik zawierający zmienne zdefiniowane globalnie
e.g
from pprint import pprint
a = 100
b = 4
def foo():
x = 100 # x is a local variable
pprint(globals())
> 'a': 100,
'b': 4
...
289
locals()
locals() function returns a dictionary containing the variables defined in the local namespace. Calling locals() in the global namespace is same as calling globals() and returns a dictionary representing the global namespace of the module.
zwraca słownik zawierający zmienne zdefiniowane lokalnie
290
all()
all() function tests whether all items in the iterable evaluates to True or not. It accepts an iterable and returns True if all the item is true, otherwise, it returns False.
e.g
all(['alpha', 'beta', ''])
> False
all(['one', 'two', 'three'])
> True
291
MySQLdb
MySQLdb is an api for accessing MySQL database using python. It is built on top of MySQL C API.
MySQLdb don't yet have support for python 3, it supports only python 2.4 - 2.7 . As a result you need to use python 2 for this tutorial. We will be using python 2.7.9, which you can download from here.
292
There are two types of files you can open and work with in Python:
- Text files
- Binary files
A text file is simply a file which stores sequences of characters using an encoding like utf-8, latin1 etc., whereas in the case of binary file data is stored in the same format as in Computer memory.
Text files: Python source code, HTML file, text file, markdown file etc.
Binary files: executable files, images, audio etc.
It is important to note that inside the disk both types of files are stored as a sequence of 1s and 0s. The only difference is that when a text file is opened the data is decoded back using the same encoding scheme they were encoded in. However, in the case of binary files no such thing happens.
293
open()
built-in function is used to open the file. Its syntax is as follows:
open(filename, mode) -> file object
filename (requires) absolute or relative path of the file to be opened.
mode (optional) mode is a string which refers to the processing mode (i.e read, write, append etc;) and file type.
Modes:
r Open the file for reading (default).
w Open the file for writing.
a Open the file in append mode i.e add new data to the end of the file.
r+ Open the file for reading and writing both
x Open the file for writing, only if it doesn't already exist.
We can also append t or b to the mode string to indicate the type of the file we will be working with. The t is used for text file and b for binary files. If neither specified, t is assumed by default:
e.g
open('todo.md', 'rt')
294
read([n])
= open("poem.txt", "r")
f.read(3) # read the first 3 characters
>'The'
Reads and returns n bytes or less (if there aren't enough characters to read) from the file as a string. If n not specified, it reads the entire file as a string and returns it.
295
readline()
Reads and returns the characters until the end of the line is reached as a string.
= open("poem.txt", "r")
f.read(4) # read the first 4 characters
>'The '
f.readline() # read until the end of the line is reached
> 'caged bird sings\n'
296
readlines()
= open("poem.txt", "r")
f.readlines()
> ['The caged bird sings\n', 'with a fearful trill\n', 'of things unknown\n', 'but longed for still\n']
297
How to read file in chunks?
Reading File in Chunks :
f = open("poem.txt", "r")
chunk = 200
while True:
data = f.read(chunk)
if not data:
break
print(data)
> The caged bird sings
with a fearful trill
of things unknown
but longed for still
298
write(s)
Writes the string s to the file and returns the number characters written.
e.g
f.write("When I think about myself, ")
> 26
Unlike the print() function the write() method doesn't add a newline character (\n) at the end of the line. If you want the newline character you have to add it manually.
299
writelines(s)
Writes all strings in the sequence s to the file.
e.g
>>> lines = [
"Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod",
"tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,"
"quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo",
"consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse",
"cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non",
"proident, sunt in culpa qui officia deserunt mollit anim id est laborum."
]
f = open("lorem.txt", "w")
f.writelines(lines)
f.close()
Notice that unlike the print() function the write() method doesn't add a newline character (\n) at the end of the line. If you want the newline character you have to add it manually (adding \n at the end of each line)
300
What is buffering?
The open() method provides an optional third argument to control the buffer.
Buffering is the process of storing data temporarily in the PC memory before it is moved to a new location (e.g. a disk)
When we use a buffer, the data is written to the disk only when the buffer becomes full or when the close() method is called.
This process is called flushing the output. You can also flush the output manually using the flush() method of the file object. Note that the flush() only saves the buffered data to the disk. It doesn't close the file.
301
Reading and writing binary file is done by appending ___ to the mode string.
b
In Python 3, the binary data is represented using a special type called bytes.
The bytes type represents an immutable sequence of numbers between 0 and 255.
e.g
binary_poem = bytes(open("poem.txt").read(), encoding="utf-8")
e.g
f = open("binary_poem", "wb")
f.write(binary_poem)
> 80
f.close()
with binary data we can't reliably use readline() and file object (as an iterator) to read the contents of a file because might be no newline character in a file. The best way to read binary data is to read it in chunks using the read() method
302
How to read and write file at specific locations using Python?
To achieve this we have to use following methods:
tell() which returns the current position of the file pointer.
seek(offset, [whence=0]) which moves the file pointer to the given offset.
offset - określa gdzie jest ustawiony file pointer. Wyraża się go w liczbie bajtów
whence - skąd (domyślnie 0, czyli od początku pliku)
When we move the pointer the next read (or write) operation will start from this point.
Możemy użyć f.seek() to przesunięcia file pointera na początek pliku w następujący sposób:
f.seek(0) # rewind the file pointer to the beginning, same as seek(0, 0)
> 0
Potem możemy sprawdzić za pomocą f.tell() czy na pewno dobrze się przesunęliśmy,
f.tell()
> 0
303
What is writing text/binary file in python?
To write to an existing file, you must add a parameter to the open() function:
"a" - Append - will append to the end of the file
"w" - Write - will overwrite any existing content
304
following three calls to open() are equivalent or not?
open file todo.md for reading in text mode
open('todo.md')
open('todo.md', 'r')
open('todo.md', 'rt')
Yes, because 'r' (read mode) and 't' (text file) are default.
The mode is optional, if not specified then the file will be opened as a text file for reading only.Note that before you can read a file, it must already exist, otherwise open() will raise FileNotFoundError exception. However, if you open a file for writing (using mode such as w, a, or r+), Python will automatically create the file for you. If the file already exists then its content will be deleted. If you want to prevent that open the file in x mode.
305
close()
Although, the file is automatically closed when the program ends but it is still a good practice to do so explicitly. Failing to close the file in a large program could be problematic and may even cause the program to crash.
To close the file call the close() method of the file object. Closing the file frees up resources associated with it and flushes the data in the buffer to the disk.
306
What is a file pointer?
When you open a file via the open() method. The operating system associates a pointer that points to a character in the file. The file pointer determines from where the read and write operation will take place.
307
EOF
End Of the File
308
tell()
Returns the current position of the file pointer.
309
seek(offset, [whence=0])
Moves the file pointer to the given offset. The offset refers to the byte count and whence determines the position relative to which the offset will move the file pointer.
The default value of whence is 0, which means that offset will move the file pointer from the beginning of the file. If whence is set to 1 or 2, the offset will move the file's pointer from the current position or from the end of the file, respectively.
310
we can use _____ method to rewind the file pointer to the beginning of the file
fseek()
e.g
f.seek(0) # rewind the file pointer to the beginning, same as seek(0, 0)
> 0
f.tell()
> 0
f.read(14) # read the first 14 characters
> b'The caged bird'
f.tell()
> 14
311
To move the file pointer from 12 bytes forward from current position call ______ method
seek()
f.tell()
> 14
f.seek(12, 1)
> 26
f.tell()
> 26
We can also move the file pointer backward. For example, the following call to seek() moves the file pointer 13 bytes backward from the current position.
f.seek(-13, 1)
> 13
312
The _____ allows us to automatically close the file once we are done working with it.
with statement
e.g
with open('poem.txt') as f:
print(f.read()) # read the entire file
313
What is CSV File?
CSV (Comma-separated values) is a common data exchange format used by the applications to produce and consume data. Some other well-known data exchange formats are XML, HTML, JSON etc.
A CSV file is a simple text file where each line contains a list of values (or fields) delimited by commas.
Although the term "Comma" appears in the format name itself, but you will encounter CSV files where data is delimited using tab (\t) or pipe (|) or any other character that can be used as a delimiter.
The first line of the CSV file represents the header containing a list of column names in the file. The header is optional but highly recommended.
314
Using commas in CSV files as delimiter:
If the values in your CSV file contains commas, then it must be enclosed inside ___________
double quotes
Name,Age,Address
Jerry,10,"2776 McDowell Street, Nashville, Tennessee"
Tom,20,"3171 Jessie Street, Westerville, Ohio"
Mike,30,"1818 Sherman Street, Hope, Kansas"
315
If you have double quotes embedded inside a field, it must be escaped with another __________ . Otherwise, they will not be interpreted correctly.
double quotes
e.g
Id,User,Comment
1,Bob,"John said ""Hello World"""
2,Tom,""The Magician""
316
The ____ module is used for reading and writing CSV files. It mainly provides following classes and functions:
reader()
writer()
DictReader()
DictWriter()
csv
317
csv module
reader()
The reader() function takes a file object and returns a _csv.reader object that can be used to iterate over the contents of a CSV file.
reader(fileobj, dialect='excel', fmtparam)
fileobj - (required) It refers to the file object, which we want to read
dialect - (optional) Dialect refers to the different ways of formatting the CSV document. By default, the csv module uses the same format as Microsoft Excel.
fmtparam - (optional) It refers to the set of keyword arguments to customize the dialect (see the next section).
e.g
import csv
with open('employees.csv', 'rt') as f:
csv_reader = csv.reader(f)
for line in csv_reader:
print(line)
> ['id', 'name', 'email', 'age', 'designation']
['1', 'John', 'john@mail.com', '24', 'programmer']
['2', 'Bob', 'bob@mail.com', '34', 'designer']
['3', 'Mary', 'mary@mail.com', '43', 'sales']
Notice that each line in the CSV file is returned as a list of strings.
318
Hoe to get data from certain fields using csv.reader() method?
Because each line in the CSV file is returned as a list of strings, to get the data from certain fields, you can use indexing.
e.g
import csv
with open('employees.csv', 'rt') as f:
csv_reader = csv.reader(f)
for line in csv_reader:
print(line[0], line[1], line[2])
> id name email
1 John john@mail.com
2 Bob bob@mail.com
3 Mary mary@mail.com
319
How to skip the heading when reading a csv file?
If you want to skip heading call the next() built-in function on the _csv.reader object and then loop over the remaining lines as usual.
e.g
import csv
with open('employees.csv', 'rt') as f:
csv_reader = csv.reader(f)
next(csv_reader) # skip the heading
for line in csv_reader:
print(line[0], line[1], line[2])
> 1 John john@mail.com
2 Bob bob@mail.com
3 Mary mary@mail.com
320
How to modify the dialect of csv files when using csv module?
By default, the csv module works according to the dialect used by Microsoft excel, but you can also define your own format using some additional arguments that you can pass to the reader() function to customize its working
e.g
delimiter - It refers to the character used to separate values (or fields) in the CSV file. It defaults to comma (,).
skipinitialspace - It controls how the space following the delimiter will be interpreted. If True , the initial whitespaces will be removed. It defaults to False
lineterminator - It refers to the character sequence used to terminate the line. It defaults to \r\n.
quotechar - It refers to the single character string that will be used to quote values if special characters (like delimiter) appears inside the field. It defaults to ".
quoting - controls when quotes should be generated by the writer or recognized by the reader. It can take one of the following constants:
csv.QUOTE_MINIMAL means add quote only when required, for example, when a field contains either the quotechar or the delimiter. This is the default.
csv.QUOTE_ALL means quotes everything regardless of the field type.
csv.QUOTE_NONNUMERIC means quotes everything except integers and floats.
csv.QUOTE_NONE means that do not quote anything on output. However, while reading quotes are included around the field values.
321
How to change delimiter using csv.reader() function?
you pass delimiter argument to the reader() function
e.g
import csv
with open('employees.csv', 'rt') as f:
csv_reader = csv.reader(f, delimiter='|')
for line in csv_reader:
print(line)
> ['id', 'name', 'email', 'age', 'designation']
['1', 'John', 'john@mail.com', '24', 'programmer']
['2', 'Bob', 'bob@mail.com', '34', 'designer']
['3', 'Mary', 'mary@mail.com', '43', 'sales']
322
How to remove spaces following the delimiter when reading csv file?
you pass skipinitialspace argument to the reader() function
e.g
import csv
with open('baseball_players.csv', 'rt') as f:
csv_reader = csv.reader(f, skipinitialspace=True)
323
When reading a csv file, how to change quote character to a single quote ?
you pass quotechar argument to the reader() function
e.g
import csv
with open('housing.csv', 'rt') as f:
csv_reader = csv.reader(f, skipinitialspace=True, quotechar="'")
for line in csv_reader:
print(line)
324
Sometimes, a csv files use some character (e.g /) as something that indicates the reader that it should escape embedded double quotes.
e.g we face something like this:
> ['Id', 'User', 'Comment']
['1', 'Bob', 'John said \\Hello World\\""']
['2', 'Tom', '\\The Magician\\""']
['3', 'Harry', '\\walk around the corner\\" she explained to the child"']
['4', 'Louis', 'He said, \\stop pulling the dog\'s tail\\""']
This output is certainly not desirable. To get the correct output change the escape character using _____ argument of _____ method
escapechar argument, csv.reader() method
e.g.
import csv
with open('employees.csv', 'rt') as f:
csv_reader = csv.reader(f, skipinitialspace=True, escapechar='\\')
for line in csv_reader:
print(line)
> ['Id', 'User', 'Comment']
['1', 'Bob', 'John said "Hello World"']
['2', 'Tom', '"The Magician"']
['3', 'Harry', '"walk around the corner" she explained to the child']
['4', 'Louis', 'He said, "stop pulling the dog\'s tail"']
325
Some files use double quote to escape the embedded double quote characters in the field.
e.g
Id, Actor, Dialogue
1, Harley Betts, "The suspect told the arresting officer, ""I was nowhere near the crime."""
2, Clyde Esparza, "John said, ""I have just finished reading Browning's 'My Last Duchess.'"""
3, Zack Campbell, "Bill asked Sandra, ""Will you marry me?"""
4, Keziah Chaney, "The librarian whispered to us, ""The sign on the wall says 'Quiet'"""
To change it set _______ argument of _______ method to _______
doublequote argument, csv.reader() method, True
e.g
import csv
with open('employees.csv', 'rt') as f:
# same as csv_reader = csv.reader(f, skipinitialspace=True)
csv_reader = csv.reader(f, skipinitialspace=True, doublequote=True)
for line in csv_reader:
print(line)
>['Id', 'Actor', 'Dialogue']
['1', 'Harley Betts', 'The suspect told the arresting officer, "I was nowhere near the crime."']
['2', 'Clyde Esparza', 'John said, "I have just finished reading Browning\'s \'My Last Duchess.\'"']
['3', 'Zack Campbell', 'Bill asked Sandra, "Will you marry me?"']
['4', 'Keziah Chaney', 'The librarian whispered to us, "The sign on the wall says \'Quiet\'"']
326
To write data to a CSV file we use the _____ function from ____ module
writer() function, csv module
It accepts the same argument as the reader()
fileobj - (required) It refers to the file object
dialect - (optional) Dialect refers to the different ways of formatting the CSV document. By default, the csv module uses the same format as Microsoft Excel.
fmtparam - (optional) Formatting parameters, work same as the reader()'s function.
The writer instance provides the following two methods to write data:
writerow(row) - Writes a single row of data and returns the number of characters written. The row must be a sequence of strings and number.
writerows(rows) - Writes multiple rows of data and returns None. The rows must be a sequence.
327
How to use writerow() and writerows methods for a csv file?
import csv
header = ['id', 'name', 'address', 'zip']
rows = [
[1, 'Hannah', '4891 Blackwell Street, Anchorage, Alaska', 99503 ],
[2, 'Walton', '4223 Half and Half Drive, Lemoore, California', 97401 ],
[3, 'Sam', '3952 Little Street, Akron, Ohio', 93704],
[4, 'Chris', '3192 Flinderation Road, Arlington Heights, Illinois', 62677],
[5, 'Doug', '3236 Walkers Ridge Way, Burr Ridge', 61257],
]
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(header) # write header
csv_writer.writerows(rows)
328
csv module
How to write a multiple rows using writerows()?
import csv
header = ['id', 'name', 'address', 'zip']
rows = [
[1, 'Hannah', '4891 Blackwell Street, Anchorage, Alaska', 99503 ],
[2, 'Walton', '4223 Half and Half Drive, Lemoore, California', 97401 ],
[3, 'Sam', '3952 Little Street, Akron, Ohio', 93704],
[4, 'Chris', '3192 Flinderation Road, Arlington Heights, Illinois', 62677],
[5, 'Doug', '3236 Walkers Ridge Way, Burr Ridge', 61257],
]
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(header) # write header
for row in rows:
csv_writer.writerow(row)
329
In the following example:
customers.csv
> id,name,address,zip
1,Hannah,"4891 Blackwell Street, Anchorage, Alaska",99503
2,Walton,"4223 Half and Half Drive, Lemoore, California",97401
3,Sam,"3952 Little Street, Akron, Ohio",93704
4,Chris,"3192 Flinderation Road, Arlington Heights, Illinois",62677
5,Doug,"3236 Walkers Ridge Way, Burr Ridge",61257
only the address field is wrapped around double quotes. What can be the cause?
This is because by default the quoting argument of the csv.writer() method is set to QUOTE_MINIMAL.
In other words, fields will be quoted only when quotechar or delimiter appears in the data.
If you want double quotes around all textual data set quoting argument to QUOTE_NONNUMERIC.
e.g
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
> "id","name","address","zip"
1,"Hannah","4891 Blackwell Street, Anchorage, Alaska",99503
2,"Walton","4223 Half and Half Drive, Lemoore, California",97401
3,"Sam","3952 Little Street, Akron, Ohio",93704
4,"Chris","3192 Flinderation Road, Arlington Heights, Illinois",62677
5,"Doug","3236 Walkers Ridge Way, Burr Ridge",61257
330
If you want to quote all fields regardless of data type set ______ to _______
set quoting to csv.QUOTE_ALL
e. g.
import csv
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f, quoting=csv.QUOTE_ALL)
> "id","name","address","zip"
"1","Hannah","4891 Blackwell Street, Anchorage, Alaska","99503"
"2","Walton","4223 Half and Half Drive, Lemoore, California","97401"
"3","Sam","3952 Little Street, Akron, Ohio","93704"
"4","Chris","3192 Flinderation Road, Arlington Heights, Illinois","62677"
"5","Doug","3236 Walkers Ridge Way, Burr Ridge","61257"
331
How to change the quote character from double quote (") to a single quote (') when using csv.writer()?
import csv
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f, quotechar="'")
> id,name,address,zip
1,Hannah,'4891 Blackwell Street, Anchorage, Alaska',99503
2,Walton,'4223 Half and Half Drive, Lemoore, California',97401
3,Sam,'3952 Little Street, Akron, Ohio',93704
4,Chris,'3192 Flinderation Road, Arlington Heights, Illinois',62677
5,Doug,'3236 Walkers Ridge Way, Burr Ridge',61257
332
How to turn off quoting all-together when using csv.writer()?
We can turn off quoting all-together by setting quoting to csv.QUOTE_NONE. However, if you do that and delimiter character appears in the data then you will get an error like this:
> Traceback (most recent call last):
...
csv_writer.writerows(rows)
_csv.Error: need to escape, but no escapechar set
The problem is that the address field contains embedded commas (,) and since we have turned off the ability to quote fields, the csv module doesn't know how to escape them properly.
333
csv module
How to escape fields properly, when we just used csv.QUOTE_NONE?
This where the escapechar argument comes into play. We have to set the escapechar to e. g backslash (\):
import csv
with open('customers.csv', 'wt') as f:
csv_writer = csv.writer(f, quoting=csv.QUOTE_NONE, escapechar='\\')
> id,name,address,zip
1,Hannah,4891 Blackwell Street\, Anchorage\, Alaska,99503
2,Walton,4223 Half and Half Drive\, Lemoore\, California,97401
3,Sam,3952 Little Street\, Akron\, Ohio,93704
4,Chris,3192 Flinderation Road\, Arlington Heights\, Illinois,62677
5,Doug,3236 Walkers Ridge Way\, Burr Ridge,61257
334
What is DictReader?
DictReader is used for reading a csv files. It works almost exactly like reader() but instead of retuning a line as a list, it returns a dictionary.
Syntax:
DictReader(fileobj, fieldnames=None, restkey=None, restval=None, dialect='excel', fmtparam)
fileobj - (required) It refers to the file object.
fieldnames - (optional) It refers to the list of keys that will be used in the returned dictionary in order. If omitted, the field names are inferred from the first row of the CSV file.
restkey - (optional) If the row has more fields than specified in the fieldnames parameter, then the remaining fields is stored as a sequence keyed by the value of restkey argument.
restval - (optional) It provides value to fields which are missing from the input.
dialect - (optional) Dialect refers to the different ways of formatting the CSV document. By default, the csv module uses the same format as Microsoft excel.
fmtparam - It refers to formatting arguments and works exactly like reader() and writer()
335
How to use DictReader?
import csv
with open('customers.csv', 'rt') as f:
csv_reader = csv.DictReader(f)
for row in csv_reader:
print(row)
>'id': '1', 'name': 'Hannah', 'zip': '99503', 'address': '4891 Blackwell Street, Anchorage, Alaska'}
{'id': '2', 'name': 'Walton', 'zip': '97401', 'address': '4223 Half and Half Drive, Lemoore, California'}
{'id': '3', 'name': 'Sam', 'zip': '93704', 'address': '3952 Little Street, Akron, Ohio'}
{'id': '4', 'name': 'Chris', 'zip': '62677', 'address': '3192 Flinderation Road, Arlington Heights, Illinois'}
{'id': '5', 'name': 'Doug', 'zip': '61257', 'address': '3236 Walkers Ridge Way, Burr Ridge'}
Headears become keys
336
How to use fieldnames parameter in DictReader()?
import csv
with open('customers.csv', 'rt') as f:
fields = ['id', 'name', 'address', 'zip']
csv_reader = csv.DictReader(f, fieldnames=fields)
for row in csv_reader:
print(row)
> {'name': 'Hannah', 'zip': '99503', 'id': '1', 'address': '4891 Blackwell Street, Anchorage, Alaska'}
{'name': 'Walton', 'zip': '97401', 'id': '2', 'address': '4223 Half and Half Drive, Lemoore, California'}
{'name': 'Sam', 'zip': '93704', 'id': '3', 'address': '3952 Little Street, Akron, Ohio'}
{'name': 'Chris', 'zip': '62677', 'id': '4', 'address': '3192 Flinderation Road, Arlington Heights, Illinois'}
{'name': 'Doug', 'zip': '61257', 'id': '5', 'address': '3236 Walkers Ridge Way, Burr Ridge'}
337
How to use restkey parameter in DictReader()?
We can use optional restkey parameter to a DictReader() function when the row has more fields than specified in the fieldnames parameter
e.g
import csv
with open('customers.csv', 'rt') as f:
fields = ['id','name',]
csv_reader = csv.DictReader(f, fieldnames=fields, restkey='extra')
for row in csv_reader:
print(row)
> {'id': '1', 'name': 'Hannah', 'extra': ['4891 Blackwell Street, Anchorage, Alaska', '99503']}
{'id': '2', 'name': 'Walton', 'extra': ['4223 Half and Half Drive, Lemoore, California', '97401']}
{'id': '3', 'name': 'Sam', 'extra': ['3952 Little Street, Akron, Ohio', '93704']}
{'id': '4', 'name': 'Chris', 'extra': ['3192 Flinderation Road, Arlington Heights, Illinois', '62677']}
{'id': '5', 'name': 'Doug', 'extra': ['3236 Walkers Ridge Way, Burr Ridge', '61257']}
338
How to use restval parameter in DictReader()?
We can use optional resval parameter to a DictReader() function when we want to save the place for additional data which we don't yet have.
e.g we have a csv file with id, name, and address of customers, but we want an additional place for email and phone
import csv
with open('customers.csv', 'rt') as f:
fields = ['id','name', 'address', 'zip', 'phone', 'email'] # two extra fields
csv_reader = csv.DictReader(f, fieldnames=fields, restkey='extra', restval='NA')
for row in csv_reader:
print(row)
> {'id': '1', 'name': 'Hannah', 'email': 'NA', 'phone': 'NA', 'address': '4891 Blackwell Street, Anchorage, Alaska', 'zip': '99503'}
{'id': '2', 'name': 'Walton', 'email': 'NA', 'phone': 'NA', 'address': '4223 Half and Half Drive, Lemoore, California', 'zip': '97401'}
{'id': '3', 'name': 'Sam', 'email': 'NA', 'phone': 'NA', 'address': '3952 Little Street, Akron, Ohio', 'zip': '93704'}
{'id': '4', 'name': 'Chris', 'email': 'NA', 'phone': 'NA', 'address': '3192 Flinderation Road, Arlington Heights, Illinois', 'zip': '62677'}
{'id': '5', 'name': 'Doug', 'email': 'NA', 'phone': 'NA', 'address': '3236 Walkers Ridge Way, Burr Ridge', 'zip': '61257'}
339
What is DictWriter?
The DictWriter object writes a dictionary to a CSV file. Its syntax is as follows:
DictWriter(fileobj, fieldnames, restval='', extrasaction='raise', dialect='excel', fmtparam)
fileobj - It refers to the file object
fieldnames - It refers to the field names and the order in which they will be written the file.
restval - It provides the missing value for the keys which doesn't exist in the dictionary.
extrasaction - It controls what action to take if the dictionary contains a key, that is not found in the fieldnames argument. By default, extrasaction is set to raise, which means an exception will be raised in such an event. If you want to ignore the extra values set extrasaction to ignore.
The DictWriter provides the following three methods to write data.
writeheader()
writerow(row)
writerows(rows)
340
writeheader()
DictWriter metod
Writes the header (i.e fieldnames) to the CSV file and returns None.
341
writerow(row)
DictWriter metod
Writes a single row of data and returns the number of characters written. The row must be a sequence of strings and number.
342
writerows(rows)
DictWriter metod
Writes multiple rows of data and returns None. The rows must be a sequence.
343
How to use DictWriter()?
import csv
header = ['id', 'name', 'address', 'zip', 'email'] # an extra field email
rows = [
{'id': 1, 'name': 'Hannah', 'address': '4891 Blackwell Street, Anchorage, Alaska', 'zip': 99503 },
{'id': 2, 'name': 'Walton', 'address': '4223 Half and Half Drive, Lemoore, California', 'zip': 97401 },
{'id': 3, 'name': 'Sam', 'address': '3952 Little Street, Akron, Ohio', 'zip': 93704 },
{'id': 4, 'name': 'Chris', 'address': '3192 Flinderation Road, Arlington Heights, Illinois', 'zip': 62677},
{'id': 5, 'name': 'Doug', 'address': '3236 Walkers Ridge Way, Burr Ridge', 'zip': 61257},
]
with open('dictcustomers.csv', 'wt') as f:
csv_writer = csv.DictWriter(f, fieldnames=header, restval="NA")
csv_writer.writeheader() # write header
csv_writer.writerows(rows)
344
What to do, when we want to use DictWrite() on a csv file, but in this file we have more headers than we want in our dictionary?
To prevent the exception from being raised we have set extrasaction to ignore.
e.g
import csv
header = ['id', 'name', 'address'] # notice zip is missing
rows = [
{'id': 1, 'name': 'Hannah', 'address': '4891 Blackwell Street, Anchorage, Alaska', 'zip': 99503 },
{'id': 2, 'name': 'Walton', 'address': '4223 Half and Half Drive, Lemoore, California', 'zip': 97401 },
{'id': 3, 'name': 'Sam', 'address': '3952 Little Street, Akron, Ohio', 'zip': 93704 },
{'id': 4, 'name': 'Chris', 'address': '3192 Flinderation Road, Arlington Heights, Illinois', 'zip': 62677},
{'id': 5, 'name': 'Doug', 'address': '3236 Walkers Ridge Way, Burr Ridge', 'zip': 61257},
]
with open('dictcustomers.csv', 'wt') as f:
csv_writer = csv.DictWriter(
f,
fieldnames=header,
restval="NA",
extrasaction='ignore' # ignore extra values in the dictionary
)
csv_writer.writeheader() # write header
csv_writer.writerows(rows)
345
What is a Dialect when considering a csv file?
A dialect object or (simply dialect) is a way to group various formatting parameters. Once you have created the dialect object, simply pass it to the reader or writer, rather than passing each formatting argument separately.
To create a new dialect, we use register_dialect() function. It accepts dialect name as a string and one or more formatting parameters as keyword arguments.
346
Consider formatting your csf file (creating a Dialect). Do you know what following terms mean? And what are their default values?
delimiter, skipinitialspace, lineterminator, quotechar, quoting, escapechar, doublequote
delimiter (,) - It refers to the character used to separate values (or fields) in the CSV file.
skipinitialspace (False) - It controls how the space following the delimiter will be interpreted. If True, the initial whitespaces will be removed.
lineterminator (\r\n) - It refers to the character sequence used to terminate the line.
quotechar (") - It refers to the single character string that will be used to quote values if special characters (like delimiter) appears inside the field.
quoting (csv.QUOTE_NONE) - controls when quotes should be generated by the writer or recognized by the reader (see above for other options).
escapechar (None) - It refers to the one-character string used to escape the delimiter when quoting is set to csv.QUOTE_NONE
doublequote (True) - controls the handling of quotes inside fields. When True, two consecutive quotes are interpreted as one during read, and when writing, each quote character embedded in the data is written as two quotes.
347
How to use register_dialect() function?
import csv
create and register new dialect
csv.register_dialect('psv', delimiter='|', quoting=csv.QUOTE_NONNUMERIC)
header = ['id', 'year', 'age', 'name', 'movie']
rows = [
{'id': 1, 'year': 2013, 'age': 55, 'name': "Daniel Day-Lewis", 'movie': "Lincoln" },
{'id': 2, 'year': 2014, 'age': 44, 'name': "Matthew McConaughey", 'movie': "Dallas Buyers Club" },
{'id': 3, 'year': 2015, 'age': 33, 'name': "Eddie Redmayne", 'movie': "The Theory of Everything" },
{'id': 4, 'year': 2016, 'age': 41, 'name': "Leonardo DiCaprio", 'movie': "The Revenant" }
]
with open('oscars.csv', 'wt') as f:
csv_writer = csv.DictWriter(
f,
fieldnames=header,
dialect='psv', # pass the new dialect
extrasaction='ignore'
)
csv_writer.writeheader() # write header
csv_writer.writerows(rows)
348
What is JSON?
JSON (JavaScript Object Notation) is language-neutral data interchange format. It was created and popularized by Douglas Crockford. In its short history, JSON has become a defacto standard for data transfer across the web.
JSON is a text-based format which is derived from JavaScript object syntax. However, it is completely independent of JavaScript, so you don't need to know any JavaScript to use JSON.
JSON is commonly used by web applications to transfer data between client and server. If you are using a web service then there are good chances that data will be returned to you in JSON format, by default.
JSON format closely resembles to dictionaries in Python.
349
What is XML?
Before the inception of JSON, XML was predominantly used to send and receive data between the client and the server. The problem with XML is that that it is verbose, heavy and not easy to parse. However, this is not the case with JSON, as you will see soon.
350
What is Serialization?
Serialization: The process of converting an object into a special format which is suitable for transmitting over the network or storing in file or database is called Serialization.
In the case of JSON, when we serializing objects, we essentially convert a Python object into a JSON string and deserialization builds up the Python object from its JSON string representation.
351
What is Deserialization?
Deserialization: It is the reverse of serialization. It converts the special format returned by the serialization back into a usable object.
In the case of JSON, when we serializing objects, we essentially convert a Python object into a JSON string and deserialization builds up the Python object from its JSON string representation.
352
Python provides a built-in module called ______ for serializing and deserializing objects
json
The json module mainly provides the following functions for serializing and deserializing:
1. dump(obj, fileobj)
2. dumps(obj)
3. load(fileobj)
4. loads(s)
353
json module:
dump()
The dump() function is used to serialize data. It takes a Python object, serializes it and writes the output (which is a JSON string) to a file like object.
e.g
import json
python object
person = {
'first_name': "John",
"isAlive": True,
"age": 27,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"hasMortgage": None
}
writing JSON object
with open('person.json', 'w') as f:
json.dump(person, f)
354
Python Type to JSON Type conversion examples
dict - object
list, tuple - array
int - number
float - number
str - string
True - true
False - false
None - null
355
json module:
load()
The load() function deserializes the JSON object from the file like object and returns it.
e.g
with open('person.json', 'r') as f:
person = json.load(f)
356
json module:
dumps()
# python object
The dumps() function works exactly like dump() but instead of sending the output to a file-like object, it returns the output as a string.
e.g
# python object
person = {
'first_name': "John",
"isAlive": True,
"age": 27,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"hasMortgage": None
}
#serialize to string
data = json.dumps(person)
#json data shown as a string
> data
'{"hasMortgage": null, "isAlive": true, "age": 27, "address": {"state": "NY", "streetAddress": "21 2nd Street", "city": "New York", "postalCode": "10021-3100"}, "first_name": "John"}'
357
json module:
loads()
loads() function is as same as load() but instead of deserializing the JSON string from a file, it deserializes from a string.
e.g
# json shown as string
> data
'{"hasMortgage": null, "isAlive": true, "age": 27, "address": {"state": "NY", "streetAddress": "21 2nd Street", "city": "New York", "postalCode": "10021-3100"}, "first_name": "John"}'
deserialize from string
person = json.loads(data)
358
List some optional keyword arguments that can be passed to the dumps or dump() function to customize the Serializer.
indent, sort_keys, skipkeys, separators
359
dump() function optional arguments:
indent
indent - a positive integer which determines the amount of indentation of key-value pairs at each level. The indent arguments come in handy to prettify the output if you have deeply nested data structures. The default value of indent is None.
e.g
print(json.dumps(person, indent=4)) # with 4 levels of indentation
360
dump() function optional arguments:
sort_keys
A boolean flag, if set to True returns a JSON string ordered by keys, instead of being randomly ordered. Its default value is False.
e.g
# print JSON string in order by keys
print(json.dumps(person, indent=4, sort_keys=True))
361
dump() function optional arguments:
skipkeys
JSON format expects the keys to be a string, if you try to use a type which can't be converted to a string (like tuple) then a TypeError exception will be raised. To prevent the exception from being raised and skip the non-string keys set the skipkeys argument to True.
e.g
data = {'one': 1, 'two': 2, (1,2): 3}
json.dumps(data, indent=4)
> Traceback (most recent call last):
...
TypeError: key (1, 2) is not a string
To prevent the exception from being raised and skip the non-string keys use the skipkeys argument
print(json.dumps(data, indent=4, skipkeys=True))
362
dump() function optional arguments:
separators
It refers to a tuple of the form (item_separator, key_separator). The item_separator is a string which is used to separate items in a list. The key_separator is also a string and is used to separate keys and values in a dictionary. By default, the separators set to (',', ': ').
e.g
#The following example changes and item_separator and key_separator to pipe (|) and dash (-) characters respectively:
print(json.dumps(employee, indent=4, skipkeys=True, separators=('|', '-')))
363
To serialize custom objects or built-in types, we have to ________________________________________________
create our own serialization function:
def serialize_objects(obj):
... if isinstance(obj, datetime):
... return {
... '__class__': datetime.__name__,
... '__value__': str(obj)
... }
... raise TypeError(str(obj) + ' is not JSON serializable')
The function takes a single argument named obj, check the type of the object using the isinstance() function.
Next, it creates a dictionary with two keys: __class__ and __value__.
The __class__ key stores the original name of the class and will be used to deserialize the data.
The __value__ key stores the value of the object, in this case, we are simply converting datetime.datetime object to its string representation using the built-in str() function.
In the last line, we raise TypeError exception. This is necessary otherwise our serialization function wouldn't report errors for objects it can't serialize.
We can pass custom serialization function to dumps() or dump() using the default keyword argument
e.g
# python object:
employee = {
'first_name': "Mike",
"designation": 'Manager',
"doj": datetime(year=2016, month=5, day=2), # date of joining
}
emp_json = json.dumps(employee, indent=4, default=serialize_objects)
364
To deserialize custom objects or built-in types, we have to ________________________________________________
create our own deserialization function:
def deserialize_objects(obj):
... if '__class__' in obj:
... if obj['__class__'] == 'datetime':
... return datetime.strptime(obj['__value__'], "%Y-%m-%d %H:%M:%S")
... # if obj['__class__'] == 'Employee':
... # return Employee(obj['__value__'])
... return obj
365
It turns out that the_____ module is not the only way to serialize data. Python provides another module called ______ to serialize and deserialize data.
json, pickle
366
4 main differences between the json and pickle module
1. Pickle is Python specific
2. Pickle serializes to binary
3. Pickle allows to serialize custom objects
4. Pickle is fast
Ad 1. The pickle module is Python-specific which means that once the object is serialized you can't deserialize it using another language like PHP, Java, Perl etc. If interoperability is what you need stick to the json module.
Ad 2. Unlike json module which serializes objects as human-readable JSON string, the pickle module serializes data in the binary format.
Ad 3. The json module allows us to serialize only the basic Python types (like int, str, dict, list etc.). If you need to serialize custom objects you would have to supply your own serialization function. However, the pickle module works with a wide variety of Python types right out of the box, including the custom objects you define.
Ad 4. Most of the pickle module is coded in C. So it provides a great performance boost while handling large data sets as compared to the json module.
367
How to serialize data with pickle?
# First, we open the file in binary mode instead of text mode:
Pickling data is done via the dump() function. It accepts data and a file object. The dump() function then serializes the data and writes it to the file.
e.g
import pickle
f = open("my_pickle", "wb")
Second, the dump() function is able to serialize the data:
pickle.dump("a string", f)
f.close()
368
How to deserialize data with pickle?
The load() function takes a file object, reconstruct the objects from the pickled representation, and returns it.
f = open("my_pickle", "rb")
pickle.load(f)
'a string'
Once you have unpickled the data you can use it like an ordinary Python object.
369
pickle module
dumps()
The dumps() works exactly like dump() but instead of sending the output to a file, it returns the pickled data as a string.
e.g
# python object
employee = {
"first_name": "Mike",
"designation": 'Manager',
"doj": datetime(year=2016, month=5, day=2), # date of joining
... }
pickled_emp = pickle.dumps(employee)
> pickled_emp
b'\x80\x03}q\x00(X\x0b\x00\x00\x00designationq\x01X\x07\x00\x00\x00Managerq\x02X\x03\x00\x00\x00dojq\x03cdatetime\ndatetime\nq\x04C\n\x07\xe0\x05\x02\x00\x00\x00\x00\x00\x00q\x05\x85q\x06Rq\x07X\n\x00\x00\x00first_nameq\x08X\x04\x00\x00\x00Mikeq\tu.'
370
pickle module
loads()
Similarly, the loads() function is same as load(), but instead of reading pickled data from a file, it reads from a string.
e.g
employee = {
"first_name": "Mike",
"designation": 'Manager',
"doj": datetime(year=2016, month=5, day=2), # date of joining
... }
pickled_emp = pickle.dumps(employee)
> pickled_emp
b'\x80\x03}q\x00(X\x0b\x00\x00\x00designationq\x01X\x07\x00\x00\x00Managerq\x02X\x03\x00\x00\x00dojq\x03cdatetime\ndatetime\nq\x04C\n\x07\xe0\x05\x02\x00\x00\x00\x00\x00\x00q\x05\x85q\x06Rq\x07X\n\x00\x00\x00first_nameq\x08X\x04\x00\x00\x00Mikeq\tu.'
pickle.loads(pickled_emp)
> pickled_emp
{'designation': 'Manager', 'doj': datetime.datetime(2016, 5, 2, 0, 0), 'first_name': 'Mike'}
Keep in mind that, when you unpickle data, objects spring into life, so never try to process pickled data from untrusted sources. A malicious user can use such technique to execute arbitrary commands on the system.
371
372
Unit testing is the testing of single components of the application to check 1 functionality at a time
TPG, PJs list
flashcards
Decks in class (3)
# Cards
