Introduction

In the previous article we had the comprehensive overview of the “Python programming language and its built-in data structure.” We also discussed the importance of learning Python to stay relevant in today’s competitive data science market, where many people have already been laid off due to automation of tasks and the rise of Gen-AI and LLMs.

In this article I will help you understand the core of the Advanced Python Topics, such as Classes And Generators, as well as some additional important topics from a data scientist’s perspective, along with an example.

By the end of this article, you will have a solid understanding of the Python programming language, which will be useful for both interview preparation and your day-to-day work as a data scientist and Python developer. By implementing these tips, you will write efficient code and increase your productivity while working with a team.

Overview

1. Discover advanced Python concepts like classes, generators, and more, especially for data scientists.
2. Learn how to create custom objects and manipulate them effectively in Python.
3. Discover the power of Python generators to save memory and streamline iteration processes.
4. Gain insight into various Python literals, including strings, numbers, and Booleans.
5. Improve your coding efficiency with Python’s built-in functions and error management techniques.
6. Build a strong Python foundation, covering everything from basic to advanced topics, with practical examples for real-world applications.

What is Advanced Python Programming?

Advanced Python Programming is the study and application of advanced Python concepts that go beyond basic programming. It includes topics such as object-oriented programming (OOP), decorators, generators, context managers, and metaclasses. It also includes advanced data structures, algorithms, concurrency, parallelism, and techniques for optimizing code performance. Mastering these concepts enables developers to write more efficient, scalable, and maintainable code suitable for complex applications in fields such as data science, machine learning, web development, and software engineering.

A. Python classes

Python allows the developer to create custom objects using the `class` keyword. The blueprint of the object can contain attributes or encapsulated data, the methods or behavior of the class.

Class brackets optional, but not function brackets

• It shows two ways to define a class: `class Container():` and `class Container:.`
• The parentheses after Container are optional for classes, but are required if you are inheriting from another class.
• In contrast, function brackets are always required when defining a function.

class Container():
    def __init__(self, data):
        self.data = data

class Container: 
    def __init__(self, data):
        self.data = data

Wrapping a primitive to change within a function

The container is a simple class that encapsulates a primitive (in this case an integer).

# The code defines a class called `Container` with a constructor method `__init__` 
# that takes a parameter `data` and assigns it to an instance variable `self.data`.
class Container:
    def __init__(self, data):
        self.data = data
    
def calculate(input):
    input.data **= 5
    
container = Container(5)
calculate(container)
print(container.data)

Output

3125

Compare identity with the operator “is”

c1 = Container(5)
c2 = Container(5)

print(id(c1), id(c2))
print(id(c1) == id(c2))  # returns False because they are different objects.
print(c1 is c2) # same objects but returns False because they are distinct instances.

Output

1274963509840 1274946106128
False
False
False

Compare now on value

The `eq method` dynamically added in the previous step is used for the equality check (c1 == c2).

c1 = Container(5)
c2 = Container(5)

print(c1 == c2)  # Compares by value
# This time, the result is True because the custom __eq__ method compares 
# the values inside the Container instances.

print(c1 is c2)  # Compares by identity (address)
# The is operator still returns False because it checks for identity.

Output

True
False

B. Python generator

Generators are a special type of iterators, created using a function with the `yield` keyword, used to generate values ​​directly.

Save memory with “Generators”

import sys

my_list = (i for i in range(1000))
print(sum(my_list))
print("Size of list", sys.getsizeof(my_list), "bytes")


my_gen = (i for i in range(1000))
print(sum(my_gen))
print("Size of generator", sys.getsizeof(my_gen), "bytes")

Output

499500
Size of list 8856 bytes
499500
Size of generator 112 bytes

Fibonacci generator and yield

• `to lie` is a generator function that generates Fibonacci numbers up to a certain amount.
• `gene` is an instance of the generator and next is used to get the next values.
• The second loop shows how to use a generator in a for loop to print the first 20 Fibonacci numbers.

def fib(count):
    a, b = 0, 1
    while count:
        yield a
        a, b = b, b + a
        count -= 1

gen = fib(100)
print(next(gen), next(gen), next(gen), next(gen), next(gen))

for i in fib(20):
    print(i, end=" ")

Output

0 1 1 2 3
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181

Infinite number and infinite generator

• `mathematics.inf` stands for positive infinity.
• The `to lie` generator is used with an infinite loop, and values ​​are printed until the condition i >= 200 is met.

This code demonstrates positive infinity, a Fibonacci number generator, and how to break the generator loop based on a condition.

import math

# Printing Infinity: special floating-point representation 
print(math.inf)

# Assign infinity to a variable and perform an operation
inf = math.inf
print(inf, inf - 1)  # Always infinity, Even when subtracting 1, the result is still infinity


# Fibonacci Generator:
def fib(count):
    a, b = 0, 1
    while count:
        yield a
        a, b = b, b + a
        count -= 1

# Using the Fibonacci Generator:
# Use the Fibonacci generator with an infinite count
f = fib(math.inf)

# Iterate through the Fibonacci numbers until a condition is met
for i in f:
    if i >= 200:
        break
    print(i, end=" ")

Output

inf
inf inf
0 1 1 2 3 5 8 13 21 34 55 89 144

List of Generator

import math

# The code is creating a Fibonacci sequence generator using a generator function called `fib`.
def fib(count):
    a, b = 0, 1
    while count:
        yield a
        a, b = b, b + a
        count -= 1

# The `fib` function takes a parameter `count` which determines the number of Fibonacci numbers to generate.
f = fib(10)

# This code generates Fibonacci numbers and creates a list containing the square root of each Fibonacci number.
data = (round(math.sqrt(i), 3) for i in f)
print(data)

Output

(0.0, 1.0, 1.0, 1.414, 1.732, 2.236, 2.828, 3.606, 4.583, 5.831)

Simple infinite generator with “itertools”

The generator function could be simpler without having to take a max count property. This can be easily done with itertools.

import itertools

def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, b + a

# itertools.islice is used to get the first 20 values from an infinite generator.
print(list(itertools.islice(fib(), 20)))

Output

(0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181)

Iterate through custom type with iter

# Defines a custom LinkedList class with a Node class as an element.

class Node:
    def __init__(self, data, next_node=None):
        self.data = data
        self.next = next_node

class LinkedList:
    def __init__(self, start):
        self.start = start
    
    # The __iter__ method is implemented to allow iteration over the linked list.
    def __iter__(self):
        node = self.start
        while node:
            yield node
            node = node.next
            
ll = LinkedList(Node(5, Node(10, Node(15, Node(20)))))
for node in ll:
    print(node.data)

Output

5
10
15
20

C. Python literals

Literals are the constants that provide variable values, which can later be used directly in expressions. They are simply a syntax used in Python to express a fixed value of a specific data type.

For example:

4x - 7 = 9

    # 4 : Coefficient
    # x : Variable
    # - and = : Operator
    # 7 and 9 : Literals (constants)

Types of Literals in Python

Python supports several types of literals, such as

In my previous article I discussed the collection literals, which you can refer to here. In this article we will discuss:

• String literals
• Numeric literal values
• Boolean literals
• Special literal words

Python String/Character Literals

A literal string is created by writing a text (i.e., the group of characters) between single quotes (‘ ‘), double quotes (” “), or triple quotes (to store multi-line strings).

For example,

# in single quote
s="AnalyticalNikita.io"
print(s)

# in double quotes
d = "AnalyticalNikita.io"
print(d)

# multi-line String
m = '''Analytical
              Nikita.
                      io'''
print(m)

# Character Literal
char = "A"
print(char)

# Unicode Literal
unicodes = u"\u0041"
print(unicodes)

# Raw String
raw_str = r"raw \n string"
print(raw_str)

Output

AnalyticalNikita.io
AnalyticalNikita.io
Analytical
               Nikita.
                          io
A
A
raw \n string

Numeric literals in Python

There are three types of numeric literals in Python that are inherently immutable, namely:

1. Integer: These are both positive and negative numbers, including 0: a decimal literal, a binary literal, an octal literal, and a hexadecimal literal. Remark: When using the print function to display a value or get the output, these literals are converted to decimals by default.
2. Rapid: These are the real numbers with both integer and decimal parts.
3. Complex: These numbers resemble the complex numbers from mathematics and are denoted in the form of `a + bj`, where ‘a’ is the real part and ‘b’ is the complex part.

# integer literal

# Binary Literals
a = 0b10100

# Decimal Literal
b = 50

# Octal Literal
c = 0o320

# Hexadecimal Literal
d = 0x12b

print(a, b, c, d)


# Float Literal
e = 24.8

print(e)

# Complex Literal
f = 2+3j
print(f)

Output

20 50 208 299
24.8
(2+3j)

Python Boolean literals

Like other programming languages, Python has only two Boolean literals, namely: Where (or 1) And False (or 0)Python treats Boolean values ​​as numbers in mathematical expressions.

Such as:

a = (1 == True)
b = (1 == False)
c = True + 4
d = False + 10

print("a is", a)
print("b is", b)
print("c:", c)
print("d:", d)

Output

a is True
b is False
c: 5
d: 10

Python Special Literals

Generally, ‘No‘ is used to define a null variable.

hi = None
print(hi)

Output

None

Remark: If we compare ‘None’ with anything other than ‘None’, it will always return Incorrect.

hi = None
bye = "ok"
print(hi == bye)

Output

False

It is known as a special literal because it is also used in Python for variable declaration. If you do not know the number of variables, you can use `None` as it will not produce any errors.

k = None
a = 7
print("Program is running..")

Output

Program is running..

D. Zip function

We’ve seen this function before in relation to Python’s built-in data structure, when we had iterables of equal length (such as lists, dictionaries, etc.) as arguments and `row()` aggregates the iterators of each of the iterables.

Using Zip with an equal number of iterators

# Example using zip with two lists
numbers = (1, 2, 3)
letters = ('a', 'b', 'c')

# Zip combines corresponding elements from both lists
zipped_result = zip(numbers, letters)

# Iterate over the zipped result
for number, letter in zipped_result:
    print(f"Number: {number}, Letter: {letter}")

Output

Number: 1, Letter: a
Number: 2, Letter: b
Number: 3, Letter: c

But what if we have an unequal number of iterators? In this case we use `zip_longest()` of the `itertools` module to aggregate the elements. If two lists have different lengths, it will aggregate `N/A`.

Using zip_longest from itertools

from itertools import zip_longest

# Example using zip_longest with two lists of different lengths
numbers = (1, 2, 3)
letters = ('a', 'b')

# zip_longest fills missing values with a specified fillvalue (default is None)
zipped_longest_result = zip_longest(numbers, letters, fillvalue="N/A")

# Iterate over the zipped_longest result
for number, letter in zipped_longest_result:
    print(f"Number: {number}, Letter: {letter}")

Output

Number: 1, Letter: a
Number: 2, Letter: b
Number: 3, Letter: N/A

Standard arguments

If you have default values, you can pass arguments by name; positional arguments must remain left.

from itertools import zip_longest


def zip_lists(list1=(), list2=(), longest=True):
    if longest:
        return (list(item) for item in zip_longest(list1, list2))
    else:
        return (list(item) for item in zip(list1, list2))
    
names = ('Alice', 'Bob', 'Eva', 'David', 'Sam', 'Ace')
points = (100, 250, 30, 600)

print(zip_lists(names, points))

Output

(('Alice', 100), ('Bob', 250), ('Eva', 30), ('David', 600), ('Sam', None), ('Ace', None))

Keyword Arguments

You can pass named arguments in any order and even skip them.

from itertools import zip_longest


def zip_lists(list1=(), list2=(), longest=True):
    if longest:
        return (list(item) for item in zip_longest(list1, list2))
    else:
        return (list(item) for item in zip(list1, list2))


print(zip_lists(longest=True, list2=('Eva')))

Output

((None, 'Eva'))

E. General functions

“do-while” loop in Python

while True:

    print("""Choose an option:
          1. Do this
          2. Do that
          3. Do this and that
          4. Quit""")
    
    # if input() == "4":
    if True: 
        break

Output

Choose an option:
1. Do this
2. Do that
3. Do this and that
4. Quit

enumerate() function instead of range(len())

fruits = ('apple', 'banana', 'kiwi', 'orange')

# Using enumerate to iterate over the list with both index and value
for index, fruit in enumerate(fruits):
    print(f"Index {index}: {fruit}")

print("\n")
# You can also specify a start index (default is 0)
for index, fruit in enumerate(fruits, start=1):
    print(f"Index {index}: {fruit}")

Output

Index 0: apple
Index 1: banana
Index 2: kiwi
Index 3: orange
Index 1: apple
Index 2: banana
Index 3: kiwi
Index 4: orange

Wait with `time.sleep()`

import time

def done():
    print("done")
   
def do_something(callback):
    time.sleep(2) # it will print output after some time for ex 2 means 2.0s
    print("Doing things....") # callback functions as an argument and prints "Doing things...." before calling the provided callback.
    callback() # Call the provided callback function 

    
# Call do_something with the done function as the callback
do_something(done)

Output

Doing things....
done

Sort complex iterables with `sorted()`

dictionary_data = ({"name": "Max", "age": 6},
                   {"name": "Max", "age": 61},
                   {"name": "Max", "age": 36},
                   )

sorted_data = sorted(dictionary_data, key=lambda x : x("age"))
print("Sorted data: ", sorted_data)

Output

Sorted data: ({'name': 'Max', 'age': 6}, {'name': 'Max', 'age': 36}, {'name': 'Max', 'age': 61}) 

Download the Python version

If you’re curious about the Python version you’re working with, you can use this code:

from platform import python_version

print(python_version())

Output

3.9.13

Get the Docstring of the objects

We can also use: `__doc__` to return the functions document, which contains all the details of the object, explaining its parameters and default behavior.

print(print.__doc__)

Defining default values ​​in dictionaries with .get() and .setdefault()

You can use `.set default()` function to insert a key with a specified default value if the key is not already in your dictionary. Otherwise `.to get()` will return Noif the item does not have a specified key.

my_dict = {"name": "Max", "age": 6}                   
count = my_dict.get("count")
print("Count is there or not:", count)

# Setting default value if count is none
count = my_dict.setdefault("count", 9)
print("Count is there or not:", count)
print("Updated my_dict:", my_dict)

Output

Count is there or not: None
Count is there or not: 9
Updated my_dict: {'name': 'Max', 'age': 6, 'count': 9}

Using “Counter” from collections

• counter(): returns a dictionary containing the number of elements in an iterable.

from collections import Counter

my_list = (1,2,1,2,2,2,4,3,4,4,5,4)
counter = Counter(my_list)
print("Count of the numbers are: ", counter)

most_commmon = counter.most_common(2) # passed in Number will denotes how many common numbers we want (counting starts from 1-n)
print("Most Common Number is: ", most_commmon(0)) # printin zeroth index element from 2 most common ones

Output

Count of the numbers are: Counter({2: 4, 4: 4, 1: 2, 3: 1, 5: 1})
Most Common Number is: (2, 4)

Merge two dictionaries using **

d1 = {"name": "Max", "age": 6}   
d2 = {"name": "Max", "city": "NY"}   

merged_dict = {**d1, **d2}
print("Here is merged dictionary: ", merged_dict)

Output

Here is merged dictionary: {'name': 'Max', 'age': 6, 'city': 'NY'}

F. Syntax error vs. runtime error

There are mainly two types of errors that can occur in a program, namely:

1. Syntax errors: These types of errors occur during compilation due to incorrect syntax.
2. Runtime errors: These types of errors occur during program execution and are also called exceptions in Python.

For practical experience and better understanding, choose the Learn Python for Data Science course

Conclusion

Congratulations! I think you have a solid foundation in Python programming by now. We covered everything from Python Basics, including operators and literals (numbers, strings, lists, dictionaries, sets, tuples), to advanced Python topics like classes and generators.

To improve your production-level coding skills, I also covered the following topics: two types of errors that can occur while writing a program. This way you are aware of it and can also refer to it article, where I discuss how to debug these errors.

Additionally, I have collected all the codes in a Jupyter Notebook, which you can find here . These codes serve as a quick syntax reference for the future.

Frequently Asked Questions