Programming in Python (IT-605) - Unit 4 Short Notes

1. File Input/Output (I/O) Operations

1.1. File Objects & Modes

• Opening Files: open(filename, mode) returns a file object.

• File Modes:

  | Mode | Description | |------|-------------| | 'r' | Read (default). | | 'w' | Write (truncates existing file). | | 'a' | Append (writes to end). | | 'x' | Exclusive creation (fails if file exists). | | 'b' | Binary mode. | | 't' | Text mode (default). | | '+' | Open for updating (read + write). |

• Key Methods:

  • read(size=-1): Reads entire file or size bytes.

  • readline(): Reads one line (including \n).

  • readlines(): Reads all lines into a list.

  • write(str): Writes a string to file.

  • writelines(list_of_str): Writes a list of strings (no newlines added automatically).

• Closing: file.close() releases resources. Implicit closing on program exit is unreliable.

[!TIP] Always use 'with' (see 1.2) to avoid forgetting close(), which can cause resource leaks and data corruption.

1.2. The with Statement & Context Managers

• Syntax: with open('file.txt', 'r') as f: ...

• Purpose: Ensures the file is automatically closed when the block exits, even if an exception occurs.

• Benefit: Eliminates manual try-finally for cleanup, leading to cleaner, safer code.

1.3. Working with Text Files

• Reading:

  • Entire content: content = f.read() (loads whole file into memory → O(n) space).

  • Line-by-line iteration: for line in f: (memory efficient, O(1) space per iteration).

• Writing: f.write("Hello\n") — newlines must be explicit.

• Encoding: Specify encoding='utf-8' (or other) to handle non-ASCII characters reliably.

1.4. Working with CSV Data

• Module: import csv

• Reading:

  
  with open('data.csv') as f:
  
      reader = csv.reader(f, delimiter=',')  # Default delimiter is ','
  
      for row in reader:  # Each row is a list of strings
  
          process(row)
  
  

• DictReader: csv.DictReader(f) returns each row as a dict (keys from header row).

• Writing:

  
  with open('out.csv', 'w', newline='') as f:
  
      writer = csv.writer(f)
  
      writer.writerow(['col1', 'col2'])
  
      writer.writerows(list_of_rows)
  
  

• DictWriter: csv.DictWriter(f, fieldnames=...) writes dictionaries.

1.5. Working with JSON Data

• Module: import json

• Parsing:

  • json.load(f): Reads JSON from file → Python object (dict/list).

  • json.loads(json_string): Parses JSON string.

• Serialization:

  • json.dump(obj, f): Writes Python object as JSON to file.

  • json.dumps(obj): Returns JSON string.

• Key Point: JSON supports only basic types (dict, list, str, int, float, bool, None).

1.6. Binary File Operations

• Mode: Use 'rb' (read binary) or 'wb' (write binary).

• Data: Read/write bytes objects (e.g., b'\x00\x01').

• Use Cases:

  • Images, audio, video.

  • Pickled Python objects (import pickle).

  • Custom binary protocols.

1.7. File & Directory Management (os & pathlib)

• os Module (legacy, string-based paths):

  • os.getcwd(), os.listdir(path)

  • os.mkdir(path), os.remove(path), os.rename(src, dst)

• pathlib Module (modern, object-oriented):

  
  from pathlib import Path
  
  p = Path('folder/file.txt')
  
  p.exists(), p.is_file(), p.is_dir()
  
  p.mkdir(parents=True, exist_ok=True)
  
  p.unlink()  # Delete file
  
  p.rename('new_name.txt')
  
  p.read_text(), p.write_text('data')
  
  

• Properties: p.stat().st_size (size), p.stat().st_mtime (modification time).

2. Exception Handling

2.1. Basics of Try-Except

• Syntax:

  
  try:
  
      risky_operation()
  
  except SomeException as e:
  
      handle_error(e)  # e is the exception object
  
  

• Catching:

  • Specific: except FileNotFoundError:

  • Generic: except Exception: (catches most exceptions, but avoid bare except:).

• Access Error: str(e) or e.args gives error message.

2.2. Multiple Except Clauses

• Order matters: subclasses before superclasses.

  
  try:
  
      ...
  
  except ValueError:   # Specific
  
      ...
  
  except Exception:    # General (catches ValueError too if placed first!)
  
      ...
  
  

2.3. The else and finally Clauses

• else: Executes only if no exception occurred in try block.

  
  try:
  
      data = f.read()
  
  except FileNotFoundError:
  
      ...
  
  else:
  
      process(data)  # Runs if file opened successfully
  
  

• finally: Always executes (even if exception is raised or caught). Used for cleanup (e.g., closing files, releasing locks).

  
  try:
  
      f = open('file.txt')
  
  finally:
  
      f.close()  # Guaranteed to run
  
  

2.4. Raising Exceptions

• raise: Re-raises current exception (in except block) or raises a new one.

  
  if x < 0:
  
      raise ValueError("x must be non-negative")
  
  

• Custom Exceptions:

  
  class InsufficientFundsError(Exception):
  
      pass
  
  raise InsufficientFundsError("Balance too low")
  
  

2.5. Common Built-in Exceptions

[!TIP] Always catch the most specific exception possible. Avoid catching Exception or BaseException unless re-raising or logging.

3. Modular Programming

3.1. Modules & Import System

• Module: Any .py file (e.g., utils.py).

• Import Statements:

  
  import module           # Access via module.func()
  
  from module import func # Access via func() directly
  
  from module import *    # Imports all public names (discouraged: pollutes namespace)
  
  

• Namespace: Each module has its own global namespace. Prevents name collisions.

3.2. The __name__ Variable

• __name__ is set by Python:

  • "__main__" if the script is run directly.

  • Module name (e.g., "utils") if imported.

• Idiom:

  
  # In module.py
  
  def main():
  
      ...
  
  if __name__ == "__main__":
  
      main()  # Runs only when executed directly, not when imported
  
  

3.3. Packages

• Package: Directory containing __init__.py (can be empty) and modules/subpackages.

• Structure:

  
  mypackage/
  
      __init__.py
  
      module1.py
  
      subpackage/
  
          __init__.py
  
          module2.py
  
  

• Importing:

  
  import mypackage.module1
  
  from mypackage import module1
  
  from mypackage.subpackage import module2
  
  

• Relative Imports (inside package):

  
  from . import module1      # Same directory
  
  from .. import module2    # Parent directory
  
  

3.4. Standard Library Exploration

• Common Modules:

  • math: sqrt(), pi, ceil().

  • random: randint(), choice(), shuffle().

  • datetime: datetime.now(), timedelta.

  • statistics: mean(), median().

  • collections: defaultdict, Counter, namedtuple.

• Documentation: Use help(module) or pydoc module in terminal.

4. Advanced Function Concepts

4.1. Variable Scope & Namespaces (LEGB Rule)

• LEGB: Python searches scopes in order:

  1. Local: Inside current function.

  2. Enclosing: In enclosing (nested) functions.

  3. Global: At module level.

  4. Built-in: Python’s built-in names (e.g., len).

• global: Assign to global variable from local scope.

  
  x = 10
  
  def f():
  
      global x
  
      x = 20  # Modifies global x
  
  

• nonlocal: Assign to variable in enclosing (non-global) scope.

  
  def outer():
  
      x = 10
  
      def inner():
  
          nonlocal x
  
          x = 20  # Modifies outer's x
  
  

4.2. Lambda Functions

• Syntax: lambda args: expression

• Characteristics:

  • Single expression (no statements like if, for).

  • Returns value of expression automatically.

• Use Cases:

  • key functions: sorted(lst, key=lambda x: x[1])

  • map(func, iterable): Applies func to each item → O(n) time.

  • filter(func, iterable): Filters items where func returns True → O(n) time.

• Limitation: Less readable for complex logic; use def for multi-line functions.

4.3. First-Class Functions & Closures

• First-Class: Functions can be:

  • Assigned to variables: f = len

  • Passed as arguments: def apply(func, x): return func(x)

  • Returned from functions: def make_adder(n): return lambda x: x + n

• Closure: Nested function that captures variables from enclosing scope.

  
  def make_multiplier(factor):
  
      def multiplier(x):
  
          return x * factor  # 'factor' is captured from enclosing scope
  
      return multiplier
  
  double = make_multiplier(2)
  
  double(5)  # Returns 10
  
  

  • Captured variables are stored even after outer function returns.

4.4. Decorators (Conceptual Introduction)

• Purpose: Modify or enhance function behavior without changing its code.

• Syntax: @decorator above function definition.

  
  def decorator(func):
  
      def wrapper(*args, **kwargs):
  
          # Do something before
  
          result = func(*args, **kwargs)
  
          # Do something after
  
          return result
  
      return wrapper
  
  @decorator
  
  def my_func():
  
      ...
  
  

  Equivalent to: my_func = decorator(my_func).

• Common Use Cases:

  • Logging: Log function calls/arguments.

  • Timing: Measure execution time.

  • Access Control: Check permissions before execution.

  • Caching/Memoization: Store results of expensive calls.

• Note: Decorators are higher-order functions that take a function and return a wrapped function.

5. Best Practices & Code Quality

5.1. PEP 8 Style Guide

• Indentation: 4 spaces (no tabs).

• Naming:

  • snake_case for functions/variables.

  • CamelCase for classes.

  • UPPER_SNAKE_CASE for constants.

• Imports: Group in order: standard library, third-party, local. One per line.

• Whitespace: Around operators, after commas, but not inside brackets.

• Line Length: Max 79 characters (PEP 8) or 99 (PEP 8 relaxed).

5.2. Docstrings

• Purpose: Describe what a module/function/class does, its parameters, return values, and exceptions.

• Common Formats:

  • Google Style:

    
    def func(a, b):
    
        """Computes the sum of a and b.
    
        Args:
    
            a (int): First number.
    
            b (int): Second number.
    
        Returns:
    
            int: Sum of a and b.
    
        """
    
        return a + b
    
    

  • NumPy/SciPy Style: More structured sections (Parameters, Returns, Examples).

  • reStructuredText (reST): Used by Sphinx.

• Access: func.__doc__ or help(func).

5.3. Basic Debugging Techniques

• print() Tracing: Insert print statements to inspect variables/flow.

• pdb Module (Python Debugger):

  • Insert import pdb; pdb.set_trace() to break.

  • Key Commands:

    • n (next): Execute current line, stop at next.

    • s (step): Step into function calls.

    • c (continue): Resume execution until next breakpoint.

    • l (list): Show source code around current line.

    • p expr (print): Evaluate and print expr.

• Traceback:

  • Read from bottom up: Last line is error type/message.

  • Above lines show call stack (most recent call last).

  • Identify file, line number, and code snippet where error occurred.

[!TIP] Use try-except with specific exceptions and log the error (logging module) instead of bare except: which hides bugs.