File I/O & JSON Processing

Introduction to File I/O in Dart

Working with files is a fundamental skill for any developer building command-line tools, server applications, or data processing pipelines. Dart’s dart:io library provides a comprehensive set of classes for reading, writing, and manipulating files and directories. Combined with dart:convert for JSON processing, you have everything you need to build robust data-driven applications.

Reading Files

The File class from dart:io is your primary tool for file operations. You can read files either synchronously (blocking) or asynchronously (non-blocking). Asynchronous reads are preferred in most cases because they do not freeze your application while waiting for disk I/O.

Reading Files Asynchronously

The readAsString() method returns a Future<String> containing the entire file contents. This is the most common way to read text files.

import 'dart:io';

Future<void> main() async {
  final file = File('config.txt');

  // Check if file exists before reading
  if (await file.exists()) {
    // Read entire file as a string
    String contents = await file.readAsString();
    print('File contents:\n$contents');

    // Read file as list of lines
    List<String> lines = await file.readAsLines();
    print('Total lines: ${lines.length}');

    for (var i = 0; i < lines.length; i++) {
      print('Line $i: ${lines[i]}');
    }

    // Read file as raw bytes
    List<int> bytes = await file.readAsBytes();
    print('File size: ${bytes.length} bytes');
  } else {
    print('File not found!');
  }
}

Reading Files Synchronously

Synchronous reads block the current thread until the file is fully read. Use these only in simple scripts or when you specifically need blocking behavior (for example, reading a config file at startup before anything else runs).

import 'dart:io';

void main() {
  final file = File('data.txt');

  // Synchronous versions — block until complete
  if (file.existsSync()) {
    String contents = file.readAsStringSync();
    List<String> lines = file.readAsLinesSync();
    List<int> bytes = file.readAsBytesSync();

    print('Read ${lines.length} lines, ${bytes.length} bytes');
  }
}

Streaming Large Files

For very large files, reading everything into memory at once is wasteful or even impossible. Dart lets you read files as a stream of chunks, processing data piece by piece.

import 'dart:io';
import 'dart:convert';

Future<void> main() async {
  final file = File('large_log.txt');

  // Open a stream that reads the file in chunks
  final stream = file.openRead();

  // Transform byte chunks into UTF-8 strings, then split into lines
  await stream
      .transform(utf8.decoder)
      .transform(const LineSplitter())
      .forEach((line) {
    if (line.contains('ERROR')) {
      print('Found error: $line');
    }
  });

  print('Finished scanning log file.');
}

Writing Files

Writing files follows the same pattern as reading: you can write synchronously or asynchronously, and you can write strings, lines, or raw bytes.

import 'dart:io';

Future<void> main() async {
  final file = File('output.txt');

  // Write a string (overwrites existing content)
  await file.writeAsString('Hello, Dart file I/O!\n');

  // Append to existing file
  await file.writeAsString(
    'This line is appended.\n',
    mode: FileMode.append,
  );

  // Write multiple lines at once
  final lines = ['Line 1', 'Line 2', 'Line 3'];
  await file.writeAsString(lines.join('\n') + '\n');

  // Write raw bytes
  final bytes = [72, 101, 108, 108, 111]; // "Hello" in ASCII
  await File('binary.dat').writeAsBytes(bytes);

  // Synchronous writing
  File('sync_output.txt').writeAsStringSync('Written synchronously.\n');

  print('All files written successfully.');
}

Writing with IOSink (Streaming Writes)

For writing large amounts of data, or when you want to write incrementally, use an IOSink. This is especially useful for generating reports, logs, or large output files.

import 'dart:io';

Future<void> main() async {
  final file = File('report.txt');
  final sink = file.openWrite();

  // Write header
  sink.writeln('=== Daily Report ===');
  sink.writeln('Generated: ${DateTime.now()}');
  sink.writeln('');

  // Write data rows
  for (var i = 1; i <= 1000; i++) {
    sink.writeln('Entry #$i: value=${i * 3.14}');
  }

  // Write footer
  sink.writeln('');
  sink.writeln('=== End of Report ===');

  // IMPORTANT: Always close the sink when done
  await sink.flush();
  await sink.close();

  print('Report written with ${await file.length()} bytes.');
}

Directory Operations

The Directory class lets you create, list, and manage directories. Combined with File, you can build complete file management utilities.

import 'dart:io';

Future<void> main() async {
  // Create a directory (including parents)
  final dir = Directory('output/reports/2024');
  await dir.create(recursive: true);
  print('Directory created: ${dir.path}');

  // List directory contents
  final currentDir = Directory.current;
  print('Current directory: ${currentDir.path}');

  await for (var entity in currentDir.list()) {
    if (entity is File) {
      print('  File: ${entity.path}');
    } else if (entity is Directory) {
      print('  Dir:  ${entity.path}');
    } else if (entity is Link) {
      print('  Link: ${entity.path}');
    }
  }

  // List recursively (all subdirectories)
  await for (var entity in currentDir.list(recursive: true)) {
    if (entity is File && entity.path.endsWith('.dart')) {
      print('Dart file: ${entity.path}');
    }
  }

  // Check if directory exists
  if (await dir.exists()) {
    // Delete directory and contents
    await dir.delete(recursive: true);
    print('Directory deleted.');
  }
}

Path Manipulation

Dart provides basic path utilities through dart:io’s Platform class and the file/directory objects themselves. For advanced path manipulation, the path package is the standard choice.

import 'dart:io';

void main() {
  // Get information from file paths
  final file = File('/home/user/documents/report.pdf');
  print('Path: ${file.path}');
  print('Parent: ${file.parent.path}');
  print('Absolute: ${file.absolute.path}');

  // Platform-specific path separator
  print('Path separator: ${Platform.pathSeparator}');

  // Using the path package (add to pubspec.yaml: path: ^1.8.0)
  // import 'package:path/path.dart' as p;
  // print(p.basename('/home/user/file.txt'));    // file.txt
  // print(p.extension('/home/user/file.txt'));   // .txt
  // print(p.dirname('/home/user/file.txt'));     // /home/user
  // print(p.join('home', 'user', 'file.txt'));  // home/user/file.txt
  // print(p.normalize('a/b/../c'));              // a/c

  // Temporary directory
  final tempDir = Directory.systemTemp;
  print('Temp directory: ${tempDir.path}');

  // Create a temp file
  final tempFile = File(
    '${tempDir.path}${Platform.pathSeparator}myapp_temp.txt'
  );
  tempFile.writeAsStringSync('Temporary data');
  print('Temp file: ${tempFile.path}');
}

JSON Processing with dart:convert

JSON (JavaScript Object Notation) is the most common data interchange format on the web. Dart’s built-in dart:convert library provides jsonDecode and jsonEncode functions for converting between JSON strings and Dart objects.

Basic JSON Encoding and Decoding

import 'dart:convert';

void main() {
  // === Decoding (JSON string -> Dart object) ===

  // JSON object -> Map<String, dynamic>
  String jsonString = '{"name": "Alice", "age": 30, "active": true}';
  Map<String, dynamic> user = jsonDecode(jsonString);
  print('Name: ${user['name']}');   // Alice
  print('Age: ${user['age']}');     // 30
  print('Type: ${user.runtimeType}'); // _Map<String, dynamic>

  // JSON array -> List<dynamic>
  String jsonArray = '[1, 2, 3, "four", true]';
  List<dynamic> items = jsonDecode(jsonArray);
  print('Items: $items');

  // === Encoding (Dart object -> JSON string) ===
  Map<String, dynamic> product = {
    'id': 42,
    'name': 'Dart Handbook',
    'price': 29.99,
    'tags': ['programming', 'dart'],
  };

  // Compact JSON
  String compact = jsonEncode(product);
  print(compact);
  // {"id":42,"name":"Dart Handbook","price":29.99,"tags":["programming","dart"]}

  // Pretty-printed JSON
  String pretty = const JsonEncoder.withIndent('  ').convert(product);
  print(pretty);
}

JSON to Model Mapping

In real applications, you rarely work with raw Map<String, dynamic>. Instead, you create model classes with fromJson and toJson methods to ensure type safety and better code organization.

import 'dart:convert';

class User {
  final String name;
  final String email;
  final int age;
  final bool isActive;

  User({
    required this.name,
    required this.email,
    required this.age,
    required this.isActive,
  });

  // Factory constructor to create User from JSON map
  factory User.fromJson(Map<String, dynamic> json) {
    return User(
      name: json['name'] as String,
      email: json['email'] as String,
      age: json['age'] as int,
      isActive: json['is_active'] as bool,
    );
  }

  // Convert User to JSON map
  Map<String, dynamic> toJson() {
    return {
      'name': name,
      'email': email,
      'age': age,
      'is_active': isActive,
    };
  }

  @override
  String toString() => 'User($name, $email, age: $age)';
}

void main() {
  // Decode JSON to User
  String jsonStr = '{"name":"Bob","email":"bob@test.com","age":25,"is_active":true}';
  var user = User.fromJson(jsonDecode(jsonStr));
  print(user); // User(Bob, bob@test.com, age: 25)

  // Encode User to JSON
  String encoded = jsonEncode(user.toJson());
  print(encoded);

  // List of users
  String usersJson = '[{"name":"A","email":"a@t.com","age":20,"is_active":true}]';
  List<User> users = (jsonDecode(usersJson) as List)
      .map((json) => User.fromJson(json))
      .toList();
  print('Users: $users');
}

Handling Nested JSON

Real-world APIs often return deeply nested JSON structures. Your model classes need to handle nested objects and arrays gracefully.

import 'dart:convert';

class Address {
  final String street;
  final String city;
  final String country;

  Address({required this.street, required this.city, required this.country});

  factory Address.fromJson(Map<String, dynamic> json) => Address(
    street: json['street'] as String,
    city: json['city'] as String,
    country: json['country'] as String,
  );

  Map<String, dynamic> toJson() => {
    'street': street,
    'city': city,
    'country': country,
  };
}

class Order {
  final int id;
  final String product;
  final double price;

  Order({required this.id, required this.product, required this.price});

  factory Order.fromJson(Map<String, dynamic> json) => Order(
    id: json['id'] as int,
    product: json['product'] as String,
    price: (json['price'] as num).toDouble(),
  );

  Map<String, dynamic> toJson() => {
    'id': id,
    'product': product,
    'price': price,
  };
}

class Customer {
  final String name;
  final Address address;
  final List<Order> orders;

  Customer({
    required this.name,
    required this.address,
    required this.orders,
  });

  factory Customer.fromJson(Map<String, dynamic> json) => Customer(
    name: json['name'] as String,
    address: Address.fromJson(json['address'] as Map<String, dynamic>),
    orders: (json['orders'] as List)
        .map((o) => Order.fromJson(o as Map<String, dynamic>))
        .toList(),
  );

  Map<String, dynamic> toJson() => {
    'name': name,
    'address': address.toJson(),
    'orders': orders.map((o) => o.toJson()).toList(),
  };

  double get totalSpent => orders.fold(0, (sum, o) => sum + o.price);
}

void main() {
  final jsonStr = '''
  {
    "name": "Alice",
    "address": {
      "street": "123 Main St",
      "city": "Springfield",
      "country": "US"
    },
    "orders": [
      {"id": 1, "product": "Laptop", "price": 999.99},
      {"id": 2, "product": "Mouse", "price": 29.99}
    ]
  }
  ''';

  final customer = Customer.fromJson(jsonDecode(jsonStr));
  print('${customer.name} spent \$${customer.totalSpent}');
  // Alice spent $1029.98

  // Round-trip: encode back to JSON
  print(const JsonEncoder.withIndent('  ').convert(customer.toJson()));
}

Practical Example: Config File Reader

Let’s build a reusable configuration file reader that loads JSON config files with defaults and validation.

import 'dart:io';
import 'dart:convert';

class AppConfig {
  final String appName;
  final int port;
  final String dbHost;
  final bool debugMode;
  final List<String> allowedOrigins;

  AppConfig({
    required this.appName,
    required this.port,
    required this.dbHost,
    required this.debugMode,
    required this.allowedOrigins,
  });

  factory AppConfig.fromJson(Map<String, dynamic> json) {
    return AppConfig(
      appName: json['app_name'] as String? ?? 'MyApp',
      port: json['port'] as int? ?? 8080,
      dbHost: json['db_host'] as String? ?? 'localhost',
      debugMode: json['debug'] as bool? ?? false,
      allowedOrigins: (json['allowed_origins'] as List<dynamic>?)
              ?.cast<String>() ??
          ['http://localhost'],
    );
  }

  static Future<AppConfig> load(String path) async {
    final file = File(path);
    if (!await file.exists()) {
      print('Config file not found. Using defaults.');
      return AppConfig.fromJson({});
    }

    try {
      final contents = await file.readAsString();
      final json = jsonDecode(contents) as Map<String, dynamic>;
      return AppConfig.fromJson(json);
    } on FormatException catch (e) {
      print('Invalid JSON in config: $e');
      return AppConfig.fromJson({});
    }
  }

  @override
  String toString() =>
      'AppConfig(app: $appName, port: $port, db: $dbHost, debug: $debugMode)';
}

Future<void> main() async {
  final config = await AppConfig.load('config.json');
  print(config);
  print('Allowed origins: ${config.allowedOrigins}');
}

Practical Example: CSV Parser

This example demonstrates reading a CSV file, parsing it into structured data, and converting it to JSON.

import 'dart:io';
import 'dart:convert';

class CsvParser {
  final String separator;

  CsvParser({this.separator = ','});

  /// Parse a CSV file into a list of maps.
  /// The first row is treated as column headers.
  Future<List<Map<String, String>>> parse(String filePath) async {
    final file = File(filePath);
    final lines = await file.readAsLines();

    if (lines.isEmpty) return [];

    // First line = headers
    final headers = lines.first
        .split(separator)
        .map((h) => h.trim())
        .toList();

    // Remaining lines = data rows
    return lines.skip(1).where((line) => line.trim().isNotEmpty).map((line) {
      final values = line.split(separator).map((v) => v.trim()).toList();
      return Map.fromIterables(
        headers,
        values.length >= headers.length
            ? values.sublist(0, headers.length)
            : [...values, ...List.filled(headers.length - values.length, '')],
      );
    }).toList();
  }

  /// Convert parsed CSV data to a JSON string.
  String toJson(List<Map<String, String>> data) {
    return const JsonEncoder.withIndent('  ').convert(data);
  }
}

Future<void> main() async {
  // Create a sample CSV file
  final csvFile = File('employees.csv');
  await csvFile.writeAsString(
    'name,department,salary\n'
    'Alice,Engineering,95000\n'
    'Bob,Marketing,72000\n'
    'Charlie,Engineering,88000\n',
  );

  // Parse and convert
  final parser = CsvParser();
  final data = await parser.parse('employees.csv');
  print('Parsed ${data.length} rows:');
  print(parser.toJson(data));

  // Write JSON output
  await File('employees.json').writeAsString(parser.toJson(data));
  print('JSON file written.');

  // Cleanup
  await csvFile.delete();
}

Practical Example: Log File Writer

This example creates a structured logging utility that writes timestamped entries to a log file.

import 'dart:io';
import 'dart:convert';

enum LogLevel { debug, info, warning, error }

class LogEntry {
  final DateTime timestamp;
  final LogLevel level;
  final String message;
  final Map<String, dynamic>? metadata;

  LogEntry({
    required this.level,
    required this.message,
    this.metadata,
  }) : timestamp = DateTime.now();

  Map<String, dynamic> toJson() => {
    'timestamp': timestamp.toIso8601String(),
    'level': level.name,
    'message': message,
    if (metadata != null) 'metadata': metadata,
  };
}

class FileLogger {
  final File _logFile;
  IOSink? _sink;

  FileLogger(String path) : _logFile = File(path);

  Future<void> open() async {
    // Create parent directories if needed
    await _logFile.parent.create(recursive: true);
    _sink = _logFile.openWrite(mode: FileMode.append);
  }

  void log(LogLevel level, String message, {Map<String, dynamic>? meta}) {
    final entry = LogEntry(level: level, message: message, metadata: meta);
    _sink?.writeln(jsonEncode(entry.toJson()));
  }

  void debug(String msg) => log(LogLevel.debug, msg);
  void info(String msg, {Map<String, dynamic>? meta}) =>
      log(LogLevel.info, msg, meta: meta);
  void warning(String msg) => log(LogLevel.warning, msg);
  void error(String msg, {Map<String, dynamic>? meta}) =>
      log(LogLevel.error, msg, meta: meta);

  Future<void> close() async {
    await _sink?.flush();
    await _sink?.close();
    _sink = null;
  }

  /// Read and parse all log entries.
  Future<List<Map<String, dynamic>>> readEntries() async {
    if (!await _logFile.exists()) return [];
    final lines = await _logFile.readAsLines();
    return lines
        .where((l) => l.trim().isNotEmpty)
        .map((l) => jsonDecode(l) as Map<String, dynamic>)
        .toList();
  }
}

Future<void> main() async {
  final logger = FileLogger('logs/app.log');
  await logger.open();

  logger.info('Application started', meta: {'version': '1.0.0'});
  logger.debug('Loading configuration...');
  logger.info('Server listening on port 8080');
  logger.warning('High memory usage detected');
  logger.error('Failed to connect to database', meta: {
    'host': 'db.example.com',
    'error_code': 'ECONNREFUSED',
  });

  await logger.close();

  // Read back and filter entries
  final entries = await logger.readEntries();
  final errors = entries.where((e) => e['level'] == 'error').toList();
  print('Total entries: ${entries.length}');
  print('Errors: ${errors.length}');
  print(const JsonEncoder.withIndent('  ').convert(errors));
}

Error Handling Best Practices for File I/O

File operations can fail for many reasons: permission denied, disk full, network drive unavailable, or file locked by another process. Robust error handling is essential.

import 'dart:io';
import 'dart:convert';

Future<Map<String, dynamic>?> safeReadJson(String path) async {
  try {
    final file = File(path);
    if (!await file.exists()) {
      print('File does not exist: $path');
      return null;
    }

    final contents = await file.readAsString();
    return jsonDecode(contents) as Map<String, dynamic>;
  } on FormatException catch (e) {
    print('Invalid JSON in $path: $e');
    return null;
  } on FileSystemException catch (e) {
    print('File system error reading $path: ${e.message}');
    return null;
  } catch (e) {
    print('Unexpected error reading $path: $e');
    return null;
  }
}

Future<bool> safeWriteJson(
  String path,
  Map<String, dynamic> data,
) async {
  try {
    final file = File(path);
    await file.parent.create(recursive: true);

    // Write to temp file first, then rename (atomic write)
    final tempFile = File('$path.tmp');
    await tempFile.writeAsString(
      const JsonEncoder.withIndent('  ').convert(data),
    );
    await tempFile.rename(path);
    return true;
  } on FileSystemException catch (e) {
    print('Failed to write $path: ${e.message}');
    return false;
  }
}

Future<void> main() async {
  final data = {'key': 'value', 'count': 42};

  if (await safeWriteJson('data/config.json', data)) {
    print('Written successfully.');
    final loaded = await safeReadJson('data/config.json');
    print('Loaded: $loaded');
  }
}

Summary

In this lesson, you learned the complete toolkit for file-based data processing in Dart:

• File reading with readAsString, readAsLines, readAsBytes (both async and sync)
• Stream-based reading for processing large files efficiently with openRead()
• File writing with writeAsString, writeAsBytes, and IOSink for streaming writes
• Directory operations for creating, listing, and deleting directories
• Path manipulation for working with file paths across platforms
• JSON processing with jsonDecode/jsonEncode and model class serialization
• Nested JSON handling with composed model classes
• Practical patterns like config readers, CSV parsers, and structured loggers
• Error handling and atomic writes for robust file operations