第14章：可迭代的对象、迭代器和生成器.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 可迭代的对象、迭代器和生成器\n",
    "\n",
    "> 当我在自己的程序中发现用到了模式，我觉得这就表明某个地方出错了。程序的形式应该仅仅反映它所要解决的问题。代码中其他任何外加的形式都是一个信号，（至少对我来说）表明我对问题的抽象还不够深——这通常意味着自己正在手动完成事情，本应该通过写代码来让宏的扩展自动实现。\n",
    "> ——Paul Graham, Lisp 黑客和风险投资人\n",
    "\n",
    "Python 内置了迭代器模式，用于进行**惰性运算**，按需求一次获取一个数据项，避免不必要的提前计算。\n",
    "\n",
    "迭代器在 Python 中并不是一个具体类型的对象，更多地使指一个具体**协议**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 迭代器协议\n",
    "Python 解释器在迭代一个对象时，会自动调用 `iter(x)`。  \n",
    "内置的 `iter` 函数会做以下操作：\n",
    "1. 检查对象是否实现了 `__iter__` 方法（`abc.Iterable`），若实现，且返回的结果是个迭代器（`abc.Iterator`），则调用它，获取迭代器并返回；\n",
    "2. 若没实现，但实现了 `__getitem__` 方法（`abc.Sequence`），若实现则尝试从 0 开始按顺序获取元素并返回；\n",
    "3. 以上尝试失败，抛出 `TypeError`，表明对象不可迭代。\n",
    "\n",
    "判断一个对象是否可迭代，最好的方法不是用 `isinstance` 来判断，而应该直接尝试调用 `iter` 函数。\n",
    "\n",
    "注：可迭代对象和迭代器不一样。从鸭子类型的角度看，可迭代对象 `Iterable` 要实现 `__iter__`，而迭代器 `Iterator` 要实现 `__next__`. 不过，迭代器上也实现了 `__iter__`，用于[返回自身](https://github.com/python/cpython/blob/3.7/Lib/_collections_abc.py#L268)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 迭代器的具体实现\n",
    "《设计模式：可复用面向对象软件的基础》一书讲解迭代器设计模式时，在“适用性”一 节中说：\n",
    "迭代器模式可用来：\n",
    "* 访问一个聚合对象的内容而无需暴露它的内部表示\n",
    "* 支持对聚合对象的多种遍历\n",
    "* 为遍历不同的聚合结构提供一个统一的接口（即支持多态迭代）\n",
    "\n",
    "为了“支持多种遍历”，必须能从同一个可迭代的实例中获取多个**独立**的迭代器，而且各个迭代器要能维护自身的内部状态，因此这一模式正确的实现方式是，每次调用 `iter(my_iterable)` 都新建一个独立的迭代器。这就是为什么这个示例需要定义 `SentenceIterator` 类。\n",
    "\n",
    "所以，不应该把 Sentence 本身作为一个迭代器，否则每次调用 `iter(sentence)` 时返回的都是自身，就无法进行多次迭代了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Return·a·list·of·all·non·overlapping·matches·in·the·string·"
     ]
    }
   ],
   "source": [
    "# 通过实现迭代器协议，让一个对象变得可迭代\n",
    "import re\n",
    "from collections import abc\n",
    "\n",
    "\n",
    "class Sentence:\n",
    "    def  __init__(self, sentence):\n",
    "        self.sentence = sentence\n",
    "        self.words = re.findall(r'\\w+', sentence)\n",
    "\n",
    "    def __iter__(self):\n",
    "        \"\"\"返回 iter(self) 的结果\"\"\"\n",
    "        return SentenceIterator(self.words)\n",
    "        \n",
    "\n",
    "# 推荐的做法是对迭代器对象进行单独实现\n",
    "class SentenceIterator(abc.Iterator):\n",
    "    def __init__(self, words):\n",
    "        self.words = words\n",
    "        self._index = 0\n",
    "\n",
    "    def __next__(self):\n",
    "        \"\"\"调用时返回下一个对象\"\"\"\n",
    "        try:\n",
    "            word = self.words[self._index]\n",
    "        except IndexError:\n",
    "            raise StopIteration()\n",
    "        else:\n",
    "            self._index += 1\n",
    "        \n",
    "        return word\n",
    "\n",
    "\n",
    "    \n",
    "sentence = Sentence('Return a list of all non-overlapping matches in the string.')\n",
    "assert isinstance(sentence, abc.Iterable)      # 实现了 __iter__，就支持 Iterable 协议\n",
    "assert isinstance(iter(sentence), abc.Iterator)\n",
    "for word in sentence:\n",
    "    print(word, end='·')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的例子中，我们的 `SentenceIterator` 对象继承自 `abc.Iterator` 通过了迭代器测试。而且 `Iterator` 替我们实现了 `__iter__` 方法。  \n",
    "但是，如果我们不继承它，我们就需要同时实现 `__next__` 抽象方法和*实际迭代中并不会用到的* `__iter__` 非抽象方法，才能通过 `Iterator` 测试。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 生成器函数\n",
    "如果懒得自己写一个迭代器，可以直接用 Python 的生成器函数来在调用 `__iter__` 时生成一个迭代器。\n",
    "\n",
    "注：在 Python 社区中，大家并没有对“生成器”和“迭代器”两个概念做太多区分，很多人是混着用的。不过无所谓啦。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用生成器函数来帮我们创建迭代器\n",
    "import re\n",
    "\n",
    "\n",
    "class Sentence:\n",
    "    def  __init__(self, sentence):\n",
    "        self.sentence = sentence\n",
    "        self.words = re.findall(r'\\w+', sentence)\n",
    "\n",
    "    def __iter__(self):\n",
    "        for word in self.words:\n",
    "            yield word\n",
    "        return\n",
    "\n",
    "sentence = Sentence('Return a list of all non-overlapping matches in the string.')\n",
    "for word in sentence:\n",
    "    print(word, end='·')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用 re.finditer 来惰性生成值\n",
    "# 使用生成器表达式（很久没用过了）\n",
    "import re\n",
    "\n",
    "\n",
    "class Sentence:\n",
    "    def  __init__(self, sentence):\n",
    "        self.re_word = re.compile(r'\\w+')\n",
    "        self.sentence = sentence\n",
    "\n",
    "    def __iter__(self):\n",
    "        return (match.group()\n",
    "                for match in self.re_word.finditer(self.sentence))\n",
    "\n",
    "sentence = Sentence('Return a list of all non-overlapping matches in the string.')\n",
    "for word in sentence:\n",
    "    print(word, end='·')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 实用模块\n",
    "import itertools\n",
    "\n",
    "# takewhile & dropwhile\n",
    "print(list(itertools.takewhile(lambda x: x < 3, [1, 5, 2, 4, 3])))\n",
    "print(list(itertools.dropwhile(lambda x: x < 3, [1, 5, 2, 4, 3])))\n",
    "# zip\n",
    "print(list(zip(range(5), range(3))))\n",
    "print(list(itertools.zip_longest(range(5), range(3))))\n",
    "\n",
    "# itertools.groupby\n",
    "animals = ['rat', 'bear', 'duck', 'bat', 'eagle', 'shark', 'dolphin', 'lion']\n",
    "# groupby 需要假定输入的可迭代对象已经按照分组标准进行排序（至少同组的元素要连在一起）\n",
    "print('----')\n",
    "for length, animal in itertools.groupby(animals, len):\n",
    "    print(length, list(animal))\n",
    "print('----')\n",
    "animals.sort(key=len)\n",
    "for length, animal in itertools.groupby(animals, len):\n",
    "    print(length, list(animal))\n",
    "print('---')\n",
    "# tee\n",
    "g1, g2 = itertools.tee('abc', 2)\n",
    "print(list(zip(g1, g2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5] [1, 2, 3, 4, 5]\n"
     ]
    }
   ],
   "source": [
    "# 使用 yield from 语句可以在生成器函数中直接迭代一个迭代器\n",
    "from itertools import chain\n",
    "\n",
    "def my_itertools_chain(*iterators):\n",
    "    for iterator in iterators:\n",
    "        yield from iterator\n",
    "\n",
    "chain1 = my_itertools_chain([1, 2], [3, 4, 5])\n",
    "chain2 = chain([1, 2, 3], [4, 5])\n",
    "print(list(chain1), list(chain2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`iter` 函数还有一个鲜为人知的用法：传入两个参数，使用常规的函数或任何可调用的对象创建迭代器。这样使用时，第一个参数必须是可调用的对象，用于不断调用（没有参数），产出各个值；第二个值是哨符，这是个标记值，当可调用的对象返回这个值时，触发迭代器抛出 StopIteration 异常，而不产出哨符。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# iter 的神奇用法\n",
    "# iter(callable, sentinel)\n",
    "import random\n",
    "\n",
    "def rand():\n",
    "    return random.randint(1, 6)\n",
    "# 不停调用 rand(), 直到产出一个 5\n",
    "print(list(iter(rand, 5)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}