linux-kernel-labs.github.io/tmp/labs/memory_mapping.html at master · devzero2000/linux-kernel-labs.github.io

751 lines (629 loc) · 51.5 KB
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Memory mapping &mdash; The Linux Kernel  documentation</title>
  <script type="text/javascript" src="../_static/js/modernizr.min.js"></script>
      <script type="text/javascript">
          var DOCUMENTATION_OPTIONS = {
              URL_ROOT:'../',
              VERSION:'',
              LANGUAGE:'None',
              COLLAPSE_INDEX:false,
              FILE_SUFFIX:'.html',
              HAS_SOURCE:  true,
              SOURCELINK_SUFFIX: '.txt'
      </script>
        <script type="text/javascript" src="../_static/jquery.js"></script>
        <script type="text/javascript" src="../_static/underscore.js"></script>
        <script type="text/javascript" src="../_static/doctools.js"></script>
        <script type="text/javascript" src="../_static/asciinema-player.js"></script>
    <script type="text/javascript" src="../_static/js/theme.js"></script>
  <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
  <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../_static/asciinema-player.css" type="text/css" />
  <link rel="stylesheet" href="../_static/theme_overrides.css" type="text/css" />
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
    <link rel="next" title="Linux Device Model" href="device_model.html" />
    <link rel="prev" title="Networking" href="networking.html" /> 
<body class="wy-body-for-nav">
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >
            <a href="../index.html" class="icon icon-home"> The Linux Kernel
          </a>
              <div class="version">
              </div>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
        </div>
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
              <p class="caption"><span class="caption-text">Lectures</span></p>
<li class="toctree-l1"><a class="reference internal" href="../lectures/so2.cs.pub.ro.html">Sisteme de operare 2</a></li>
<li class="toctree-l1"><a class="reference internal" href="../lectures/intro.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="../lectures/syscalls.html">System Calls</a></li>
<li class="toctree-l1"><a class="reference internal" href="../lectures/interrupts.html">Interrupts</a></li>
<li class="toctree-l1"><a class="reference internal" href="../lectures/smp.html">Symmetric Multi-Processing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../lectures/debugging.html">Debugging</a></li>
<li class="toctree-l1"><a class="reference internal" href="vm.html">Virtual Machine Setup</a></li>
<p class="caption"><span class="caption-text">Labs</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="infrastructure.html">Infrastructure</a></li>
<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="kernel_modules.html">Kernel modules</a></li>
<li class="toctree-l1"><a class="reference internal" href="kernel_api.html">Kernel API</a></li>
<li class="toctree-l1"><a class="reference internal" href="device_drivers.html">Character device drivers</a></li>
<li class="toctree-l1"><a class="reference internal" href="interrupts.html">I/O access and Interrupts</a></li>
<li class="toctree-l1"><a class="reference internal" href="deferred_work.html">Deferred work</a></li>
<li class="toctree-l1"><a class="reference internal" href="block_device_drivers.html">Block Device Drivers</a></li>
<li class="toctree-l1"><a class="reference internal" href="filesystems_part1.html">File system drivers (Part 1)</a></li>
<li class="toctree-l1"><a class="reference internal" href="filesystems_part2.html">File system drivers (Part 2)</a></li>
<li class="toctree-l1"><a class="reference internal" href="networking.html">Networking</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Memory mapping</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#lab-objectives">Lab objectives</a></li>
<li class="toctree-l2"><a class="reference internal" href="#overview">Overview</a></li>
<li class="toctree-l2"><a class="reference internal" href="#structures-used-for-memory-mapping">Structures used for memory mapping</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#struct-page"><code class="docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#struct-vm-area-struct"><code class="docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="#struct-mm-struct"><code class="docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code></a></li>
<li class="toctree-l2"><a class="reference internal" href="#device-driver-memory-mapping">Device driver memory mapping</a></li>
<li class="toctree-l2"><a class="reference internal" href="#further-reading">Further reading</a></li>
<li class="toctree-l2"><a class="reference internal" href="#exercises">Exercises</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#mapping-contiguous-physical-memory-to-userspace">1. Mapping contiguous physical memory to userspace</a></li>
<li class="toctree-l3"><a class="reference internal" href="#mapping-non-contiguous-physical-memory-to-userspace">2. Mapping non-contiguous physical memory to userspace</a></li>
<li class="toctree-l3"><a class="reference internal" href="#read-write-operations-in-mapped-memory">3. Read / write operations in mapped memory</a></li>
<li class="toctree-l3"><a class="reference internal" href="#display-memory-mapped-in-procfs">4. Display memory mapped in procfs</a></li>
<li class="toctree-l1"><a class="reference internal" href="device_model.html">Linux Device Model</a></li>
        </div>
      </div>
    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
      <nav class="wy-nav-top" aria-label="top navigation">
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../index.html">The Linux Kernel</a>
      </nav>
      <div class="wy-nav-content">
        <div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../index.html">Docs</a> &raquo;</li>
      <li>Memory mapping</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../_sources/labs/memory_mapping.rst.txt" rel="nofollow"> View page source</a>
      </li>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
  <div class="section" id="memory-mapping">
<h1>Memory mapping<a class="headerlink" href="#memory-mapping" title="Permalink to this headline">¶</a></h1>
<div class="section" id="lab-objectives">
<h2>Lab objectives<a class="headerlink" href="#lab-objectives" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li>Understand address space mapping mechanisms</li>
<li>Learn about the most important structures related to memory management</li>
<p>Keywords:</p>
<ul class="simple">
<li>address space</li>
<li><code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code></li>
<li><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code></li>
<li><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code></li>
<li><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_struct</span></code></li>
<li><code class="xref c c-type docutils literal"><span class="pre">remap_pfn_range</span></code></li>
<li><code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code></li>
<li><code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code></li>
<div class="section" id="overview">
<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2>
<p>In the Linux kernel it is possible to map a kernel address space to a
user address space. This eliminates the overhead of copying user space
information into the kernel space and vice versa. This can be done
through a device driver and the user space device interface
(<code class="file docutils literal"><span class="pre">/dev</span></code>).</p>
<p>This feature can be used by implementing the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> operation
in the device driver’s <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">file_operations</span></code> and using the
<code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> system call in user space.</p>
<p>The basic unit for virtual memory management is a page, which size is
usually 4K, but it can be up to 64K on some platforms. Whenever we
work with virtual memory we work with two types of addresses: virtual
address and physical address. All CPU access (including from kernel
space) uses virtual addresses that are translated by the MMU into
physical addresses with the help of page tables.</p>
<p>A physical page of memory is identified by the Page Frame Number
(PFN). The PFN can be easily computed from the physical address by
dividing it with the size of the page (or by shifting the physical
address with PAGE_SHIFT bits to the right).</p>
<a class="reference internal image-reference" href="../_images/paging.png"><img alt="../_images/paging.png" src="../_images/paging.png" style="width: 49%;" /></a>
<p>For efficiency reasons, the virtual address space is divided into
user space and kernel space. For the same reason, the kernel space
contains a memory mapped zone, called <strong>lowmem</strong>, which is contiguously
mapped in physical memory, starting from the lowest possible physical
address (usually 0). The virtual address where lowmem is mapped is
defined by <code class="xref c c-macro docutils literal"><span class="pre">PAGE_OFFSET</span></code>.</p>
<p>On a 32bit system, not all available memory can be mapped in lowmem and
because of that there is a separate zone in kernel space called
<strong>highmem</strong> which can be used to arbitrarily map physical memory.</p>
<p>Memory allocated by <code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code> resides in lowmem and it is
physically contiguous. Memory allocated by <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code> is not
contiguous and does not reside in lowmem (it has a dedicated zone in
highmem).</p>
<a class="reference internal image-reference" href="../_images/kernel-virtmem-map.png"><img alt="../_images/kernel-virtmem-map.png" src="../_images/kernel-virtmem-map.png" style="width: 49%;" /></a>
<div class="section" id="structures-used-for-memory-mapping">
<h2>Structures used for memory mapping<a class="headerlink" href="#structures-used-for-memory-mapping" title="Permalink to this headline">¶</a></h2>
<p>Before discussing the mechanism of memory-mapping a device, we will
present some of the basic structures related to the memory management
subsystem of the Linux kernel.</p>
<p>Before discussing about the memory mapping mechanism over a device,
we will present some of the basic structures used by the Linux memory
management subsystem.
Some of the basic structures are: <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code>,
<code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code>, <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code>.</p>
<div class="section" id="struct-page">
<h3><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code><a class="headerlink" href="#struct-page" title="Permalink to this headline">¶</a></h3>
<p><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code> is used to embed information about all physical
pages in the system. The kernel has a <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code> structure
for all pages in the system.</p>
<p>There are many functions that interact with this structure:</p>
<ul class="simple">
<li><code class="xref c c-func docutils literal"><span class="pre">virt_to_page()</span></code> returns the page associated with a virtual
address</li>
<li><code class="xref c c-func docutils literal"><span class="pre">pfn_to_page()</span></code> returns the page associated with a page frame
number</li>
<li><code class="xref c c-func docutils literal"><span class="pre">page_to_pfn()</span></code> return the page frame number associated with a
<code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">page</span></code></li>
<li><code class="xref c c-func docutils literal"><span class="pre">page_address()</span></code> returns the virtual address of a
<code class="xref c c-type docutils literal"><span class="pre">struc</span> <span class="pre">page</span></code>; this functions can be called only for pages from
lowmem</li>
<li><code class="xref c c-func docutils literal"><span class="pre">kmap()</span></code> creates a mapping in kernel for an arbitrary physical
page (can be from highmem) and returns a virtual address that can be
used to directly reference the page</li>
<div class="section" id="struct-vm-area-struct">
<h3><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code><a class="headerlink" href="#struct-vm-area-struct" title="Permalink to this headline">¶</a></h3>
<p><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code> holds information about a contiguous
virtual memory area. The memory areas of a process can be viewed by
inspecting the <em>maps</em> attribute of the process via procfs:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>root@qemux86:~# cat /proc/1/maps
<span class="c1">#address          perms offset  device inode     pathname</span>
<span class="m">08048000</span>-08050000 r-xp <span class="m">00000000</span> fe:00 <span class="m">761</span>        /sbin/init.sysvinit
<span class="m">08050000</span>-08051000 r--p <span class="m">00007000</span> fe:00 <span class="m">761</span>        /sbin/init.sysvinit
<span class="m">08051000</span>-08052000 rw-p <span class="m">00008000</span> fe:00 <span class="m">761</span>        /sbin/init.sysvinit
092e1000-09302000 rw-p <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>          <span class="o">[</span>heap<span class="o">]</span>
4480c000-4482e000 r-xp <span class="m">00000000</span> fe:00 <span class="m">576</span>        /lib/ld-2.25.so
4482e000-4482f000 r--p <span class="m">00021000</span> fe:00 <span class="m">576</span>        /lib/ld-2.25.so
4482f000-44830000 rw-p <span class="m">00022000</span> fe:00 <span class="m">576</span>        /lib/ld-2.25.so
<span class="m">44832000</span>-449a9000 r-xp <span class="m">00000000</span> fe:00 <span class="m">581</span>        /lib/libc-2.25.so
449a9000-449ab000 r--p <span class="m">00176000</span> fe:00 <span class="m">581</span>        /lib/libc-2.25.so
449ab000-449ac000 rw-p <span class="m">00178000</span> fe:00 <span class="m">581</span>        /lib/libc-2.25.so
449ac000-449af000 rw-p <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>
b7761000-b7763000 rw-p <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>
b7763000-b7766000 r--p <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>          <span class="o">[</span>vvar<span class="o">]</span>
b7766000-b7767000 r-xp <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>          <span class="o">[</span>vdso<span class="o">]</span>
bfa15000-bfa36000 rw-p <span class="m">00000000</span> <span class="m">00</span>:00 <span class="m">0</span>          <span class="o">[</span>stack<span class="o">]</span>
</pre></div>
<p>A memory area is characterized by a start address, a stop address,
length, permissions.</p>
<p>A <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code> is created at each <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code>
call issued from user space. A driver that supports the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code>
operation must complete and initialize the associated
<code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code>. The most important fields of this
structure are:</p>
<ul class="simple">
<li><code class="xref c c-member docutils literal"><span class="pre">vm_start</span></code>, <code class="xref c c-member docutils literal"><span class="pre">vm_end</span></code> - the beginning and the end of
the memory area, respectively (these fields also appear in
<code class="file docutils literal"><span class="pre">/proc/&lt;pid&gt;/maps</span></code>);</li>
<li><code class="xref c c-member docutils literal"><span class="pre">vm_file</span></code> - the pointer to the associated file structure (if any);</li>
<li><code class="xref c c-member docutils literal"><span class="pre">vm_pgoff</span></code> - the offset of the area within the file;</li>
<li><code class="xref c c-member docutils literal"><span class="pre">vm_flags</span></code> - a set of flags;</li>
<li><code class="xref c c-member docutils literal"><span class="pre">vm_ops</span></code> - a set of working functions for this area</li>
<li><code class="xref c c-member docutils literal"><span class="pre">vm_next</span></code>, <code class="xref c c-member docutils literal"><span class="pre">vm_prev</span></code> - the areas of the same process
are chained by a list structure</li>
<div class="section" id="struct-mm-struct">
<h3><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code><a class="headerlink" href="#struct-mm-struct" title="Permalink to this headline">¶</a></h3>
<p><code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code> encompasses all memory areas associated
with a process. The <code class="xref c c-member docutils literal"><span class="pre">mm</span></code> field of <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">task_struct</span></code>
is a pointer to the <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code> of the current process.</p>
<div class="section" id="device-driver-memory-mapping">
<h2>Device driver memory mapping<a class="headerlink" href="#device-driver-memory-mapping" title="Permalink to this headline">¶</a></h2>
<p>Memory mapping is one of the most interesting features of a Unix
system. From a driver’s point of view, the memory-mapping facility
allows direct memory access to a user space device.</p>
<p>To assign a <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> operation to a driver, the <code class="xref c c-member docutils literal"><span class="pre">mmap</span></code>
field of the device driver’s <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">file_operations</span></code> must be
implemented. If that is the case, the user space process can then use
the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> system call on a file descriptor associated with
the device.</p>
<p>The mmap system call takes the following parameters:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="kt">void</span> <span class="o">*</span><span class="nf">mmap</span><span class="p">(</span><span class="n">caddr_t</span> <span class="n">addr</span><span class="p">,</span> <span class="kt">size_t</span> <span class="n">len</span><span class="p">,</span> <span class="kt">int</span> <span class="n">prot</span><span class="p">,</span>
           <span class="kt">int</span> <span class="n">flags</span><span class="p">,</span> <span class="kt">int</span> <span class="n">fd</span><span class="p">,</span> <span class="kt">off_t</span> <span class="n">offset</span><span class="p">);</span>
</pre></div>
<p>To map memory between a device and user space, the user process must
open the device and issue the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> system call with the resulting
file descriptor.</p>
<p>The device driver <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> operation has the following signature:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="kt">int</span> <span class="p">(</span><span class="o">*</span><span class="n">mmap</span><span class="p">)(</span><span class="k">struct</span> <span class="n">file</span> <span class="o">*</span><span class="n">filp</span><span class="p">,</span> <span class="k">struct</span> <span class="n">vm_area_struct</span> <span class="o">*</span><span class="n">vma</span><span class="p">);</span>
</pre></div>
<p>The <em>filp</em> field is a pointer to a <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">file</span></code> created when
the device is opened from user space. The <em>vma</em> field is used to
indicate the virtual address space where the memory should be mapped
by the device. A driver should allocate memory (using
<code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code>, <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>, <code class="xref c c-func docutils literal"><span class="pre">alloc_pages()</span></code>) and then
map it to the user address space as indicated by the <em>vma</em> parameter
using helper functions such as <code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code>.</p>
<p><code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code> will map a contiguous physical address space
into the virtual space represented by <code class="xref c c-type docutils literal"><span class="pre">vm_area_struct</span></code>:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="kt">int</span> <span class="nf">remap_pfn_range</span> <span class="p">(</span><span class="n">structure</span> <span class="n">vm_area_struct</span> <span class="o">*</span><span class="n">vma</span><span class="p">,</span> <span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">addr</span><span class="p">,</span>
                     <span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">pfn</span><span class="p">,</span> <span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">size</span><span class="p">,</span> <span class="n">pgprot_t</span> <span class="n">prot</span><span class="p">);</span>
</pre></div>
<p><code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code> expects the following parameters:</p>
<ul class="simple">
<li><em>vma</em>  - the virtual memory space in which mapping is made;</li>
<li><em>addr</em> - the virtual address space from where remapping begins; page
tables for the virtual address space between addr and addr + size
will be formed as needed</li>
<li><em>pfn</em> the page frame number to which the virtual address should be
mapped</li>
<li><em>size</em> - the size (in bytes) of the memory to be mapped</li>
<li><em>prot</em> - protection flags for this mapping</li>
<p>Here is an example of using this function that contiguously maps the
physical memory starting at page frame number <em>pfn</em> (memory that was
previously allocated) to the <em>vma-&gt;vm_start</em> virtual address:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="k">struct</span> <span class="n">vm_area_struct</span> <span class="o">*</span><span class="n">vma</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">len</span> <span class="o">=</span> <span class="n">vma</span><span class="o">-&gt;</span><span class="n">vm_end</span> <span class="o">-</span> <span class="n">vma</span><span class="o">-&gt;</span><span class="n">vm_start</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">ret</span> <span class="p">;</span>
<span class="n">ret</span> <span class="o">=</span> <span class="n">remap_pfn_range</span><span class="p">(</span><span class="n">vma</span><span class="p">,</span> <span class="n">vma</span><span class="o">-&gt;</span><span class="n">vm_start</span><span class="p">,</span> <span class="n">pfn</span><span class="p">,</span> <span class="n">len</span><span class="p">,</span> <span class="n">vma</span><span class="o">-&gt;</span><span class="n">vm_page_prot</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">ret</span> <span class="o">&lt;</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
    <span class="n">pr_err</span><span class="p">(</span><span class="s">&quot;could not map the address area</span><span class="se">\n</span><span class="s">&quot;</span><span class="p">);</span>
    <span class="k">return</span> <span class="o">-</span><span class="n">EIO</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>To obtain the page frame number of the physical memory we must
consider how the memory allocation was performed. For each
<code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code>, <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>, <code class="xref c c-func docutils literal"><span class="pre">alloc_pages()</span></code>, we must
used a different approach. For <code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code> we can use something
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">char</span> <span class="o">*</span><span class="n">kmalloc_area</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">pfn</span> <span class="o">=</span> <span class="n">virt_to_phys</span><span class="p">((</span><span class="kt">void</span> <span class="o">*</span><span class="p">)</span><span class="n">kmalloc_area</span><span class="p">)</span><span class="o">&gt;&gt;</span><span class="n">PAGE_SHIFT</span><span class="p">;</span>
</pre></div>
<p>while for <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="k">static</span> <span class="kt">char</span> <span class="o">*</span><span class="n">vmalloc_area</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">pfn</span> <span class="o">=</span> <span class="n">vmalloc_to_pfn</span><span class="p">(</span><span class="n">vmalloc_area</span><span class="p">);</span>
</pre></div>
<p>and finally for <code class="xref c c-func docutils literal"><span class="pre">alloc_pages()</span></code>:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="k">struct</span> <span class="n">page</span> <span class="o">*</span><span class="n">page</span><span class="p">;</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">pfn</span> <span class="o">=</span> <span class="n">page_to_pfn</span><span class="p">(</span><span class="n">page</span><span class="p">);</span>
</pre></div>
<div class="admonition attention">
<p class="first admonition-title">Attention</p>
<p class="last">Note that memory allocated with <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code> is not
physically contiguous so if we want to map a range alocated
with <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>, we have to map each page individually
and compute the physical address for each page.</p>
<p>Since the pages are mapped to user space, they might be swapped
out. To avoid this we must set the PG_reserved bit on the page.
Enabling is done using <code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code> while reseting it
(which must be done before freeing the memory) is done with
<code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code>:</p>
<div class="highlight-c"><div class="highlight"><pre><span></span><span class="kt">void</span> <span class="nf">alloc_mmap_pages</span><span class="p">(</span><span class="kt">int</span> <span class="n">npages</span><span class="p">)</span>
<span class="p">{</span>
    <span class="kt">int</span> <span class="n">i</span><span class="p">;</span>
    <span class="kt">char</span> <span class="o">*</span><span class="n">mem</span> <span class="o">=</span> <span class="n">kmalloc</span><span class="p">(</span><span class="n">PAGE_SIZE</span> <span class="o">*</span> <span class="n">npages</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">mem</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">mem</span><span class="p">;</span>
    <span class="k">for</span><span class="p">(</span><span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">npages</span> <span class="o">*</span> <span class="n">PAGE_SIZE</span><span class="p">;</span> <span class="n">i</span> <span class="o">+=</span> <span class="n">PAGE_SIZE</span><span class="p">)</span> <span class="p">{</span>
        <span class="n">SetPageReserved</span><span class="p">(</span><span class="n">virt_to_page</span><span class="p">(((</span><span class="kt">unsigned</span> <span class="kt">long</span><span class="p">)</span><span class="n">mem</span><span class="p">)</span> <span class="o">+</span> <span class="n">i</span><span class="p">));</span>
    <span class="k">return</span> <span class="n">mem</span><span class="p">;</span>
<span class="p">}</span>
<span class="kt">void</span> <span class="n">free_mmap_pages</span><span class="p">(</span><span class="kt">void</span> <span class="o">*</span><span class="n">mem</span><span class="p">,</span> <span class="kt">int</span> <span class="n">npages</span><span class="p">)</span>
<span class="p">{</span>
    <span class="kt">int</span> <span class="n">i</span><span class="p">;</span>
    <span class="k">for</span><span class="p">(</span><span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">npages</span> <span class="o">*</span> <span class="n">PAGE_SIZE</span><span class="p">;</span> <span class="n">i</span> <span class="o">+=</span> <span class="n">PAGE_SIZE</span><span class="p">)</span> <span class="p">{</span>
        <span class="n">ClearPageReserved</span><span class="p">(</span><span class="n">virt_to_page</span><span class="p">(((</span><span class="kt">unsigned</span> <span class="kt">long</span><span class="p">)</span><span class="n">mem</span><span class="p">)</span> <span class="o">+</span> <span class="n">i</span><span class="p">));</span>
    <span class="n">kfree</span><span class="p">(</span><span class="n">mem</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<div class="section" id="further-reading">
<h2>Further reading<a class="headerlink" href="#further-reading" title="Permalink to this headline">¶</a></h2>
<ul class="simple">
<li><a class="reference external" href="http://lwn.net/images/pdf/LDD3/ch15.pdf">Linux Device Drivers 3rd Edition - Chapter 15. Memory Mapping and DMA</a></li>
<li><a class="reference external" href="http://www.xml.com/ldd/chapter/book/ch13.html">Linux Device Driver mmap Skeleton</a></li>
<li><a class="reference external" href="http://lwn.net/Articles/28746/">Driver porting: supporting mmap ()</a></li>
<li><a class="reference external" href="http://www.linuxjournal.com/article/1287">Device Drivers Concluded</a></li>
<li><a class="reference external" href="http://en.wikipedia.org/wiki/Mmap">mmap</a></li>
<div class="section" id="exercises">
<h2>Exercises<a class="headerlink" href="#exercises" title="Permalink to this headline">¶</a></h2>
<div class="admonition important">
<p class="first admonition-title">Important</p>
<p>To solve exercises, you need to perform these steps:</p>
<blockquote>
<div><ul class="simple">
<li>prepare skeletons from templates</li>
<li>build modules</li>
<li>copy modules to the VM</li>
<li>start the VM and test the module in the VM.</li>
</div></blockquote>
<p>The current lab name is memory_mapping. See the exercises for the task name.</p>
<p>The skeleton code is generated from full source examples located in
<code class="file docutils literal"><span class="pre">tools/labs/templates</span></code>. To solve the tasks, start by generating
the skeleton code for a complete lab:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>tools/labs $ make clean
tools/labs $ <span class="nv">LABS</span><span class="o">=</span>&lt;lab name&gt; make skels
</pre></div>
<p>You can also generate the skeleton for a single task, using</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>tools/labs $ <span class="nv">LABS</span><span class="o">=</span>&lt;lab name&gt;/&lt;task name&gt; make skels
</pre></div>
<p>Once the skeleton drivers are generated, build the source:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>tools/labs $ make build
</pre></div>
<p>Then, copy the modules and start the VM:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>tools/labs $ make copy
tools/labs $ make boot
</pre></div>
<p>The modules are placed in /home/root/skels/memory_mapping/&lt;task_name&gt;.</p>
<p>Alternatively, we can copy files via <strong class="command">scp</strong>, in order to avoid restarting the VM.
For additional details about connecting to the VM via the network, please check <a class="reference internal" href="vm.html#vm-interaction-link"><span class="std std-ref">Connecting to the VM</span></a>.</p>
<p class="last">Review the <a class="reference internal" href="#exercises">Exercises</a> section for more detailed information.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p>Before starting the exercises or generating the skeletons, please run <strong>git pull</strong> inside the Linux repo,
to make sure you have the latest version of the exercises.</p>
<p>If you have local changes, the pull command will fail. Check for local changes using <code class="docutils literal"><span class="pre">git</span> <span class="pre">status</span></code>.
If you want to keep them, run <code class="docutils literal"><span class="pre">git</span> <span class="pre">stash</span></code> before <code class="docutils literal"><span class="pre">pull</span></code> and <code class="docutils literal"><span class="pre">git</span> <span class="pre">stash</span> <span class="pre">pop</span></code> after.
To discard the changes, run <code class="docutils literal"><span class="pre">git</span> <span class="pre">reset</span> <span class="pre">--hard</span> <span class="pre">master</span></code>.</p>
<p class="last">If you already generated the skeleton before <code class="docutils literal"><span class="pre">git</span> <span class="pre">pull</span></code> you will need to generate it again.</p>
<div class="section" id="mapping-contiguous-physical-memory-to-userspace">
<h3>1. Mapping contiguous physical memory to userspace<a class="headerlink" href="#mapping-contiguous-physical-memory-to-userspace" title="Permalink to this headline">¶</a></h3>
<p>Implement a device driver that maps contiguous physical memory
(e.g. obtained via <code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code>) to userspace.</p>
<p>Review the <a class="reference internal" href="#device-driver-memory-mapping">Device driver memory mapping</a> section, generate the
skeleton for the task named <strong>kmmap</strong> and fill in the areas marked
with <strong>TODO 1</strong>.</p>
<p>Start with allocating a NPAGES+2 memory area page using <code class="xref c c-func docutils literal"><span class="pre">kmalloc()</span></code>
in the module init function and find the first address in the area that is
aligned to a page boundary.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p>The size of a page is <em>PAGE_SIZE</em>.</p>
<p>Store the allocated area in <em>kmalloc_ptr</em> and the page
aligned address in <em>kmalloc_area</em>:</p>
<p class="last">Use <code class="xref c c-func docutils literal"><span class="pre">PAGE_ALIGN()</span></code> to determine <em>kmalloc_area</em>.</p>
<p>Enable the PG_reserved bit of each page with
<code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code>. Clear the bit with
<code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code> before freeing the memory.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">Use <code class="xref c c-func docutils literal"><span class="pre">virt_to_page()</span></code> to translate virtual pages into
physical pages, as required by <code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code>
and <code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code>.</p>
<p>For verification purpose (using the test below), fill in the first 4
bytes of each page with the following values: 0xaa, 0xbb, 0xcc, 0xdd.</p>
<p>Implement the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> driver function.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p>For mapping, use <code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code>. The third
argument for <code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code> is a page frame number (PFN).</p>
<p>To convert from virtual kernel address to physical address,
use <code class="xref c c-func docutils literal"><span class="pre">virt_to_phys()</span></code>.</p>
<p class="last">To convert a physical address to its PFN, shift the address
with PAGE_SHIFT bits to the right.</p>
<p>For testing, use <code class="file docutils literal"><span class="pre">test/mmap-test</span></code>. If everything goes well, the test
will show “matched” messages.</p>
<div class="section" id="mapping-non-contiguous-physical-memory-to-userspace">
<h3>2. Mapping non-contiguous physical memory to userspace<a class="headerlink" href="#mapping-non-contiguous-physical-memory-to-userspace" title="Permalink to this headline">¶</a></h3>
<p>Implement a device driver that maps non-contiguous physical memory
(e.g. obtained via <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>) to userspace.</p>
<p>Review the <a class="reference internal" href="#device-driver-memory-mapping">Device driver memory mapping</a> section, generate the
skeleton for the task named <strong>vmmap</strong> and fill in the areas marked
with <strong>TODO 2</strong>.</p>
<p>Allocate a memory area of NPAGES with <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code>.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">The size of a page is <em>PAGE_SIZE</em>.
Store the allocated area in <em>vmalloc_area</em>.
Memory allocated by <code class="xref c c-func docutils literal"><span class="pre">vmalloc()</span></code> is paged aligned.</p>
<p>Enable the PG_reserved bit of each page with
<code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code>. Clear the bit with
<code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code> before freeing the memory.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">Use <code class="xref c c-func docutils literal"><span class="pre">vmalloc_to_page()</span></code> to translate virtual pages
into physical pages used by the functions
<code class="xref c c-func docutils literal"><span class="pre">SetPageReserved()</span></code> and <code class="xref c c-func docutils literal"><span class="pre">ClearPageReserved()</span></code>.</p>
<p>For verification purpose (using the test below), fill in the first 4
bytes of each page with the following values: 0xaa, 0xbb, 0xcc, 0xdd.</p>
<p>Implement the mmap driver function.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">To convert from virtual vmalloc address to physical address,
use <code class="xref c c-func docutils literal"><span class="pre">vmalloc_to_pfn()</span></code> which returns a PFN directly.</p>
<div class="admonition attention">
<p class="first admonition-title">Attention</p>
<p>vmalloc pages are not physically contiguous so it is
needed to use <code class="xref c c-func docutils literal"><span class="pre">remap_pfn_range()</span></code> for each page.</p>
<p>Loop through all virtual pages and for each:
* determine the physical address
* map it with <code class="xref c c-func docutils literal"><span class="pre">remap_fpn_range()</span></code></p>
<p class="last">Make sure the that you determine the physical address
each time and that you use a range of one page for mapping.</p>
<p>For testing, use <em>test/mmap-test</em>. If everything goes well the test
will show “matched” messages.</p>
<div class="section" id="read-write-operations-in-mapped-memory">
<h3>3. Read / write operations in mapped memory<a class="headerlink" href="#read-write-operations-in-mapped-memory" title="Permalink to this headline">¶</a></h3>
<p>Modify one of the previous modules to allow read / write operations on
your device. This is a didactic exercise to see that the same space
can also be used with the <code class="xref c c-func docutils literal"><span class="pre">mmap()</span></code> call and with <code class="xref c c-func docutils literal"><span class="pre">read()</span></code>
and <code class="xref c c-func docutils literal"><span class="pre">write()</span></code> calls.</p>
<p>Fill in areas marked with <strong>TODO 3</strong>.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The offset parameter sent to the read / write operation can
be ignored as all reads / writes from the test program will
be done with 0 offsets.</p>
<p>For testing run <code class="file docutils literal"><span class="pre">test/mmap-test</span></code> with 3 as parameter:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>root@qemux86:~# skels/memory_mapping/test/mmap-test <span class="m">3</span>
</pre></div>
<div class="section" id="display-memory-mapped-in-procfs">
<h3>4. Display memory mapped in procfs<a class="headerlink" href="#display-memory-mapped-in-procfs" title="Permalink to this headline">¶</a></h3>
<p>Using one of the previous modules, create a procfs file in which you
display the total memory mapped by the calling process.</p>
<p>Fill in the areas marked with <strong>TODO 4</strong>.</p>
<p>Create a new entry in procfs (<code class="xref c c-macro docutils literal"><span class="pre">PROC_ENTRY_NAME</span></code>, defined in
<code class="file docutils literal"><span class="pre">mmap-test.h</span></code>) that will show the total memory mapped by the process
that called the <code class="xref c c-func docutils literal"><span class="pre">read()</span></code> on that file.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">Use <code class="xref c c-func docutils literal"><span class="pre">proc_create()</span></code>. For the mode parameter, use 0,
and for the parent parameter use NULL. Use
<code class="xref c c-func docutils literal"><span class="pre">my_proc_file_ops()</span></code> for operations.</p>
<p>In the module exit function, delete the <code class="xref c c-macro docutils literal"><span class="pre">PROC_ENTRY_NAME</span></code> entry
using <code class="xref c c-func docutils literal"><span class="pre">remove_proc_entry()</span></code>.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>A (complex) use and description of the <code class="xref c c-type docutils literal"><span class="pre">struct</span>
<span class="pre">seq_file</span></code> interface can be found here in this <a class="reference external" href="http://tldp.org/LDP/lkmpg/2.6/html/x861.html">example</a> .</p>
<p class="last">For this exercise, just a simple use of the interface
described <a class="reference external" href="http://lwn.net/Articles/22355/">here</a> is
sufficient. Check the “extra-simple” API described there.</p>
<p>In the <code class="xref c c-func docutils literal"><span class="pre">my_seq_show()</span></code> function you will need to:</p>
<li><p class="first">Obtain the <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code> structure of the current process
using the <code class="xref c c-func docutils literal"><span class="pre">get_task_mm()</span></code> function.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">The current process is available via the <em>current</em> variable
of type <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">task_struct*</span></code>.</p>
<li><p class="first">Iterate through the entire <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code> list
associated with the process.</p>
<div class="admonition hint">
<p class="first admonition-title">Hint</p>
<p class="last">Use the variable <code class="xref c c-data docutils literal"><span class="pre">vma_iterator</span></code> and start from
<code class="xref c c-data docutils literal"><span class="pre">mm-&gt;mmap</span></code>. Use the <code class="xref c c-member docutils literal"><span class="pre">vm_next</span></code> field of
the <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">vm_area_struct</span></code> to navigate through
the list of memory areas. Stop when you reach <code class="xref c c-macro docutils literal"><span class="pre">NULL</span></code>.</p>
<li><p class="first">Use <em>vm_start</em> and <em>vm_end</em> for each area to compute the total size.</p>
<li><p class="first">Use <code class="xref c c-func docutils literal"><span class="pre">pr_info(&quot;%lx</span> <span class="pre">%lxn,</span> <span class="pre">...)()</span></code> to print <em>vm_start</em> and <em>vm_end</em> for
each area.</p>
<li><p class="first">To release <code class="xref c c-type docutils literal"><span class="pre">struct</span> <span class="pre">mm_struct</span></code>, decrement the reference
counter of the structure using <code class="xref c c-func docutils literal"><span class="pre">mmput()</span></code>.</p>
<li><p class="first">Use <code class="xref c c-func docutils literal"><span class="pre">seq_printf()</span></code> to write to the file. Show only the total count,
no other messages. Do not even show newline (n).</p>
<p>In <code class="xref c c-func docutils literal"><span class="pre">my_seq_open()</span></code> register the display function
(<code class="xref c c-func docutils literal"><span class="pre">my_seq_show()</span></code>) using <code class="xref c c-func docutils literal"><span class="pre">single_open()</span></code>.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last"><code class="xref c c-func docutils literal"><span class="pre">single_open()</span></code> can use <code class="xref c c-macro docutils literal"><span class="pre">NULL</span></code> as its third argument.</p>
<p>For testing run <code class="file docutils literal"><span class="pre">kmmap/mmap-test</span></code> with 4 as parameter:</p>
<div class="highlight-shell"><div class="highlight"><pre><span></span>root@qemux86:~# skels/memory_mapping/kmmap/mmap-test <span class="m">4</span>
</pre></div>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The test waits for a while (it has an internal sleep
instruction). As long as the test waits, use the
<strong class="command">pmap</strong> command in another console to see the
mappings of the test and compare those to the test results.</p>
           </div>
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="device_model.html" class="btn btn-neutral float-right" title="Linux Device Model" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
        <a href="networking.html" class="btn btn-neutral float-left" title="Networking" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
  <div role="contentinfo">
        &copy; Copyright The kernel development community
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 
        </div>
      </div>
    </section>
  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
  </script>
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

memory_mapping.html

Latest commit

History

memory_mapping.html

File metadata and controls
