N: Rik van Riel
E: riel@redhat.com
W: http://www.surriel.com/
-D: Linux-MM site, Documentation/sysctl/*, swap/mm readaround
+D: Linux-MM site, Documentation/admin-guide/sysctl/*, swap/mm readaround
D: kswapd fixes, random kernel hacker, rmap VM,
D: nl.linux.org administrator, minor scheduler additions
S: Red Hat Boston
Kernel code may export it for complete or partial access.
GPIOs are identified as they are inside the kernel, using integers in
- the range 0..INT_MAX. See Documentation/gpio for more information.
+ the range 0..INT_MAX. See Documentation/admin-guide/gpio for more information.
/sys/class/gpio
/export ... asks the kernel to export a GPIO to userspace
rfkill - radio frequency (RF) connector kill switch support
-For details to this subsystem look at Documentation/rfkill.txt.
+For details to this subsystem look at Documentation/driver-api/rfkill.rst.
What: /sys/class/rfkill/rfkill[0-9]+/claim
Date: 09-Jul-2007
rfkill - radio frequency (RF) connector kill switch support
-For details to this subsystem look at Documentation/rfkill.txt.
+For details to this subsystem look at Documentation/driver-api/rfkill.rst.
For the deprecated /sys/class/rfkill/*/claim knobs of this interface look in
Documentation/ABI/removed/sysfs-class-rfkill.
Contact: Linux Memory Management list <linux-mm@kvack.org>
Description:
The node's hit/miss statistics, in units of pages.
- See Documentation/numastat.txt
+ See Documentation/admin-guide/numastat.rst
What: /sys/devices/system/node/nodeX/distance
Date: October 2002
17 - sectors discarded
18 - time spent discarding
- For more details refer to Documentation/iostats.txt
+ For more details refer to Documentation/admin-guide/iostats.rst
9 - I/Os currently in progress
10 - time spent doing I/Os (ms)
11 - weighted time spent doing I/Os (ms)
- For more details refer Documentation/iostats.txt
+ For more details refer Documentation/admin-guide/iostats.rst
What: /sys/block/<disk>/<part>/stat
- Values below -2 are rejected with -EINVAL
For more information, see
- Documentation/laptops/disk-shock-protection.txt
+ Documentation/admin-guide/laptops/disk-shock-protection.rst
What: /sys/block/*/device/ncq_prio_enable
switchtec - Microsemi Switchtec PCI Switch Management Endpoint
-For details on this subsystem look at Documentation/switchtec.txt.
+For details on this subsystem look at Documentation/driver-api/switchtec.rst.
What: /sys/class/switchtec
Date: 05-Jan-2017
present: cpus that have been identified as being present in
the system.
- See Documentation/cputopology.txt for more information.
+ See Documentation/admin-guide/cputopology.rst for more information.
What: /sys/devices/system/cpu/probe
thread_siblings_list: human-readable list of cpu#'s hardware
threads within the same core as cpu#
- See Documentation/cputopology.txt for more information.
+ See Documentation/admin-guide/cputopology.rst for more information.
What: /sys/devices/system/cpu/cpuidle/current_driver
To control the LED display, use the following :
echo 0x0T000DDD > /sys/devices/platform/asus_laptop/
where T control the 3 letters display, and DDD the 3 digits display.
- The DDD table can be found in Documentation/laptops/asus-laptop.txt
+ The DDD table can be found in Documentation/admin-guide/laptops/asus-laptop.rst
What: /sys/devices/platform/asus_laptop/bluetooth
Date: January 2007
Description:
Resume on lid open. 1 means on, 0 means off.
-What: /sys/devices/platform/<platform>/fan_mode
-Date: Apr 2019
-KernelVersion: 5.2
+What: /sys/devices/platform/<platform>/fan_boost_mode
+Date: Sep 2019
+KernelVersion: 5.3
Contact: "Yurii Pavlovskyi" <yurii.pavlovskyi@gmail.com>
Description:
Fan boost mode:
If the device only supports 32-bit addressing for descriptors in the
coherent allocations, but supports full 64-bits for streaming mappings
-it would look like this:
+it would look like this::
if (dma_set_mask(dev, DMA_BIT_MASK(64))) {
dev_warn(dev, "mydev: No suitable DMA available\n");
+==================
+Control Groupstats
+==================
+
Control Groupstats is inspired by the discussion at
http://lkml.org/lkml/2007/4/11/187 and implements per cgroup statistics as
suggested by Andrew Morton in http://lkml.org/lkml/2007/4/11/263.
information will not be available.
To extract cgroup statistics a utility very similar to getdelays.c
-has been developed, the sample output of the utility is shown below
+has been developed, the sample output of the utility is shown below::
-~/balbir/cgroupstats # ./getdelays -C "/sys/fs/cgroup/a"
-sleeping 1, blocked 0, running 1, stopped 0, uninterruptible 0
-~/balbir/cgroupstats # ./getdelays -C "/sys/fs/cgroup"
-sleeping 155, blocked 0, running 1, stopped 0, uninterruptible 2
+ ~/balbir/cgroupstats # ./getdelays -C "/sys/fs/cgroup/a"
+ sleeping 1, blocked 0, running 1, stopped 0, uninterruptible 0
+ ~/balbir/cgroupstats # ./getdelays -C "/sys/fs/cgroup"
+ sleeping 155, blocked 0, running 1, stopped 0, uninterruptible 2
+================
Delay accounting
-----------------
+================
Tasks encounter delays in execution when they wait
for some kernel resource to become available e.g. a
generic data structure to userspace corresponding to per-pid and per-tgid
statistics. The delay accounting functionality populates specific fields of
this structure. See
+
include/linux/taskstats.h
+
for a description of the fields pertaining to delay accounting.
It will generally be in the form of counters returning the cumulative
delay seen for cpu, sync block I/O, swapin, memory reclaim etc.
Usage
-----
-Compile the kernel with
+Compile the kernel with::
+
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASKSTATS=y
Delay accounting is enabled by default at boot up.
-To disable, add
+To disable, add::
+
nodelayacct
+
to the kernel boot options. The rest of the instructions
below assume this has not been done.
executed and the corresponding delays to be
seen.
-General format of the getdelays command
+General format of the getdelays command::
-getdelays [-t tgid] [-p pid] [-c cmd...]
+ getdelays [-t tgid] [-p pid] [-c cmd...]
-Get delays, since system boot, for pid 10
-# ./getdelays -p 10
-(output similar to next case)
+Get delays, since system boot, for pid 10::
-Get sum of delays, since system boot, for all pids with tgid 5
-# ./getdelays -t 5
+ # ./getdelays -p 10
+ (output similar to next case)
+Get sum of delays, since system boot, for all pids with tgid 5::
-CPU count real total virtual total delay total
- 7876 92005750 100000000 24001500
-IO count delay total
- 0 0
-SWAP count delay total
- 0 0
-RECLAIM count delay total
- 0 0
+ # ./getdelays -t 5
+
+
+ CPU count real total virtual total delay total
+ 7876 92005750 100000000 24001500
+ IO count delay total
+ 0 0
+ SWAP count delay total
+ 0 0
+ RECLAIM count delay total
+ 0 0
+
+Get delays seen in executing a given simple command::
-Get delays seen in executing a given simple command
-# ./getdelays -c ls /
+ # ./getdelays -c ls /
-bin data1 data3 data5 dev home media opt root srv sys usr
-boot data2 data4 data6 etc lib mnt proc sbin subdomain tmp var
+ bin data1 data3 data5 dev home media opt root srv sys usr
+ boot data2 data4 data6 etc lib mnt proc sbin subdomain tmp var
-CPU count real total virtual total delay total
+ CPU count real total virtual total delay total
6 4000250 4000000 0
-IO count delay total
+ IO count delay total
0 0
-SWAP count delay total
+ SWAP count delay total
0 0
-RECLAIM count delay total
+ RECLAIM count delay total
0 0
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==========
+Accounting
+==========
+
+.. toctree::
+ :maxdepth: 1
+
+ cgroupstats
+ delay-accounting
+ psi
+ taskstats
+ taskstats-struct
Pressure information for each resource is exported through the
respective file in /proc/pressure/ -- cpu, memory, and io.
-The format for CPU is as such:
+The format for CPU is as such::
-some avg10=0.00 avg60=0.00 avg300=0.00 total=0
+ some avg10=0.00 avg60=0.00 avg300=0.00 total=0
-and for memory and IO:
+and for memory and IO::
-some avg10=0.00 avg60=0.00 avg300=0.00 total=0
-full avg10=0.00 avg60=0.00 avg300=0.00 total=0
+ some avg10=0.00 avg60=0.00 avg300=0.00 total=0
+ full avg10=0.00 avg60=0.00 avg300=0.00 total=0
The "some" line indicates the share of time in which at least some
tasks are stalled on a given resource.
/proc/pressure/ representing the resource to be monitored and write the
desired threshold and time window. The open file descriptor should be
used to wait for trigger events using select(), poll() or epoll().
-The following format is used:
+The following format is used::
-<some|full> <stall amount in us> <time window in us>
+ <some|full> <stall amount in us> <time window in us>
For example writing "some 150000 1000000" into /proc/pressure/memory
would add 150ms threshold for partial memory stall measured within
Userspace monitor usage example
===============================
-#include <errno.h>
-#include <fcntl.h>
-#include <stdio.h>
-#include <poll.h>
-#include <string.h>
-#include <unistd.h>
-
-/*
- * Monitor memory partial stall with 1s tracking window size
- * and 150ms threshold.
- */
-int main() {
+::
+
+ #include <errno.h>
+ #include <fcntl.h>
+ #include <stdio.h>
+ #include <poll.h>
+ #include <string.h>
+ #include <unistd.h>
+
+ /*
+ * Monitor memory partial stall with 1s tracking window size
+ * and 150ms threshold.
+ */
+ int main() {
const char trig[] = "some 150000 1000000";
struct pollfd fds;
int n;
}
return 0;
-}
+ }
Cgroup2 interface
=================
+====================
The struct taskstats
---------------------
+====================
This document contains an explanation of the struct taskstats fields.
the common fields and basic accounting fields are collected for
delivery at do_exit() of a task.
2) Delay accounting fields
- These fields are placed between
- /* Delay accounting fields start */
- and
- /* Delay accounting fields end */
+ These fields are placed between::
+
+ /* Delay accounting fields start */
+
+ and::
+
+ /* Delay accounting fields end */
+
Their values are collected if CONFIG_TASK_DELAY_ACCT is set.
3) Extended accounting fields
- These fields are placed between
- /* Extended accounting fields start */
- and
- /* Extended accounting fields end */
+ These fields are placed between::
+
+ /* Extended accounting fields start */
+
+ and::
+
+ /* Extended accounting fields end */
+
Their values are collected if CONFIG_TASK_XACCT is set.
4) Per-task and per-thread context switch count statistics
Future extension should add fields to the end of the taskstats struct, and
should not change the relative position of each field within the struct.
+::
-struct taskstats {
+ struct taskstats {
+
+1) Common and basic accounting fields::
-1) Common and basic accounting fields:
/* The version number of this struct. This field is always set to
* TAKSTATS_VERSION, which is defined in <linux/taskstats.h>.
* Each time the struct is changed, the value should be incremented.
*/
__u16 version;
- /* The exit code of a task. */
+ /* The exit code of a task. */
__u32 ac_exitcode; /* Exit status */
- /* The accounting flags of a task as defined in <linux/acct.h>
+ /* The accounting flags of a task as defined in <linux/acct.h>
* Defined values are AFORK, ASU, ACOMPAT, ACORE, and AXSIG.
*/
__u8 ac_flag; /* Record flags */
- /* The value of task_nice() of a task. */
+ /* The value of task_nice() of a task. */
__u8 ac_nice; /* task_nice */
- /* The name of the command that started this task. */
+ /* The name of the command that started this task. */
char ac_comm[TS_COMM_LEN]; /* Command name */
- /* The scheduling discipline as set in task->policy field. */
+ /* The scheduling discipline as set in task->policy field. */
__u8 ac_sched; /* Scheduling discipline */
__u8 ac_pad[3];
__u32 ac_pid; /* Process ID */
__u32 ac_ppid; /* Parent process ID */
- /* The time when a task begins, in [secs] since 1970. */
+ /* The time when a task begins, in [secs] since 1970. */
__u32 ac_btime; /* Begin time [sec since 1970] */
- /* The elapsed time of a task, in [usec]. */
+ /* The elapsed time of a task, in [usec]. */
__u64 ac_etime; /* Elapsed time [usec] */
- /* The user CPU time of a task, in [usec]. */
+ /* The user CPU time of a task, in [usec]. */
__u64 ac_utime; /* User CPU time [usec] */
- /* The system CPU time of a task, in [usec]. */
+ /* The system CPU time of a task, in [usec]. */
__u64 ac_stime; /* System CPU time [usec] */
- /* The minor page fault count of a task, as set in task->min_flt. */
+ /* The minor page fault count of a task, as set in task->min_flt. */
__u64 ac_minflt; /* Minor Page Fault Count */
/* The major page fault count of a task, as set in task->maj_flt. */
__u64 ac_majflt; /* Major Page Fault Count */
-2) Delay accounting fields:
+2) Delay accounting fields::
+
/* Delay accounting fields start
*
* All values, until the comment "Delay accounting fields end" are
/* version 1 ends here */
-3) Extended accounting fields
+3) Extended accounting fields::
+
/* Extended accounting fields start */
/* Accumulated RSS usage in duration of a task, in MBytes-usecs.
*/
__u64 coremem; /* accumulated RSS usage in MB-usec */
- /* Accumulated virtual memory usage in duration of a task.
+ /* Accumulated virtual memory usage in duration of a task.
* Same as acct_rss_mem1 above except that we keep track of VM usage.
*/
__u64 virtmem; /* accumulated VM usage in MB-usec */
- /* High watermark of RSS usage in duration of a task, in KBytes. */
+ /* High watermark of RSS usage in duration of a task, in KBytes. */
__u64 hiwater_rss; /* High-watermark of RSS usage */
- /* High watermark of VM usage in duration of a task, in KBytes. */
+ /* High watermark of VM usage in duration of a task, in KBytes. */
__u64 hiwater_vm; /* High-water virtual memory usage */
/* The following four fields are I/O statistics of a task. */
/* Extended accounting fields end */
-4) Per-task and per-thread statistics
+4) Per-task and per-thread statistics::
+
__u64 nvcsw; /* Context voluntary switch counter */
__u64 nivcsw; /* Context involuntary switch counter */
-5) Time accounting for SMT machines
+5) Time accounting for SMT machines::
+
__u64 ac_utimescaled; /* utime scaled on frequency etc */
__u64 ac_stimescaled; /* stime scaled on frequency etc */
__u64 cpu_scaled_run_real_total; /* scaled cpu_run_real_total */
-6) Extended delay accounting fields for memory reclaim
+6) Extended delay accounting fields for memory reclaim::
+
/* Delay waiting for memory reclaim */
__u64 freepages_count;
__u64 freepages_delay_total;
-}
+
+::
+
+ }
+=============================
Per-task statistics interface
------------------------------
+=============================
Taskstats is a netlink-based interface for sending per-task and
The data exchanged between user and kernel space is a netlink message belonging
to the NETLINK_GENERIC family and using the netlink attributes interface.
-The messages are in the format
+The messages are in the format::
+----------+- - -+-------------+-------------------+
| nlmsghdr | Pad | genlmsghdr | taskstats payload |
To avoid losing statistics, userspace should do one or more of the following:
- increase the receive buffer sizes for the netlink sockets opened by
-listeners to receive exit data.
+ listeners to receive exit data.
- create more listeners and reduce the number of cpus being listened to by
-each listener. In the extreme case, there could be one listener for each cpu.
-Users may also consider setting the cpu affinity of the listener to the subset
-of cpus to which it listens, especially if they are listening to just one cpu.
+ each listener. In the extreme case, there could be one listener for each cpu.
+ Users may also consider setting the cpu affinity of the listener to the subset
+ of cpus to which it listens, especially if they are listening to just one cpu.
Despite these measures, if the userspace receives ENOBUFS error messages
indicated overflow of receive buffers, it should take measures to handle the
loss of data.
-
-----
http://aoetools.sourceforge.net/
-The scripts in this Documentation/aoe directory are intended to
+The scripts in this Documentation/admin-guide/aoe directory are intended to
document the use of the driver and are not necessary if you install
the aoetools.
a convenient way. Users with aoetools should use the aoe-stat
command::
- root@makki root# sh Documentation/aoe/status.sh
+ root@makki root# sh Documentation/admin-guide/aoe/status.sh
e10.0 eth3 up
e10.1 eth3 up
e10.2 eth3 up
-:orphan:
-
=======================
ATA over Ethernet (AoE)
=======================
# udev_rules="/etc/udev/rules.d/"
# bash# ls /etc/udev/rules.d/
# 10-wacom.rules 50-udev.rules
-# bash# cp /path/to/linux/Documentation/aoe/udev.txt \
+# bash# cp /path/to/linux/Documentation/admin-guide/aoe/udev.txt \
# /etc/udev/rules.d/60-aoe.rules
#
+================================
+kernel data structure for DRBD-9
+================================
+
This describes the in kernel data structure for DRBD-9. Starting with
Linux v3.14 we are reorganizing DRBD to use this data structure.
The DRBD objects are interconnected to form a matrix as depicted below; a
drbd_peer_device object sits at each intersection between a drbd_device and a
-drbd_connection:
+drbd_connection::
/--------------+---------------+.....+---------------\
| resource | device | | device |
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+.. The here included files are intended to help understand the implementation
+
+Data flows that Relate some functions, and write packets
+========================================================
+
+.. kernel-figure:: DRBD-8.3-data-packets.svg
+ :alt: DRBD-8.3-data-packets.svg
+ :align: center
+
+.. kernel-figure:: DRBD-data-packets.svg
+ :alt: DRBD-data-packets.svg
+ :align: center
+
+
+Sub graphs of DRBD's state transitions
+======================================
+
+.. kernel-figure:: conn-states-8.dot
+ :alt: conn-states-8.dot
+ :align: center
+
+.. kernel-figure:: disk-states-8.dot
+ :alt: disk-states-8.dot
+ :align: center
+
+.. kernel-figure:: node-states-8.dot
+ :alt: node-states-8.dot
+ :align: center
+==========================================
+Distributed Replicated Block Device - DRBD
+==========================================
+
Description
+===========
DRBD is a shared-nothing, synchronously replicated block device. It
is designed to serve as a building block for high availability
Please visit http://www.drbd.org to find out more.
-The here included files are intended to help understand the implementation
-
-DRBD-8.3-data-packets.svg, DRBD-data-packets.svg
- relates some functions, and write packets.
+.. toctree::
+ :maxdepth: 1
-conn-states-8.dot, disk-states-8.dot, node-states-8.dot
- The sub graphs of DRBD's state transitions
+ data-structure-v9
+ figures
Unknown -> Primary [ label = "connected" ]
Unknown -> Secondary [ label = "connected" ]
}
-
-This file describes the floppy driver.
+=============
+Floppy Driver
+=============
FAQ list:
=========
- A FAQ list may be found in the fdutils package (see below), and also
+A FAQ list may be found in the fdutils package (see below), and also
at <http://fdutils.linux.lu/faq.html>.
LILO configuration options (Thinkpad users, read this)
======================================================
- The floppy driver is configured using the 'floppy=' option in
+The floppy driver is configured using the 'floppy=' option in
lilo. This option can be typed at the boot prompt, or entered in the
lilo configuration file.
- Example: If your kernel is called linux-2.6.9, type the following line
-at the lilo boot prompt (if you have a thinkpad):
+Example: If your kernel is called linux-2.6.9, type the following line
+at the lilo boot prompt (if you have a thinkpad)::
linux-2.6.9 floppy=thinkpad
You may also enter the following line in /etc/lilo.conf, in the description
-of linux-2.6.9:
+of linux-2.6.9::
append = "floppy=thinkpad"
- Several floppy related options may be given, example:
+Several floppy related options may be given, example::
linux-2.6.9 floppy=daring floppy=two_fdc
append = "floppy=daring floppy=two_fdc"
- If you give options both in the lilo config file and on the boot
+If you give options both in the lilo config file and on the boot
prompt, the option strings of both places are concatenated, the boot
prompt options coming last. That's why there are also options to
restore the default behavior.
Module configuration options
============================
- If you use the floppy driver as a module, use the following syntax:
-modprobe floppy floppy="<options>"
+If you use the floppy driver as a module, use the following syntax::
-Example:
- modprobe floppy floppy="omnibook messages"
+ modprobe floppy floppy="<options>"
- If you need certain options enabled every time you load the floppy driver,
-you can put:
+Example::
- options floppy floppy="omnibook messages"
+ modprobe floppy floppy="omnibook messages"
+
+If you need certain options enabled every time you load the floppy driver,
+you can put::
+
+ options floppy floppy="omnibook messages"
in a configuration file in /etc/modprobe.d/.
- The floppy driver related options are:
+The floppy driver related options are:
floppy=asus_pci
Sets the bit mask to allow only units 0 and 1. (default)
Tells the floppy driver that you have only one floppy controller.
(default)
- floppy=two_fdc
- floppy=<address>,two_fdc
+ floppy=two_fdc / floppy=<address>,two_fdc
Tells the floppy driver that you have two floppy controllers.
The second floppy controller is assumed to be at <address>.
This option is not needed if the second controller is at address
floppy=0,thinkpad
Tells the floppy driver that you don't have a Thinkpad.
- floppy=omnibook
- floppy=nodma
+ floppy=omnibook / floppy=nodma
Tells the floppy driver not to use Dma for data transfers.
This is needed on HP Omnibooks, which don't have a workable
DMA channel for the floppy driver. This option is also useful
described in the physical CMOS), or if your BIOS uses
non-standard CMOS types. The CMOS types are:
- 0 - Use the value of the physical CMOS
- 1 - 5 1/4 DD
- 2 - 5 1/4 HD
- 3 - 3 1/2 DD
- 4 - 3 1/2 HD
- 5 - 3 1/2 ED
- 6 - 3 1/2 ED
- 16 - unknown or not installed
+ == ==================================
+ 0 Use the value of the physical CMOS
+ 1 5 1/4 DD
+ 2 5 1/4 HD
+ 3 3 1/2 DD
+ 4 3 1/2 HD
+ 5 3 1/2 ED
+ 6 3 1/2 ED
+ 16 unknown or not installed
+ == ==================================
(Note: there are two valid types for ED drives. This is because 5 was
initially chosen to represent floppy *tapes*, and 6 for ED drives.
Print a warning message when an unexpected interrupt is received.
(default)
- floppy=no_unexpected_interrupts
- floppy=L40SX
+ floppy=no_unexpected_interrupts / floppy=L40SX
Don't print a message when an unexpected interrupt is received. This
is needed on IBM L40SX laptops in certain video modes. (There seems
to be an interaction between video and floppy. The unexpected
Sets the floppy DMA channel to <nr> instead of 2.
floppy=slow
- Use PS/2 stepping rate:
- " PS/2 floppies have much slower step rates than regular floppies.
+ Use PS/2 stepping rate::
+
+ PS/2 floppies have much slower step rates than regular floppies.
It's been recommended that take about 1/4 of the default speed
- in some more extreme cases."
+ in some more extreme cases.
Supporting utilities and additional documentation:
==================================================
- Additional parameters of the floppy driver can be configured at
+Additional parameters of the floppy driver can be configured at
runtime. Utilities which do this can be found in the fdutils package.
This package also contains a new version of mtools which allows to
access high capacity disks (up to 1992K on a high density 3 1/2 disk!).
It also contains additional documentation about the floppy driver.
The latest version can be found at fdutils homepage:
+
http://fdutils.linux.lu
The fdutils releases can be found at:
+
http://fdutils.linux.lu/download.html
+
http://www.tux.org/pub/knaff/fdutils/
+
ftp://metalab.unc.edu/pub/Linux/utils/disk-management/
Reporting problems about the floppy driver
==========================================
- If you have a question or a bug report about the floppy driver, mail
+If you have a question or a bug report about the floppy driver, mail
me at Alain.Knaff@poboxes.com . If you post to Usenet, preferably use
comp.os.linux.hardware. As the volume in these groups is rather high,
be sure to include the word "floppy" (or "FLOPPY") in the subject
line. If the reported problem happens when mounting floppy disks, be
sure to mention also the type of the filesystem in the subject line.
- Be sure to read the FAQ before mailing/posting any bug reports!
+Be sure to read the FAQ before mailing/posting any bug reports!
- Alain
+Alain
Changelog
=========
-10-30-2004 : Cleanup, updating, add reference to module configuration.
+10-30-2004 :
+ Cleanup, updating, add reference to module configuration.
James Nelson <james4765@gmail.com>
-6-3-2000 : Original Document
+6-3-2000 :
+ Original Document
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===========================
+The Linux RapidIO Subsystem
+===========================
+
+.. toctree::
+ :maxdepth: 1
+
+ floppy
+ nbd
+ paride
+ ramdisk
+ zram
+
+ drbd/index
+==================================
Network Block Device (TCP version)
==================================
nbds_max
Number of block devices that should be initialized (default: 16).
-
-
- Linux and parallel port IDE devices
+===================================
+Linux and parallel port IDE devices
+===================================
PARIDE v1.03 (c) 1997-8 Grant Guenther <grant@torque.net>
1. Introduction
+===============
Owing to the simplicity and near universality of the parallel port interface
to personal computers, many external devices such as portable hard-disk,
CD-ROM, LS-120 and tape drives use the parallel port to connect to their
host computer. While some devices (notably scanners) use ad-hoc methods
-to pass commands and data through the parallel port interface, most
+to pass commands and data through the parallel port interface, most
external devices are actually identical to an internal model, but with
a parallel-port adapter chip added in. Some of the original parallel port
adapters were little more than mechanisms for multiplexing a SCSI bus.
find a standard ATAPI CD-ROM drive, a power supply, and a single adapter
that interconnected a standard PC parallel port cable and a standard
IDE cable. It is usually possible to exchange the CD-ROM device with
-any other device using the IDE interface.
+any other device using the IDE interface.
The document describes the support in Linux for parallel port IDE
devices. It does not cover parallel port SCSI devices, "ditto" tape
-drives or scanners. Many different devices are supported by the
+drives or scanners. Many different devices are supported by the
parallel port IDE subsystem, including:
- MicroSolutions backpack CD-ROM
- MicroSolutions backpack PD/CD
- MicroSolutions backpack hard-drives
- MicroSolutions backpack 8000t tape drive
- SyQuest EZ-135, EZ-230 & SparQ drives
- Avatar Shark
- Imation Superdisk LS-120
- Maxell Superdisk LS-120
- FreeCom Power CD
- Hewlett-Packard 5GB and 8GB tape drives
- Hewlett-Packard 7100 and 7200 CD-RW drives
+ - MicroSolutions backpack CD-ROM
+ - MicroSolutions backpack PD/CD
+ - MicroSolutions backpack hard-drives
+ - MicroSolutions backpack 8000t tape drive
+ - SyQuest EZ-135, EZ-230 & SparQ drives
+ - Avatar Shark
+ - Imation Superdisk LS-120
+ - Maxell Superdisk LS-120
+ - FreeCom Power CD
+ - Hewlett-Packard 5GB and 8GB tape drives
+ - Hewlett-Packard 7100 and 7200 CD-RW drives
as well as most of the clone and no-name products on the market.
To support such a wide range of devices, PARIDE, the parallel port IDE
subsystem, is actually structured in three parts. There is a base
paride module which provides a registry and some common methods for
-accessing the parallel ports. The second component is a set of
-high-level drivers for each of the different types of supported devices:
+accessing the parallel ports. The second component is a set of
+high-level drivers for each of the different types of supported devices:
+ === =============
pd IDE disk
pcd ATAPI CD-ROM
pf ATAPI disk
pt ATAPI tape
pg ATAPI generic
+ === =============
(Currently, the pg driver is only used with CD-R drives).
The high-level drivers function according to the relevant standards.
The third component of PARIDE is a set of low-level protocol drivers
for each of the parallel port IDE adapter chips. Thanks to the interest
-and encouragement of Linux users from many parts of the world,
+and encouragement of Linux users from many parts of the world,
support is available for almost all known adapter protocols:
+ ==== ====================================== ====
aten ATEN EH-100 (HK)
bpck Microsolutions backpack (US)
comm DataStor (old-type) "commuter" adapter (TW)
ktti KT Technology PHd adapter (SG)
on20 OnSpec 90c20 (US)
on26 OnSpec 90c26 (US)
+ ==== ====================================== ====
2. Using the PARIDE subsystem
+=============================
While configuring the Linux kernel, you may choose either to build
the PARIDE drivers into your kernel, or to build them as modules.
as well as at least one of the high-level drivers and at least one
of the parallel port communication protocols. If you do not know
what kind of parallel port adapter is used in your drive, you could
-begin by checking the file names and any text files on your DOS
+begin by checking the file names and any text files on your DOS
installation floppy. Alternatively, you can look at the markings on
the adapter chip itself. That's usually sufficient to identify the
-correct device.
+correct device.
You can actually select all the protocol modules, and allow the PARIDE
subsystem to try them all for you.
For the "brand-name" products listed above, here are the protocol
and high-level drivers that you would use:
+ ================ ============ ====== ========
Manufacturer Model Driver Protocol
-
+ ================ ============ ====== ========
MicroSolutions CD-ROM pcd bpck
MicroSolutions PD drive pf bpck
MicroSolutions hard-drive pd bpck
Hewlett-Packard 5GB Tape pt epat
Hewlett-Packard 7200e (CD) pcd epat
Hewlett-Packard 7200e (CD-R) pg epat
+ ================ ============ ====== ========
2.1 Configuring built-in drivers
+---------------------------------
We recommend that you get to know how the drivers work and how to
configure them as loadable modules, before attempting to compile a
chain ID. While your system is booting, a number of messages are
displayed on the console. Like all such messages, they can be
reviewed with the 'dmesg' command. Among those messages will be
-some lines like:
+some lines like::
paride: bpck registered as protocol 0
paride: epat registered as protocol 1
documentation about this).
As an example, let's assume that you have a MicroSolutions PD/CD drive
-with unit ID number 36 connected to the parallel port at 0x378, a SyQuest
-EZ-135 connected to the chained port on the PD/CD drive and also an
-Imation Superdisk connected to port 0x278. You could give the following
-options on your boot command:
+with unit ID number 36 connected to the parallel port at 0x378, a SyQuest
+EZ-135 connected to the chained port on the PD/CD drive and also an
+Imation Superdisk connected to port 0x278. You could give the following
+options on your boot command::
pd.drive0=0x378,1 pf.drive0=0x278,1 pf.drive1=0x378,0,36
is the parallel port base address, the 0 is the protocol registration
number and 36 is the chain ID.
-Please note: while PARIDE will work both with and without the
+Please note: while PARIDE will work both with and without the
PARPORT parallel port sharing system that is included by the
"Parallel port support" option, PARPORT must be included and enabled
if you want to use chains of devices on the same parallel port.
2.2 Loading and configuring PARIDE as modules
+----------------------------------------------
It is much faster and simpler to get to understand the PARIDE drivers
-if you use them as loadable kernel modules.
+if you use them as loadable kernel modules.
-Note 1: using these drivers with the "kerneld" automatic module loading
-system is not recommended for beginners, and is not documented here.
+Note 1:
+ using these drivers with the "kerneld" automatic module loading
+ system is not recommended for beginners, and is not documented here.
-Note 2: if you build PARPORT support as a loadable module, PARIDE must
-also be built as loadable modules, and PARPORT must be loaded before the
-PARIDE modules.
+Note 2:
+ if you build PARPORT support as a loadable module, PARIDE must
+ also be built as loadable modules, and PARPORT must be loaded before
+ the PARIDE modules.
-To use PARIDE, you must begin by
+To use PARIDE, you must begin by::
insmod paride
Then, load as many of the protocol modules as you think you might need.
As you load each module, it will register the protocols that it supports,
-and print a log message to your kernel log file and your console. For
-example:
+and print a log message to your kernel log file and your console. For
+example::
# insmod epat
paride: epat registered as protocol 0
paride: k971 registered as protocol 2
Finally, you can load high-level drivers for each kind of device that
-you have connected. By default, each driver will autoprobe for a single
+you have connected. By default, each driver will autoprobe for a single
device, but you can support up to four similar devices by giving their
individual co-ordinates when you load the driver.
For example, if you had two no-name CD-ROM drives both using the
KingByte KBIC-951A adapter, one on port 0x378 and the other on 0x3bc
-you could give the following command:
+you could give the following command::
# insmod pcd drive0=0x378,1 drive1=0x3bc,1
For most adapters, giving a port address and protocol number is sufficient,
-but check the source files in linux/drivers/block/paride for more
+but check the source files in linux/drivers/block/paride for more
information. (Hopefully someone will write some man pages one day !).
As another example, here's what happens when PARPORT is installed, and
-a SyQuest EZ-135 is attached to port 0x378:
+a SyQuest EZ-135 is attached to port 0x378::
# insmod paride
paride: version 1.0 installed
scanner - in this case it reports that it has found a disk with one partition.
2.3 Using a PARIDE device
+--------------------------
Once the drivers have been loaded, you can access PARIDE devices in the
same way as their traditional counterparts. You will probably need to
create the device "special files". Here is a simple script that you can
-cut to a file and execute:
-
-#!/bin/bash
-#
-# mkd -- a script to create the device special files for the PARIDE subsystem
-#
-function mkdev {
- mknod $1 $2 $3 $4 ; chmod 0660 $1 ; chown root:disk $1
-}
-#
-function pd {
- D=$( printf \\$( printf "x%03x" $[ $1 + 97 ] ) )
- mkdev pd$D b 45 $[ $1 * 16 ]
- for P in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- do mkdev pd$D$P b 45 $[ $1 * 16 + $P ]
- done
-}
-#
-cd /dev
-#
-for u in 0 1 2 3 ; do pd $u ; done
-for u in 0 1 2 3 ; do mkdev pcd$u b 46 $u ; done
-for u in 0 1 2 3 ; do mkdev pf$u b 47 $u ; done
-for u in 0 1 2 3 ; do mkdev pt$u c 96 $u ; done
-for u in 0 1 2 3 ; do mkdev npt$u c 96 $[ $u + 128 ] ; done
-for u in 0 1 2 3 ; do mkdev pg$u c 97 $u ; done
-#
-# end of mkd
+cut to a file and execute::
+
+ #!/bin/bash
+ #
+ # mkd -- a script to create the device special files for the PARIDE subsystem
+ #
+ function mkdev {
+ mknod $1 $2 $3 $4 ; chmod 0660 $1 ; chown root:disk $1
+ }
+ #
+ function pd {
+ D=$( printf \\$( printf "x%03x" $[ $1 + 97 ] ) )
+ mkdev pd$D b 45 $[ $1 * 16 ]
+ for P in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
+ do mkdev pd$D$P b 45 $[ $1 * 16 + $P ]
+ done
+ }
+ #
+ cd /dev
+ #
+ for u in 0 1 2 3 ; do pd $u ; done
+ for u in 0 1 2 3 ; do mkdev pcd$u b 46 $u ; done
+ for u in 0 1 2 3 ; do mkdev pf$u b 47 $u ; done
+ for u in 0 1 2 3 ; do mkdev pt$u c 96 $u ; done
+ for u in 0 1 2 3 ; do mkdev npt$u c 96 $[ $u + 128 ] ; done
+ for u in 0 1 2 3 ; do mkdev pg$u c 97 $u ; done
+ #
+ # end of mkd
With the device files and drivers in place, you can access PARIDE devices
-like any other Linux device. For example, to mount a CD-ROM in pcd0, use:
+like any other Linux device. For example, to mount a CD-ROM in pcd0, use::
mount /dev/pcd0 /cdrom
If you have a fresh Avatar Shark cartridge, and the drive is pda, you
-might do something like:
+might do something like::
fdisk /dev/pda -- make a new partition table with
partition 1 of type 83
Devices like the Imation superdisk work in the same way, except that
they do not have a partition table. For example to make a 120MB
-floppy that you could share with a DOS system:
+floppy that you could share with a DOS system::
mkdosfs /dev/pf0
mount /dev/pf0 /mnt
2.4 The pf driver
+------------------
The pf driver is intended for use with parallel port ATAPI disk
devices. The most common devices in this category are PD drives
and LS-120 drives. Traditionally, media for these devices are not
partitioned. Consequently, the pf driver does not support partitioned
-media. This may be changed in a future version of the driver.
+media. This may be changed in a future version of the driver.
2.5 Using the pt driver
+------------------------
The pt driver for parallel port ATAPI tape drives is a minimal driver.
-It does not yet support many of the standard tape ioctl operations.
+It does not yet support many of the standard tape ioctl operations.
For best performance, a block size of 32KB should be used. You will
probably want to set the parallel port delay to 0, if you can.
2.6 Using the pg driver
+------------------------
The pg driver can be used in conjunction with the cdrecord program
to create CD-ROMs. Please get cdrecord version 1.6.1 or later
-from ftp://ftp.fokus.gmd.de/pub/unix/cdrecord/ . To record CD-R media
-your parallel port should ideally be set to EPP mode, and the "port delay"
-should be set to 0. With those settings it is possible to record at 2x
+from ftp://ftp.fokus.gmd.de/pub/unix/cdrecord/ . To record CD-R media
+your parallel port should ideally be set to EPP mode, and the "port delay"
+should be set to 0. With those settings it is possible to record at 2x
speed without any buffer underruns. If you cannot get the driver to work
in EPP mode, try to use "bidirectional" or "PS/2" mode and 1x speeds only.
3. Troubleshooting
+==================
3.1 Use EPP mode if you can
+----------------------------
The most common problems that people report with the PARIDE drivers
concern the parallel port CMOS settings. At this time, none of the
using your CMOS setup procedure.
3.2 Check the port delay
+-------------------------
Some parallel ports cannot reliably transfer data at full speed. To
offset the errors, the PARIDE protocol modules introduce a "port
linux/drivers/block/paride.
3.3 Some drives need a printer reset
+-------------------------------------
There appear to be a number of "noname" external drives on the market
that do not always power up correctly. We have noticed this with some
drives based on OnSpec and older Freecom adapters. In these rare cases,
the adapter can often be reinitialised by issuing a "printer reset" on
-the parallel port. As the reset operation is potentially disruptive in
-multiple device environments, the PARIDE drivers will not do it
-automatically. You can however, force a printer reset by doing:
+the parallel port. As the reset operation is potentially disruptive in
+multiple device environments, the PARIDE drivers will not do it
+automatically. You can however, force a printer reset by doing::
insmod lp reset=1
rmmod lp
If you have one of these marginal cases, you should probably build
your paride drivers as modules, and arrange to do the printer reset
-before loading the PARIDE drivers.
+before loading the PARIDE drivers.
3.4 Use the verbose option and dmesg if you need help
+------------------------------------------------------
While a lot of testing has gone into these drivers to make them work
as smoothly as possible, problems will arise. If you do have problems,
and that either (a) PARPORT is enabled or (b) no other device driver
is using your parallel port (check in /proc/ioports). Then, load the
appropriate drivers (you can load several protocol modules if you want)
-as in:
+as in::
# insmod paride
# insmod epat
and post your report there.
3.5 For more information or help
+---------------------------------
You can join the linux-parport mailing list by sending a mail message
-to
+to:
+
linux-parport-request@torque.net
-with the single word
+with the single word::
subscribe
You might also find some useful information on the linux-parport
web pages (although they are not always up to date) at
- http://web.archive.org/web/*/http://www.torque.net/parport/
-
-
+ http://web.archive.org/web/%2E/http://www.torque.net/parport/
+==========================================
Using the RAM disk block device with Linux
-------------------------------------------
+==========================================
-Contents:
+.. Contents:
1) Overview
2) Kernel Command Line Parameters
2a) Kernel Command Line Parameters
ramdisk_size=N
- ==============
+ Size of the ramdisk.
This parameter tells the RAM disk driver to set up RAM disks of N k size. The
default is 4096 (4 MB).
2b) Module parameters
rd_nr
- =====
- /dev/ramX devices created.
+ /dev/ramX devices created.
max_part
- ========
- Maximum partition number.
+ Maximum partition number.
rd_size
- =======
- See ramdisk_size.
+ See ramdisk_size.
3) Using "rdev -r"
------------------
prompt/wait sequence is to be given before trying to read the RAM disk. Since
the RAM disk dynamically grows as data is being written into it, a size field
is not required. Bits 11 to 13 are not currently used and may as well be zero.
-These numbers are no magical secrets, as seen below:
+These numbers are no magical secrets, as seen below::
-./arch/x86/kernel/setup.c:#define RAMDISK_IMAGE_START_MASK 0x07FF
-./arch/x86/kernel/setup.c:#define RAMDISK_PROMPT_FLAG 0x8000
-./arch/x86/kernel/setup.c:#define RAMDISK_LOAD_FLAG 0x4000
+ ./arch/x86/kernel/setup.c:#define RAMDISK_IMAGE_START_MASK 0x07FF
+ ./arch/x86/kernel/setup.c:#define RAMDISK_PROMPT_FLAG 0x8000
+ ./arch/x86/kernel/setup.c:#define RAMDISK_LOAD_FLAG 0x4000
Consider a typical two floppy disk setup, where you will have the
kernel on disk one, and have already put a RAM disk image onto disk #2.
The command line equivalent is: "prompt_ramdisk=1"
Putting that together gives 2^15 + 2^14 + 0 = 49152 for an rdev word.
-So to create disk one of the set, you would do:
+So to create disk one of the set, you would do::
/usr/src/linux# cat arch/x86/boot/zImage > /dev/fd0
/usr/src/linux# rdev /dev/fd0 /dev/fd0
/usr/src/linux# rdev -r /dev/fd0 49152
-If you make a boot disk that has LILO, then for the above, you would use:
+If you make a boot disk that has LILO, then for the above, you would use::
+
append = "ramdisk_start=0 load_ramdisk=1 prompt_ramdisk=1"
-Since the default start = 0 and the default prompt = 1, you could use:
+
+Since the default start = 0 and the default prompt = 1, you could use::
+
append = "load_ramdisk=1"
4) An Example of Creating a Compressed RAM Disk
-----------------------------------------------
+-----------------------------------------------
To create a RAM disk image, you will need a spare block device to
construct it on. This can be the RAM disk device itself, or an
Create it by writing to the RAM disk device. (This step is not currently
required, but may be in the future.) It is wise to zero out the
area (esp. for disks) so that maximal compression is achieved for
- the unused blocks of the image that you are about to create.
+ the unused blocks of the image that you are about to create::
dd if=/dev/zero of=/dev/ram0 bs=1k count=2048
-b) Make a filesystem on it. Say ext2fs for this example.
+b) Make a filesystem on it. Say ext2fs for this example::
mke2fs -vm0 /dev/ram0 2048
d) Compress the contents of the RAM disk. The level of compression
will be approximately 50% of the space used by the files. Unused
- space on the RAM disk will compress to almost nothing.
+ space on the RAM disk will compress to almost nothing::
dd if=/dev/ram0 bs=1k count=2048 | gzip -v9 > /tmp/ram_image.gz
-e) Put the kernel onto the floppy
+e) Put the kernel onto the floppy::
dd if=zImage of=/dev/fd0 bs=1k
(possibly larger) kernel onto the same floppy later without overlapping
the RAM disk image. An offset of 400 kB for kernels about 350 kB in
size would be reasonable. Make sure offset+size of ram_image.gz is
- not larger than the total space on your floppy (usually 1440 kB).
+ not larger than the total space on your floppy (usually 1440 kB)::
dd if=/tmp/ram_image.gz of=/dev/fd0 bs=1k seek=400
g) Use "rdev" to set the boot device, RAM disk offset, prompt flag, etc.
For prompt_ramdisk=1, load_ramdisk=1, ramdisk_start=400, one would
- have 2^15 + 2^14 + 400 = 49552.
+ have 2^15 + 2^14 + 400 = 49552::
rdev /dev/fd0 /dev/fd0
rdev -r /dev/fd0 49552
That is it. You now have your boot/root compressed RAM disk floppy. Some
users may wish to combine steps (d) and (f) by using a pipe.
---------------------------------------------------------------------------
+
Paul Gortmaker 12/95
Changelog:
----------
-10-22-04 : Updated to reflect changes in command line options, remove
+10-22-04 :
+ Updated to reflect changes in command line options, remove
obsolete references, general cleanup.
James Nelson (james4765@gmail.com)
-12-95 : Original Document
+12-95 :
+ Original Document
+========================================
zram: Compressed RAM based block devices
-----------------------------------------
+========================================
-* Introduction
+Introduction
+============
The zram module creates RAM based block devices named /dev/zram<id>
(<id> = 0, 1, ...). Pages written to these disks are compressed and stored
Statistics for individual zram devices are exported through sysfs nodes at
/sys/block/zram<id>/
-* Usage
+Usage
+=====
There are several ways to configure and manage zram device(-s):
+
a) using zram and zram_control sysfs attributes
b) using zramctl utility, provided by util-linux (util-linux@vger.kernel.org).
IOW, zram and zram_control sysfs attributes.
In order to get a better idea about zramctl please consult util-linux
-documentation, zramctl man-page or `zramctl --help'. Please be informed
+documentation, zramctl man-page or `zramctl --help`. Please be informed
that zram maintainers do not develop/maintain util-linux or zramctl, should
you have any questions please contact util-linux@vger.kernel.org
WARNING
=======
+
For the sake of simplicity we skip error checking parts in most of the
examples below. However, it is your sole responsibility to handle errors.
zram sysfs attributes always return negative values in case of errors.
The list of possible return codes:
--EBUSY -- an attempt to modify an attribute that cannot be changed once
-the device has been initialised. Please reset device first;
--ENOMEM -- zram was not able to allocate enough memory to fulfil your
-needs;
--EINVAL -- invalid input has been provided.
+
+======== =============================================================
+-EBUSY an attempt to modify an attribute that cannot be changed once
+ the device has been initialised. Please reset device first;
+-ENOMEM zram was not able to allocate enough memory to fulfil your
+ needs;
+-EINVAL invalid input has been provided.
+======== =============================================================
If you use 'echo', the returned value that is changed by 'echo' utility,
-and, in general case, something like:
+and, in general case, something like::
echo 3 > /sys/block/zram0/max_comp_streams
if [ $? -ne 0 ];
should suffice.
-1) Load Module:
+1) Load Module
+==============
+
+::
+
modprobe zram num_devices=4
This creates 4 devices: /dev/zram{0,1,2,3}
pre-created. Default: 1.
2) Set max number of compression streams
+========================================
+
Regardless the value passed to this attribute, ZRAM will always
allocate multiple compression streams - one per online CPUs - thus
allowing several concurrent compression operations. The number of
become offline. There is no single-compression-stream mode anymore,
unless you are running a UP system or has only 1 CPU online.
-To find out how many streams are currently available:
+To find out how many streams are currently available::
+
cat /sys/block/zram0/max_comp_streams
3) Select compression algorithm
+===============================
+
Using comp_algorithm device attribute one can see available and
currently selected (shown in square brackets) compression algorithms,
change selected compression algorithm (once the device is initialised
there is no way to change compression algorithm).
-Examples:
+Examples::
+
#show supported compression algorithms
cat /sys/block/zram0/comp_algorithm
lzo [lz4]
#select lzo compression algorithm
echo lzo > /sys/block/zram0/comp_algorithm
-For the time being, the `comp_algorithm' content does not necessarily
+For the time being, the `comp_algorithm` content does not necessarily
show every compression algorithm supported by the kernel. We keep this
list primarily to simplify device configuration and one can configure
a new device with a compression algorithm that is not listed in
-`comp_algorithm'. The thing is that, internally, ZRAM uses Crypto API
+`comp_algorithm`. The thing is that, internally, ZRAM uses Crypto API
and, if some of the algorithms were built as modules, it's impossible
to list all of them using, for instance, /proc/crypto or any other
method. This, however, has an advantage of permitting the usage of
custom crypto compression modules (implementing S/W or H/W compression).
4) Set Disksize
+===============
+
Set disk size by writing the value to sysfs node 'disksize'.
The value can be either in bytes or you can use mem suffixes.
-Examples:
+Examples::
+
# Initialize /dev/zram0 with 50MB disksize
echo $((50*1024*1024)) > /sys/block/zram0/disksize
size of the disk when not in use so a huge zram is wasteful.
5) Set memory limit: Optional
+=============================
+
Set memory limit by writing the value to sysfs node 'mem_limit'.
The value can be either in bytes or you can use mem suffixes.
In addition, you could change the value in runtime.
-Examples:
+Examples::
+
# limit /dev/zram0 with 50MB memory
echo $((50*1024*1024)) > /sys/block/zram0/mem_limit
# To disable memory limit
echo 0 > /sys/block/zram0/mem_limit
-6) Activate:
+6) Activate
+===========
+
+::
+
mkswap /dev/zram0
swapon /dev/zram0
mount /dev/zram1 /tmp
7) Add/remove zram devices
+==========================
zram provides a control interface, which enables dynamic (on-demand) device
addition and removal.
attribute. This will return either new device's device id (meaning that you
can use /dev/zram<id>) or error code.
-Example:
+Example::
+
cat /sys/class/zram-control/hot_add
1
To remove the existing /dev/zramX device (where X is a device id)
-execute
+execute::
+
echo X > /sys/class/zram-control/hot_remove
-8) Stats:
+8) Stats
+========
+
Per-device statistics are exported as various nodes under /sys/block/zram<id>/
A brief description of exported device attributes. For more details please
read Documentation/ABI/testing/sysfs-block-zram.
+====================== ====== ===============================================
Name access description
----- ------ -----------
+====================== ====== ===============================================
disksize RW show and set the device's disk size
initstate RO shows the initialization state of the device
reset WO trigger device reset
-mem_used_max WO reset the `mem_used_max' counter (see later)
-mem_limit WO specifies the maximum amount of memory ZRAM can use
- to store the compressed data
-writeback_limit WO specifies the maximum amount of write IO zram can
- write out to backing device as 4KB unit
+mem_used_max WO reset the `mem_used_max` counter (see later)
+mem_limit WO specifies the maximum amount of memory ZRAM can
+ use to store the compressed data
+writeback_limit WO specifies the maximum amount of write IO zram
+ can write out to backing device as 4KB unit
writeback_limit_enable RW show and set writeback_limit feature
-max_comp_streams RW the number of possible concurrent compress operations
+max_comp_streams RW the number of possible concurrent compress
+ operations
comp_algorithm RW show and change the compression algorithm
compact WO trigger memory compaction
debug_stat RO this file is used for zram debugging purposes
backing_dev RW set up backend storage for zram to write out
idle WO mark allocated slot as idle
+====================== ====== ===============================================
User space is advised to use the following files to read the device statistics.
File /sys/block/zram<id>/stat
-Represents block layer statistics. Read Documentation/block/stat.txt for
+Represents block layer statistics. Read Documentation/block/stat.rst for
details.
File /sys/block/zram<id>/io_stat
layer and, thus, not available in zram<id>/stat file. It consists of a
single line of text and contains the following stats separated by
whitespace:
- failed_reads the number of failed reads
- failed_writes the number of failed writes
- invalid_io the number of non-page-size-aligned I/O requests
+
+ ============= =============================================================
+ failed_reads The number of failed reads
+ failed_writes The number of failed writes
+ invalid_io The number of non-page-size-aligned I/O requests
notify_free Depending on device usage scenario it may account
+
a) the number of pages freed because of swap slot free
- notifications or b) the number of pages freed because of
- REQ_OP_DISCARD requests sent by bio. The former ones are
- sent to a swap block device when a swap slot is freed,
- which implies that this disk is being used as a swap disk.
+ notifications
+ b) the number of pages freed because of
+ REQ_OP_DISCARD requests sent by bio. The former ones are
+ sent to a swap block device when a swap slot is freed,
+ which implies that this disk is being used as a swap disk.
+
The latter ones are sent by filesystem mounted with
discard option, whenever some data blocks are getting
discarded.
+ ============= =============================================================
File /sys/block/zram<id>/mm_stat
The stat file represents device's mm statistics. It consists of a single
line of text and contains the following stats separated by whitespace:
+
+ ================ =============================================================
orig_data_size uncompressed size of data stored in this disk.
This excludes same-element-filled pages (same_pages) since
no memory is allocated for them.
No memory is allocated for such pages.
pages_compacted the number of pages freed during compaction
huge_pages the number of incompressible pages
+ ================ =============================================================
File /sys/block/zram<id>/bd_stat
The stat file represents device's backing device statistics. It consists of
a single line of text and contains the following stats separated by whitespace:
+
+ ============== =============================================================
bd_count size of data written in backing device.
Unit: 4K bytes
bd_reads the number of reads from backing device
Unit: 4K bytes
bd_writes the number of writes to backing device
Unit: 4K bytes
+ ============== =============================================================
+
+9) Deactivate
+=============
+
+::
-9) Deactivate:
swapoff /dev/zram0
umount /dev/zram1
-10) Reset:
- Write any positive value to 'reset' sysfs node
- echo 1 > /sys/block/zram0/reset
- echo 1 > /sys/block/zram1/reset
+10) Reset
+=========
+
+ Write any positive value to 'reset' sysfs node::
+
+ echo 1 > /sys/block/zram0/reset
+ echo 1 > /sys/block/zram1/reset
This frees all the memory allocated for the given device and
resets the disksize to zero. You must set the disksize again
before reusing the device.
-* Optional Feature
+Optional Feature
+================
-= writeback
+writeback
+---------
With CONFIG_ZRAM_WRITEBACK, zram can write idle/incompressible page
to backing storage rather than keeping it in memory.
-To use the feature, admin should set up backing device via
+To use the feature, admin should set up backing device via::
- "echo /dev/sda5 > /sys/block/zramX/backing_dev"
+ echo /dev/sda5 > /sys/block/zramX/backing_dev
before disksize setting. It supports only partition at this moment.
-If admin want to use incompressible page writeback, they could do via
+If admin want to use incompressible page writeback, they could do via::
- "echo huge > /sys/block/zramX/write"
+ echo huge > /sys/block/zramX/write
To use idle page writeback, first, user need to declare zram pages
-as idle.
+as idle::
- "echo all > /sys/block/zramX/idle"
+ echo all > /sys/block/zramX/idle
From now on, any pages on zram are idle pages. The idle mark
will be removed until someone request access of the block.
IOW, unless there is access request, those pages are still idle pages.
-Admin can request writeback of those idle pages at right timing via
+Admin can request writeback of those idle pages at right timing via::
- "echo idle > /sys/block/zramX/writeback"
+ echo idle > /sys/block/zramX/writeback
With the command, zram writeback idle pages from memory to the storage.
To overcome the concern, zram supports "writeback_limit" feature.
The "writeback_limit_enable"'s default value is 0 so that it doesn't limit
any writeback. IOW, if admin want to apply writeback budget, he should
-enable writeback_limit_enable via
+enable writeback_limit_enable via::
$ echo 1 > /sys/block/zramX/writeback_limit_enable
assigned via /sys/block/zramX/writeback_limit is meaninless.)
If admin want to limit writeback as per-day 400M, he could do it
-like below.
+like below::
$ MB_SHIFT=20
$ 4K_SHIFT=12
$ echo 1 > /sys/block/zram0/writeback_limit_enable
If admin want to allow further write again once the bugdet is exausted,
-he could do it like below
+he could do it like below::
$ echo $((400<<MB_SHIFT>>4K_SHIFT)) > \
/sys/block/zram0/writeback_limit
-If admin want to see remaining writeback budget since he set,
+If admin want to see remaining writeback budget since he set::
$ cat /sys/block/zramX/writeback_limit
-If admin want to disable writeback limit, he could do
+If admin want to disable writeback limit, he could do::
$ echo 0 > /sys/block/zramX/writeback_limit_enable
If admin want to measure writeback count in a certain period, he could
know it via /sys/block/zram0/bd_stat's 3rd column.
-= memory tracking
+memory tracking
+===============
With CONFIG_ZRAM_MEMORY_TRACKING, user can know information of the
zram block. It could be useful to catch cold or incompressible
pages of the process with*pagemap.
+
If you enable the feature, you could see block state via
-/sys/kernel/debug/zram/zram0/block_state". The output is as follows,
+/sys/kernel/debug/zram/zram0/block_state". The output is as follows::
300 75.033841 .wh.
301 63.806904 s...
302 63.806919 ..hi
-First column is zram's block index.
-Second column is access time since the system was booted
-Third column is state of the block.
-(s: same page
-w: written page to backing store
-h: huge page
-i: idle page)
+First column
+ zram's block index.
+Second column
+ access time since the system was booted
+Third column
+ state of the block:
+
+ s:
+ same page
+ w:
+ written page to backing store
+ h:
+ huge page
+ i:
+ idle page
First line of above example says 300th block is accessed at 75.033841sec
and the block's state is huge so it is written back to the backing
run a null modem to a second machine and capture the output there
using your favourite communication program. Minicom works well.
-(3) Use Kdump (see Documentation/kdump/kdump.rst),
+(3) Use Kdump (see Documentation/admin-guide/kdump/kdump.rst),
extract the kernel ring buffer from old memory with using dmesg
- gdbmacro in Documentation/kdump/gdbmacros.txt.
+ gdbmacro in Documentation/admin-guide/kdump/gdbmacros.txt.
Finding the bug's location
--------------------------
==============
Written by Paul Menage <menage@google.com> based on
-Documentation/cgroup-v1/cpusets.rst
+Documentation/admin-guide/cgroup-v1/cpusets.rst
Original copyright statements from cpusets.txt:
tracking. The intention is that other subsystems hook into the generic
cgroup support to provide new attributes for cgroups, such as
accounting/limiting the resources which processes in a cgroup can
-access. For example, cpusets (see Documentation/cgroup-v1/cpusets.rst) allow
+access. For example, cpusets (see Documentation/admin-guide/cgroup-v1/cpusets.rst) allow
you to associate a set of CPUs and a set of memory nodes with the
tasks in each cgroup.
job placement on large systems.
Cpusets use the generic cgroup subsystem described in
-Documentation/cgroup-v1/cgroups.rst.
+Documentation/admin-guide/cgroup-v1/cgroups.rst.
Requests by a task, using the sched_setaffinity(2) system call to
include CPUs in its CPU affinity mask, and using the mbind(2) and
-:orphan:
-
========================
Control Groups version 1
========================
is complex. This is a document for memcg's internal behavior.
Please note that implementation details can be changed.
-(*) Topics on API should be in Documentation/cgroup-v1/memory.rst)
+(*) Topics on API should be in Documentation/admin-guide/cgroup-v1/memory.rst)
0. How to record usage ?
========================
You can see charges have been moved by reading ``*.usage_in_bytes`` or
memory.stat of both A and B.
- See 8.2 of Documentation/cgroup-v1/memory.rst to see what value should
+ See 8.2 of Documentation/admin-guide/cgroup-v1/memory.rst to see what value should
be written to move_charge_at_immigrate.
9.10 Memory thresholds
conventions of cgroup v2. It describes all userland-visible aspects
of cgroup including core and specific controller behaviors. All
future changes must be reflected in this document. Documentation for
-v1 is available under Documentation/cgroup-v1/.
+v1 is available under Documentation/admin-guide/cgroup-v1/.
.. CONTENTS
A read-only nested-key file which exists on non-root cgroups.
Shows pressure stall information for CPU. See
- Documentation/accounting/psi.txt for details.
+ Documentation/accounting/psi.rst for details.
Memory
A read-only nested-key file which exists on non-root cgroups.
Shows pressure stall information for memory. See
- Documentation/accounting/psi.txt for details.
+ Documentation/accounting/psi.rst for details.
Usage Guidelines
A read-only nested-key file which exists on non-root cgroups.
Shows pressure stall information for IO. See
- Documentation/accounting/psi.txt for details.
+ Documentation/accounting/psi.rst for details.
Writeback
-:orphan:
-
=============
Device Mapper
=============
The I/O statistics counters for each step-sized area of a region are
in the same format as `/sys/block/*/stat` or `/proc/diskstats` (see:
-Documentation/iostats.txt). But two extra counters (12 and 13) are
+Documentation/admin-guide/iostats.rst). But two extra counters (12 and 13) are
provided: total time spent reading and writing. When the histogram
argument is used, the 14th parameter is reported that represents the
histogram of latencies. All these counters may be accessed by sending
The first 11 counters have the same meaning as
`/sys/block/*/stat or /proc/diskstats`.
- Please refer to Documentation/iostats.txt for details.
+ Please refer to Documentation/admin-guide/iostats.rst for details.
1. the number of reads completed
2. the number of reads merged
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
====
gpio
For further information about confining guests to a single or to a group
of cores consult the cpusets documentation:
- https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.rst
+ https://www.kernel.org/doc/Documentation/admin-guide/cgroup-v1/cpusets.rst
.. _interrupt_isolation:
README
kernel-parameters
devices
+ sysctl/index
This section describes CPU vulnerabilities and their mitigations.
ramoops
dynamic-debug-howto
init
+ kdump/index
+ perf/index
This is the beginning of a section with information of interest to
application developers. Documents covering various aspects of the kernel
initrd
cgroup-v2
+ cgroup-v1/index
serial-console
braille-console
parport
md
module-signing
+ rapidio
sysrq
unicode
vga-softcursor
java
ras
bcache
+ blockdev/index
ext4
binderfs
+ xfs
pm/index
thunderbolt
LSM/index
mm/index
+ namespaces/index
perf-security
acpi/index
+ aoe/index
+ btmrvl
+ clearing-warn-once
+ cpu-load
+ cputopology
+ device-mapper/index
+ efi-stub
+ gpio/index
+ highuid
+ hw_random
+ iostats
+ kernel-per-CPU-kthreads
+ laptops/index
+ lcd-panel-cgram
+ ldm
+ lockup-watchdogs
+ numastat
+ pnp
+ rtc
+ svga
+ video-output
.. only:: subproject and html
-:orphan:
================================================================
Documentation for Kdump - The kexec-based Crash Dumping Solution
LOOP Loopback device support is enabled.
M68k M68k architecture is enabled.
These options have more detailed description inside of
- Documentation/m68k/kernel-options.txt.
+ Documentation/m68k/kernel-options.rst.
MDA MDA console support is enabled.
MIPS MIPS architecture is enabled.
MOUSE Appropriate mouse support is enabled.
blkdevparts= Manual partition parsing of block device(s) for
embedded devices based on command line input.
- See Documentation/block/cmdline-partition.txt
+ See Documentation/block/cmdline-partition.rst
boot_delay= Milliseconds to delay each printk during boot.
Values larger than 10 seconds (10000) are changed to
[KNL, x86_64] select a region under 4G first, and
fall back to reserve region above 4G when '@offset'
hasn't been specified.
- See Documentation/kdump/kdump.rst for further details.
+ See Documentation/admin-guide/kdump/kdump.rst for further details.
crashkernel=range1:size1[,range2:size2,...][@offset]
[KNL] Same as above, but depends on the memory
in the running system. The syntax of range is
start-[end] where start and end are both
a memory unit (amount[KMG]). See also
- Documentation/kdump/kdump.rst for an example.
+ Documentation/admin-guide/kdump/kdump.rst for an example.
crashkernel=size[KMG],high
[KNL, x86_64] range could be above 4G. Allow kernel
edid/1680x1050.bin, or edid/1920x1080.bin is given
and no file with the same name exists. Details and
instructions how to build your own EDID data are
- available in Documentation/EDID/howto.rst. An EDID
+ available in Documentation/driver-api/edid.rst. An EDID
data set will only be used for a particular connector,
if its name and a colon are prepended to the EDID
name. Each connector may use a unique EDID data
elevator= [IOSCHED]
Format: { "mq-deadline" | "kyber" | "bfq" }
- See Documentation/block/deadline-iosched.txt,
- Documentation/block/kyber-iosched.txt and
- Documentation/block/bfq-iosched.txt for details.
+ See Documentation/block/deadline-iosched.rst,
+ Documentation/block/kyber-iosched.rst and
+ Documentation/block/bfq-iosched.rst for details.
elfcorehdr=[size[KMG]@]offset[KMG] [IA64,PPC,SH,X86,S390]
Specifies physical address of start of kernel core
image elf header and optionally the size. Generally
kexec loader will pass this option to capture kernel.
- See Documentation/kdump/kdump.rst for details.
+ See Documentation/admin-guide/kdump/kdump.rst for details.
enable_mtrr_cleanup [X86]
The kernel tries to adjust MTRR layout from continuous
See also Documentation/fault-injection/.
floppy= [HW]
- See Documentation/blockdev/floppy.txt.
+ See Documentation/admin-guide/blockdev/floppy.rst.
force_pal_cache_flush
[IA-64] Avoid check_sal_cache_flush which may hang on
Built with CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF=y,
the default is off.
+ kprobe_event=[probe-list]
+ [FTRACE] Add kprobe events and enable at boot time.
+ The probe-list is a semicolon delimited list of probe
+ definitions. Each definition is same as kprobe_events
+ interface, but the parameters are comma delimited.
+ For example, to add a kprobe event on vfs_read with
+ arg1 and arg2, add to the command line;
+
+ kprobe_event=p,vfs_read,$arg1,$arg2
+
+ See also Documentation/trace/kprobetrace.rst "Kernel
+ Boot Parameter" section.
+
kpti= [ARM64] Control page table isolation of user
and kernel address spaces.
Default: enabled on cores which need mitigation.
memblock=debug [KNL] Enable memblock debug messages.
load_ramdisk= [RAM] List of ramdisks to load from floppy
- See Documentation/blockdev/ramdisk.txt.
+ See Documentation/admin-guide/blockdev/ramdisk.rst.
lockd.nlm_grace_period=P [NFS] Assign grace period.
Format: <integer>
/sys/module/printk/parameters/console_suspend) to
turn on/off it dynamically.
+ novmcoredd [KNL,KDUMP]
+ Disable device dump. Device dump allows drivers to
+ append dump data to vmcore so you can collect driver
+ specified debug info. Drivers can append the data
+ without any limit and this data is stored in memory,
+ so this may cause significant memory stress. Disabling
+ device dump can help save memory but the driver debug
+ data will be no longer available. This parameter
+ is only available when CONFIG_PROC_VMCORE_DEVICE_DUMP
+ is set.
+
noaliencache [MM, NUMA, SLAB] Disables the allocation of alien
caches in the slab allocator. Saves per-node memory,
but will impact performance.
numa_zonelist_order= [KNL, BOOT] Select zonelist order for NUMA.
'node', 'default' can be specified
This can be set from sysctl after boot.
- See Documentation/sysctl/vm.txt for details.
+ See Documentation/admin-guide/sysctl/vm.rst for details.
ohci1394_dma=early [HW] enable debugging via the ohci1394 driver.
See Documentation/debugging-via-ohci1394.txt for more
pcd. [PARIDE]
See header of drivers/block/paride/pcd.c.
- See also Documentation/blockdev/paride.txt.
+ See also Documentation/admin-guide/blockdev/paride.rst.
pci=option[,option...] [PCI] various PCI subsystem options.
needed on a platform with proper driver support.
pd. [PARIDE]
- See Documentation/blockdev/paride.txt.
+ See Documentation/admin-guide/blockdev/paride.rst.
pdcchassis= [PARISC,HW] Disable/Enable PDC Chassis Status codes at
boot time.
and performance comparison.
pf. [PARIDE]
- See Documentation/blockdev/paride.txt.
+ See Documentation/admin-guide/blockdev/paride.rst.
pg. [PARIDE]
- See Documentation/blockdev/paride.txt.
+ See Documentation/admin-guide/blockdev/paride.rst.
pirq= [SMP,APIC] Manual mp-table setup
See Documentation/x86/i386/IO-APIC.rst.
prompt_ramdisk= [RAM] List of RAM disks to prompt for floppy disk
before loading.
- See Documentation/blockdev/ramdisk.txt.
+ See Documentation/admin-guide/blockdev/ramdisk.rst.
psi= [KNL] Enable or disable pressure stall information
tracking.
pstore.backend= Specify the name of the pstore backend to use
pt. [PARIDE]
- See Documentation/blockdev/paride.txt.
+ See Documentation/admin-guide/blockdev/paride.rst.
pti= [X86_64] Control Page Table Isolation of user and
kernel address spaces. Disabling this feature
See Documentation/admin-guide/md.rst.
ramdisk_size= [RAM] Sizes of RAM disks in kilobytes
- See Documentation/blockdev/ramdisk.txt.
+ See Documentation/admin-guide/blockdev/ramdisk.rst.
random.trust_cpu={on,off}
[KNL] Enable or disable trusting the use of the
relax_domain_level=
[KNL, SMP] Set scheduler's default relax_domain_level.
- See Documentation/cgroup-v1/cpusets.rst.
+ See Documentation/admin-guide/cgroup-v1/cpusets.rst.
reserve= [KNL,BUGS] Force kernel to ignore I/O ports or memory
Format: <base1>,<size1>[,<base2>,<size2>,...]
Format: <integer>
sonypi.*= [HW] Sony Programmable I/O Control Device driver
- See Documentation/laptops/sonypi.txt
+ See Documentation/admin-guide/laptops/sonypi.rst
spectre_v2= [X86] Control mitigation of Spectre variant 2
(indirect branch speculation) vulnerability.
swapaccount=[0|1]
[KNL] Enable accounting of swap in memory resource
controller if no parameter or 1 is given or disable
- it if 0 is given (See Documentation/cgroup-v1/memory.rst)
+ it if 0 is given (See Documentation/admin-guide/cgroup-v1/memory.rst)
swiotlb= [ARM,IA-64,PPC,MIPS,X86]
Format: { <int> | force | noforce }
vga= [BOOT,X86-32] Select a particular video mode
See Documentation/x86/boot.rst and
- Documentation/svga.txt.
+ Documentation/admin-guide/svga.rst.
Use vga=ask for menu.
This is actually a boot loader parameter; the value is
passed to the kernel using a special protocol.
- Documentation/IRQ-affinity.txt: Binding interrupts to sets of CPUs.
-- Documentation/cgroup-v1: Using cgroups to bind tasks to sets of CPUs.
+- Documentation/admin-guide/cgroup-v1: Using cgroups to bind tasks to sets of CPUs.
- man taskset: Using the taskset command to bind tasks to sets
of CPUs.
+==================
Asus Laptop Extras
+==================
Version 0.1
+
August 6, 2009
Corentin Chary <corentincj@iksaif.net>
It may also support some MEDION, JVC or VICTOR laptops (such as MEDION 9675 or
VICTOR XP7210 for example). It makes all the extra buttons generate input
events (like keyboards).
+
On some models adds support for changing the display brightness and output,
switching the LCD backlight on and off, and most importantly, allows you to
blink those fancy LEDs intended for reporting mail and wireless status.
-This driver supercedes the old asus_acpi driver.
+This driver supersedes the old asus_acpi driver.
Requirements
------------
see some lines like this :
Asus Laptop Extras version 0.42
- L2D model detected.
+ - L2D model detected.
If it is not the output you have on your laptop, send it (and the laptop's
DSDT) to me.
LEDs
----
- You can modify LEDs be echoing values to /sys/class/leds/asus::*/brightness :
+ You can modify LEDs be echoing values to `/sys/class/leds/asus/*/brightness`::
+
echo 1 > /sys/class/leds/asus::mail/brightness
+
will switch the mail LED on.
+
You can also know if they are on/off by reading their content and use
kernel triggers like disk-activity or heartbeat.
/sys/class/backlight/asus-laptop/. Brightness Values are between 0 and 15.
Wireless devices
----------------
+----------------
You can turn the internal Bluetooth adapter on/off with the bluetooth entry
(only on models with Bluetooth). This usually controls the associated LED.
Note: the display switching code is currently considered EXPERIMENTAL.
Switching works for the following models:
- L3800C
- A2500H
- L5800C
- M5200N
- W1000N (albeit with some glitches)
- M6700R
- A6JC
- F3J
+
+ - L3800C
+ - A2500H
+ - L5800C
+ - M5200N
+ - W1000N (albeit with some glitches)
+ - M6700R
+ - A6JC
+ - F3J
Switching doesn't work for the following:
- M3700N
- L2X00D (locks the laptop under certain conditions)
+
+ - M3700N
+ - L2X00D (locks the laptop under certain conditions)
To switch the displays, echo values from 0 to 15 to
/sys/devices/platform/asus-laptop/display. The significance of those values
+-------+-----+-----+-----+-----+-----+
| Bin | Val | DVI | TV | CRT | LCD |
+-------+-----+-----+-----+-----+-----+
- + 0000 + 0 + + + + +
+ | 0000 | 0 | | | | |
+-------+-----+-----+-----+-----+-----+
- + 0001 + 1 + + + + X +
+ | 0001 | 1 | | | | X |
+-------+-----+-----+-----+-----+-----+
- + 0010 + 2 + + + X + +
+ | 0010 | 2 | | | X | |
+-------+-----+-----+-----+-----+-----+
- + 0011 + 3 + + + X + X +
+ | 0011 | 3 | | | X | X |
+-------+-----+-----+-----+-----+-----+
- + 0100 + 4 + + X + + +
+ | 0100 | 4 | | X | | |
+-------+-----+-----+-----+-----+-----+
- + 0101 + 5 + + X + + X +
+ | 0101 | 5 | | X | | X |
+-------+-----+-----+-----+-----+-----+
- + 0110 + 6 + + X + X + +
+ | 0110 | 6 | | X | X | |
+-------+-----+-----+-----+-----+-----+
- + 0111 + 7 + + X + X + X +
+ | 0111 | 7 | | X | X | X |
+-------+-----+-----+-----+-----+-----+
- + 1000 + 8 + X + + + +
+ | 1000 | 8 | X | | | |
+-------+-----+-----+-----+-----+-----+
- + 1001 + 9 + X + + + X +
+ | 1001 | 9 | X | | | X |
+-------+-----+-----+-----+-----+-----+
- + 1010 + 10 + X + + X + +
+ | 1010 | 10 | X | | X | |
+-------+-----+-----+-----+-----+-----+
- + 1011 + 11 + X + + X + X +
+ | 1011 | 11 | X | | X | X |
+-------+-----+-----+-----+-----+-----+
- + 1100 + 12 + X + X + + +
+ | 1100 | 12 | X | X | | |
+-------+-----+-----+-----+-----+-----+
- + 1101 + 13 + X + X + + X +
+ | 1101 | 13 | X | X | | X |
+-------+-----+-----+-----+-----+-----+
- + 1110 + 14 + X + X + X + +
+ | 1110 | 14 | X | X | X | |
+-------+-----+-----+-----+-----+-----+
- + 1111 + 15 + X + X + X + X +
+ | 1111 | 15 | X | X | X | X |
+-------+-----+-----+-----+-----+-----+
In most cases, the appropriate displays must be plugged in for the above
combinations to work. TV-Out may need to be initialized at boot time.
Debugging:
+
1) Check whether the Fn+F8 key:
+
a) does not lock the laptop (try a boot with noapic / nolapic if it does)
b) generates events (0x6n, where n is the value corresponding to the
configuration above)
c) actually works
+
Record the disp value at every configuration.
2) Echo values from 0 to 15 to /sys/devices/platform/asus-laptop/display.
Record its value, note any change. If nothing changes, try a broader range,
Note: on some machines (e.g. L3C), after the module has been loaded, only 0x6n
events are generated and no actual switching occurs. In such a case, a line
- like:
+ like::
echo $((10#$arg-60)) > /sys/devices/platform/asus-laptop/display
several items of information.
LED display works for the following models:
- W1000N
- W1J
- To control the LED display, use the following :
+ - W1000N
+ - W1J
+
+ To control the LED display, use the following::
echo 0x0T000DDD > /sys/devices/platform/asus-laptop/
where T control the 3 letters display, and DDD the 3 digits display,
- according to the tables below.
+ according to the tables below::
DDD (digits)
000 to 999 = display digits
For example "echo 0x01000001 >/sys/devices/platform/asus-laptop/ledd"
would display "DVD001".
-Driver options:
----------------
+Driver options
+--------------
Options can be passed to the asus-laptop driver using the standard
module argument syntax (<param>=<value> when passing the option to the
wapf: WAPF defines the behavior of the Fn+Fx wlan key
The significance of values is yet to be found, but
most of the time:
+
- 0x0 should do nothing
- 0x1 should allow to control the device with Fn+Fx key.
- 0x4 should send an ACPI event (0x88) while pressing the Fn+Fx key
- ASUS L7300G
- ASUS L8400
-Patches, Errors, Questions:
+Patches, Errors, Questions
--------------------------
I appreciate any success or failure
Any other comments or patches are also more than welcome.
acpi4asus-user@lists.sourceforge.net
- http://sourceforge.net/projects/acpi4asus
+ http://sourceforge.net/projects/acpi4asus
+==========================
Hard disk shock protection
==========================
Author: Elias Oltmanns <eo@nebensachen.de>
+
Last modified: 2008-10-03
-0. Contents
------------
+.. 0. Contents
-1. Intro
-2. The interface
-3. References
-4. CREDITS
+ 1. Intro
+ 2. The interface
+ 3. References
+ 4. CREDITS
1. Intro
----------------
For each ATA device, the kernel exports the file
-block/*/device/unload_heads in sysfs (here assumed to be mounted under
-/sys). Access to /sys/block/*/device/unload_heads is denied with
+`block/*/device/unload_heads` in sysfs (here assumed to be mounted under
+/sys). Access to `/sys/block/*/device/unload_heads` is denied with
-EOPNOTSUPP if the device does not support the unload feature.
Otherwise, writing an integer value to this file will take the heads
of the respective drive off the platter and block all I/O operations
immediately by specifying a timeout of 0. Values below -2 are rejected
with -EINVAL (see below for the special meaning of -1 and -2). If the
timeout specified for a recent head park request has not yet expired,
-reading from /sys/block/*/device/unload_heads will report the number
+reading from `/sys/block/*/device/unload_heads` will report the number
of milliseconds remaining until normal operation will be resumed;
otherwise, reading the unload_heads attribute will return 0.
For example, do the following in order to park the heads of drive
-/dev/sda and stop all I/O operations for five seconds:
+/dev/sda and stop all I/O operations for five seconds::
-# echo 5000 > /sys/block/sda/device/unload_heads
+ # echo 5000 > /sys/block/sda/device/unload_heads
-A simple
+A simple::
-# cat /sys/block/sda/device/unload_heads
+ # cat /sys/block/sda/device/unload_heads
will show you how many milliseconds are left before normal operation
will be resumed.
support the unload feature (for instance, because the vendor of your
laptop or the hard drive itself told you so), then you can tell the
kernel to enable the usage of this feature for that drive by writing
-the special value -1 to the unload_heads attribute:
+the special value -1 to the unload_heads attribute::
-# echo -1 > /sys/block/sda/device/unload_heads
+ # echo -1 > /sys/block/sda/device/unload_heads
will enable the feature for /dev/sda, and giving -2 instead of -1 will
disable it again.
of my ignorance.
- http://www.thinkwiki.org/wiki/HDAPS
+
See this page for information about Linux support of the hard disk
active protection system as implemented in IBM/Lenovo Thinkpads.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==============
+Laptop Drivers
+==============
+
+.. toctree::
+ :maxdepth: 1
+
+ asus-laptop
+ disk-shock-protection
+ laptop-mode
+ lg-laptop
+ sony-laptop
+ sonypi
+ thinkpad-acpi
+ toshiba_haps
+===============================================
How to conserve battery power using laptop-mode
------------------------------------------------
+===============================================
Document Author: Bart Samwel (bart@samwel.tk)
+
Date created: January 2, 2004
+
Last modified: December 06, 2004
Introduction
to conserve battery power on laptops. It has been reported to cause significant
power savings.
-Contents
---------
+.. Contents
-* Introduction
-* Installation
-* Caveats
-* The Details
-* Tips & Tricks
-* Control script
-* ACPI integration
-* Monitoring tool
+ * Introduction
+ * Installation
+ * Caveats
+ * The Details
+ * Tips & Tricks
+ * Control script
+ * ACPI integration
+ * Monitoring tool
Installation
laptop mode will automatically be started when you're on battery. For
your convenience, a tarball containing an installer can be downloaded at:
-http://www.samwel.tk/laptop_mode/laptop_mode/
+ http://www.samwel.tk/laptop_mode/laptop_mode/
To configure laptop mode, you need to edit the configuration file, which is
located in /etc/default/laptop-mode on Debian-based systems, or in
this on powerbooks too. I hope that this is a piece of information that
might be useful to the Laptop Mode patch or its users."
-* In syslog.conf, you can prefix entries with a dash ``-'' to omit syncing the
+* In syslog.conf, you can prefix entries with a dash `-` to omit syncing the
file after every logging. When you're using laptop-mode and your disk doesn't
spin down, this is a likely culprit.
It should be installed as /etc/default/laptop-mode on Debian, and as
/etc/sysconfig/laptop-mode on Red Hat, SUSE, Mandrake, and other work-alikes.
---------------------CONFIG FILE BEGIN-------------------------------------------
-# Maximum time, in seconds, of hard drive spindown time that you are
-# comfortable with. Worst case, it's possible that you could lose this
-# amount of work if your battery fails you while in laptop mode.
-#MAX_AGE=600
-
-# Automatically disable laptop mode when the number of minutes of battery
-# that you have left goes below this threshold.
-MINIMUM_BATTERY_MINUTES=10
-
-# Read-ahead, in 512-byte sectors. You can spin down the disk while playing MP3/OGG
-# by setting the disk readahead to 8MB (READAHEAD=16384). Effectively, the disk
-# will read a complete MP3 at once, and will then spin down while the MP3/OGG is
-# playing.
-#READAHEAD=4096
-
-# Shall we remount journaled fs. with appropriate commit interval? (1=yes)
-#DO_REMOUNTS=1
-
-# And shall we add the "noatime" option to that as well? (1=yes)
-#DO_REMOUNT_NOATIME=1
-
-# Dirty synchronous ratio. At this percentage of dirty pages the process
-# which
-# calls write() does its own writeback
-#DIRTY_RATIO=40
-
-#
-# Allowed dirty background ratio, in percent. Once DIRTY_RATIO has been
-# exceeded, the kernel will wake flusher threads which will then reduce the
-# amount of dirty memory to dirty_background_ratio. Set this nice and low,
-# so once some writeout has commenced, we do a lot of it.
-#
-#DIRTY_BACKGROUND_RATIO=5
-
-# kernel default dirty buffer age
-#DEF_AGE=30
-#DEF_UPDATE=5
-#DEF_DIRTY_BACKGROUND_RATIO=10
-#DEF_DIRTY_RATIO=40
-#DEF_XFS_AGE_BUFFER=15
-#DEF_XFS_SYNC_INTERVAL=30
-#DEF_XFS_BUFD_INTERVAL=1
-
-# This must be adjusted manually to the value of HZ in the running kernel
-# on 2.4, until the XFS people change their 2.4 external interfaces to work in
-# centisecs. This can be automated, but it's a work in progress that still
-# needs# some fixes. On 2.6 kernels, XFS uses USER_HZ instead of HZ for
-# external interfaces, and that is currently always set to 100. So you don't
-# need to change this on 2.6.
-#XFS_HZ=100
-
-# Should the maximum CPU frequency be adjusted down while on battery?
-# Requires CPUFreq to be setup.
-# See Documentation/admin-guide/pm/cpufreq.rst for more info
-#DO_CPU=0
-
-# When on battery what is the maximum CPU speed that the system should
-# use? Legal values are "slowest" for the slowest speed that your
-# CPU is able to operate at, or a value listed in:
-# /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
-# Only applicable if DO_CPU=1.
-#CPU_MAXFREQ=slowest
-
-# Idle timeout for your hard drive (man hdparm for valid values, -S option)
-# Default is 2 hours on AC (AC_HD=244) and 20 seconds for battery (BATT_HD=4).
-#AC_HD=244
-#BATT_HD=4
-
-# The drives for which to adjust the idle timeout. Separate them by a space,
-# e.g. HD="/dev/hda /dev/hdb".
-#HD="/dev/hda"
-
-# Set the spindown timeout on a hard drive?
-#DO_HD=1
-
---------------------CONFIG FILE END---------------------------------------------
+Config file::
+
+ # Maximum time, in seconds, of hard drive spindown time that you are
+ # comfortable with. Worst case, it's possible that you could lose this
+ # amount of work if your battery fails you while in laptop mode.
+ #MAX_AGE=600
+
+ # Automatically disable laptop mode when the number of minutes of battery
+ # that you have left goes below this threshold.
+ MINIMUM_BATTERY_MINUTES=10
+
+ # Read-ahead, in 512-byte sectors. You can spin down the disk while playing MP3/OGG
+ # by setting the disk readahead to 8MB (READAHEAD=16384). Effectively, the disk
+ # will read a complete MP3 at once, and will then spin down while the MP3/OGG is
+ # playing.
+ #READAHEAD=4096
+
+ # Shall we remount journaled fs. with appropriate commit interval? (1=yes)
+ #DO_REMOUNTS=1
+
+ # And shall we add the "noatime" option to that as well? (1=yes)
+ #DO_REMOUNT_NOATIME=1
+
+ # Dirty synchronous ratio. At this percentage of dirty pages the process
+ # which
+ # calls write() does its own writeback
+ #DIRTY_RATIO=40
+
+ #
+ # Allowed dirty background ratio, in percent. Once DIRTY_RATIO has been
+ # exceeded, the kernel will wake flusher threads which will then reduce the
+ # amount of dirty memory to dirty_background_ratio. Set this nice and low,
+ # so once some writeout has commenced, we do a lot of it.
+ #
+ #DIRTY_BACKGROUND_RATIO=5
+
+ # kernel default dirty buffer age
+ #DEF_AGE=30
+ #DEF_UPDATE=5
+ #DEF_DIRTY_BACKGROUND_RATIO=10
+ #DEF_DIRTY_RATIO=40
+ #DEF_XFS_AGE_BUFFER=15
+ #DEF_XFS_SYNC_INTERVAL=30
+ #DEF_XFS_BUFD_INTERVAL=1
+
+ # This must be adjusted manually to the value of HZ in the running kernel
+ # on 2.4, until the XFS people change their 2.4 external interfaces to work in
+ # centisecs. This can be automated, but it's a work in progress that still
+ # needs# some fixes. On 2.6 kernels, XFS uses USER_HZ instead of HZ for
+ # external interfaces, and that is currently always set to 100. So you don't
+ # need to change this on 2.6.
+ #XFS_HZ=100
+
+ # Should the maximum CPU frequency be adjusted down while on battery?
+ # Requires CPUFreq to be setup.
+ # See Documentation/admin-guide/pm/cpufreq.rst for more info
+ #DO_CPU=0
+
+ # When on battery what is the maximum CPU speed that the system should
+ # use? Legal values are "slowest" for the slowest speed that your
+ # CPU is able to operate at, or a value listed in:
+ # /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
+ # Only applicable if DO_CPU=1.
+ #CPU_MAXFREQ=slowest
+
+ # Idle timeout for your hard drive (man hdparm for valid values, -S option)
+ # Default is 2 hours on AC (AC_HD=244) and 20 seconds for battery (BATT_HD=4).
+ #AC_HD=244
+ #BATT_HD=4
+
+ # The drives for which to adjust the idle timeout. Separate them by a space,
+ # e.g. HD="/dev/hda /dev/hdb".
+ #HD="/dev/hda"
+
+ # Set the spindown timeout on a hard drive?
+ #DO_HD=1
Control script
Please note that this control script works for the Linux 2.4 and 2.6 series (thanks
to Kiko Piris).
---------------------CONTROL SCRIPT BEGIN----------------------------------------
-#!/bin/bash
-
-# start or stop laptop_mode, best run by a power management daemon when
-# ac gets connected/disconnected from a laptop
-#
-# install as /sbin/laptop_mode
-#
-# Contributors to this script: Kiko Piris
-# Bart Samwel
-# Micha Feigin
-# Andrew Morton
-# Herve Eychenne
-# Dax Kelson
-#
-# Original Linux 2.4 version by: Jens Axboe
-
-#############################################################################
-
-# Source config
-if [ -f /etc/default/laptop-mode ] ; then
+Control script::
+
+ #!/bin/bash
+
+ # start or stop laptop_mode, best run by a power management daemon when
+ # ac gets connected/disconnected from a laptop
+ #
+ # install as /sbin/laptop_mode
+ #
+ # Contributors to this script: Kiko Piris
+ # Bart Samwel
+ # Micha Feigin
+ # Andrew Morton
+ # Herve Eychenne
+ # Dax Kelson
+ #
+ # Original Linux 2.4 version by: Jens Axboe
+
+ #############################################################################
+
+ # Source config
+ if [ -f /etc/default/laptop-mode ] ; then
# Debian
. /etc/default/laptop-mode
-elif [ -f /etc/sysconfig/laptop-mode ] ; then
+ elif [ -f /etc/sysconfig/laptop-mode ] ; then
# Others
- . /etc/sysconfig/laptop-mode
-fi
-
-# Don't raise an error if the config file is incomplete
-# set defaults instead:
-
-# Maximum time, in seconds, of hard drive spindown time that you are
-# comfortable with. Worst case, it's possible that you could lose this
-# amount of work if your battery fails you while in laptop mode.
-MAX_AGE=${MAX_AGE:-'600'}
-
-# Read-ahead, in kilobytes
-READAHEAD=${READAHEAD:-'4096'}
-
-# Shall we remount journaled fs. with appropriate commit interval? (1=yes)
-DO_REMOUNTS=${DO_REMOUNTS:-'1'}
-
-# And shall we add the "noatime" option to that as well? (1=yes)
-DO_REMOUNT_NOATIME=${DO_REMOUNT_NOATIME:-'1'}
-
-# Shall we adjust the idle timeout on a hard drive?
-DO_HD=${DO_HD:-'1'}
-
-# Adjust idle timeout on which hard drive?
-HD="${HD:-'/dev/hda'}"
-
-# spindown time for HD (hdparm -S values)
-AC_HD=${AC_HD:-'244'}
-BATT_HD=${BATT_HD:-'4'}
-
-# Dirty synchronous ratio. At this percentage of dirty pages the process which
-# calls write() does its own writeback
-DIRTY_RATIO=${DIRTY_RATIO:-'40'}
-
-# cpu frequency scaling
-# See Documentation/admin-guide/pm/cpufreq.rst for more info
-DO_CPU=${CPU_MANAGE:-'0'}
-CPU_MAXFREQ=${CPU_MAXFREQ:-'slowest'}
-
-#
-# Allowed dirty background ratio, in percent. Once DIRTY_RATIO has been
-# exceeded, the kernel will wake flusher threads which will then reduce the
-# amount of dirty memory to dirty_background_ratio. Set this nice and low,
-# so once some writeout has commenced, we do a lot of it.
-#
-DIRTY_BACKGROUND_RATIO=${DIRTY_BACKGROUND_RATIO:-'5'}
-
-# kernel default dirty buffer age
-DEF_AGE=${DEF_AGE:-'30'}
-DEF_UPDATE=${DEF_UPDATE:-'5'}
-DEF_DIRTY_BACKGROUND_RATIO=${DEF_DIRTY_BACKGROUND_RATIO:-'10'}
-DEF_DIRTY_RATIO=${DEF_DIRTY_RATIO:-'40'}
-DEF_XFS_AGE_BUFFER=${DEF_XFS_AGE_BUFFER:-'15'}
-DEF_XFS_SYNC_INTERVAL=${DEF_XFS_SYNC_INTERVAL:-'30'}
-DEF_XFS_BUFD_INTERVAL=${DEF_XFS_BUFD_INTERVAL:-'1'}
-
-# This must be adjusted manually to the value of HZ in the running kernel
-# on 2.4, until the XFS people change their 2.4 external interfaces to work in
-# centisecs. This can be automated, but it's a work in progress that still needs
-# some fixes. On 2.6 kernels, XFS uses USER_HZ instead of HZ for external
-# interfaces, and that is currently always set to 100. So you don't need to
-# change this on 2.6.
-XFS_HZ=${XFS_HZ:-'100'}
-
-#############################################################################
-
-KLEVEL="$(uname -r |
- {
+ . /etc/sysconfig/laptop-mode
+ fi
+
+ # Don't raise an error if the config file is incomplete
+ # set defaults instead:
+
+ # Maximum time, in seconds, of hard drive spindown time that you are
+ # comfortable with. Worst case, it's possible that you could lose this
+ # amount of work if your battery fails you while in laptop mode.
+ MAX_AGE=${MAX_AGE:-'600'}
+
+ # Read-ahead, in kilobytes
+ READAHEAD=${READAHEAD:-'4096'}
+
+ # Shall we remount journaled fs. with appropriate commit interval? (1=yes)
+ DO_REMOUNTS=${DO_REMOUNTS:-'1'}
+
+ # And shall we add the "noatime" option to that as well? (1=yes)
+ DO_REMOUNT_NOATIME=${DO_REMOUNT_NOATIME:-'1'}
+
+ # Shall we adjust the idle timeout on a hard drive?
+ DO_HD=${DO_HD:-'1'}
+
+ # Adjust idle timeout on which hard drive?
+ HD="${HD:-'/dev/hda'}"
+
+ # spindown time for HD (hdparm -S values)
+ AC_HD=${AC_HD:-'244'}
+ BATT_HD=${BATT_HD:-'4'}
+
+ # Dirty synchronous ratio. At this percentage of dirty pages the process which
+ # calls write() does its own writeback
+ DIRTY_RATIO=${DIRTY_RATIO:-'40'}
+
+ # cpu frequency scaling
+ # See Documentation/admin-guide/pm/cpufreq.rst for more info
+ DO_CPU=${CPU_MANAGE:-'0'}
+ CPU_MAXFREQ=${CPU_MAXFREQ:-'slowest'}
+
+ #
+ # Allowed dirty background ratio, in percent. Once DIRTY_RATIO has been
+ # exceeded, the kernel will wake flusher threads which will then reduce the
+ # amount of dirty memory to dirty_background_ratio. Set this nice and low,
+ # so once some writeout has commenced, we do a lot of it.
+ #
+ DIRTY_BACKGROUND_RATIO=${DIRTY_BACKGROUND_RATIO:-'5'}
+
+ # kernel default dirty buffer age
+ DEF_AGE=${DEF_AGE:-'30'}
+ DEF_UPDATE=${DEF_UPDATE:-'5'}
+ DEF_DIRTY_BACKGROUND_RATIO=${DEF_DIRTY_BACKGROUND_RATIO:-'10'}
+ DEF_DIRTY_RATIO=${DEF_DIRTY_RATIO:-'40'}
+ DEF_XFS_AGE_BUFFER=${DEF_XFS_AGE_BUFFER:-'15'}
+ DEF_XFS_SYNC_INTERVAL=${DEF_XFS_SYNC_INTERVAL:-'30'}
+ DEF_XFS_BUFD_INTERVAL=${DEF_XFS_BUFD_INTERVAL:-'1'}
+
+ # This must be adjusted manually to the value of HZ in the running kernel
+ # on 2.4, until the XFS people change their 2.4 external interfaces to work in
+ # centisecs. This can be automated, but it's a work in progress that still needs
+ # some fixes. On 2.6 kernels, XFS uses USER_HZ instead of HZ for external
+ # interfaces, and that is currently always set to 100. So you don't need to
+ # change this on 2.6.
+ XFS_HZ=${XFS_HZ:-'100'}
+
+ #############################################################################
+
+ KLEVEL="$(uname -r |
+ {
IFS='.' read a b c
echo $a.$b
}
-)"
-case "$KLEVEL" in
+ )"
+ case "$KLEVEL" in
"2.4"|"2.6")
;;
*)
echo "Unhandled kernel version: $KLEVEL ('uname -r' = '$(uname -r)')" >&2
exit 1
;;
-esac
+ esac
-if [ ! -e /proc/sys/vm/laptop_mode ] ; then
+ if [ ! -e /proc/sys/vm/laptop_mode ] ; then
echo "Kernel is not patched with laptop_mode patch." >&2
exit 1
-fi
+ fi
-if [ ! -w /proc/sys/vm/laptop_mode ] ; then
+ if [ ! -w /proc/sys/vm/laptop_mode ] ; then
echo "You do not have enough privileges to enable laptop_mode." >&2
exit 1
-fi
+ fi
-# Remove an option (the first parameter) of the form option=<number> from
-# a mount options string (the rest of the parameters).
-parse_mount_opts () {
+ # Remove an option (the first parameter) of the form option=<number> from
+ # a mount options string (the rest of the parameters).
+ parse_mount_opts () {
OPT="$1"
shift
echo ",$*," | sed \
-e 's/,,*/,/g' \
-e 's/^,//' \
-e 's/,$//'
-}
+ }
-# Remove an option (the first parameter) without any arguments from
-# a mount option string (the rest of the parameters).
-parse_nonumber_mount_opts () {
+ # Remove an option (the first parameter) without any arguments from
+ # a mount option string (the rest of the parameters).
+ parse_nonumber_mount_opts () {
OPT="$1"
shift
echo ",$*," | sed \
-e 's/,,*/,/g' \
-e 's/^,//' \
-e 's/,$//'
-}
-
-# Find out the state of a yes/no option (e.g. "atime"/"noatime") in
-# fstab for a given filesystem, and use this state to replace the
-# value of the option in another mount options string. The device
-# is the first argument, the option name the second, and the default
-# value the third. The remainder is the mount options string.
-#
-# Example:
-# parse_yesno_opts_wfstab /dev/hda1 atime atime defaults,noatime
-#
-# If fstab contains, say, "rw" for this filesystem, then the result
-# will be "defaults,atime".
-parse_yesno_opts_wfstab () {
+ }
+
+ # Find out the state of a yes/no option (e.g. "atime"/"noatime") in
+ # fstab for a given filesystem, and use this state to replace the
+ # value of the option in another mount options string. The device
+ # is the first argument, the option name the second, and the default
+ # value the third. The remainder is the mount options string.
+ #
+ # Example:
+ # parse_yesno_opts_wfstab /dev/hda1 atime atime defaults,noatime
+ #
+ # If fstab contains, say, "rw" for this filesystem, then the result
+ # will be "defaults,atime".
+ parse_yesno_opts_wfstab () {
L_DEV="$1"
OPT="$2"
DEF_OPT="$3"
# option not specified in fstab -- choose the default.
echo "$PARSEDOPTS1,$DEF_OPT"
fi
-}
-
-# Find out the state of a numbered option (e.g. "commit=NNN") in
-# fstab for a given filesystem, and use this state to replace the
-# value of the option in another mount options string. The device
-# is the first argument, and the option name the second. The
-# remainder is the mount options string in which the replacement
-# must be done.
-#
-# Example:
-# parse_mount_opts_wfstab /dev/hda1 commit defaults,commit=7
-#
-# If fstab contains, say, "commit=3,rw" for this filesystem, then the
-# result will be "rw,commit=3".
-parse_mount_opts_wfstab () {
+ }
+
+ # Find out the state of a numbered option (e.g. "commit=NNN") in
+ # fstab for a given filesystem, and use this state to replace the
+ # value of the option in another mount options string. The device
+ # is the first argument, and the option name the second. The
+ # remainder is the mount options string in which the replacement
+ # must be done.
+ #
+ # Example:
+ # parse_mount_opts_wfstab /dev/hda1 commit defaults,commit=7
+ #
+ # If fstab contains, say, "commit=3,rw" for this filesystem, then the
+ # result will be "rw,commit=3".
+ parse_mount_opts_wfstab () {
L_DEV="$1"
OPT="$2"
shift 2
# option not specified in fstab: set it to 0
echo "$PARSEDOPTS1,$OPT=0"
fi
-}
+ }
-deduce_fstype () {
+ deduce_fstype () {
MP="$1"
# My root filesystem unfortunately has
# type "unknown" in /etc/mtab. If we encounter
exit 0
fi
done
-}
+ }
-if [ $DO_REMOUNT_NOATIME -eq 1 ] ; then
+ if [ $DO_REMOUNT_NOATIME -eq 1 ] ; then
NOATIME_OPT=",noatime"
-fi
+ fi
-case "$1" in
+ case "$1" in
start)
AGE=$((100*$MAX_AGE))
XFS_AGE=$(($XFS_HZ*$MAX_AGE))
exit 1
;;
-esac
+ esac
-exit 0
---------------------CONTROL SCRIPT END------------------------------------------
+ exit 0
ACPI integration
automatically disables laptop mode when the battery is low was
written by Jan Topinski.
------------------/etc/acpi/events/ac_adapter BEGIN------------------------------
-event=ac_adapter
-action=/etc/acpi/actions/ac.sh %e
-----------------/etc/acpi/events/ac_adapter END---------------------------------
+/etc/acpi/events/ac_adapter::
+
+ event=ac_adapter
+ action=/etc/acpi/actions/ac.sh %e
+
+/etc/acpi/events/battery::
+ event=battery.*
+ action=/etc/acpi/actions/battery.sh %e
------------------/etc/acpi/events/battery BEGIN---------------------------------
-event=battery.*
-action=/etc/acpi/actions/battery.sh %e
-----------------/etc/acpi/events/battery END------------------------------------
+/etc/acpi/actions/ac.sh::
+ #!/bin/bash
-----------------/etc/acpi/actions/ac.sh BEGIN-----------------------------------
-#!/bin/bash
+ # ac on/offline event handler
-# ac on/offline event handler
+ status=`awk '/^state: / { print $2 }' /proc/acpi/ac_adapter/$2/state`
-status=`awk '/^state: / { print $2 }' /proc/acpi/ac_adapter/$2/state`
+ case $status in
+ "on-line")
+ /sbin/laptop_mode stop
+ exit 0
+ ;;
+ "off-line")
+ /sbin/laptop_mode start
+ exit 0
+ ;;
+ esac
-case $status in
- "on-line")
- /sbin/laptop_mode stop
- exit 0
- ;;
- "off-line")
- /sbin/laptop_mode start
- exit 0
- ;;
-esac
----------------------------/etc/acpi/actions/ac.sh END--------------------------
+/etc/acpi/actions/battery.sh::
----------------------------/etc/acpi/actions/battery.sh BEGIN-------------------
-#! /bin/bash
+ #! /bin/bash
-# Automatically disable laptop mode when the battery almost runs out.
+ # Automatically disable laptop mode when the battery almost runs out.
-BATT_INFO=/proc/acpi/battery/$2/state
+ BATT_INFO=/proc/acpi/battery/$2/state
-if [[ -f /proc/sys/vm/laptop_mode ]]
-then
- LM=`cat /proc/sys/vm/laptop_mode`
- if [[ $LM -gt 0 ]]
- then
- if [[ -f $BATT_INFO ]]
+ if [[ -f /proc/sys/vm/laptop_mode ]]
+ then
+ LM=`cat /proc/sys/vm/laptop_mode`
+ if [[ $LM -gt 0 ]]
then
- # Source the config file only now that we know we need
- if [ -f /etc/default/laptop-mode ] ; then
- # Debian
- . /etc/default/laptop-mode
- elif [ -f /etc/sysconfig/laptop-mode ] ; then
- # Others
- . /etc/sysconfig/laptop-mode
- fi
- MINIMUM_BATTERY_MINUTES=${MINIMUM_BATTERY_MINUTES:-'10'}
-
- ACTION="`cat $BATT_INFO | grep charging | cut -c 26-`"
- if [[ ACTION -eq "discharging" ]]
- then
- PRESENT_RATE=`cat $BATT_INFO | grep "present rate:" | sed "s/.* \([0-9][0-9]* \).*/\1/" `
- REMAINING=`cat $BATT_INFO | grep "remaining capacity:" | sed "s/.* \([0-9][0-9]* \).*/\1/" `
- fi
- if (($REMAINING * 60 / $PRESENT_RATE < $MINIMUM_BATTERY_MINUTES))
- then
- /sbin/laptop_mode stop
- fi
- else
- logger -p daemon.warning "You are using laptop mode and your battery interface $BATT_INFO is missing. This may lead to loss of data when the battery runs out. Check kernel ACPI support and /proc/acpi/battery folder, and edit /etc/acpi/battery.sh to set BATT_INFO to the correct path."
+ if [[ -f $BATT_INFO ]]
+ then
+ # Source the config file only now that we know we need
+ if [ -f /etc/default/laptop-mode ] ; then
+ # Debian
+ . /etc/default/laptop-mode
+ elif [ -f /etc/sysconfig/laptop-mode ] ; then
+ # Others
+ . /etc/sysconfig/laptop-mode
+ fi
+ MINIMUM_BATTERY_MINUTES=${MINIMUM_BATTERY_MINUTES:-'10'}
+
+ ACTION="`cat $BATT_INFO | grep charging | cut -c 26-`"
+ if [[ ACTION -eq "discharging" ]]
+ then
+ PRESENT_RATE=`cat $BATT_INFO | grep "present rate:" | sed "s/.* \([0-9][0-9]* \).*/\1/" `
+ REMAINING=`cat $BATT_INFO | grep "remaining capacity:" | sed "s/.* \([0-9][0-9]* \).*/\1/" `
+ fi
+ if (($REMAINING * 60 / $PRESENT_RATE < $MINIMUM_BATTERY_MINUTES))
+ then
+ /sbin/laptop_mode stop
+ fi
+ else
+ logger -p daemon.warning "You are using laptop mode and your battery interface $BATT_INFO is missing. This may lead to loss of data when the battery runs out. Check kernel ACPI support and /proc/acpi/battery folder, and edit /etc/acpi/battery.sh to set BATT_INFO to the correct path."
+ fi
fi
- fi
-fi
----------------------------/etc/acpi/actions/battery.sh END--------------------
+ fi
Monitoring tool
.. SPDX-License-Identifier: GPL-2.0+
-:orphan:
LG Gram laptop extra features
=============================
+=========================================
Sony Notebook Control Driver (SNC) Readme
------------------------------------------
- Copyright (C) 2004- 2005 Stelian Pop <stelian@popies.net>
- Copyright (C) 2007 Mattia Dongili <malattia@linux.it>
+=========================================
+
+ - Copyright (C) 2004- 2005 Stelian Pop <stelian@popies.net>
+ - Copyright (C) 2007 Mattia Dongili <malattia@linux.it>
This mini-driver drives the SNC and SPIC device present in the ACPI BIOS of the
Sony Vaio laptops. This driver mixes both devices functions under the same
Fn keys (hotkeys):
------------------
+
Some models report hotkeys through the SNC or SPIC devices, such events are
reported both through the ACPI subsystem as acpi events and through the INPUT
subsystem. See the logs of /proc/bus/input/devices to find out what those
/sys/class/backlight/sony/
directory. You will be able to query and set the current screen
brightness:
+
+ ====================== =========================================
brightness get/set screen brightness (an integer
between 0 and 7)
actual_brightness reading from this file will query the HW
to get real brightness value
max_brightness the maximum brightness value
+ ====================== =========================================
Platform specific:
standard UNIX tools.
The files are:
+
+ ====================== ==========================================
brightness_default screen brightness which will be set
when the laptop will be rebooted
cdpower power on/off the internal CD drive
(only in debug mode)
bluetoothpower power on/off the internal bluetooth device
fanspeed get/set the fan speed
+ ====================== ==========================================
Note that some files may be missing if they are not supported
by your particular laptop model.
-Example usage:
+Example usage::
+
# echo "1" > /sys/devices/platform/sony-laptop/brightness_default
-sets the lowest screen brightness for the next and later reboots,
+
+sets the lowest screen brightness for the next and later reboots
+
+::
+
# echo "8" > /sys/devices/platform/sony-laptop/brightness_default
-sets the highest screen brightness for the next and later reboots,
+
+sets the highest screen brightness for the next and later reboots
+
+::
+
# cat /sys/devices/platform/sony-laptop/brightness_default
-retrieves the value.
+
+retrieves the value
+
+::
# echo "0" > /sys/devices/platform/sony-laptop/audiopower
-powers off the sound card,
+
+powers off the sound card
+
+::
+
# echo "1" > /sys/devices/platform/sony-laptop/audiopower
+
powers on the sound card.
More recent Vaio models expose a consistent set of ACPI methods to
control radio frequency emitting devices. If you are a lucky owner of
such a laptop you will find the necessary rfkill devices under
-/sys/class/rfkill. Check those starting with sony-* in
+/sys/class/rfkill. Check those starting with sony-* in::
+
# grep . /sys/class/rfkill/*/{state,name}
your ACPI BIOS could have on your laptop), load the driver and
pass the option 'debug=1'.
-REPEAT: DON'T DO THIS IF YOU DON'T LIKE RISKY BUSINESS.
+REPEAT:
+ **DON'T DO THIS IF YOU DON'T LIKE RISKY BUSINESS.**
In your kernel logs you will find the list of all ACPI methods
the SNC device has on your laptop.
* For new models you will see a long list of meaningless method names,
-reading the DSDT table source should reveal that:
+ reading the DSDT table source should reveal that:
+
(1) the SNC device uses an internal capability lookup table
(2) SN00 is used to find values in the lookup table
(3) SN06 and SN07 are used to call into the real methods based on
offsets you can obtain iterating the table using SN00
(4) SN02 used to enable events.
+
Some values in the capability lookup table are more or less known, see
the code for all sony_call_snc_handle calls, others are more obscure.
* For old models you can see the GCDP/GCDP methods used to pwer on/off
-the CD drive, but there are others and they are usually different from
-model to model.
+ the CD drive, but there are others and they are usually different from
+ model to model.
-I HAVE NO IDEA WHAT THOSE METHODS DO.
+**I HAVE NO IDEA WHAT THOSE METHODS DO.**
The sony-laptop driver creates, for some of those methods (the most
current ones found on several Vaio models), an entry under
+==================================================
Sony Programmable I/O Control Device Driver Readme
---------------------------------------------------
- Copyright (C) 2001-2004 Stelian Pop <stelian@popies.net>
- Copyright (C) 2001-2002 Alcôve <www.alcove.com>
- Copyright (C) 2001 Michael Ashley <m.ashley@unsw.edu.au>
- Copyright (C) 2001 Junichi Morita <jun1m@mars.dti.ne.jp>
- Copyright (C) 2000 Takaya Kinjo <t-kinjo@tc4.so-net.ne.jp>
- Copyright (C) 2000 Andrew Tridgell <tridge@samba.org>
+==================================================
+
+ - Copyright (C) 2001-2004 Stelian Pop <stelian@popies.net>
+ - Copyright (C) 2001-2002 Alcôve <www.alcove.com>
+ - Copyright (C) 2001 Michael Ashley <m.ashley@unsw.edu.au>
+ - Copyright (C) 2001 Junichi Morita <jun1m@mars.dti.ne.jp>
+ - Copyright (C) 2000 Takaya Kinjo <t-kinjo@tc4.so-net.ne.jp>
+ - Copyright (C) 2000 Andrew Tridgell <tridge@samba.org>
This driver enables access to the Sony Programmable I/O Control Device which
can be found in many Sony Vaio laptops. Some newer Sony laptops (seems to be
It will give access (through a user space utility) to some events those laptops
generate, like:
+
- jogdial events (the small wheel on the side of Vaios)
- capture button events (only on Vaio Picturebook series)
- Fn keys
module or sonypi.<param>=<value> on the kernel boot line when sonypi is
statically linked into the kernel). Those options are:
- minor: minor number of the misc device /dev/sonypi,
+ =============== =======================================================
+ minor: minor number of the misc device /dev/sonypi,
default is -1 (automatic allocation, see /proc/misc
or kernel logs)
set to 0xffffffff, meaning that all possible events
will be tried. You can use the following bits to
construct your own event mask (from
- drivers/char/sonypi.h):
- SONYPI_JOGGER_MASK 0x0001
- SONYPI_CAPTURE_MASK 0x0002
- SONYPI_FNKEY_MASK 0x0004
- SONYPI_BLUETOOTH_MASK 0x0008
- SONYPI_PKEY_MASK 0x0010
- SONYPI_BACK_MASK 0x0020
- SONYPI_HELP_MASK 0x0040
- SONYPI_LID_MASK 0x0080
- SONYPI_ZOOM_MASK 0x0100
- SONYPI_THUMBPHRASE_MASK 0x0200
+ drivers/char/sonypi.h)::
+
+ SONYPI_JOGGER_MASK 0x0001
+ SONYPI_CAPTURE_MASK 0x0002
+ SONYPI_FNKEY_MASK 0x0004
+ SONYPI_BLUETOOTH_MASK 0x0008
+ SONYPI_PKEY_MASK 0x0010
+ SONYPI_BACK_MASK 0x0020
+ SONYPI_HELP_MASK 0x0040
+ SONYPI_LID_MASK 0x0080
+ SONYPI_ZOOM_MASK 0x0100
+ SONYPI_THUMBPHRASE_MASK 0x0200
SONYPI_MEYE_MASK 0x0400
SONYPI_MEMORYSTICK_MASK 0x0800
SONYPI_BATTERY_MASK 0x1000
created, one which interprets the jogdial events as
mouse events, the other one which acts like a
keyboard reporting the pressing of the special keys.
+ =============== =======================================================
Module use:
-----------
In order to automatically load the sonypi module on use, you can put those
-lines a configuration file in /etc/modprobe.d/:
+lines a configuration file in /etc/modprobe.d/::
alias char-major-10-250 sonypi
options sonypi minor=250
-This supposes the use of minor 250 for the sonypi device:
+This supposes the use of minor 250 for the sonypi device::
# mknod /dev/sonypi c 10 250
http://www.acc.umu.se/~erikw/program/smartdimmer-0.1.tar.bz2
- since all development was done by reverse engineering, there is
- _absolutely no guarantee_ that this driver will not crash your
+ *absolutely no guarantee* that this driver will not crash your
laptop. Permanently.
- ThinkPad ACPI Extras Driver
+===========================
+ThinkPad ACPI Extras Driver
+===========================
- Version 0.25
- October 16th, 2013
+Version 0.25
- Borislav Deianov <borislav@users.sf.net>
- Henrique de Moraes Holschuh <hmh@hmh.eng.br>
- http://ibm-acpi.sf.net/
+October 16th, 2013
+- Borislav Deianov <borislav@users.sf.net>
+- Henrique de Moraes Holschuh <hmh@hmh.eng.br>
+
+http://ibm-acpi.sf.net/
This is a Linux driver for the IBM and Lenovo ThinkPad laptops. It
supports various features of these laptops which are accessible
and any and all userspace programs must deal with it.
-Notes about the sysfs interface:
+Notes about the sysfs interface
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Unlike what was done with the procfs interface, correctness when talking
to the sysfs interfaces will be enforced, as will correctness in the
--------------
procfs: /proc/acpi/ibm/driver
+
sysfs driver attribute: version
The driver name and version. No commands can be written to this file.
Version of the thinkpad-acpi sysfs interface, as an unsigned long
(output in hex format: 0xAAAABBCC), where:
- AAAA - major revision
- BB - minor revision
- CC - bugfix revision
+
+ AAAA
+ - major revision
+ BB
+ - minor revision
+ CC
+ - bugfix revision
The sysfs interface version changelog for the driver can be found at the
end of this document. Changes to the sysfs interface done by the kernel
--------
procfs: /proc/acpi/ibm/hotkey
+
sysfs device attribute: hotkey_*
In a ThinkPad, the ACPI HKEY handler is responsible for communicating
The driver enables the HKEY ("hot key") event reporting automatically
when loaded, and disables it when it is removed.
-The driver will report HKEY events in the following format:
+The driver will report HKEY events in the following format::
ibm/hotkey HKEY 00000080 0000xxxx
polling the "CMOS NVRAM" at least 10 times per second. The driver
attempts to enables this functionality automatically when required.
-procfs notes:
+procfs notes
+^^^^^^^^^^^^
-The following commands can be written to the /proc/acpi/ibm/hotkey file:
+The following commands can be written to the /proc/acpi/ibm/hotkey file::
echo 0xffffffff > /proc/acpi/ibm/hotkey -- enable all hot keys
echo 0 > /proc/acpi/ibm/hotkey -- disable all possible hot keys
echo reset > /proc/acpi/ibm/hotkey -- restore the recommended mask
The following commands have been deprecated and will cause the kernel
-to log a warning:
+to log a warning::
echo enable > /proc/acpi/ibm/hotkey -- does nothing
echo disable > /proc/acpi/ibm/hotkey -- returns an error
nor does it allow one to manipulate the hot key mask when the firmware
does not support masks at all, even if NVRAM polling is in use.
-sysfs notes:
+sysfs notes
+^^^^^^^^^^^
hotkey_bios_enabled:
DEPRECATED, WILL BE REMOVED SOON.
This attribute has poll()/select() support.
-input layer notes:
+input layer notes
+^^^^^^^^^^^^^^^^^
A Hot key is mapped to a single input layer EV_KEY event, possibly
followed by an EV_MSC MSC_SCAN event that shall contain that key's scan
The events are available in an input device, with the following id:
- Bus: BUS_HOST
- vendor: 0x1014 (PCI_VENDOR_ID_IBM) or
+ ============== ==============================
+ Bus BUS_HOST
+ vendor 0x1014 (PCI_VENDOR_ID_IBM) or
0x17aa (PCI_VENDOR_ID_LENOVO)
- product: 0x5054 ("TP")
- version: 0x4101
+ product 0x5054 ("TP")
+ version 0x4101
+ ============== ==============================
The version will have its LSB incremented if the keymap changes in a
backwards-compatible way. The MSB shall always be 0x41 for this input
Thinkpad-acpi Hot Key event map (version 0x4101):
+======= ======= ============== ==============================================
ACPI Scan
event code Key Notes
-
+======= ======= ============== ==============================================
0x1001 0x00 FN+F1 -
0x1002 0x01 FN+F2 IBM: battery (rare)
or toggle screen expand
0x1009 0x08 FN+F9 -
- .. .. ..
+
+... ... ... ...
+
0x100B 0x0A FN+F11 -
0x100C 0x0B FN+F12 Sleep to disk. You are always
0x1018 0x17 THINKPAD ThinkPad/Access IBM/Lenovo key
0x1019 0x18 unknown
-.. .. ..
+
+... ... ...
+
0x1020 0x1F unknown
+======= ======= ============== ==============================================
The ThinkPad firmware does not allow one to differentiate when most hot
keys are pressed or released (either that, or we don't know how to, yet).
In addition to the EV_KEY events, thinkpad-acpi may also issue EV_SW
events for switches:
+============== ==============================================
SW_RFKILL_ALL T60 and later hardware rfkill rocker switch
SW_TABLET_MODE Tablet ThinkPads HKEY events 0x5009 and 0x500A
+============== ==============================================
-Non hotkey ACPI HKEY event map:
--------------------------------
+Non hotkey ACPI HKEY event map
+------------------------------
Events that are never propagated by the driver:
+====== ==================================================
0x2304 System is waking up from suspend to undock
0x2305 System is waking up from suspend to eject bay
0x2404 System is waking up from hibernation to undock
0x6000 KEYBOARD: Numlock key pressed
0x6005 KEYBOARD: Fn key pressed (TO BE VERIFIED)
0x7000 Radio Switch may have changed state
+====== ==================================================
Events that are propagated by the driver to userspace:
+====== =====================================================
0x2313 ALARM: System is waking up from suspend because
the battery is nearly empty
0x2413 ALARM: System is waking up from hibernation because
0x6040 Nvidia Optimus/AC adapter related (TO BE VERIFIED)
0x60C0 X1 Yoga 2016, Tablet mode status changed
0x60F0 Thermal Transformation changed (GMTS, Windows)
+====== =====================================================
Battery nearly empty alarms are a last resort attempt to get the
operating system to hibernate or shutdown cleanly (0x2313), or shutdown
happens.
-Brightness hotkey notes:
+Brightness hotkey notes
+^^^^^^^^^^^^^^^^^^^^^^^
Don't mess with the brightness hotkeys in a Thinkpad. If you want
notifications for OSD, use the sysfs backlight class event support.
---------
procfs: /proc/acpi/ibm/bluetooth
+
sysfs device attribute: bluetooth_enable (deprecated)
+
sysfs rfkill class: switch "tpacpi_bluetooth_sw"
This feature shows the presence and current state of a ThinkPad
If the ThinkPad supports it, the Bluetooth state is stored in NVRAM,
so it is kept across reboots and power-off.
-Procfs notes:
+Procfs notes
+^^^^^^^^^^^^
-If Bluetooth is installed, the following commands can be used:
+If Bluetooth is installed, the following commands can be used::
echo enable > /proc/acpi/ibm/bluetooth
echo disable > /proc/acpi/ibm/bluetooth
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
If the Bluetooth CDC card is installed, it can be enabled /
disabled through the "bluetooth_enable" thinkpad-acpi device
attribute, and its current status can also be queried.
enable:
- 0: disables Bluetooth / Bluetooth is disabled
- 1: enables Bluetooth / Bluetooth is enabled.
+
+ - 0: disables Bluetooth / Bluetooth is disabled
+ - 1: enables Bluetooth / Bluetooth is enabled.
Note: this interface has been superseded by the generic rfkill
class. It has been deprecated, and it will be removed in year
2010.
rfkill controller switch "tpacpi_bluetooth_sw": refer to
- Documentation/rfkill.txt for details.
+ Documentation/driver-api/rfkill.rst for details.
Video output control -- /proc/acpi/ibm/video
--------------------------------------------
This feature allows control over the devices used for video output -
-LCD, CRT or DVI (if available). The following commands are available:
+LCD, CRT or DVI (if available). The following commands are available::
echo lcd_enable > /proc/acpi/ibm/video
echo lcd_disable > /proc/acpi/ibm/video
echo expand_toggle > /proc/acpi/ibm/video
echo video_switch > /proc/acpi/ibm/video
-NOTE: Access to this feature is restricted to processes owning the
-CAP_SYS_ADMIN capability for safety reasons, as it can interact badly
-enough with some versions of X.org to crash it.
+NOTE:
+ Access to this feature is restricted to processes owning the
+ CAP_SYS_ADMIN capability for safety reasons, as it can interact badly
+ enough with some versions of X.org to crash it.
Each video output device can be enabled or disabled individually.
Reading /proc/acpi/ibm/video shows the status of each device.
------------------
procfs: /proc/acpi/ibm/light
+
sysfs attributes: as per LED class, for the "tpacpi::thinklight" LED
-procfs notes:
+procfs notes
+^^^^^^^^^^^^
The ThinkLight status can be read and set through the procfs interface. A
few models which do not make the status available will show the ThinkLight
-status as "unknown". The available commands are:
+status as "unknown". The available commands are::
echo on > /proc/acpi/ibm/light
echo off > /proc/acpi/ibm/light
-sysfs notes:
+sysfs notes
+^^^^^^^^^^^
The ThinkLight sysfs interface is documented by the LED class
documentation, in Documentation/leds/leds-class.rst. The ThinkLight LED name
-----------------
procfs: /proc/acpi/ibm/cmos
+
sysfs device attribute: cmos_command
This feature is mostly used internally by the ACPI firmware to keep the legacy
effect and the behavior varies from model to model. Here is the behavior
on the X40 (tpb is the ThinkPad Buttons utility):
- 0 - Related to "Volume down" key press
- 1 - Related to "Volume up" key press
- 2 - Related to "Mute on" key press
- 3 - Related to "Access IBM" key press
- 4 - Related to "LCD brightness up" key press
- 5 - Related to "LCD brightness down" key press
- 11 - Related to "toggle screen expansion" key press/function
- 12 - Related to "ThinkLight on"
- 13 - Related to "ThinkLight off"
- 14 - Related to "ThinkLight" key press (toggle ThinkLight)
+ - 0 - Related to "Volume down" key press
+ - 1 - Related to "Volume up" key press
+ - 2 - Related to "Mute on" key press
+ - 3 - Related to "Access IBM" key press
+ - 4 - Related to "LCD brightness up" key press
+ - 5 - Related to "LCD brightness down" key press
+ - 11 - Related to "toggle screen expansion" key press/function
+ - 12 - Related to "ThinkLight on"
+ - 13 - Related to "ThinkLight off"
+ - 14 - Related to "ThinkLight" key press (toggle ThinkLight)
The cmos command interface is prone to firmware split-brain problems, as
in newer ThinkPads it is just a compatibility layer. Do not use it, it is
Audio mute and microphone mute LEDs are supported, but currently not
visible to userspace. They are used by the snd-hda-intel audio driver.
-procfs notes:
+procfs notes
+^^^^^^^^^^^^
-The available commands are:
+The available commands are::
echo '<LED number> on' >/proc/acpi/ibm/led
echo '<LED number> off' >/proc/acpi/ibm/led
controlled varies from model to model. Here is the common ThinkPad
mapping:
- 0 - power
- 1 - battery (orange)
- 2 - battery (green)
- 3 - UltraBase/dock
- 4 - UltraBay
- 5 - UltraBase battery slot
- 6 - (unknown)
- 7 - standby
- 8 - dock status 1
- 9 - dock status 2
- 10, 11 - (unknown)
- 12 - thinkvantage
- 13, 14, 15 - (unknown)
+ - 0 - power
+ - 1 - battery (orange)
+ - 2 - battery (green)
+ - 3 - UltraBase/dock
+ - 4 - UltraBay
+ - 5 - UltraBase battery slot
+ - 6 - (unknown)
+ - 7 - standby
+ - 8 - dock status 1
+ - 9 - dock status 2
+ - 10, 11 - (unknown)
+ - 12 - thinkvantage
+ - 13, 14, 15 - (unknown)
All of the above can be turned on and off and can be made to blink.
-sysfs notes:
+sysfs notes
+^^^^^^^^^^^
The ThinkPad LED sysfs interface is described in detail by the LED class
documentation, in Documentation/leds/leds-class.rst.
audible alerts in various situations. This feature allows the same
sounds to be triggered manually.
-The commands are non-negative integer numbers:
+The commands are non-negative integer numbers::
echo <number> >/proc/acpi/ibm/beep
and the sounds vary from model to model. Here is the behavior on the
X40:
- 0 - stop a sound in progress (but use 17 to stop 16)
- 2 - two beeps, pause, third beep ("low battery")
- 3 - single beep
- 4 - high, followed by low-pitched beep ("unable")
- 5 - single beep
- 6 - very high, followed by high-pitched beep ("AC/DC")
- 7 - high-pitched beep
- 9 - three short beeps
- 10 - very long beep
- 12 - low-pitched beep
- 15 - three high-pitched beeps repeating constantly, stop with 0
- 16 - one medium-pitched beep repeating constantly, stop with 17
- 17 - stop 16
+ - 0 - stop a sound in progress (but use 17 to stop 16)
+ - 2 - two beeps, pause, third beep ("low battery")
+ - 3 - single beep
+ - 4 - high, followed by low-pitched beep ("unable")
+ - 5 - single beep
+ - 6 - very high, followed by high-pitched beep ("AC/DC")
+ - 7 - high-pitched beep
+ - 9 - three short beeps
+ - 10 - very long beep
+ - 12 - low-pitched beep
+ - 15 - three high-pitched beeps repeating constantly, stop with 0
+ - 16 - one medium-pitched beep repeating constantly, stop with 17
+ - 17 - stop 16
Temperature sensors
-------------------
procfs: /proc/acpi/ibm/thermal
+
sysfs device attributes: (hwmon "thinkpad") temp*_input
Most ThinkPads include six or more separate temperature sensors but only
ThinkPads, and up to sixteen different sensors on newer ThinkPads.
For example, on the X40, a typical output may be:
-temperatures: 42 42 45 41 36 -128 33 -128
+
+temperatures:
+ 42 42 45 41 36 -128 33 -128
On the T43/p, a typical output may be:
-temperatures: 48 48 36 52 38 -128 31 -128 48 52 48 -128 -128 -128 -128 -128
+
+temperatures:
+ 48 48 36 52 38 -128 31 -128 48 52 48 -128 -128 -128 -128 -128
The mapping of thermal sensors to physical locations varies depending on
system-board model (and thus, on ThinkPad model).
Most (newer?) models seem to follow this pattern:
-1: CPU
-2: (depends on model)
-3: (depends on model)
-4: GPU
-5: Main battery: main sensor
-6: Bay battery: main sensor
-7: Main battery: secondary sensor
-8: Bay battery: secondary sensor
-9-15: (depends on model)
+- 1: CPU
+- 2: (depends on model)
+- 3: (depends on model)
+- 4: GPU
+- 5: Main battery: main sensor
+- 6: Bay battery: main sensor
+- 7: Main battery: secondary sensor
+- 8: Bay battery: secondary sensor
+- 9-15: (depends on model)
For the R51 (source: Thomas Gruber):
-2: Mini-PCI
-3: Internal HDD
+
+- 2: Mini-PCI
+- 3: Internal HDD
For the T43, T43/p (source: Shmidoax/Thinkwiki.org)
http://thinkwiki.org/wiki/Thermal_Sensors#ThinkPad_T43.2C_T43p
-2: System board, left side (near PCMCIA slot), reported as HDAPS temp
-3: PCMCIA slot
-9: MCH (northbridge) to DRAM Bus
-10: Clock-generator, mini-pci card and ICH (southbridge), under Mini-PCI
- card, under touchpad
-11: Power regulator, underside of system board, below F2 key
+
+- 2: System board, left side (near PCMCIA slot), reported as HDAPS temp
+- 3: PCMCIA slot
+- 9: MCH (northbridge) to DRAM Bus
+- 10: Clock-generator, mini-pci card and ICH (southbridge), under Mini-PCI
+ card, under touchpad
+- 11: Power regulator, underside of system board, below F2 key
The A31 has a very atypical layout for the thermal sensors
(source: Milos Popovic, http://thinkwiki.org/wiki/Thermal_Sensors#ThinkPad_A31)
-1: CPU
-2: Main Battery: main sensor
-3: Power Converter
-4: Bay Battery: main sensor
-5: MCH (northbridge)
-6: PCMCIA/ambient
-7: Main Battery: secondary sensor
-8: Bay Battery: secondary sensor
+- 1: CPU
+- 2: Main Battery: main sensor
+- 3: Power Converter
+- 4: Bay Battery: main sensor
+- 5: MCH (northbridge)
+- 6: PCMCIA/ambient
+- 7: Main Battery: secondary sensor
+- 8: Bay Battery: secondary sensor
+
+
+Procfs notes
+^^^^^^^^^^^^
-Procfs notes:
Readings from sensors that are not available return -128.
No commands can be written to this file.
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
+
Sensors that are not available return the ENXIO error. This
status may change at runtime, as there are hotplug thermal
sensors, like those inside the batteries and docks.
Use it to determine the register holding the fan
speed on some models. To do that, do the following:
+
- make sure the battery is fully charged
- make sure the fan is running
- use above mentioned tool to read out the EC
----------------------
procfs: /proc/acpi/ibm/brightness
+
sysfs backlight device "thinkpad_screen"
This feature allows software control of the LCD brightness on ThinkPad
forces it to be enabled when available, even if the standard ACPI
interface is also available.
-Procfs notes:
+Procfs notes
+^^^^^^^^^^^^
- The available commands are:
+The available commands are::
echo up >/proc/acpi/ibm/brightness
echo down >/proc/acpi/ibm/brightness
echo 'level <level>' >/proc/acpi/ibm/brightness
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
The interface is implemented through the backlight sysfs class, which is
poorly documented at this time.
--------------------------------------
procfs: /proc/acpi/ibm/volume
+
ALSA: "ThinkPad Console Audio Control", default ID: "ThinkPadEC"
NOTE: by default, the volume control interface operates in read-only
mixer.
-About the ThinkPad Console Audio control:
+About the ThinkPad Console Audio control
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ThinkPads have a built-in amplifier and muting circuit that drives the
console headphone and speakers. This circuit is after the main AC97
involved).
-The ThinkPad-ACPI volume control:
+The ThinkPad-ACPI volume control
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The preferred way to interact with the Console Audio control is the
ALSA interface.
The legacy procfs interface allows one to read the current state,
-and if volume control is enabled, accepts the following commands:
+and if volume control is enabled, accepts the following commands::
echo up >/proc/acpi/ibm/volume
echo down >/proc/acpi/ibm/volume
---------------------------------------------------------
procfs: /proc/acpi/ibm/fan
-sysfs device attributes: (hwmon "thinkpad") fan1_input, pwm1,
- pwm1_enable, fan2_input
+
+sysfs device attributes: (hwmon "thinkpad") fan1_input, pwm1, pwm1_enable, fan2_input
+
sysfs hwmon driver attributes: fan_watchdog
-NOTE NOTE NOTE: fan control operations are disabled by default for
-safety reasons. To enable them, the module parameter "fan_control=1"
-must be given to thinkpad-acpi.
+NOTE NOTE NOTE:
+ fan control operations are disabled by default for
+ safety reasons. To enable them, the module parameter "fan_control=1"
+ must be given to thinkpad-acpi.
This feature attempts to show the current fan speed, control mode and
other fan data that might be available. The speed is read directly
Some Lenovo ThinkPads support a secondary fan. This fan cannot be
controlled separately, it shares the main fan control.
-Fan levels:
+Fan levels
+^^^^^^^^^^
Most ThinkPad fans work in "levels" at the firmware interface. Level 0
stops the fan. The higher the level, the higher the fan speed, although
means other than the "enable", "disable", and "level" procfs fan
commands, or the hwmon fan control sysfs interface.
-Procfs notes:
+Procfs notes
+^^^^^^^^^^^^
-The fan may be enabled or disabled with the following commands:
+The fan may be enabled or disabled with the following commands::
echo enable >/proc/acpi/ibm/fan
echo disable >/proc/acpi/ibm/fan
Placing a fan on level 0 is the same as disabling it. Enabling a fan
will try to place it in a safe level if it is too slow or disabled.
-The fan level can be controlled with the command:
+The fan level can be controlled with the command::
echo 'level <level>' > /proc/acpi/ibm/fan
On the X31 and X40 (and ONLY on those models), the fan speed can be
controlled to a certain degree. Once the fan is running, it can be
-forced to run faster or slower with the following command:
+forced to run faster or slower with the following command::
echo 'speed <speed>' > /proc/acpi/ibm/fan
fan cannot be stopped or started with this command. This functionality
is incomplete, and not available through the sysfs interface.
-To program the safety watchdog, use the "watchdog" command.
+To program the safety watchdog, use the "watchdog" command::
echo 'watchdog <interval in seconds>' > /proc/acpi/ibm/fan
If you want to disable the watchdog, use 0 as the interval.
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
The sysfs interface follows the hwmon subsystem guidelines for the most
part, and the exception is the fan safety watchdog.
Features not yet implemented by the driver return ENOSYS.
hwmon device attribute pwm1_enable:
- 0: PWM offline (fan is set to full-speed mode)
- 1: Manual PWM control (use pwm1 to set fan level)
- 2: Hardware PWM control (EC "auto" mode)
- 3: reserved (Software PWM control, not implemented yet)
+ - 0: PWM offline (fan is set to full-speed mode)
+ - 1: Manual PWM control (use pwm1 to set fan level)
+ - 2: Hardware PWM control (EC "auto" mode)
+ - 3: reserved (Software PWM control, not implemented yet)
Modes 0 and 2 are not supported by all ThinkPads, and the
driver is not always able to detect this. If it does know a
---
procfs: /proc/acpi/ibm/wan
+
sysfs device attribute: wwan_enable (deprecated)
+
sysfs rfkill class: switch "tpacpi_wwan_sw"
This feature shows the presence and current state of the built-in
It was tested on a Lenovo ThinkPad X60. It should probably work on other
ThinkPad models which come with this module installed.
-Procfs notes:
+Procfs notes
+^^^^^^^^^^^^
-If the W-WAN card is installed, the following commands can be used:
+If the W-WAN card is installed, the following commands can be used::
echo enable > /proc/acpi/ibm/wan
echo disable > /proc/acpi/ibm/wan
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
If the W-WAN card is installed, it can be enabled /
disabled through the "wwan_enable" thinkpad-acpi device
attribute, and its current status can also be queried.
enable:
- 0: disables WWAN card / WWAN card is disabled
- 1: enables WWAN card / WWAN card is enabled.
+ - 0: disables WWAN card / WWAN card is disabled
+ - 1: enables WWAN card / WWAN card is enabled.
Note: this interface has been superseded by the generic rfkill
class. It has been deprecated, and it will be removed in year
2010.
rfkill controller switch "tpacpi_wwan_sw": refer to
- Documentation/rfkill.txt for details.
+ Documentation/driver-api/rfkill.rst for details.
EXPERIMENTAL: UWB
This feature exports an rfkill controller for the UWB device, if one is
present and enabled in the BIOS.
-Sysfs notes:
+Sysfs notes
+^^^^^^^^^^^
rfkill controller switch "tpacpi_uwb_sw": refer to
- Documentation/rfkill.txt for details.
+ Documentation/driver-api/rfkill.rst for details.
Adaptive keyboard
-----------------
Lenovo X1 Carbon 2nd gen (2014)'s adaptive keyboard. The value can be read
and set.
-1 = Home mode
-2 = Web-browser mode
-3 = Web-conference mode
-4 = Function mode
-5 = Layflat mode
+- 1 = Home mode
+- 2 = Web-browser mode
+- 3 = Web-conference mode
+- 4 = Function mode
+- 5 = Layflat mode
For more details about which buttons will appear depending on the mode, please
review the laptop's user guide:
------------------------------------
Multiple commands can be written to the proc files in one shot by
-separating them with commas, for example:
+separating them with commas, for example::
echo enable,0xffff > /proc/acpi/ibm/hotkey
echo lcd_disable,crt_enable > /proc/acpi/ibm/video
Commands can also be specified when loading the thinkpad-acpi module,
-for example:
+for example::
modprobe thinkpad_acpi hotkey=enable,0xffff video=auto_disable
-------------------------
The module takes a debug parameter which can be used to selectively
-enable various classes of debugging output, for example:
+enable various classes of debugging output, for example::
modprobe thinkpad_acpi debug=0xffff
will enable all debugging output classes. It takes a bitmask, so
to enable more than one output class, just add their values.
+ ============= ======================================
Debug bitmask Description
+ ============= ======================================
0x8000 Disclose PID of userspace programs
accessing some functions of the driver
0x0001 Initialization and probing
0x0010 Fan control
0x0020 Backlight brightness
0x0040 Audio mixer/volume control
+ ============= ======================================
There is also a kernel build option to enable more debugging
information, which may be necessary to debug driver problems.
not, please contact ibm-acpi-devel@lists.sourceforge.net with a report.
-Sysfs interface changelog:
+Sysfs interface changelog
+^^^^^^^^^^^^^^^^^^^^^^^^^
+========= ===============================================================
0x000100: Initial sysfs support, as a single platform driver and
device.
0x000200: Hot key support for 32 hot keys, and radio slider switch
0x030000: Thermal and fan sysfs attributes were moved to the hwmon
device instead of being attached to the backing platform
device.
+========= ===============================================================
-Kernel driver toshiba_haps
+====================================
Toshiba HDD Active Protection Sensor
====================================
+Kernel driver: toshiba_haps
+
Author: Azael Avalos <coproscefalo@gmail.com>
-0. Contents
------------
+.. 0. Contents
-1. Description
-2. Interface
-3. Accelerometer axes
-4. Supported devices
-5. Usage
+ 1. Description
+ 2. Interface
+ 3. Accelerometer axes
+ 4. Supported devices
+ 5. Usage
1. Description
------------
This device comes with 3 methods:
-_STA - Checks existence of the device, returning Zero if the device does not
+
+==== =====================================================================
+_STA Checks existence of the device, returning Zero if the device does not
exists or is not supported.
-PTLV - Sets the desired protection level.
-RSSS - Shuts down the HDD protection interface for a few seconds,
+PTLV Sets the desired protection level.
+RSSS Shuts down the HDD protection interface for a few seconds,
then restores normal operation.
+==== =====================================================================
Note:
-The presence of Solid State Drives (SSD) can make this driver to fail loading,
-given the fact that such drives have no movable parts, and thus, not requiring
-any "protection" as well as failing during the evaluation of the _STA method
-found under this device.
+ The presence of Solid State Drives (SSD) can make this driver to fail loading,
+ given the fact that such drives have no movable parts, and thus, not requiring
+ any "protection" as well as failing during the evaluation of the _STA method
+ found under this device.
3. Accelerometer axes
--------
The sysfs files under /sys/devices/LNXSYSTM:00/LNXSYBUS:00/TOS620A:00/ are:
-protection_level - The protection_level is readable and writeable, and
+
+================ ============================================================
+protection_level The protection_level is readable and writeable, and
provides a way to let userspace query the current protection
level, as well as set the desired protection level, the
- available protection levels are:
- 0 - Disabled | 1 - Low | 2 - Medium | 3 - High
-reset_protection - The reset_protection entry is writeable only, being "1"
+ available protection levels are::
+
+ ============ ======= ========== ========
+ 0 - Disabled 1 - Low 2 - Medium 3 - High
+ ============ ======= ========== ========
+
+reset_protection The reset_protection entry is writeable only, being "1"
the only parameter it accepts, it is used to trigger
a reset of the protection interface.
+================ ============================================================
+======================================
+Parallel port LCD/Keypad Panel support
+======================================
+
Some LCDs allow you to define up to 8 characters, mapped to ASCII
characters 0 to 7. The escape code to define a new character is
'\e[LG' followed by one digit from 0 to 7, representing the character
top of the character to the bottom. On a 5x7 matrix, only the 5 lower
bits of the 7 first bytes are used for each character. If the string
is incomplete, only complete lines will be redefined. Here are some
-examples :
+examples::
printf "\e[LG0010101050D1F0C04;" => 0 = [enter]
printf "\e[LG1040E1F0000000000;" => 1 = [up]
printf "\e[LG00002061E1E060200;" => small speaker
Willy
-
+=====================
+CMA Debugfs Interface
+=====================
+
The CMA debugfs interface is useful to retrieve basic information out of the
different CMA areas and to test allocation/release in each of the areas.
- [RO] count: Amount of memory in the CMA area.
- [RO] order_per_bit: Order of pages represented by one bit.
- [RO] bitmap: The bitmap of page states in the zone.
- - [WO] alloc: Allocate N pages from that CMA area. For example:
+ - [WO] alloc: Allocate N pages from that CMA area. For example::
echo 5 > <debugfs>/cma/cma-2/alloc
Linux memory management is a complex system with many configurable
settings. Most of these settings are available via ``/proc``
filesystem and can be quired and adjusted using ``sysctl``. These APIs
-are described in Documentation/sysctl/vm.txt and in `man 5 proc`_.
+are described in Documentation/admin-guide/sysctl/vm.rst and in `man 5 proc`_.
.. _man 5 proc: http://man7.org/linux/man-pages/man5/proc.5.html
:maxdepth: 1
concepts
+ cma_debugfs
hugetlbpage
idle_page_tracking
ksm
If a region of memory must be split into at least one new MADV_MERGEABLE
or MADV_UNMERGEABLE region, the madvise may return ENOMEM if the process
-will exceed ``vm.max_map_count`` (see Documentation/sysctl/vm.txt).
+will exceed ``vm.max_map_count`` (see Documentation/admin-guide/sysctl/vm.rst).
Like other madvise calls, they are intended for use on mapped areas of
the user address space: they will report ENOMEM if the specified range
support.
Memory policies should not be confused with cpusets
-(``Documentation/cgroup-v1/cpusets.rst``)
+(``Documentation/admin-guide/cgroup-v1/cpusets.rst``)
which is an administrative mechanism for restricting the nodes from which
memory may be allocated by a set of processes. Memory policies are a
programming interface that a NUMA-aware application can take advantage of. When
- Namespaces compatibility list
+=============================
+Namespaces compatibility list
+=============================
This document contains the information about the problems user
may have when creating tasks living in different namespaces.
occur when tasks share some namespace (the columns) while living
in different other namespaces (the rows):
- UTS IPC VFS PID User Net
+==== === === === === ==== ===
+- UTS IPC VFS PID User Net
+==== === === === === ==== ===
UTS X
IPC X 1
VFS X
PID 1 1 X
User 2 2 X
Net X
+==== === === === === ==== ===
1. Both the IPC and the PID namespaces provide IDs to address
object inside the kernel. E.g. semaphore with IPCID or
even having equal UIDs.
But currently this is not so.
-
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==========
+Namespaces
+==========
+
+.. toctree::
+ :maxdepth: 1
+
+ compatibility-list
+ resource-control
+===========================
+Namespaces research control
+===========================
+
There are a lot of kinds of objects in the kernel that don't have
individual limits or that have limits that are ineffective when a set
of processes is allowed to switch user ids. With user namespaces
+==========================
ARM Cache Coherent Network
==========================
require "xp" and "vc" as as above plus "port" (device port index),
"dir" (transmit/receive direction), comparator values ("cmp_l"
and "cmp_h") and "mask", being index of the comparator mask.
+
Masks are defined separately from the event description
(due to limited number of the config values) in the "cmp_mask"
directory, with first 8 configurable by user and additional
will be overwritten anyway). In case of this processor being offlined,
the events are migrated to another one and the attribute is updated.
-Example of perf tool use:
+Example of perf tool use::
-/ # perf list | grep ccn
- ccn/cycles/ [Kernel PMU event]
-<...>
- ccn/xp_valid_flit,xp=?,port=?,vc=?,dir=?/ [Kernel PMU event]
-<...>
+ / # perf list | grep ccn
+ ccn/cycles/ [Kernel PMU event]
+ <...>
+ ccn/xp_valid_flit,xp=?,port=?,vc=?,dir=?/ [Kernel PMU event]
+ <...>
-/ # perf stat -a -e ccn/cycles/,ccn/xp_valid_flit,xp=1,port=0,vc=1,dir=1/ \
- sleep 1
+ / # perf stat -a -e ccn/cycles/,ccn/xp_valid_flit,xp=1,port=0,vc=1,dir=1/ \
+ sleep 1
The driver does not support sampling, therefore "perf record" will
not work. Per-task (without "-a") perf sessions are not supported.
+==================================
ARM DynamIQ Shared Unit (DSU) PMU
==================================
The DSU provides a bitmap for a subset of implemented events via hardware
registers. There is no way for the driver to determine if the other events
are available or not. Hence the driver exposes only those events advertised
-by the DSU, in "events" directory under :
+by the DSU, in "events" directory under::
/sys/bus/event_sources/devices/arm_dsu_<N>/
The driver also exposes the CPUs connected to the DSU instance in "associated_cpus".
-e.g usage :
+e.g usage::
perf stat -a -e arm_dsu_0/cycles/
+======================================================
HiSilicon SoC uncore Performance Monitoring Unit (PMU)
======================================================
+
The HiSilicon SoC chip includes various independent system device PMUs
such as L3 cache (L3C), Hydra Home Agent (HHA) and DDRC. These PMUs are
independent and have hardware logic to gather statistics and performance
two HHAs (0 - 1) and four DDRCs (0 - 3), respectively.
HiSilicon SoC uncore PMU driver
----------------------------------------
+-------------------------------
+
Each device PMU has separate registers for event counting, control and
interrupt, and the PMU driver shall register perf PMU drivers like L3C,
HHA and DDRC etc. The available events and configuration options shall
-be described in the sysfs, see :
+be described in the sysfs, see:
+
/sys/devices/hisi_sccl{X}_<l3c{Y}/hha{Y}/ddrc{Y}>/, or
/sys/bus/event_source/devices/hisi_sccl{X}_<l3c{Y}/hha{Y}/ddrc{Y}>.
The "perf list" command shall list the available events from sysfs.
name will appear in event listing as hisi_sccl<sccl-id>_module<index-id>.
where "sccl-id" is the identifier of the SCCL and "index-id" is the index of
module.
+
e.g. hisi_sccl3_l3c0/rd_hit_cpipe is READ_HIT_CPIPE event of L3C index #0 in
SCCL ID #3.
+
e.g. hisi_sccl1_hha0/rx_operations is RX_OPERATIONS event of HHA index #0 in
SCCL ID #1.
The driver also provides a "cpumask" sysfs attribute, which shows the CPU core
ID used to count the uncore PMU event.
-Example usage of perf:
-$# perf list
-hisi_sccl3_l3c0/rd_hit_cpipe/ [kernel PMU event]
-------------------------------------------
-hisi_sccl3_l3c0/wr_hit_cpipe/ [kernel PMU event]
-------------------------------------------
-hisi_sccl1_l3c0/rd_hit_cpipe/ [kernel PMU event]
-------------------------------------------
-hisi_sccl1_l3c0/wr_hit_cpipe/ [kernel PMU event]
-------------------------------------------
-
-$# perf stat -a -e hisi_sccl3_l3c0/rd_hit_cpipe/ sleep 5
-$# perf stat -a -e hisi_sccl3_l3c0/config=0x02/ sleep 5
+Example usage of perf::
+
+ $# perf list
+ hisi_sccl3_l3c0/rd_hit_cpipe/ [kernel PMU event]
+ ------------------------------------------
+ hisi_sccl3_l3c0/wr_hit_cpipe/ [kernel PMU event]
+ ------------------------------------------
+ hisi_sccl1_l3c0/rd_hit_cpipe/ [kernel PMU event]
+ ------------------------------------------
+ hisi_sccl1_l3c0/wr_hit_cpipe/ [kernel PMU event]
+ ------------------------------------------
+
+ $# perf stat -a -e hisi_sccl3_l3c0/rd_hit_cpipe/ sleep 5
+ $# perf stat -a -e hisi_sccl3_l3c0/config=0x02/ sleep 5
The current driver does not support sampling. So "perf record" is unsupported.
Also attach to a task is unsupported as the events are all uncore.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===========================
+Performance monitor support
+===========================
+
+.. toctree::
+ :maxdepth: 1
+
+ hisi-pmu
+ qcom_l2_pmu
+ qcom_l3_pmu
+ arm-ccn
+ xgene-pmu
+ arm_dsu_pmu
+ thunderx2-pmu
+=====================================================================
Qualcomm Technologies Level-2 Cache Performance Monitoring Unit (PMU)
=====================================================================
consisting of one CPU per cluster which will be used to handle all the PMU
events on that cluster.
-Examples for use with perf:
+Examples for use with perf::
perf stat -e l2cache_0/config=0x001/,l2cache_0/config=0x042/ -a sleep 1
+===========================================================================
Qualcomm Datacenter Technologies L3 Cache Performance Monitoring Unit (PMU)
===========================================================================
exposed via the "event" format attribute. In addition to the 32bit physical
counters the driver supports virtual 64bit hardware counters by using hardware
counter chaining. This feature is exposed via the "lc" (long counter) format
-flag. E.g.:
+flag. E.g.::
perf stat -e l3cache_0_0/read-miss,lc/
+=============================================================
Cavium ThunderX2 SoC Performance Monitoring Unit (PMU UNCORE)
=============================================================
The driver does not support sampling, therefore "perf record" will not
work. Per-task perf sessions are also not supported.
-Examples:
+Examples::
-# perf stat -a -e uncore_dmc_0/cnt_cycles/ sleep 1
+ # perf stat -a -e uncore_dmc_0/cnt_cycles/ sleep 1
-# perf stat -a -e \
-uncore_dmc_0/cnt_cycles/,\
-uncore_dmc_0/data_transfers/,\
-uncore_dmc_0/read_txns/,\
-uncore_dmc_0/write_txns/ sleep 1
+ # perf stat -a -e \
+ uncore_dmc_0/cnt_cycles/,\
+ uncore_dmc_0/data_transfers/,\
+ uncore_dmc_0/read_txns/,\
+ uncore_dmc_0/write_txns/ sleep 1
-# perf stat -a -e \
-uncore_l3c_0/read_request/,\
-uncore_l3c_0/read_hit/,\
-uncore_l3c_0/inv_request/,\
-uncore_l3c_0/inv_hit/ sleep 1
+ # perf stat -a -e \
+ uncore_l3c_0/read_request/,\
+ uncore_l3c_0/read_hit/,\
+ uncore_l3c_0/inv_request/,\
+ uncore_l3c_0/inv_hit/ sleep 1
+================================================
APM X-Gene SoC Performance Monitoring Unit (PMU)
================================================
Each perf driver also provides a "cpumask" sysfs attribute, which contains a
single CPU ID of the processor which will be used to handle all the PMU events.
-Example for perf tool use:
+Example for perf tool use::
/ # perf list | grep -e l3c -e iob -e mcb -e mc
l3c0/ackq-full/ [Kernel PMU event]
--- /dev/null
+================================
+Documentation for /proc/sys/abi/
+================================
+
+kernel version 2.6.0.test2
+
+Copyright (c) 2003, Fabian Frederick <ffrederick@users.sourceforge.net>
+
+For general info: index.rst.
+
+------------------------------------------------------------------------------
+
+This path is binary emulation relevant aka personality types aka abi.
+When a process is executed, it's linked to an exec_domain whose
+personality is defined using values available from /proc/sys/abi.
+You can find further details about abi in include/linux/personality.h.
+
+Here are the files featuring in 2.6 kernel:
+
+- defhandler_coff
+- defhandler_elf
+- defhandler_lcall7
+- defhandler_libcso
+- fake_utsname
+- trace
+
+defhandler_coff
+---------------
+
+defined value:
+ PER_SCOSVR3::
+
+ 0x0003 | STICKY_TIMEOUTS | WHOLE_SECONDS | SHORT_INODE
+
+defhandler_elf
+--------------
+
+defined value:
+ PER_LINUX::
+
+ 0
+
+defhandler_lcall7
+-----------------
+
+defined value :
+ PER_SVR4::
+
+ 0x0001 | STICKY_TIMEOUTS | MMAP_PAGE_ZERO,
+
+defhandler_libsco
+-----------------
+
+defined value:
+ PER_SVR4::
+
+ 0x0001 | STICKY_TIMEOUTS | MMAP_PAGE_ZERO,
+
+fake_utsname
+------------
+
+Unused
+
+trace
+-----
+
+Unused
-Documentation for /proc/sys/fs/* kernel version 2.2.10
- (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
- (c) 2009, Shen Feng<shen@cn.fujitsu.com>
+===============================
+Documentation for /proc/sys/fs/
+===============================
-For general info and legal blurb, please look in README.
+kernel version 2.2.10
-==============================================================
+Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+
+Copyright (c) 2009, Shen Feng<shen@cn.fujitsu.com>
+
+For general info and legal blurb, please look in intro.rst.
+
+------------------------------------------------------------------------------
This file contains documentation for the sysctl files in
/proc/sys/fs/ and is valid for Linux kernel version 2.2.
before actually making adjustments.
1. /proc/sys/fs
-----------------------------------------------------------
+===============
Currently, these files are in /proc/sys/fs:
+
- aio-max-nr
- aio-nr
- dentry-state
- super-max
- super-nr
-==============================================================
-aio-nr & aio-max-nr:
+aio-nr & aio-max-nr
+-------------------
aio-nr is the running total of the number of events specified on the
io_setup system call for all currently active aio contexts. If aio-nr
raising aio-max-nr does not result in the pre-allocation or re-sizing
of any kernel data structures.
-==============================================================
-dentry-state:
+dentry-state
+------------
-From linux/include/linux/dcache.h:
---------------------------------------------------------------
-struct dentry_stat_t dentry_stat {
+From linux/include/linux/dcache.h::
+
+ struct dentry_stat_t dentry_stat {
int nr_dentry;
int nr_unused;
int age_limit; /* age in seconds */
int want_pages; /* pages requested by system */
int nr_negative; /* # of unused negative dentries */
int dummy; /* Reserved for future use */
-};
---------------------------------------------------------------
+ };
Dentries are dynamically allocated and deallocated.
they help speeding up rejection of non-existing files provided
by the users.
-==============================================================
-dquot-max & dquot-nr:
+dquot-max & dquot-nr
+--------------------
The file dquot-max shows the maximum number of cached disk
quota entries.
you have some awesome number of simultaneous system users,
you might want to raise the limit.
-==============================================================
-file-max & file-nr:
+file-max & file-nr
+------------------
The value in file-max denotes the maximum number of file-
handles that the Linux kernel will allocate. When you get lots
Attempts to allocate more file descriptors than file-max are
reported with printk, look for "VFS: file-max limit <number>
reached".
-==============================================================
-nr_open:
+
+nr_open
+-------
This denotes the maximum number of file-handles a process can
allocate. Default value is 1024*1024 (1048576) which should be
enough for most machines. Actual limit depends on RLIMIT_NOFILE
resource limit.
-==============================================================
-inode-max, inode-nr & inode-state:
+inode-max, inode-nr & inode-state
+---------------------------------
As with file handles, the kernel allocates the inode structures
dynamically, but can't free them yet.
system needs to prune the inode list instead of allocating
more.
-==============================================================
-overflowgid & overflowuid:
+overflowgid & overflowuid
+-------------------------
Some filesystems only support 16-bit UIDs and GIDs, although in Linux
UIDs and GIDs are 32 bits. When one of these filesystems is mounted
These sysctls allow you to change the value of the fixed UID and GID.
The default is 65534.
-==============================================================
-pipe-user-pages-hard:
+pipe-user-pages-hard
+--------------------
Maximum total number of pages a non-privileged user may allocate for pipes.
Once this limit is reached, no new pipes may be allocated until usage goes
below the limit again. When set to 0, no limit is applied, which is the default
setting.
-==============================================================
-pipe-user-pages-soft:
+pipe-user-pages-soft
+--------------------
Maximum total number of pages a non-privileged user may allocate for pipes
before the pipe size gets limited to a single page. Once this limit is reached,
allocate up to 1024 pipes at their default size. When set to 0, no limit is
applied.
-==============================================================
-protected_fifos:
+protected_fifos
+---------------
The intent of this protection is to avoid unintentional writes to
an attacker-controlled FIFO, where a program expected to create a regular
This protection is based on the restrictions in Openwall.
-==============================================================
-protected_hardlinks:
+protected_hardlinks
+--------------------
A long-standing class of security issues is the hardlink-based
time-of-check-time-of-use race, most commonly seen in world-writable
This protection is based on the restrictions in Openwall and grsecurity.
-==============================================================
-protected_regular:
+protected_regular
+-----------------
This protection is similar to protected_fifos, but it
avoids writes to an attacker-controlled regular file, where a program
When set to "2" it also applies to group writable sticky directories.
-==============================================================
-protected_symlinks:
+protected_symlinks
+------------------
A long-standing class of security issues is the symlink-based
time-of-check-time-of-use race, most commonly seen in world-writable
This protection is based on the restrictions in Openwall and grsecurity.
-==============================================================
suid_dumpable:
+--------------
This value can be used to query and set the core dump mode for setuid
or otherwise protected/tainted binaries. The modes are
-0 - (default) - traditional behaviour. Any process which has changed
- privilege levels or is execute only will not be dumped.
-1 - (debug) - all processes dump core when possible. The core dump is
- owned by the current user and no security is applied. This is
- intended for system debugging situations only. Ptrace is unchecked.
- This is insecure as it allows regular users to examine the memory
- contents of privileged processes.
-2 - (suidsafe) - any binary which normally would not be dumped is dumped
- anyway, but only if the "core_pattern" kernel sysctl is set to
- either a pipe handler or a fully qualified path. (For more details
- on this limitation, see CVE-2006-2451.) This mode is appropriate
- when administrators are attempting to debug problems in a normal
- environment, and either have a core dump pipe handler that knows
- to treat privileged core dumps with care, or specific directory
- defined for catching core dumps. If a core dump happens without
- a pipe handler or fully qualifid path, a message will be emitted
- to syslog warning about the lack of a correct setting.
-
-==============================================================
-
-super-max & super-nr:
+= ========== ===============================================================
+0 (default) traditional behaviour. Any process which has changed
+ privilege levels or is execute only will not be dumped.
+1 (debug) all processes dump core when possible. The core dump is
+ owned by the current user and no security is applied. This is
+ intended for system debugging situations only.
+ Ptrace is unchecked.
+ This is insecure as it allows regular users to examine the
+ memory contents of privileged processes.
+2 (suidsafe) any binary which normally would not be dumped is dumped
+ anyway, but only if the "core_pattern" kernel sysctl is set to
+ either a pipe handler or a fully qualified path. (For more
+ details on this limitation, see CVE-2006-2451.) This mode is
+ appropriate when administrators are attempting to debug
+ problems in a normal environment, and either have a core dump
+ pipe handler that knows to treat privileged core dumps with
+ care, or specific directory defined for catching core dumps.
+ If a core dump happens without a pipe handler or fully
+ qualified path, a message will be emitted to syslog warning
+ about the lack of a correct setting.
+= ========== ===============================================================
+
+
+super-max & super-nr
+--------------------
These numbers control the maximum number of superblocks, and
thus the maximum number of mounted filesystems the kernel
mount more filesystems than the current value in super-max
allows you to.
-==============================================================
-aio-nr & aio-max-nr:
+aio-nr & aio-max-nr
+-------------------
aio-nr shows the current system-wide number of asynchronous io
requests. aio-max-nr allows you to change the maximum value
aio-nr can grow to.
-==============================================================
-mount-max:
+mount-max
+---------
This denotes the maximum number of mounts that may exist
in a mount namespace.
-==============================================================
2. /proc/sys/fs/binfmt_misc
-----------------------------------------------------------
+===========================
Documentation for the files in /proc/sys/fs/binfmt_misc is
in Documentation/admin-guide/binfmt-misc.rst.
3. /proc/sys/fs/mqueue - POSIX message queues filesystem
-----------------------------------------------------------
+========================================================
+
The "mqueue" filesystem provides the necessary kernel features to enable the
creation of a user space library that implements the POSIX message queues
exceed msgsize_max, the default value is initialized msgsize_max.
4. /proc/sys/fs/epoll - Configuration options for the epoll interface
---------------------------------------------------------
+=====================================================================
This directory contains configuration options for the epoll(7) interface.
on a 64bit one.
The current default value for max_user_watches is the 1/32 of the available
low memory, divided for the "watch" cost in bytes.
-
-Documentation for /proc/sys/ kernel version 2.2.10
- (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+===========================
+Documentation for /proc/sys
+===========================
+
+Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+
+------------------------------------------------------------------------------
'Why', I hear you ask, 'would anyone even _want_ documentation
for them sysctl files? If anybody really needs it, it's all in
Furthermore, the programmers who built sysctl have built it to
be actually used, not just for the fun of programming it :-)
-==============================================================
+------------------------------------------------------------------------------
Legal blurb:
As usual, there are two main things to consider:
+
1. you get what you pay for
2. it's free
Rik van Riel.
-==============================================================
+--------------------------------------------------------------
-Introduction:
+Introduction
+============
Sysctl is a means of configuring certain aspects of the kernel
at run-time, and the /proc/sys/ directory is there so that you
don't even need special tools to do it!
In fact, there are only four things needed to use these config
facilities:
+
- a running Linux system
- root access
- common sense (this is especially hard to come by these days)
one part of the kernel, so you can do configuration on a piece
by piece basis, or just some 'thematic frobbing'.
-The subdirs are about:
+This documentation is about:
+
+=============== ===============================================================
abi/ execution domains & personalities
debug/ <empty>
dev/ device specific information (eg dev/cdrom/info)
vm/ memory management tuning
buffer and cache management
user/ Per user per user namespace limits
+=============== ===============================================================
These are the subdirs I have on my system. There might be more
or other subdirs in another setup. If you see another dir, I'd
really like to hear about it :-)
+
+.. toctree::
+ :maxdepth: 1
+
+ abi
+ fs
+ kernel
+ net
+ sunrpc
+ user
+ vm
-Documentation for /proc/sys/kernel/* kernel version 2.2.10
- (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
- (c) 2009, Shen Feng<shen@cn.fujitsu.com>
+===================================
+Documentation for /proc/sys/kernel/
+===================================
-For general info and legal blurb, please look in README.
+kernel version 2.2.10
-==============================================================
+Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+
+Copyright (c) 2009, Shen Feng<shen@cn.fujitsu.com>
+
+For general info and legal blurb, please look in index.rst.
+
+------------------------------------------------------------------------------
This file contains documentation for the sysctl files in
/proc/sys/kernel/ and is valid for Linux kernel version 2.2.
- watchdog_thresh
- version
-==============================================================
acct:
+=====
highwater lowwater frequency
if we got >=4%; consider information about amount of free space
valid for 30 seconds.
-==============================================================
acpi_video_flags:
+=================
flags
See Doc*/kernel/power/video.txt, it allows mode of video boot to be
set during run time.
-==============================================================
auto_msgmni:
+============
This variable has no effect and may be removed in future kernel
releases. Reading it always returns 0.
Echoing "0" turned it off. auto_msgmni default value was 1.
-==============================================================
-
bootloader_type:
+================
x86 bootloader identification
See the type_of_loader and ext_loader_type fields in
Documentation/x86/boot.rst for additional information.
-==============================================================
bootloader_version:
+===================
x86 bootloader version
See the type_of_loader and ext_loader_ver fields in
Documentation/x86/boot.rst for additional information.
-==============================================================
-cap_last_cap
+cap_last_cap:
+=============
Highest valid capability of the running kernel. Exports
CAP_LAST_CAP from the kernel.
-==============================================================
core_pattern:
+=============
core_pattern is used to specify a core dumpfile pattern name.
-. max length 127 characters; default value is "core"
-. core_pattern is used as a pattern template for the output filename;
+
+* max length 127 characters; default value is "core"
+* core_pattern is used as a pattern template for the output filename;
certain string patterns (beginning with '%') are substituted with
their actual values.
-. backward compatibility with core_uses_pid:
+* backward compatibility with core_uses_pid:
+
If core_pattern does not include "%p" (default does not)
and core_uses_pid is set, then .PID will be appended to
the filename.
-. corename format specifiers:
+
+* corename format specifiers::
+
%<NUL> '%' is dropped
%% output one '%'
%p pid
%e executable filename (may be shortened)
%E executable path
%<OTHER> both are dropped
-. If the first character of the pattern is a '|', the kernel will treat
+
+* If the first character of the pattern is a '|', the kernel will treat
the rest of the pattern as a command to run. The core dump will be
written to the standard input of that program instead of to a file.
-==============================================================
core_pipe_limit:
+================
This sysctl is only applicable when core_pattern is configured to pipe
core files to a user space helper (when the first character of
process is not guaranteed access to /proc/<crashing pid>/). This
value defaults to 0.
-==============================================================
core_uses_pid:
+==============
The default coredump filename is "core". By setting
core_uses_pid to 1, the coredump filename becomes core.PID.
and core_uses_pid is set, then .PID will be appended to
the filename.
-==============================================================
ctrl-alt-del:
+=============
When the value in this file is 0, ctrl-alt-del is trapped and
sent to the init(1) program to handle a graceful restart.
Nerve Pinch (tm) will be an immediate reboot, without even
syncing its dirty buffers.
-Note: when a program (like dosemu) has the keyboard in 'raw'
-mode, the ctrl-alt-del is intercepted by the program before it
-ever reaches the kernel tty layer, and it's up to the program
-to decide what to do with it.
+Note:
+ when a program (like dosemu) has the keyboard in 'raw'
+ mode, the ctrl-alt-del is intercepted by the program before it
+ ever reaches the kernel tty layer, and it's up to the program
+ to decide what to do with it.
-==============================================================
dmesg_restrict:
+===============
This toggle indicates whether unprivileged users are prevented
from using dmesg(8) to view messages from the kernel's log buffer.
The kernel config option CONFIG_SECURITY_DMESG_RESTRICT sets the
default value of dmesg_restrict.
-==============================================================
domainname & hostname:
+======================
These files can be used to set the NIS/YP domainname and the
hostname of your box in exactly the same way as the commands
-domainname and hostname, i.e.:
-# echo "darkstar" > /proc/sys/kernel/hostname
-# echo "mydomain" > /proc/sys/kernel/domainname
-has the same effect as
-# hostname "darkstar"
-# domainname "mydomain"
+domainname and hostname, i.e.::
+
+ # echo "darkstar" > /proc/sys/kernel/hostname
+ # echo "mydomain" > /proc/sys/kernel/domainname
+
+has the same effect as::
+
+ # hostname "darkstar"
+ # domainname "mydomain"
Note, however, that the classic darkstar.frop.org has the
hostname "darkstar" and DNS (Internet Domain Name Server)
domain names are in general different. For a detailed discussion
see the hostname(1) man page.
-==============================================================
+
hardlockup_all_cpu_backtrace:
+=============================
This value controls the hard lockup detector behavior when a hard
lockup condition is detected as to whether or not to gather further
0: do nothing. This is the default behavior.
1: on detection capture more debug information.
-==============================================================
+
hardlockup_panic:
+=================
This parameter can be used to control whether the kernel panics
when a hard lockup is detected.
0 - don't panic on hard lockup
1 - panic on hard lockup
-See Documentation/lockup-watchdogs.txt for more information. This can
+See Documentation/admin-guide/lockup-watchdogs.rst for more information. This can
also be set using the nmi_watchdog kernel parameter.
-==============================================================
hotplug:
+========
Path for the hotplug policy agent.
Default value is "/sbin/hotplug".
-==============================================================
hung_task_panic:
+================
Controls the kernel's behavior when a hung task is detected.
This file shows up if CONFIG_DETECT_HUNG_TASK is enabled.
1: panic immediately.
-==============================================================
hung_task_check_count:
+======================
The upper bound on the number of tasks that are checked.
This file shows up if CONFIG_DETECT_HUNG_TASK is enabled.
-==============================================================
hung_task_timeout_secs:
+=======================
When a task in D state did not get scheduled
for more than this value report a warning.
This file shows up if CONFIG_DETECT_HUNG_TASK is enabled.
0: means infinite timeout - no checking done.
+
Possible values to set are in range {0..LONG_MAX/HZ}.
-==============================================================
hung_task_check_interval_secs:
+==============================
Hung task check interval. If hung task checking is enabled
(see hung_task_timeout_secs), the check is done every
0 (default): means use hung_task_timeout_secs as checking interval.
Possible values to set are in range {0..LONG_MAX/HZ}.
-==============================================================
hung_task_warnings:
+===================
The maximum number of warnings to report. During a check interval
if a hung task is detected, this value is decreased by 1.
-1: report an infinite number of warnings.
-==============================================================
hyperv_record_panic_msg:
+========================
Controls whether the panic kmsg data should be reported to Hyper-V.
1: report the panic kmsg data. This is the default behavior.
-==============================================================
kexec_load_disabled:
+====================
A toggle indicating if the kexec_load syscall has been disabled. This
value defaults to 0 (false: kexec_load enabled), but can be set to 1
later use) an image without it being altered. Generally used together
with the "modules_disabled" sysctl.
-==============================================================
kptr_restrict:
+==============
This toggle indicates whether restrictions are placed on
exposing kernel addresses via /proc and other interfaces.
When kptr_restrict is set to (2), kernel pointers printed using
%pK will be replaced with 0's regardless of privileges.
-==============================================================
l2cr: (PPC only)
+================
This flag controls the L2 cache of G3 processor boards. If
0, the cache is disabled. Enabled if nonzero.
-==============================================================
modules_disabled:
+=================
A toggle value indicating if modules are allowed to be loaded
in an otherwise modular kernel. This toggle defaults to off
neither loaded nor unloaded, and the toggle cannot be set back
to false. Generally used with the "kexec_load_disabled" toggle.
-==============================================================
msg_next_id, sem_next_id, and shm_next_id:
+==========================================
These three toggles allows to specify desired id for next allocated IPC
object: message, semaphore or shared memory respectively.
Possible values to set are in range {0..INT_MAX}.
Notes:
-1) kernel doesn't guarantee, that new object will have desired id. So,
-it's up to userspace, how to handle an object with "wrong" id.
-2) Toggle with non-default value will be set back to -1 by kernel after
-successful IPC object allocation. If an IPC object allocation syscall
-fails, it is undefined if the value remains unmodified or is reset to -1.
+ 1) kernel doesn't guarantee, that new object will have desired id. So,
+ it's up to userspace, how to handle an object with "wrong" id.
+ 2) Toggle with non-default value will be set back to -1 by kernel after
+ successful IPC object allocation. If an IPC object allocation syscall
+ fails, it is undefined if the value remains unmodified or is reset to -1.
-==============================================================
nmi_watchdog:
+=============
This parameter can be used to control the NMI watchdog
(i.e. the hard lockup detector) on x86 systems.
- 0 - disable the hard lockup detector
- 1 - enable the hard lockup detector
+0 - disable the hard lockup detector
+
+1 - enable the hard lockup detector
The hard lockup detector monitors each CPU for its ability to respond to
timer interrupts. The mechanism utilizes CPU performance counter registers
while a CPU is busy. Hence, the alternative name 'NMI watchdog'.
The NMI watchdog is disabled by default if the kernel is running as a guest
-in a KVM virtual machine. This default can be overridden by adding
+in a KVM virtual machine. This default can be overridden by adding::
nmi_watchdog=1
to the guest kernel command line (see Documentation/admin-guide/kernel-parameters.rst).
-==============================================================
-numa_balancing
+numa_balancing:
+===============
Enables/disables automatic page fault based NUMA memory
balancing. Memory is moved automatically to nodes
numa_balancing_scan_delay_ms, numa_balancing_scan_period_max_ms,
numa_balancing_scan_size_mb, and numa_balancing_settle_count sysctls.
-==============================================================
+numa_balancing_scan_period_min_ms, numa_balancing_scan_delay_ms, numa_balancing_scan_period_max_ms, numa_balancing_scan_size_mb
+===============================================================================================================================
-numa_balancing_scan_period_min_ms, numa_balancing_scan_delay_ms,
-numa_balancing_scan_period_max_ms, numa_balancing_scan_size_mb
Automatic NUMA balancing scans tasks address space and unmaps pages to
detect if pages are properly placed or if the data should be migrated to a
numa_balancing_scan_size_mb is how many megabytes worth of pages are
scanned for a given scan.
-==============================================================
osrelease, ostype & version:
+============================
+
+::
-# cat osrelease
-2.1.88
-# cat ostype
-Linux
-# cat version
-#5 Wed Feb 25 21:49:24 MET 1998
+ # cat osrelease
+ 2.1.88
+ # cat ostype
+ Linux
+ # cat version
+ #5 Wed Feb 25 21:49:24 MET 1998
The files osrelease and ostype should be clear enough. Version
needs a little more clarification however. The '#5' means that
date behind it indicates the time the kernel was built.
The only way to tune these values is to rebuild the kernel :-)
-==============================================================
overflowgid & overflowuid:
+==========================
if your architecture did not always support 32-bit UIDs (i.e. arm,
i386, m68k, sh, and sparc32), a fixed UID and GID will be returned to
These sysctls allow you to change the value of the fixed UID and GID.
The default is 65534.
-==============================================================
panic:
+======
The value in this file represents the number of seconds the kernel
waits before rebooting on a panic. When you use the software watchdog,
the recommended setting is 60.
-==============================================================
panic_on_io_nmi:
+================
Controls the kernel's behavior when a CPU receives an NMI caused by
an IO error.
servers issue this sort of NMI when the dump button is pushed,
and you can use this option to take a crash dump.
-==============================================================
panic_on_oops:
+==============
Controls the kernel's behaviour when an oops or BUG is encountered.
0: try to continue operation
-1: panic immediately. If the `panic' sysctl is also non-zero then the
+1: panic immediately. If the `panic` sysctl is also non-zero then the
machine will be rebooted.
-==============================================================
panic_on_stackoverflow:
+=======================
Controls the kernel's behavior when detecting the overflows of
kernel, IRQ and exception stacks except a user stack.
1: panic immediately.
-==============================================================
panic_on_unrecovered_nmi:
+=========================
The default Linux behaviour on an NMI of either memory or unknown is
to continue operation. For many environments such as scientific
such as power management so the default is off. That sysctl works like
the existing panic controls already in that directory.
-==============================================================
panic_on_warn:
+==============
Calls panic() in the WARN() path when set to 1. This is useful to avoid
a kernel rebuild when attempting to kdump at the location of a WARN().
1: call panic() after printing out WARN() location.
-==============================================================
panic_print:
+============
Bitmask for printing system info when panic happens. User can chose
combination of the following bits:
-bit 0: print all tasks info
-bit 1: print system memory info
-bit 2: print timer info
-bit 3: print locks info if CONFIG_LOCKDEP is on
-bit 4: print ftrace buffer
+===== ========================================
+bit 0 print all tasks info
+bit 1 print system memory info
+bit 2 print timer info
+bit 3 print locks info if CONFIG_LOCKDEP is on
+bit 4 print ftrace buffer
+===== ========================================
+
+So for example to print tasks and memory info on panic, user can::
-So for example to print tasks and memory info on panic, user can:
echo 3 > /proc/sys/kernel/panic_print
-==============================================================
panic_on_rcu_stall:
+===================
When set to 1, calls panic() after RCU stall detection messages. This
is useful to define the root cause of RCU stalls using a vmcore.
1: panic() after printing RCU stall messages.
-==============================================================
perf_cpu_time_max_percent:
+==========================
Hints to the kernel how much CPU time it should be allowed to
use to handle perf sampling events. If the perf subsystem
stacked up next to each other so much that nothing else is
allowed to execute.
-0: disable the mechanism. Do not monitor or correct perf's
+0:
+ disable the mechanism. Do not monitor or correct perf's
sampling rate no matter how CPU time it takes.
-1-100: attempt to throttle perf's sample rate to this
+1-100:
+ attempt to throttle perf's sample rate to this
percentage of CPU. Note: the kernel calculates an
"expected" length of each sample event. 100 here means
100% of that expected length. Even if this is set to
length is exceeded. Set to 0 if you truly do not care
how much CPU is consumed.
-==============================================================
perf_event_paranoid:
+====================
Controls use of the performance events system by unprivileged
users (without CAP_SYS_ADMIN). The default value is 2.
- -1: Allow use of (almost) all events by all users
+=== ==================================================================
+ -1 Allow use of (almost) all events by all users
+
Ignore mlock limit after perf_event_mlock_kb without CAP_IPC_LOCK
->=0: Disallow ftrace function tracepoint by users without CAP_SYS_ADMIN
+
+>=0 Disallow ftrace function tracepoint by users without CAP_SYS_ADMIN
+
Disallow raw tracepoint access by users without CAP_SYS_ADMIN
->=1: Disallow CPU event access by users without CAP_SYS_ADMIN
->=2: Disallow kernel profiling by users without CAP_SYS_ADMIN
-==============================================================
+>=1 Disallow CPU event access by users without CAP_SYS_ADMIN
+
+>=2 Disallow kernel profiling by users without CAP_SYS_ADMIN
+=== ==================================================================
+
perf_event_max_stack:
+=====================
Controls maximum number of stack frames to copy for (attr.sample_type &
PERF_SAMPLE_CALLCHAIN) configured events, for instance, when using
The default value is 127.
-==============================================================
perf_event_mlock_kb:
+====================
Control size of per-cpu ring buffer not counted agains mlock limit.
The default value is 512 + 1 page
-==============================================================
perf_event_max_contexts_per_stack:
+==================================
Controls maximum number of stack frame context entries for
(attr.sample_type & PERF_SAMPLE_CALLCHAIN) configured events, for
The default value is 8.
-==============================================================
pid_max:
+========
PID allocation wrap value. When the kernel's next PID value
reaches this value, it wraps back to a minimum PID value.
PIDs of value pid_max or larger are not allocated.
-==============================================================
ns_last_pid:
+============
The last pid allocated in the current (the one task using this sysctl
lives in) pid namespace. When selecting a pid for a next task on fork
kernel tries to allocate a number starting from this one.
-==============================================================
powersave-nap: (PPC only)
+=========================
If set, Linux-PPC will use the 'nap' mode of powersaving,
otherwise the 'doze' mode will be used.
==============================================================
printk:
+=======
The four values in printk denote: console_loglevel,
default_message_loglevel, minimum_console_loglevel and
logging error messages. See 'man 2 syslog' for more info on
the different loglevels.
-- console_loglevel: messages with a higher priority than
- this will be printed to the console
-- default_message_loglevel: messages without an explicit priority
- will be printed with this priority
-- minimum_console_loglevel: minimum (highest) value to which
- console_loglevel can be set
-- default_console_loglevel: default value for console_loglevel
+- console_loglevel:
+ messages with a higher priority than
+ this will be printed to the console
+- default_message_loglevel:
+ messages without an explicit priority
+ will be printed with this priority
+- minimum_console_loglevel:
+ minimum (highest) value to which
+ console_loglevel can be set
+- default_console_loglevel:
+ default value for console_loglevel
-==============================================================
printk_delay:
+=============
Delay each printk message in printk_delay milliseconds
Value from 0 - 10000 is allowed.
-==============================================================
printk_ratelimit:
+=================
Some warning messages are rate limited. printk_ratelimit specifies
the minimum length of time between these messages (in jiffies), by
A value of 0 will disable rate limiting.
-==============================================================
printk_ratelimit_burst:
+=======================
While long term we enforce one message per printk_ratelimit
seconds, we do allow a burst of messages to pass through.
printk_ratelimit_burst specifies the number of messages we can
send before ratelimiting kicks in.
-==============================================================
printk_devkmsg:
+===============
Control the logging to /dev/kmsg from userspace:
-ratelimit: default, ratelimited
+ratelimit:
+ default, ratelimited
+
on: unlimited logging to /dev/kmsg from userspace
+
off: logging to /dev/kmsg disabled
The kernel command line parameter printk.devkmsg= overrides this and is
a one-time setting until next reboot: once set, it cannot be changed by
this sysctl interface anymore.
-==============================================================
randomize_va_space:
+===================
This option can be used to select the type of process address
space randomization that is used in the system, for architectures
that support this feature.
-0 - Turn the process address space randomization off. This is the
+== ===========================================================================
+0 Turn the process address space randomization off. This is the
default for architectures that do not support this feature anyways,
and kernels that are booted with the "norandmaps" parameter.
-1 - Make the addresses of mmap base, stack and VDSO page randomized.
+1 Make the addresses of mmap base, stack and VDSO page randomized.
This, among other things, implies that shared libraries will be
loaded to random addresses. Also for PIE-linked binaries, the
location of code start is randomized. This is the default if the
CONFIG_COMPAT_BRK option is enabled.
-2 - Additionally enable heap randomization. This is the default if
+2 Additionally enable heap randomization. This is the default if
CONFIG_COMPAT_BRK is disabled.
There are a few legacy applications out there (such as some ancient
Systems with ancient and/or broken binaries should be configured
with CONFIG_COMPAT_BRK enabled, which excludes the heap from process
address space randomization.
+== ===========================================================================
-==============================================================
reboot-cmd: (Sparc only)
+========================
??? This seems to be a way to give an argument to the Sparc
ROM/Flash boot loader. Maybe to tell it what to do after
rebooting. ???
-==============================================================
rtsig-max & rtsig-nr:
+=====================
The file rtsig-max can be used to tune the maximum number
of POSIX realtime (queued) signals that can be outstanding
rtsig-nr shows the number of RT signals currently queued.
-==============================================================
sched_energy_aware:
+===================
Enables/disables Energy Aware Scheduling (EAS). EAS starts
automatically on platforms where it can run (that is,
requirements for EAS but you do not want to use it, change
this value to 0.
-==============================================================
sched_schedstats:
+=================
Enables/disables scheduler statistics. Enabling this feature
incurs a small amount of overhead in the scheduler but is
useful for debugging and performance tuning.
-==============================================================
sg-big-buff:
+============
This file shows the size of the generic SCSI (sg) buffer.
You can't tune it just yet, but you could change it on
you can come up with one, you probably know what you
are doing anyway :)
-==============================================================
shmall:
+=======
This parameter sets the total amount of shared memory pages that
can be used system wide. Hence, SHMALL should always be at least
If you are not sure what the default PAGE_SIZE is on your Linux
system, you can run the following command:
-# getconf PAGE_SIZE
+ # getconf PAGE_SIZE
-==============================================================
shmmax:
+=======
This value can be used to query and set the run time limit
on the maximum shared memory segment size that can be created.
Shared memory segments up to 1Gb are now supported in the
kernel. This value defaults to SHMMAX.
-==============================================================
shm_rmid_forced:
+================
Linux lets you set resource limits, including how much memory one
process can consume, via setrlimit(2). Unfortunately, shared memory
Note that if you change this from 0 to 1, already created segments
without users and with a dead originative process will be destroyed.
-==============================================================
sysctl_writes_strict:
+=====================
Control how file position affects the behavior of updating sysctl values
via the /proc/sys interface:
- -1 - Legacy per-write sysctl value handling, with no printk warnings.
+ == ======================================================================
+ -1 Legacy per-write sysctl value handling, with no printk warnings.
Each write syscall must fully contain the sysctl value to be
written, and multiple writes on the same sysctl file descriptor
will rewrite the sysctl value, regardless of file position.
- 0 - Same behavior as above, but warn about processes that perform writes
+ 0 Same behavior as above, but warn about processes that perform writes
to a sysctl file descriptor when the file position is not 0.
- 1 - (default) Respect file position when writing sysctl strings. Multiple
+ 1 (default) Respect file position when writing sysctl strings. Multiple
writes will append to the sysctl value buffer. Anything past the max
length of the sysctl value buffer will be ignored. Writes to numeric
sysctl entries must always be at file position 0 and the value must
be fully contained in the buffer sent in the write syscall.
+ == ======================================================================
-==============================================================
softlockup_all_cpu_backtrace:
+=============================
This value controls the soft lockup detector thread's behavior
when a soft lockup condition is detected as to whether or not
1: on detection capture more debug information.
-==============================================================
-soft_watchdog
+soft_watchdog:
+==============
This parameter can be used to control the soft lockup detector.
0 - disable the soft lockup detector
+
1 - enable the soft lockup detector
The soft lockup detector monitors CPUs for threads that are hogging the CPUs
the watchdog timer function, otherwise the NMI watchdog - if enabled - can
detect a hard lockup condition.
-==============================================================
-stack_erasing
+stack_erasing:
+==============
This parameter can be used to control kernel stack erasing at the end
of syscalls for kernels built with CONFIG_GCC_PLUGIN_STACKLEAK.
1: kernel stack erasing is enabled (default), it is performed before
returning to the userspace at the end of syscalls.
-==============================================================
+
tainted
+=======
Non-zero if the kernel has been tainted. Numeric values, which can be
ORed together. The letters are seen in "Tainted" line of Oops reports.
- 1 (P): proprietary module was loaded
- 2 (F): module was force loaded
- 4 (S): SMP kernel oops on an officially SMP incapable processor
- 8 (R): module was force unloaded
- 16 (M): processor reported a Machine Check Exception (MCE)
- 32 (B): bad page referenced or some unexpected page flags
- 64 (U): taint requested by userspace application
- 128 (D): kernel died recently, i.e. there was an OOPS or BUG
- 256 (A): an ACPI table was overridden by user
- 512 (W): kernel issued warning
- 1024 (C): staging driver was loaded
- 2048 (I): workaround for bug in platform firmware applied
- 4096 (O): externally-built ("out-of-tree") module was loaded
- 8192 (E): unsigned module was loaded
- 16384 (L): soft lockup occurred
- 32768 (K): kernel has been live patched
- 65536 (X): Auxiliary taint, defined and used by for distros
-131072 (T): The kernel was built with the struct randomization plugin
+====== ===== ==============================================================
+ 1 `(P)` proprietary module was loaded
+ 2 `(F)` module was force loaded
+ 4 `(S)` SMP kernel oops on an officially SMP incapable processor
+ 8 `(R)` module was force unloaded
+ 16 `(M)` processor reported a Machine Check Exception (MCE)
+ 32 `(B)` bad page referenced or some unexpected page flags
+ 64 `(U)` taint requested by userspace application
+ 128 `(D)` kernel died recently, i.e. there was an OOPS or BUG
+ 256 `(A)` an ACPI table was overridden by user
+ 512 `(W)` kernel issued warning
+ 1024 `(C)` staging driver was loaded
+ 2048 `(I)` workaround for bug in platform firmware applied
+ 4096 `(O)` externally-built ("out-of-tree") module was loaded
+ 8192 `(E)` unsigned module was loaded
+ 16384 `(L)` soft lockup occurred
+ 32768 `(K)` kernel has been live patched
+ 65536 `(X)` Auxiliary taint, defined and used by for distros
+131072 `(T)` The kernel was built with the struct randomization plugin
+====== ===== ==============================================================
See Documentation/admin-guide/tainted-kernels.rst for more information.
-==============================================================
-threads-max
+threads-max:
+============
This value controls the maximum number of threads that can be created
using fork().
a part (1/8th) of the available RAM pages.
The minimum value that can be written to threads-max is 20.
+
The maximum value that can be written to threads-max is given by the
constant FUTEX_TID_MASK (0x3fffffff).
+
If a value outside of this range is written to threads-max an error
EINVAL occurs.
thread structures would occupy too much (more than 1/8th) of the
available RAM pages threads-max is reduced accordingly.
-==============================================================
unknown_nmi_panic:
+==================
The value in this file affects behavior of handling NMI. When the
value is non-zero, unknown NMI is trapped and then panic occurs. At
NMI switch that most IA32 servers have fires unknown NMI up, for
example. If a system hangs up, try pressing the NMI switch.
-==============================================================
watchdog:
+=========
This parameter can be used to disable or enable the soft lockup detector
_and_ the NMI watchdog (i.e. the hard lockup detector) at the same time.
0 - disable both lockup detectors
+
1 - enable both lockup detectors
The soft lockup detector and the NMI watchdog can also be disabled or
enabled individually, using the soft_watchdog and nmi_watchdog parameters.
-If the watchdog parameter is read, for example by executing
+If the watchdog parameter is read, for example by executing::
cat /proc/sys/kernel/watchdog
the output of this command (0 or 1) shows the logical OR of soft_watchdog
and nmi_watchdog.
-==============================================================
watchdog_cpumask:
+=================
This value can be used to control on which cpus the watchdog may run.
The default cpumask is all possible cores, but if NO_HZ_FULL is
The argument value is the standard cpulist format for cpumasks,
so for example to enable the watchdog on cores 0, 2, 3, and 4 you
-might say:
+might say::
echo 0,2-4 > /proc/sys/kernel/watchdog_cpumask
-==============================================================
watchdog_thresh:
+================
This value can be used to control the frequency of hrtimer and NMI
events and the soft and hard lockup thresholds. The default threshold
The softlockup threshold is (2 * watchdog_thresh). Setting this
tunable to zero will disable lockup detection altogether.
-
-==============================================================
-Documentation for /proc/sys/net/*
- (c) 1999 Terrehon Bowden <terrehon@pacbell.net>
- Bodo Bauer <bb@ricochet.net>
- (c) 2000 Jorge Nerin <comandante@zaralinux.com>
- (c) 2009 Shen Feng <shen@cn.fujitsu.com>
+================================
+Documentation for /proc/sys/net/
+================================
-For general info and legal blurb, please look in README.
+Copyright
-==============================================================
+Copyright (c) 1999
+
+ - Terrehon Bowden <terrehon@pacbell.net>
+ - Bodo Bauer <bb@ricochet.net>
+
+Copyright (c) 2000
+
+ - Jorge Nerin <comandante@zaralinux.com>
+
+Copyright (c) 2009
+
+ - Shen Feng <shen@cn.fujitsu.com>
+
+For general info and legal blurb, please look in index.rst.
+
+------------------------------------------------------------------------------
This file contains the documentation for the sysctl files in
/proc/sys/net
Table : Subdirectories in /proc/sys/net
-..............................................................................
- Directory Content Directory Content
- core General parameter appletalk Appletalk protocol
- unix Unix domain sockets netrom NET/ROM
- 802 E802 protocol ax25 AX25
- ethernet Ethernet protocol rose X.25 PLP layer
- ipv4 IP version 4 x25 X.25 protocol
- ipx IPX token-ring IBM token ring
- bridge Bridging decnet DEC net
- ipv6 IP version 6 tipc TIPC
-..............................................................................
+
+ ========= =================== = ========== ==================
+ Directory Content Directory Content
+ ========= =================== = ========== ==================
+ core General parameter appletalk Appletalk protocol
+ unix Unix domain sockets netrom NET/ROM
+ 802 E802 protocol ax25 AX25
+ ethernet Ethernet protocol rose X.25 PLP layer
+ ipv4 IP version 4 x25 X.25 protocol
+ ipx IPX token-ring IBM token ring
+ bridge Bridging decnet DEC net
+ ipv6 IP version 6 tipc TIPC
+ ========= =================== = ========== ==================
1. /proc/sys/net/core - Network core options
--------------------------------------------------------
+============================================
bpf_jit_enable
--------------
through bpf(2) and passing a verifier in the kernel, a JIT will then
translate these BPF proglets into native CPU instructions. There are
two flavors of JITs, the newer eBPF JIT currently supported on:
+
- x86_64
- x86_32
- arm64
- riscv
And the older cBPF JIT supported on the following archs:
+
- mips
- ppc
- sparc
tcpdump filters, seccomp rules, etc, but not mentioned eBPF
programs loaded through bpf(2).
-Values :
- 0 - disable the JIT (default value)
- 1 - enable the JIT
- 2 - enable the JIT and ask the compiler to emit traces on kernel log.
+Values:
+
+ - 0 - disable the JIT (default value)
+ - 1 - enable the JIT
+ - 2 - enable the JIT and ask the compiler to emit traces on kernel log.
bpf_jit_harden
--------------
This enables hardening for the BPF JIT compiler. Supported are eBPF
JIT backends. Enabling hardening trades off performance, but can
mitigate JIT spraying.
-Values :
- 0 - disable JIT hardening (default value)
- 1 - enable JIT hardening for unprivileged users only
- 2 - enable JIT hardening for all users
+
+Values:
+
+ - 0 - disable JIT hardening (default value)
+ - 1 - enable JIT hardening for unprivileged users only
+ - 2 - enable JIT hardening for all users
bpf_jit_kallsyms
----------------
in /proc/kallsyms. This enables export of these addresses, which can
be used for debugging/tracing. If bpf_jit_harden is enabled, this
feature is disabled.
+
Values :
- 0 - disable JIT kallsyms export (default value)
- 1 - enable JIT kallsyms export for privileged users only
+
+ - 0 - disable JIT kallsyms export (default value)
+ - 1 - enable JIT kallsyms export for privileged users only
bpf_jit_limit
-------------
in bytes.
dev_weight
---------------
+----------
The maximum number of packets that kernel can handle on a NAPI interrupt,
it's a Per-CPU variable. For drivers that support LRO or GRO_HW, a hardware
Default: 64
dev_weight_rx_bias
---------------
+------------------
RPS (e.g. RFS, aRFS) processing is competing with the registered NAPI poll function
of the driver for the per softirq cycle netdev_budget. This parameter influences
dev_weight adaptable for asymmetric CPU needs on RX/TX side of the network stack.
(see dev_weight_tx_bias) It is effective on a per CPU basis. Determination is based
on dev_weight and is calculated multiplicative (dev_weight * dev_weight_rx_bias).
+
Default: 1
dev_weight_tx_bias
---------------
+------------------
Scales the maximum number of packets that can be processed during a TX softirq cycle.
Effective on a per CPU basis. Allows scaling of current dev_weight for asymmetric
net stack processing needs. Be careful to avoid making TX softirq processing a CPU hog.
+
Calculation is based on dev_weight (dev_weight * dev_weight_tx_bias).
+
Default: 1
default_qdisc
---------------
+-------------
The default queuing discipline to use for network devices. This allows
overriding the default of pfifo_fast with an alternative. Since the default
interfaces still use mq as root qdisc, which in turn uses this default for its
leaves. Virtual devices (like e.g. lo or veth) ignore this setting and instead
default to noqueue.
+
Default: pfifo_fast
busy_read
-----------------
+---------
+
Low latency busy poll timeout for socket reads. (needs CONFIG_NET_RX_BUSY_POLL)
Approximate time in us to busy loop waiting for packets on the device queue.
This sets the default value of the SO_BUSY_POLL socket option.
Can be set or overridden per socket by setting socket option SO_BUSY_POLL,
which is the preferred method of enabling. If you need to enable the feature
globally via sysctl, a value of 50 is recommended.
+
Will increase power usage.
+
Default: 0 (off)
busy_poll
Note that only sockets with SO_BUSY_POLL set will be busy polled,
so you want to either selectively set SO_BUSY_POLL on those sockets or set
sysctl.net.busy_read globally.
+
Will increase power usage.
+
Default: 0 (off)
rmem_default
Allow processes to receive tx timestamps looped together with the original
packet contents. If disabled, transmit timestamp requests from unprivileged
processes are dropped unless socket option SOF_TIMESTAMPING_OPT_TSONLY is set.
+
Default: 1 (on)
Some user space might need to gather its content even if drivers do not
provide ethtool -x support yet.
-myhost:~# cat /proc/sys/net/core/netdev_rss_key
-84:50:f4:00:a8:15:d1:a7:e9:7f:1d:60:35:c7:47:25:42:97:74:ca:56:bb:b6:a1:d8: ... (52 bytes total)
+::
+
+ myhost:~# cat /proc/sys/net/core/netdev_rss_key
+ 84:50:f4:00:a8:15:d1:a7:e9:7f:1d:60:35:c7:47:25:42:97:74:ca:56:bb:b6:a1:d8: ... (52 bytes total)
File contains nul bytes if no driver ever called netdev_rss_key_fill() function.
+
Note:
-/proc/sys/net/core/netdev_rss_key contains 52 bytes of key,
-but most drivers only use 40 bytes of it.
+ /proc/sys/net/core/netdev_rss_key contains 52 bytes of key,
+ but most drivers only use 40 bytes of it.
+
+::
-myhost:~# ethtool -x eth0
-RX flow hash indirection table for eth0 with 8 RX ring(s):
- 0: 0 1 2 3 4 5 6 7
-RSS hash key:
-84:50:f4:00:a8:15:d1:a7:e9:7f:1d:60:35:c7:47:25:42:97:74:ca:56:bb:b6:a1:d8:43:e3:c9:0c:fd:17:55:c2:3a:4d:69:ed:f1:42:89
+ myhost:~# ethtool -x eth0
+ RX flow hash indirection table for eth0 with 8 RX ring(s):
+ 0: 0 1 2 3 4 5 6 7
+ RSS hash key:
+ 84:50:f4:00:a8:15:d1:a7:e9:7f:1d:60:35:c7:47:25:42:97:74:ca:56:bb:b6:a1:d8:43:e3:c9:0c:fd:17:55:c2:3a:4d:69:ed:f1:42:89
netdev_tstamp_prequeue
----------------------
Default : 0 (for compatibility reasons)
devconf_inherit_init_net
-----------------------------
+------------------------
Controls if a new network namespace should inherit all current
settings under /proc/sys/net/{ipv4,ipv6}/conf/{all,default}/. By
Default : 0 (for compatibility reasons)
2. /proc/sys/net/unix - Parameters for Unix domain sockets
--------------------------------------------------------
+----------------------------------------------------------
There is only one file in this directory.
unix_dgram_qlen limits the max number of datagrams queued in Unix domain
3. /proc/sys/net/ipv4 - IPV4 settings
--------------------------------------------------------
+-------------------------------------
Please see: Documentation/networking/ip-sysctl.txt and ipvs-sysctl.txt for
descriptions of these entries.
4. Appletalk
--------------------------------------------------------
+------------
The /proc/sys/net/appletalk directory holds the Appletalk configuration data
when Appletalk is loaded. The configurable parameters are:
5. IPX
--------------------------------------------------------
+------
The IPX protocol has no tunable values in proc/sys/net.
address of the router (or Connected) for internal networks.
6. TIPC
--------------------------------------------------------
+-------
tipc_rmem
-----------
+---------
The TIPC protocol now has a tunable for the receive memory, similar to the
tcp_rmem - i.e. a vector of 3 INTEGERs: (min, default, max)
+::
+
# cat /proc/sys/net/tipc/tipc_rmem
4252725 34021800 68043600
#
preserved in order to be consistent with things like tcp_rmem.
named_timeout
---------------
+-------------
TIPC name table updates are distributed asynchronously in a cluster, without
any form of transaction handling. This means that different race scenarios are
-Documentation for /proc/sys/sunrpc/* kernel version 2.2.10
- (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+===================================
+Documentation for /proc/sys/sunrpc/
+===================================
-For general info and legal blurb, please look in README.
+kernel version 2.2.10
-==============================================================
+Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+
+For general info and legal blurb, please look in index.rst.
+
+------------------------------------------------------------------------------
This file contains the documentation for the sysctl files in
/proc/sys/sunrpc and is valid for Linux kernel version 2.2.
-Documentation for /proc/sys/user/* kernel version 4.9.0
- (c) 2016 Eric Biederman <ebiederm@xmission.com>
+=================================
+Documentation for /proc/sys/user/
+=================================
-==============================================================
+kernel version 4.9.0
+
+Copyright (c) 2016 Eric Biederman <ebiederm@xmission.com>
+
+------------------------------------------------------------------------------
This file contains the documentation for the sysctl files in
/proc/sys/user.
Currently, these files are in /proc/sys/user:
-- max_cgroup_namespaces
+max_cgroup_namespaces
+=====================
The maximum number of cgroup namespaces that any user in the current
user namespace may create.
-- max_ipc_namespaces
+max_ipc_namespaces
+==================
The maximum number of ipc namespaces that any user in the current
user namespace may create.
-- max_mnt_namespaces
+max_mnt_namespaces
+==================
The maximum number of mount namespaces that any user in the current
user namespace may create.
-- max_net_namespaces
+max_net_namespaces
+==================
The maximum number of network namespaces that any user in the
current user namespace may create.
-- max_pid_namespaces
+max_pid_namespaces
+==================
The maximum number of pid namespaces that any user in the current
user namespace may create.
-- max_user_namespaces
+max_user_namespaces
+===================
The maximum number of user namespaces that any user in the current
user namespace may create.
-- max_uts_namespaces
+max_uts_namespaces
+==================
The maximum number of user namespaces that any user in the current
user namespace may create.
-Documentation for /proc/sys/vm/* kernel version 2.6.29
- (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
- (c) 2008 Peter W. Morreale <pmorreale@novell.com>
+===============================
+Documentation for /proc/sys/vm/
+===============================
-For general info and legal blurb, please look in README.
+kernel version 2.6.29
-==============================================================
+Copyright (c) 1998, 1999, Rik van Riel <riel@nl.linux.org>
+
+Copyright (c) 2008 Peter W. Morreale <pmorreale@novell.com>
+
+For general info and legal blurb, please look in index.rst.
+
+------------------------------------------------------------------------------
This file contains the documentation for the sysctl files in
/proc/sys/vm and is valid for Linux kernel version 2.6.29.
- watermark_scale_factor
- zone_reclaim_mode
-==============================================================
admin_reserve_kbytes
+====================
The amount of free memory in the system that should be reserved for users
with the capability cap_sys_admin.
Changing this takes effect whenever an application requests memory.
-==============================================================
block_dump
+==========
block_dump enables block I/O debugging when set to a nonzero value. More
-information on block I/O debugging is in Documentation/laptops/laptop-mode.txt.
+information on block I/O debugging is in Documentation/admin-guide/laptops/laptop-mode.rst.
-==============================================================
compact_memory
+==============
Available only when CONFIG_COMPACTION is set. When 1 is written to the file,
all zones are compacted such that free memory is available in contiguous
blocks where possible. This can be important for example in the allocation of
huge pages although processes will also directly compact memory as required.
-==============================================================
compact_unevictable_allowed
+===========================
Available only when CONFIG_COMPACTION is set. When set to 1, compaction is
allowed to examine the unevictable lru (mlocked pages) for pages to compact.
acceptable trade for large contiguous free memory. Set to 0 to prevent
compaction from moving pages that are unevictable. Default value is 1.
-==============================================================
dirty_background_bytes
+======================
Contains the amount of dirty memory at which the background kernel
flusher threads will start writeback.
-Note: dirty_background_bytes is the counterpart of dirty_background_ratio. Only
-one of them may be specified at a time. When one sysctl is written it is
-immediately taken into account to evaluate the dirty memory limits and the
-other appears as 0 when read.
+Note:
+ dirty_background_bytes is the counterpart of dirty_background_ratio. Only
+ one of them may be specified at a time. When one sysctl is written it is
+ immediately taken into account to evaluate the dirty memory limits and the
+ other appears as 0 when read.
-==============================================================
dirty_background_ratio
+======================
Contains, as a percentage of total available memory that contains free pages
and reclaimable pages, the number of pages at which the background kernel
The total available memory is not equal to total system memory.
-==============================================================
dirty_bytes
+===========
Contains the amount of dirty memory at which a process generating disk writes
will itself start writeback.
value lower than this limit will be ignored and the old configuration will be
retained.
-==============================================================
dirty_expire_centisecs
+======================
This tunable is used to define when dirty data is old enough to be eligible
for writeout by the kernel flusher threads. It is expressed in 100'ths
of a second. Data which has been dirty in-memory for longer than this
interval will be written out next time a flusher thread wakes up.
-==============================================================
dirty_ratio
+===========
Contains, as a percentage of total available memory that contains free pages
and reclaimable pages, the number of pages at which a process which is
The total available memory is not equal to total system memory.
-==============================================================
dirtytime_expire_seconds
+========================
When a lazytime inode is constantly having its pages dirtied, the inode with
an updated timestamp will never get chance to be written out. And, if the
inode is old enough to be eligible for writeback by the kernel flusher threads.
And, it is also used as the interval to wakeup dirtytime_writeback thread.
-==============================================================
dirty_writeback_centisecs
+=========================
-The kernel flusher threads will periodically wake up and write `old' data
+The kernel flusher threads will periodically wake up and write `old` data
out to disk. This tunable expresses the interval between those wakeups, in
100'ths of a second.
Setting this to zero disables periodic writeback altogether.
-==============================================================
drop_caches
+===========
Writing to this will cause the kernel to drop clean caches, as well as
reclaimable slab objects like dentries and inodes. Once dropped, their
memory becomes free.
-To free pagecache:
+To free pagecache::
+
echo 1 > /proc/sys/vm/drop_caches
-To free reclaimable slab objects (includes dentries and inodes):
+
+To free reclaimable slab objects (includes dentries and inodes)::
+
echo 2 > /proc/sys/vm/drop_caches
-To free slab objects and pagecache:
+
+To free slab objects and pagecache::
+
echo 3 > /proc/sys/vm/drop_caches
This is a non-destructive operation and will not free any dirty objects.
To increase the number of objects freed by this operation, the user may run
-`sync' prior to writing to /proc/sys/vm/drop_caches. This will minimize the
+`sync` prior to writing to /proc/sys/vm/drop_caches. This will minimize the
number of dirty objects on the system and create more candidates to be
dropped.
use outside of a testing or debugging environment is not recommended.
You may see informational messages in your kernel log when this file is
-used:
+used::
cat (1234): drop_caches: 3
These are informational only. They do not mean that anything is wrong
with your system. To disable them, echo 4 (bit 2) into drop_caches.
-==============================================================
extfrag_threshold
+=================
This parameter affects whether the kernel will compact memory or direct
reclaim to satisfy a high-order allocation. The extfrag/extfrag_index file in
The kernel will not compact memory in a zone if the
fragmentation index is <= extfrag_threshold. The default value is 500.
-==============================================================
highmem_is_dirtyable
+====================
Available only for systems with CONFIG_HIGHMEM enabled (32b systems).
only use the low memory and they can fill it up with dirty data without
any throttling.
-==============================================================
hugetlb_shm_group
+=================
hugetlb_shm_group contains group id that is allowed to create SysV
shared memory segment using hugetlb page.
-==============================================================
laptop_mode
+===========
laptop_mode is a knob that controls "laptop mode". All the things that are
-controlled by this knob are discussed in Documentation/laptops/laptop-mode.txt.
+controlled by this knob are discussed in Documentation/admin-guide/laptops/laptop-mode.rst.
-==============================================================
legacy_va_layout
+================
If non-zero, this sysctl disables the new 32-bit mmap layout - the kernel
will use the legacy (2.4) layout for all processes.
-==============================================================
lowmem_reserve_ratio
+====================
For some specialised workloads on highmem machines it is dangerous for
the kernel to allow process memory to be allocated from the "lowmem"
can be fatal.
So the Linux page allocator has a mechanism which prevents allocations
-which _could_ use highmem from using too much lowmem. This means that
+which *could* use highmem from using too much lowmem. This means that
a certain amount of lowmem is defended from the possibility of being
captured into pinned user memory.
mechanism will also defend that region from allocations which could use
highmem or lowmem).
-The `lowmem_reserve_ratio' tunable determines how aggressive the kernel is
+The `lowmem_reserve_ratio` tunable determines how aggressive the kernel is
in defending these lower zones.
If you have a machine which uses highmem or ISA DMA and your
applications are using mlock(), or if you are running with no swap then
you probably should change the lowmem_reserve_ratio setting.
-The lowmem_reserve_ratio is an array. You can see them by reading this file.
--
-% cat /proc/sys/vm/lowmem_reserve_ratio
-256 256 32
--
+The lowmem_reserve_ratio is an array. You can see them by reading this file::
+
+ % cat /proc/sys/vm/lowmem_reserve_ratio
+ 256 256 32
But, these values are not used directly. The kernel calculates # of protection
pages for each zones from them. These are shown as array of protection pages
in /proc/zoneinfo like followings. (This is an example of x86-64 box).
-Each zone has an array of protection pages like this.
-
--
-Node 0, zone DMA
- pages free 1355
- min 3
- low 3
- high 4
+Each zone has an array of protection pages like this::
+
+ Node 0, zone DMA
+ pages free 1355
+ min 3
+ low 3
+ high 4
:
:
- numa_other 0
- protection: (0, 2004, 2004, 2004)
+ numa_other 0
+ protection: (0, 2004, 2004, 2004)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- pagesets
- cpu: 0 pcp: 0
- :
--
+ pagesets
+ cpu: 0 pcp: 0
+ :
+
These protections are added to score to judge whether this zone should be used
for page allocation or should be reclaimed.
normal page requirement. If requirement is DMA zone(index=0), protection[0]
(=0) is used.
-zone[i]'s protection[j] is calculated by following expression.
+zone[i]'s protection[j] is calculated by following expression::
-(i < j):
- zone[i]->protection[j]
- = (total sums of managed_pages from zone[i+1] to zone[j] on the node)
- / lowmem_reserve_ratio[i];
-(i = j):
- (should not be protected. = 0;
-(i > j):
- (not necessary, but looks 0)
+ (i < j):
+ zone[i]->protection[j]
+ = (total sums of managed_pages from zone[i+1] to zone[j] on the node)
+ / lowmem_reserve_ratio[i];
+ (i = j):
+ (should not be protected. = 0;
+ (i > j):
+ (not necessary, but looks 0)
The default values of lowmem_reserve_ratio[i] are
+
+ === ====================================
256 (if zone[i] means DMA or DMA32 zone)
- 32 (others).
+ 32 (others)
+ === ====================================
+
As above expression, they are reciprocal number of ratio.
256 means 1/256. # of protection pages becomes about "0.39%" of total managed
pages of higher zones on the node.
The minimum value is 1 (1/1 -> 100%). The value less than 1 completely
disables protection of the pages.
-==============================================================
max_map_count:
+==============
This file contains the maximum number of memory map areas a process
may have. Memory map areas are used as a side-effect of calling
The default value is 65536.
-=============================================================
memory_failure_early_kill:
+==========================
Control how to kill processes when uncorrected memory error (typically
a 2bit error in a memory module) is detected in the background by hardware
Applications can override this setting individually with the PR_MCE_KILL prctl
-==============================================================
memory_failure_recovery
+=======================
Enable memory failure recovery (when supported by the platform)
0: Always panic on a memory failure.
-==============================================================
-min_free_kbytes:
+min_free_kbytes
+===============
This is used to force the Linux VM to keep a minimum number
of kilobytes free. The VM uses this number to compute a
Setting this too high will OOM your machine instantly.
-=============================================================
-min_slab_ratio:
+min_slab_ratio
+==============
This is available only on NUMA kernels.
The process of reclaiming slab memory is currently not node specific
and may not be fast.
-=============================================================
-min_unmapped_ratio:
+min_unmapped_ratio
+==================
This is available only on NUMA kernels.
The default is 1 percent.
-==============================================================
mmap_min_addr
+=============
This file indicates the amount of address space which a user process will
be restricted from mmapping. Since kernel null dereference bugs could
vast majority of applications to work correctly and provide defense in depth
against future potential kernel bugs.
-==============================================================
-mmap_rnd_bits:
+mmap_rnd_bits
+=============
This value can be used to select the number of bits to use to
determine the random offset to the base address of vma regions
This value can be changed after boot using the
/proc/sys/vm/mmap_rnd_bits tunable
-==============================================================
-mmap_rnd_compat_bits:
+mmap_rnd_compat_bits
+====================
This value can be used to select the number of bits to use to
determine the random offset to the base address of vma regions
This value can be changed after boot using the
/proc/sys/vm/mmap_rnd_compat_bits tunable
-==============================================================
nr_hugepages
+============
Change the minimum size of the hugepage pool.
See Documentation/admin-guide/mm/hugetlbpage.rst
-==============================================================
nr_hugepages_mempolicy
+======================
Change the size of the hugepage pool at run-time on a specific
set of NUMA nodes.
See Documentation/admin-guide/mm/hugetlbpage.rst
-==============================================================
nr_overcommit_hugepages
+=======================
Change the maximum size of the hugepage pool. The maximum is
nr_hugepages + nr_overcommit_hugepages.
See Documentation/admin-guide/mm/hugetlbpage.rst
-==============================================================
nr_trim_pages
+=============
This is available only on NOMMU kernels.
See Documentation/nommu-mmap.txt for more information.
-==============================================================
numa_zonelist_order
+===================
This sysctl is only for NUMA and it is deprecated. Anything but
Node order will fail!
'where the memory is allocated from' is controlled by zonelists.
+
(This documentation ignores ZONE_HIGHMEM/ZONE_DMA32 for simple explanation.
- you may be able to read ZONE_DMA as ZONE_DMA32...)
+you may be able to read ZONE_DMA as ZONE_DMA32...)
In non-NUMA case, a zonelist for GFP_KERNEL is ordered as following.
ZONE_NORMAL -> ZONE_DMA
get memory from ZONE_DMA only when ZONE_NORMAL is not available.
In NUMA case, you can think of following 2 types of order.
-Assume 2 node NUMA and below is zonelist of Node(0)'s GFP_KERNEL
+Assume 2 node NUMA and below is zonelist of Node(0)'s GFP_KERNEL::
-(A) Node(0) ZONE_NORMAL -> Node(0) ZONE_DMA -> Node(1) ZONE_NORMAL
-(B) Node(0) ZONE_NORMAL -> Node(1) ZONE_NORMAL -> Node(0) ZONE_DMA.
+ (A) Node(0) ZONE_NORMAL -> Node(0) ZONE_DMA -> Node(1) ZONE_NORMAL
+ (B) Node(0) ZONE_NORMAL -> Node(1) ZONE_NORMAL -> Node(0) ZONE_DMA.
Type(A) offers the best locality for processes on Node(0), but ZONE_DMA
will be used before ZONE_NORMAL exhaustion. This increases possibility of
Default order is recommended unless this is causing problems for your
system/application.
-==============================================================
oom_dump_tasks
+==============
Enables a system-wide task dump (excluding kernel threads) to be produced
when the kernel performs an OOM-killing and includes such information as
The default value is 1 (enabled).
-==============================================================
oom_kill_allocating_task
+========================
This enables or disables killing the OOM-triggering task in
out-of-memory situations.
The default value is 0.
-==============================================================
-overcommit_kbytes:
+overcommit_kbytes
+=================
When overcommit_memory is set to 2, the committed address space is not
permitted to exceed swap plus this amount of physical RAM. See below.
of them may be specified at a time. Setting one disables the other (which
then appears as 0 when read).
-==============================================================
-overcommit_memory:
+overcommit_memory
+=================
This value contains a flag that enables memory overcommitment.
See Documentation/vm/overcommit-accounting.rst and
mm/util.c::__vm_enough_memory() for more information.
-==============================================================
-overcommit_ratio:
+overcommit_ratio
+================
When overcommit_memory is set to 2, the committed address
space is not permitted to exceed swap plus this percentage
of physical RAM. See above.
-==============================================================
page-cluster
+============
page-cluster controls the number of pages up to which consecutive pages
are read in from swap in a single attempt. This is the swap counterpart
extra faults and I/O delays for following faults if they would have been part of
that consecutive pages readahead would have brought in.
-=============================================================
panic_on_oom
+============
This enables or disables panic on out-of-memory feature.
system panics.
The default value is 0.
+
1 and 2 are for failover of clustering. Please select either
according to your policy of failover.
+
panic_on_oom=2+kdump gives you very strong tool to investigate
why oom happens. You can get snapshot.
-=============================================================
percpu_pagelist_fraction
+========================
This is the fraction of pages at most (high mark pcp->high) in each zone that
are allocated for each per cpu page list. The min value for this is 8. It
the high water marks for each per cpu page list. If the user writes '0' to this
sysctl, it will revert to this default behavior.
-==============================================================
stat_interval
+=============
The time interval between which vm statistics are updated. The default
is 1 second.
-==============================================================
stat_refresh
+============
Any read or write (by root only) flushes all the per-cpu vm statistics
into their global totals, for more accurate reports when testing
(At time of writing, a few stats are known sometimes to be found negative,
with no ill effects: errors and warnings on these stats are suppressed.)
-==============================================================
numa_stat
+=========
This interface allows runtime configuration of numa statistics.
When page allocation performance becomes a bottleneck and you can tolerate
some possible tool breakage and decreased numa counter precision, you can
-do:
+do::
+
echo 0 > /proc/sys/vm/numa_stat
When page allocation performance is not a bottleneck and you want all
-tooling to work, you can do:
+tooling to work, you can do::
+
echo 1 > /proc/sys/vm/numa_stat
-==============================================================
swappiness
+==========
This control is used to define how aggressive the kernel will swap
memory pages. Higher values will increase aggressiveness, lower values
The default value is 60.
-==============================================================
unprivileged_userfaultfd
+========================
This flag controls whether unprivileged users can use the userfaultfd
system calls. Set this to 1 to allow unprivileged users to use the
The default value is 1.
-==============================================================
-- user_reserve_kbytes
+user_reserve_kbytes
+===================
When overcommit_memory is set to 2, "never overcommit" mode, reserve
min(3% of current process size, user_reserve_kbytes) of free memory.
Changing this takes effect whenever an application requests memory.
-==============================================================
vfs_cache_pressure
-------------------
+==================
This percentage value controls the tendency of the kernel to reclaim
the memory which is used for caching of directory and inode objects.
directory and inode objects. With vfs_cache_pressure=1000, it will look for
ten times more freeable objects than there are.
-=============================================================
-watermark_boost_factor:
+watermark_boost_factor
+======================
This factor controls the level of reclaim when memory is being fragmented.
It defines the percentage of the high watermark of a zone that will be
smaller than a pageblock then a pageblocks worth of pages will be reclaimed
(e.g. 2MB on 64-bit x86). A boost factor of 0 will disable the feature.
-=============================================================
-watermark_scale_factor:
+watermark_scale_factor
+======================
This factor controls the aggressiveness of kswapd. It defines the
amount of memory left in a node/system before kswapd is woken up and
too small for the allocation bursts occurring in the system. This knob
can then be used to tune kswapd aggressiveness accordingly.
-==============================================================
-zone_reclaim_mode:
+zone_reclaim_mode
+=================
Zone_reclaim_mode allows someone to set more or less aggressive approaches to
reclaim memory when a zone runs out of memory. If it is set to zero then no
zone reclaim occurs. Allocations will be satisfied from other zones / nodes
in the system.
-This is value ORed together of
+This is value OR'ed together of
-1 = Zone reclaim on
-2 = Zone reclaim writes dirty pages out
-4 = Zone reclaim swaps pages
+= ===================================
+1 Zone reclaim on
+2 Zone reclaim writes dirty pages out
+4 Zone reclaim swaps pages
+= ===================================
zone_reclaim_mode is disabled by default. For file servers or workloads
that benefit from having their data cached, zone_reclaim_mode should be
Allowing regular swap effectively restricts allocations to the local
node unless explicitly overridden by memory policies or cpuset
configurations.
-
-============ End of Document =================================
+.. SPDX-License-Identifier: GPL-2.0
+======================
The SGI XFS Filesystem
======================
=============
When mounting an XFS filesystem, the following options are accepted.
-For boolean mount options, the names with the (*) suffix is the
-default behaviour.
allocsize=size
Sets the buffered I/O end-of-file preallocation size when
preallocation size, which uses a set of heuristics to
optimise the preallocation size based on the current
allocation patterns within the file and the access patterns
- to the file. Specifying a fixed allocsize value turns off
+ to the file. Specifying a fixed ``allocsize`` value turns off
the dynamic behaviour.
- attr2
- noattr2
+ attr2 or noattr2
The options enable/disable an "opportunistic" improvement to
be made in the way inline extended attributes are stored
on-disk. When the new form is used for the first time when
- attr2 is selected (either when setting or removing extended
+ ``attr2`` is selected (either when setting or removing extended
attributes) the on-disk superblock feature bit field will be
updated to reflect this format being in use.
The default behaviour is determined by the on-disk feature
- bit indicating that attr2 behaviour is active. If either
- mount option it set, then that becomes the new default used
+ bit indicating that ``attr2`` behaviour is active. If either
+ mount option is set, then that becomes the new default used
by the filesystem.
- CRC enabled filesystems always use the attr2 format, and so
- will reject the noattr2 mount option if it is set.
+ CRC enabled filesystems always use the ``attr2`` format, and so
+ will reject the ``noattr2`` mount option if it is set.
- discard
- nodiscard (*)
+ discard or nodiscard (default)
Enable/disable the issuing of commands to let the block
device reclaim space freed by the filesystem. This is
useful for SSD devices, thinly provisioned LUNs and virtual
machine images, but may have a performance impact.
- Note: It is currently recommended that you use the fstrim
- application to discard unused blocks rather than the discard
+ Note: It is currently recommended that you use the ``fstrim``
+ application to ``discard`` unused blocks rather than the ``discard``
mount option because the performance impact of this option
is quite severe.
- grpid/bsdgroups
- nogrpid/sysvgroups (*)
+ grpid/bsdgroups or nogrpid/sysvgroups (default)
These options define what group ID a newly created file
- gets. When grpid is set, it takes the group ID of the
+ gets. When ``grpid`` is set, it takes the group ID of the
directory in which it is created; otherwise it takes the
- fsgid of the current process, unless the directory has the
- setgid bit set, in which case it takes the gid from the
- parent directory, and also gets the setgid bit set if it is
+ ``fsgid`` of the current process, unless the directory has the
+ ``setgid`` bit set, in which case it takes the ``gid`` from the
+ parent directory, and also gets the ``setgid`` bit set if it is
a directory itself.
filestreams
across the entire filesystem rather than just on directories
configured to use it.
- ikeep
- noikeep (*)
- When ikeep is specified, XFS does not delete empty inode
- clusters and keeps them around on disk. When noikeep is
+ ikeep or noikeep (default)
+ When ``ikeep`` is specified, XFS does not delete empty inode
+ clusters and keeps them around on disk. When ``noikeep`` is
specified, empty inode clusters are returned to the free
space pool.
- inode32
- inode64 (*)
- When inode32 is specified, it indicates that XFS limits
+ inode32 or inode64 (default)
+ When ``inode32`` is specified, it indicates that XFS limits
inode creation to locations which will not result in inode
numbers with more than 32 bits of significance.
- When inode64 is specified, it indicates that XFS is allowed
+ When ``inode64`` is specified, it indicates that XFS is allowed
to create inodes at any location in the filesystem,
including those which will result in inode numbers occupying
- more than 32 bits of significance.
+ more than 32 bits of significance.
- inode32 is provided for backwards compatibility with older
+ ``inode32`` is provided for backwards compatibility with older
systems and applications, since 64 bits inode numbers might
cause problems for some applications that cannot handle
large inode numbers. If applications are in use which do
- not handle inode numbers bigger than 32 bits, the inode32
+ not handle inode numbers bigger than 32 bits, the ``inode32``
option should be specified.
-
- largeio
- nolargeio (*)
- If "nolargeio" is specified, the optimal I/O reported in
- st_blksize by stat(2) will be as small as possible to allow
+ largeio or nolargeio (default)
+ If ``nolargeio`` is specified, the optimal I/O reported in
+ ``st_blksize`` by **stat(2)** will be as small as possible to allow
user applications to avoid inefficient read/modify/write
I/O. This is typically the page size of the machine, as
this is the granularity of the page cache.
- If "largeio" specified, a filesystem that was created with a
- "swidth" specified will return the "swidth" value (in bytes)
- in st_blksize. If the filesystem does not have a "swidth"
- specified but does specify an "allocsize" then "allocsize"
+ If ``largeio`` is specified, a filesystem that was created with a
+ ``swidth`` specified will return the ``swidth`` value (in bytes)
+ in ``st_blksize``. If the filesystem does not have a ``swidth``
+ specified but does specify an ``allocsize`` then ``allocsize``
(in bytes) will be returned instead. Otherwise the behaviour
- is the same as if "nolargeio" was specified.
+ is the same as if ``nolargeio`` was specified.
logbufs=value
Set the number of in-memory log buffers. Valid numbers
If the memory cost of 8 log buffers is too high on small
systems, then it may be reduced at some cost to performance
- on metadata intensive workloads. The logbsize option below
+ on metadata intensive workloads. The ``logbsize`` option below
controls the size of each buffer and so is also relevant to
this case.
and 32768 (32k). Valid sizes for version 2 logs also
include 65536 (64k), 131072 (128k) and 262144 (256k). The
logbsize must be an integer multiple of the log
- stripe unit configured at mkfs time.
+ stripe unit configured at **mkfs(8)** time.
The default value for for version 1 logs is 32768, while the
default value for version 2 logs is MAX(32768, log_sunit).
noalign
Data allocations will not be aligned at stripe unit
boundaries. This is only relevant to filesystems created
- with non-zero data alignment parameters (sunit, swidth) by
- mkfs.
+ with non-zero data alignment parameters (``sunit``, ``swidth``) by
+ **mkfs(8)**.
norecovery
The filesystem will be mounted without running log recovery.
If the filesystem was not cleanly unmounted, it is likely to
- be inconsistent when mounted in "norecovery" mode.
+ be inconsistent when mounted in ``norecovery`` mode.
Some files or directories may not be accessible because of this.
- Filesystems mounted "norecovery" must be mounted read-only or
+ Filesystems mounted ``norecovery`` must be mounted read-only or
the mount will fail.
nouuid
Don't check for double mounted file systems using the file
- system uuid. This is useful to mount LVM snapshot volumes,
- and often used in combination with "norecovery" for mounting
+ system ``uuid``. This is useful to mount LVM snapshot volumes,
+ and often used in combination with ``norecovery`` for mounting
read-only snapshots.
noquota
uquota/usrquota/uqnoenforce/quota
User disk quota accounting enabled, and limits (optionally)
- enforced. Refer to xfs_quota(8) for further details.
+ enforced. Refer to **xfs_quota(8)** for further details.
gquota/grpquota/gqnoenforce
Group disk quota accounting enabled and limits (optionally)
- enforced. Refer to xfs_quota(8) for further details.
+ enforced. Refer to **xfs_quota(8)** for further details.
pquota/prjquota/pqnoenforce
Project disk quota accounting enabled and limits (optionally)
- enforced. Refer to xfs_quota(8) for further details.
+ enforced. Refer to **xfs_quota(8)** for further details.
sunit=value and swidth=value
Used to specify the stripe unit and width for a RAID device
block units. These options are only relevant to filesystems
that were created with non-zero data alignment parameters.
- The sunit and swidth parameters specified must be compatible
+ The ``sunit`` and ``swidth`` parameters specified must be compatible
with the existing filesystem alignment characteristics. In
- general, that means the only valid changes to sunit are
- increasing it by a power-of-2 multiple. Valid swidth values
- are any integer multiple of a valid sunit value.
+ general, that means the only valid changes to ``sunit`` are
+ increasing it by a power-of-2 multiple. Valid ``swidth`` values
+ are any integer multiple of a valid ``sunit`` value.
Typically the only time these mount options are necessary if
after an underlying RAID device has had it's geometry
Deprecated Mount Options
========================
+=========================== ================
Name Removal Schedule
- ---- ----------------
+=========================== ================
+=========================== ================
Removed Mount Options
=====================
+=========================== =======
Name Removed
- ---- -------
+=========================== =======
delaylog/nodelaylog v4.0
ihashsize v4.0
irixsgid v4.0
osyncisdsync/osyncisosync v4.0
barrier v4.19
nobarrier v4.19
-
+=========================== =======
sysctls
=======
fs.xfs.inherit_sync (Min: 0 Default: 1 Max: 1)
Setting this to "1" will cause the "sync" flag set
- by the xfs_io(8) chattr command on a directory to be
+ by the **xfs_io(8)** chattr command on a directory to be
inherited by files in that directory.
fs.xfs.inherit_nodump (Min: 0 Default: 1 Max: 1)
Setting this to "1" will cause the "nodump" flag set
- by the xfs_io(8) chattr command on a directory to be
+ by the **xfs_io(8)** chattr command on a directory to be
inherited by files in that directory.
fs.xfs.inherit_noatime (Min: 0 Default: 1 Max: 1)
Setting this to "1" will cause the "noatime" flag set
- by the xfs_io(8) chattr command on a directory to be
+ by the **xfs_io(8)** chattr command on a directory to be
inherited by files in that directory.
fs.xfs.inherit_nosymlinks (Min: 0 Default: 1 Max: 1)
Setting this to "1" will cause the "nosymlinks" flag set
- by the xfs_io(8) chattr command on a directory to be
+ by the **xfs_io(8)** chattr command on a directory to be
inherited by files in that directory.
fs.xfs.inherit_nodefrag (Min: 0 Default: 1 Max: 1)
Setting this to "1" will cause the "nodefrag" flag set
- by the xfs_io(8) chattr command on a directory to be
+ by the **xfs_io(8)** chattr command on a directory to be
inherited by files in that directory.
fs.xfs.rotorstep (Min: 1 Default: 1 Max: 256)
-error handlers:
Defines the behavior for a specific error.
-The filesystem behavior during an error can be set via sysfs files. Each
+The filesystem behavior during an error can be set via ``sysfs`` files. Each
error handler works independently - the first condition met by an error handler
for a specific class will cause the error to be propagated rather than reset and
retried.
handler configurations.
Note: there is no guarantee that fail_at_unmount can be set while an
- unmount is in progress. It is possible that the sysfs entries are
+ unmount is in progress. It is possible that the ``sysfs`` entries are
removed by the unmounting filesystem before a "retry forever" error
handler configuration causes unmount to hang, and hence the filesystem
must be configured appropriately before unmount begins to prevent
Each filesystem has specific error class handlers that define the error
propagation behaviour for specific errors. There is also a "default" error
handler defined, which defines the behaviour for all errors that don't have
-specific handlers defined. Where multiple retry constraints are configuredi for
+specific handlers defined. Where multiple retry constraints are configured for
a single error, the first retry configuration that expires will cause the error
to be propagated. The handler configurations are found in the directory:
Setting the value to "N" (where 0 < N < Max) will allow XFS to retry the
operation for up to "N" seconds before propagating the error.
-Note: The default behaviour for a specific error handler is dependent on both
+**Note:** The default behaviour for a specific error handler is dependent on both
the class and error context. For example, the default values for
"metadata/ENODEV" are "0" rather than "-1" so that this error handler defaults
to "fail immediately" behaviour. This is done because ENODEV is a fatal,
+++ /dev/null
-ARM Marvell SoCs
-================
-
-This document lists all the ARM Marvell SoCs that are currently
-supported in mainline by the Linux kernel. As the Marvell families of
-SoCs are large and complex, it is hard to understand where the support
-for a particular SoC is available in the Linux kernel. This document
-tries to help in understanding where those SoCs are supported, and to
-match them with their corresponding public datasheet, when available.
-
-Orion family
-------------
-
- Flavors:
- 88F5082
- 88F5181
- 88F5181L
- 88F5182
- Datasheet : http://www.embeddedarm.com/documentation/third-party/MV88F5182-datasheet.pdf
- Programmer's User Guide : http://www.embeddedarm.com/documentation/third-party/MV88F5182-opensource-manual.pdf
- User Manual : http://www.embeddedarm.com/documentation/third-party/MV88F5182-usermanual.pdf
- 88F5281
- Datasheet : http://www.ocmodshop.com/images/reviews/networking/qnap_ts409u/marvel_88f5281_data_sheet.pdf
- 88F6183
- Core: Feroceon 88fr331 (88f51xx) or 88fr531-vd (88f52xx) ARMv5 compatible
- Linux kernel mach directory: arch/arm/mach-orion5x
- Linux kernel plat directory: arch/arm/plat-orion
-
-Kirkwood family
----------------
-
- Flavors:
- 88F6282 a.k.a Armada 300
- Product Brief : http://www.marvell.com/embedded-processors/armada-300/assets/armada_310.pdf
- 88F6283 a.k.a Armada 310
- Product Brief : http://www.marvell.com/embedded-processors/armada-300/assets/armada_310.pdf
- 88F6190
- Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6190-003_WEB.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F619x_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
- 88F6192
- Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6192-003_ver1.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F619x_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
- 88F6182
- 88F6180
- Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6180-003_ver1.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F6180_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
- 88F6281
- Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6281-004_ver1.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F6281_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
- Homepage: http://www.marvell.com/embedded-processors/kirkwood/
- Core: Feroceon 88fr131 ARMv5 compatible
- Linux kernel mach directory: arch/arm/mach-mvebu
- Linux kernel plat directory: none
-
-Discovery family
-----------------
-
- Flavors:
- MV78100
- Product Brief : http://www.marvell.com/embedded-processors/discovery-innovation/assets/MV78100-003_WEB.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/discovery-innovation/assets/HW_MV78100_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/discovery-innovation/assets/FS_MV76100_78100_78200_OpenSource.pdf
- MV78200
- Product Brief : http://www.marvell.com/embedded-processors/discovery-innovation/assets/MV78200-002_WEB.pdf
- Hardware Spec : http://www.marvell.com/embedded-processors/discovery-innovation/assets/HW_MV78200_OpenSource.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/discovery-innovation/assets/FS_MV76100_78100_78200_OpenSource.pdf
- MV76100
- Not supported by the Linux kernel.
-
- Core: Feroceon 88fr571-vd ARMv5 compatible
-
- Linux kernel mach directory: arch/arm/mach-mv78xx0
- Linux kernel plat directory: arch/arm/plat-orion
-
-EBU Armada family
------------------
-
- Armada 370 Flavors:
- 88F6710
- 88F6707
- 88F6W11
- Product Brief: http://www.marvell.com/embedded-processors/armada-300/assets/Marvell_ARMADA_370_SoC.pdf
- Hardware Spec: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA370-datasheet.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA370-FunctionalSpec-datasheet.pdf
- Core: Sheeva ARMv7 compatible PJ4B
-
- Armada 375 Flavors:
- 88F6720
- Product Brief: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA_375_SoC-01_product_brief.pdf
- Core: ARM Cortex-A9
-
- Armada 38x Flavors:
- 88F6810 Armada 380
- 88F6820 Armada 385
- 88F6828 Armada 388
- Product infos: http://www.marvell.com/embedded-processors/armada-38x/
- Functional Spec: https://marvellcorp.wufoo.com/forms/marvell-armada-38x-functional-specifications/
- Core: ARM Cortex-A9
-
- Armada 39x Flavors:
- 88F6920 Armada 390
- 88F6928 Armada 398
- Product infos: http://www.marvell.com/embedded-processors/armada-39x/
- Core: ARM Cortex-A9
-
- Armada XP Flavors:
- MV78230
- MV78260
- MV78460
- NOTE: not to be confused with the non-SMP 78xx0 SoCs
- Product Brief: http://www.marvell.com/embedded-processors/armada-xp/assets/Marvell-ArmadaXP-SoC-product%20brief.pdf
- Functional Spec: http://www.marvell.com/embedded-processors/armada-xp/assets/ARMADA-XP-Functional-SpecDatasheet.pdf
- Hardware Specs:
- http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78230_OS.PDF
- http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78260_OS.PDF
- http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78460_OS.PDF
- Core: Sheeva ARMv7 compatible Dual-core or Quad-core PJ4B-MP
-
- Linux kernel mach directory: arch/arm/mach-mvebu
- Linux kernel plat directory: none
-
-EBU Armada family ARMv8
------------------------
-
- Armada 3710/3720 Flavors:
- 88F3710
- 88F3720
- Core: ARM Cortex A53 (ARMv8)
-
- Homepage: http://www.marvell.com/embedded-processors/armada-3700/
- Product Brief: http://www.marvell.com/embedded-processors/assets/PB-88F3700-FNL.pdf
- Device tree files: arch/arm64/boot/dts/marvell/armada-37*
-
- Armada 7K Flavors:
- 88F7020 (AP806 Dual + one CP110)
- 88F7040 (AP806 Quad + one CP110)
- Core: ARM Cortex A72
-
- Homepage: http://www.marvell.com/embedded-processors/armada-70xx/
- Product Brief: http://www.marvell.com/embedded-processors/assets/Armada7020PB-Jan2016.pdf
- http://www.marvell.com/embedded-processors/assets/Armada7040PB-Jan2016.pdf
- Device tree files: arch/arm64/boot/dts/marvell/armada-70*
-
- Armada 8K Flavors:
- 88F8020 (AP806 Dual + two CP110)
- 88F8040 (AP806 Quad + two CP110)
- Core: ARM Cortex A72
-
- Homepage: http://www.marvell.com/embedded-processors/armada-80xx/
- Product Brief: http://www.marvell.com/embedded-processors/assets/Armada8020PB-Jan2016.pdf
- http://www.marvell.com/embedded-processors/assets/Armada8040PB-Jan2016.pdf
- Device tree files: arch/arm64/boot/dts/marvell/armada-80*
-
-Avanta family
--------------
-
- Flavors:
- 88F6510
- 88F6530P
- 88F6550
- 88F6560
- Homepage : http://www.marvell.com/broadband/
- Product Brief: http://www.marvell.com/broadband/assets/Marvell_Avanta_88F6510_305_060-001_product_brief.pdf
- No public datasheet available.
-
- Core: ARMv5 compatible
-
- Linux kernel mach directory: no code in mainline yet, planned for the future
- Linux kernel plat directory: no code in mainline yet, planned for the future
-
-Storage family
---------------
-
- Armada SP:
- 88RC1580
- Product infos: http://www.marvell.com/storage/armada-sp/
- Core: Sheeva ARMv7 comatible Quad-core PJ4C
- (not supported in upstream Linux kernel)
-
-Dove family (application processor)
------------------------------------
-
- Flavors:
- 88AP510 a.k.a Armada 510
- Product Brief : http://www.marvell.com/application-processors/armada-500/assets/Marvell_Armada510_SoC.pdf
- Hardware Spec : http://www.marvell.com/application-processors/armada-500/assets/Armada-510-Hardware-Spec.pdf
- Functional Spec : http://www.marvell.com/application-processors/armada-500/assets/Armada-510-Functional-Spec.pdf
- Homepage: http://www.marvell.com/application-processors/armada-500/
- Core: ARMv7 compatible
-
- Directory: arch/arm/mach-mvebu (DT enabled platforms)
- arch/arm/mach-dove (non-DT enabled platforms)
-
-PXA 2xx/3xx/93x/95x family
---------------------------
-
- Flavors:
- PXA21x, PXA25x, PXA26x
- Application processor only
- Core: ARMv5 XScale1 core
- PXA270, PXA271, PXA272
- Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_pb.pdf
- Design guide : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_design_guide.pdf
- Developers manual : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_dev_man.pdf
- Specification : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_emts.pdf
- Specification update : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_spec_update.pdf
- Application processor only
- Core: ARMv5 XScale2 core
- PXA300, PXA310, PXA320
- PXA 300 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA300_PB_R4.pdf
- PXA 310 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA310_PB_R4.pdf
- PXA 320 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA320_PB_R4.pdf
- Design guide : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Design_Guide.pdf
- Developers manual : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Developers_Manual.zip
- Specifications : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_EMTS.pdf
- Specification Update : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Spec_Update.zip
- Reference Manual : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_TavorP_BootROM_Ref_Manual.pdf
- Application processor only
- Core: ARMv5 XScale3 core
- PXA930, PXA935
- Application processor with Communication processor
- Core: ARMv5 XScale3 core
- PXA955
- Application processor with Communication processor
- Core: ARMv7 compatible Sheeva PJ4 core
-
- Comments:
-
- * This line of SoCs originates from the XScale family developed by
- Intel and acquired by Marvell in ~2006. The PXA21x, PXA25x,
- PXA26x, PXA27x, PXA3xx and PXA93x were developed by Intel, while
- the later PXA95x were developed by Marvell.
-
- * Due to their XScale origin, these SoCs have virtually nothing in
- common with the other (Kirkwood, Dove, etc.) families of Marvell
- SoCs, except with the MMP/MMP2 family of SoCs.
-
- Linux kernel mach directory: arch/arm/mach-pxa
- Linux kernel plat directory: arch/arm/plat-pxa
-
-MMP/MMP2/MMP3 family (communication processor)
------------------------------------------
-
- Flavors:
- PXA168, a.k.a Armada 168
- Homepage : http://www.marvell.com/application-processors/armada-100/armada-168.jsp
- Product brief : http://www.marvell.com/application-processors/armada-100/assets/pxa_168_pb.pdf
- Hardware manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_datasheet.pdf
- Software manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_software_manual.pdf
- Specification update : http://www.marvell.com/application-processors/armada-100/assets/ARMADA16x_Spec_update.pdf
- Boot ROM manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_ref_manual.pdf
- App node package : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_app_note_package.pdf
- Application processor only
- Core: ARMv5 compatible Marvell PJ1 88sv331 (Mohawk)
- PXA910/PXA920
- Homepage : http://www.marvell.com/communication-processors/pxa910/
- Product Brief : http://www.marvell.com/communication-processors/pxa910/assets/Marvell_PXA910_Platform-001_PB_final.pdf
- Application processor with Communication processor
- Core: ARMv5 compatible Marvell PJ1 88sv331 (Mohawk)
- PXA688, a.k.a. MMP2, a.k.a Armada 610
- Product Brief : http://www.marvell.com/application-processors/armada-600/assets/armada610_pb.pdf
- Application processor only
- Core: ARMv7 compatible Sheeva PJ4 88sv581x core
- PXA2128, a.k.a. MMP3 (OLPC XO4, Linux support not upstream)
- Product Brief : http://www.marvell.com/application-processors/armada/pxa2128/assets/Marvell-ARMADA-PXA2128-SoC-PB.pdf
- Application processor only
- Core: Dual-core ARMv7 compatible Sheeva PJ4C core
- PXA960/PXA968/PXA978 (Linux support not upstream)
- Application processor with Communication Processor
- Core: ARMv7 compatible Sheeva PJ4 core
- PXA986/PXA988 (Linux support not upstream)
- Application processor with Communication Processor
- Core: Dual-core ARMv7 compatible Sheeva PJ4B-MP core
- PXA1088/PXA1920 (Linux support not upstream)
- Application processor with Communication Processor
- Core: quad-core ARMv7 Cortex-A7
- PXA1908/PXA1928/PXA1936
- Application processor with Communication Processor
- Core: multi-core ARMv8 Cortex-A53
-
- Comments:
-
- * This line of SoCs originates from the XScale family developed by
- Intel and acquired by Marvell in ~2006. All the processors of
- this MMP/MMP2 family were developed by Marvell.
-
- * Due to their XScale origin, these SoCs have virtually nothing in
- common with the other (Kirkwood, Dove, etc.) families of Marvell
- SoCs, except with the PXA family of SoCs listed above.
-
- Linux kernel mach directory: arch/arm/mach-mmp
- Linux kernel plat directory: arch/arm/plat-pxa
-
-Berlin family (Multimedia Solutions)
--------------------------------------
-
- Flavors:
- 88DE3010, Armada 1000 (no Linux support)
- Core: Marvell PJ1 (ARMv5TE), Dual-core
- Product Brief: http://www.marvell.com.cn/digital-entertainment/assets/armada_1000_pb.pdf
- 88DE3005, Armada 1500 Mini
- Design name: BG2CD
- Core: ARM Cortex-A9, PL310 L2CC
- 88DE3006, Armada 1500 Mini Plus
- Design name: BG2CDP
- Core: Dual Core ARM Cortex-A7
- 88DE3100, Armada 1500
- Design name: BG2
- Core: Marvell PJ4B-MP (ARMv7), Tauros3 L2CC
- 88DE3114, Armada 1500 Pro
- Design name: BG2Q
- Core: Quad Core ARM Cortex-A9, PL310 L2CC
- 88DE3214, Armada 1500 Pro 4K
- Design name: BG3
- Core: ARM Cortex-A15, CA15 integrated L2CC
- 88DE3218, ARMADA 1500 Ultra
- Core: ARM Cortex-A53
-
- Homepage: https://www.synaptics.com/products/multimedia-solutions
- Directory: arch/arm/mach-berlin
-
- Comments:
-
- * This line of SoCs is based on Marvell Sheeva or ARM Cortex CPUs
- with Synopsys DesignWare (IRQ, GPIO, Timers, ...) and PXA IP (SDHCI, USB, ETH, ...).
-
- * The Berlin family was acquired by Synaptics from Marvell in 2017.
-
-CPU Cores
----------
-
-The XScale cores were designed by Intel, and shipped by Marvell in the older
-PXA processors. Feroceon is a Marvell designed core that developed in-house,
-and that evolved into Sheeva. The XScale and Feroceon cores were phased out
-over time and replaced with Sheeva cores in later products, which subsequently
-got replaced with licensed ARM Cortex-A cores.
-
- XScale 1
- CPUID 0x69052xxx
- ARMv5, iWMMXt
- XScale 2
- CPUID 0x69054xxx
- ARMv5, iWMMXt
- XScale 3
- CPUID 0x69056xxx or 0x69056xxx
- ARMv5, iWMMXt
- Feroceon-1850 88fr331 "Mohawk"
- CPUID 0x5615331x or 0x41xx926x
- ARMv5TE, single issue
- Feroceon-2850 88fr531-vd "Jolteon"
- CPUID 0x5605531x or 0x41xx926x
- ARMv5TE, VFP, dual-issue
- Feroceon 88fr571-vd "Jolteon"
- CPUID 0x5615571x
- ARMv5TE, VFP, dual-issue
- Feroceon 88fr131 "Mohawk-D"
- CPUID 0x5625131x
- ARMv5TE, single-issue in-order
- Sheeva PJ1 88sv331 "Mohawk"
- CPUID 0x561584xx
- ARMv5, single-issue iWMMXt v2
- Sheeva PJ4 88sv581x "Flareon"
- CPUID 0x560f581x
- ARMv7, idivt, optional iWMMXt v2
- Sheeva PJ4B 88sv581x
- CPUID 0x561f581x
- ARMv7, idivt, optional iWMMXt v2
- Sheeva PJ4B-MP / PJ4C
- CPUID 0x562f584x
- ARMv7, idivt/idiva, LPAE, optional iWMMXt v2 and/or NEON
-
-Long-term plans
----------------
-
- * Unify the mach-dove/, mach-mv78xx0/, mach-orion5x/ into the
- mach-mvebu/ to support all SoCs from the Marvell EBU (Engineering
- Business Unit) in a single mach-<foo> directory. The plat-orion/
- would therefore disappear.
-
- * Unify the mach-mmp/ and mach-pxa/ into the same mach-pxa
- directory. The plat-pxa/ would therefore disappear.
-
-Credits
--------
-
- Maen Suleiman <maen@marvell.com>
- Lior Amsalem <alior@marvell.com>
- Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
- Andrew Lunn <andrew@lunn.ch>
- Nicolas Pitre <nico@fluxnic.net>
- Eric Miao <eric.y.miao@gmail.com>
+++ /dev/null
-NetWinder specific documentation
-================================
-
-The NetWinder is a small low-power computer, primarily designed
-to run Linux. It is based around the StrongARM RISC processor,
-DC21285 PCI bridge, with PC-type hardware glued around it.
-
-Port usage
-==========
-
-Min - Max Description
----------------------------
-0x0000 - 0x000f DMA1
-0x0020 - 0x0021 PIC1
-0x0060 - 0x006f Keyboard
-0x0070 - 0x007f RTC
-0x0080 - 0x0087 DMA1
-0x0088 - 0x008f DMA2
-0x00a0 - 0x00a3 PIC2
-0x00c0 - 0x00df DMA2
-0x0180 - 0x0187 IRDA
-0x01f0 - 0x01f6 ide0
-0x0201 Game port
-0x0203 RWA010 configuration read
-0x0220 - ? SoundBlaster
-0x0250 - ? WaveArtist
-0x0279 RWA010 configuration index
-0x02f8 - 0x02ff Serial ttyS1
-0x0300 - 0x031f Ether10
-0x0338 GPIO1
-0x033a GPIO2
-0x0370 - 0x0371 W83977F configuration registers
-0x0388 - ? AdLib
-0x03c0 - 0x03df VGA
-0x03f6 ide0
-0x03f8 - 0x03ff Serial ttyS0
-0x0400 - 0x0408 DC21143
-0x0480 - 0x0487 DMA1
-0x0488 - 0x048f DMA2
-0x0a79 RWA010 configuration write
-0xe800 - 0xe80f ide0/ide1 BM DMA
-
-
-Interrupt usage
-===============
-
-IRQ type Description
----------------------------
- 0 ISA 100Hz timer
- 1 ISA Keyboard
- 2 ISA cascade
- 3 ISA Serial ttyS1
- 4 ISA Serial ttyS0
- 5 ISA PS/2 mouse
- 6 ISA IRDA
- 7 ISA Printer
- 8 ISA RTC alarm
- 9 ISA
-10 ISA GP10 (Orange reset button)
-11 ISA
-12 ISA WaveArtist
-13 ISA
-14 ISA hda1
-15 ISA
-
-DMA usage
-=========
-
-DMA type Description
----------------------------
- 0 ISA IRDA
- 1 ISA
- 2 ISA cascade
- 3 ISA WaveArtist
- 4 ISA
- 5 ISA
- 6 ISA
- 7 ISA WaveArtist
+++ /dev/null
-Freebird-1.1 is produced by Legend(C), Inc.
-http://web.archive.org/web/*/http://www.legend.com.cn
-and software/linux maintained by Coventive(C), Inc.
-(http://www.coventive.com)
-
-Based on the Nicolas's strongarm kernel tree.
-
-===============================================================
-Maintainer:
-
-Chester Kuo <chester@coventive.com>
- <chester@linux.org.tw>
-
-Author :
-Tim wu <timwu@coventive.com>
-CIH <cih@coventive.com>
-Eric Peng <ericpeng@coventive.com>
-Jeff Lee <jeff_lee@coventive.com>
-Allen Cheng
-Tony Liu <tonyliu@coventive.com>
-
+++ /dev/null
-See ../empeg/README
-
+++ /dev/null
-The SA1100 serial port had its major/minor numbers officially assigned:
-
-> Date: Sun, 24 Sep 2000 21:40:27 -0700
-> From: H. Peter Anvin <hpa@transmeta.com>
-> To: Nicolas Pitre <nico@CAM.ORG>
-> Cc: Device List Maintainer <device@lanana.org>
-> Subject: Re: device
->
-> Okay. Note that device numbers 204 and 205 are used for "low density
-> serial devices", so you will have a range of minors on those majors (the
-> tty device layer handles this just fine, so you don't have to worry about
-> doing anything special.)
->
-> So your assignments are:
->
-> 204 char Low-density serial ports
-> 5 = /dev/ttySA0 SA1100 builtin serial port 0
-> 6 = /dev/ttySA1 SA1100 builtin serial port 1
-> 7 = /dev/ttySA2 SA1100 builtin serial port 2
->
-> 205 char Low-density serial ports (alternate device)
-> 5 = /dev/cusa0 Callout device for ttySA0
-> 6 = /dev/cusa1 Callout device for ttySA1
-> 7 = /dev/cusa2 Callout device for ttySA2
->
-
-You must create those inodes in /dev on the root filesystem used
-by your SA1100-based device:
-
- mknod ttySA0 c 204 5
- mknod ttySA1 c 204 6
- mknod ttySA2 c 204 7
- mknod cusa0 c 205 5
- mknod cusa1 c 205 6
- mknod cusa2 c 205 7
-
-In addition to the creation of the appropriate device nodes above, you
-must ensure your user space applications make use of the correct device
-name. The classic example is the content of the /etc/inittab file where
-you might have a getty process started on ttyS0. In this case:
-
-- replace occurrences of ttyS0 with ttySA0, ttyS1 with ttySA1, etc.
-
-- don't forget to add 'ttySA0', 'console', or the appropriate tty name
- in /etc/securetty for root to be allowed to login as well.
-
-
- ARM Linux 2.6
- =============
+=======================
+ARM Linux 2.6 and upper
+=======================
Please check <ftp://ftp.arm.linux.org.uk/pub/armlinux> for
updates.
line as detailed below.
If you wish to cross-compile, then alter the following lines in the top
- level make file:
+ level make file::
ARCH = <whatever>
- with
+
+ with::
+
ARCH = arm
- and
+ and::
CROSS_COMPILE=
- to
+
+ to::
+
CROSS_COMPILE=<your-path-to-your-compiler-without-gcc>
- eg.
+
+ eg.::
+
CROSS_COMPILE=arm-linux-
- Do a 'make config', followed by 'make Image' to build the kernel
- (arch/arm/boot/Image). A compressed image can be built by doing a
+ Do a 'make config', followed by 'make Image' to build the kernel
+ (arch/arm/boot/Image). A compressed image can be built by doing a
'make zImage' instead of 'make Image'.
Bug reports should be sent to linux-arm-kernel@lists.arm.linux.org.uk,
or submitted through the web form at
- http://www.arm.linux.org.uk/developer/
+ http://www.arm.linux.org.uk/developer/
When sending bug reports, please ensure that they contain all relevant
information, eg. the kernel messages that were printed before/during
which are there to reduce the clutter in the top-level directory. These
directories, and their purpose is listed below:
- arch-* machine/platform specific header files
- hardware driver-internal ARM specific data structures/definitions
- mach descriptions of generic ARM to specific machine interfaces
- proc-* processor dependent header files (currently only two
+ ============= ==========================================================
+ `arch-*` machine/platform specific header files
+ `hardware` driver-internal ARM specific data structures/definitions
+ `mach` descriptions of generic ARM to specific machine interfaces
+ `proc-*` processor dependent header files (currently only two
categories)
+ ============= ==========================================================
Machine/Platform support
HDC base to the source.
As of 31/3/96 it works with two drives (you should get the ADFS
- *configure harddrive set to 2). I've got an internal 20MB and a great
+ `*configure` harddrive set to 2). I've got an internal 20MB and a great
big external 5.25" FH 64MB drive (who could ever want more :-) ).
I've just got 240K/s off it (a dd with bs=128k); thats about half of what
are welcome.
-CONFIG_MACH_ and CONFIG_ARCH_
------------------------------
+`CONFIG_MACH_` and `CONFIG_ARCH_`
+---------------------------------
A change was made in 2003 to the macro names for new machines.
- Historically, CONFIG_ARCH_ was used for the bonafide architecture,
+ Historically, `CONFIG_ARCH_` was used for the bonafide architecture,
e.g. SA1100, as well as implementations of the architecture,
e.g. Assabet. It was decided to change the implementation macros
- to read CONFIG_MACH_ for clarity. Moreover, a retroactive fixup has
+ to read `CONFIG_MACH_` for clarity. Moreover, a retroactive fixup has
not been made because it would complicate patching.
Previous registrations may be found online.
<http://www.arm.linux.org.uk/developer/machines/>
Kernel entry (head.S)
---------------------------
+---------------------
The initial entry into the kernel is via head.S, which uses machine
independent code. The machine is selected by the value of 'r1' on
entry, which must be kept unique.
platform is DT-only, you do not need a registered machine type.
---
+
Russell King (15/03/2004)
- Booting ARM Linux
- =================
+=================
+Booting ARM Linux
+=================
Author: Russell King
+
Date : 18 May 2002
The following documentation is relevant to 2.4.18-rmk6 and beyond.
1. Setup and initialise RAM
---------------------------
-Existing boot loaders: MANDATORY
-New boot loaders: MANDATORY
+Existing boot loaders:
+ MANDATORY
+New boot loaders:
+ MANDATORY
The boot loader is expected to find and initialise all RAM that the
kernel will use for volatile data storage in the system. It performs
2. Initialise one serial port
-----------------------------
-Existing boot loaders: OPTIONAL, RECOMMENDED
-New boot loaders: OPTIONAL, RECOMMENDED
+Existing boot loaders:
+ OPTIONAL, RECOMMENDED
+New boot loaders:
+ OPTIONAL, RECOMMENDED
The boot loader should initialise and enable one serial port on the
target. This allows the kernel serial driver to automatically detect
3. Detect the machine type
--------------------------
-Existing boot loaders: OPTIONAL
-New boot loaders: MANDATORY except for DT-only platforms
+Existing boot loaders:
+ OPTIONAL
+New boot loaders:
+ MANDATORY except for DT-only platforms
The boot loader should detect the machine type its running on by some
method. Whether this is a hard coded value or some algorithm that
4. Setup boot data
------------------
-Existing boot loaders: OPTIONAL, HIGHLY RECOMMENDED
-New boot loaders: MANDATORY
+Existing boot loaders:
+ OPTIONAL, HIGHLY RECOMMENDED
+New boot loaders:
+ MANDATORY
The boot loader must provide either a tagged list or a dtb image for
passing configuration data to the kernel. The physical address of the
The boot loader must pass at a minimum the size and location of
the system memory, and root filesystem location. Therefore, the
-minimum tagged list should look:
+minimum tagged list should look::
- +-----------+
-base -> | ATAG_CORE | |
- +-----------+ |
- | ATAG_MEM | | increasing address
- +-----------+ |
- | ATAG_NONE | |
- +-----------+ v
+ +-----------+
+ base -> | ATAG_CORE | |
+ +-----------+ |
+ | ATAG_MEM | | increasing address
+ +-----------+ |
+ | ATAG_NONE | |
+ +-----------+ v
The tagged list should be stored in system RAM.
5. Load initramfs.
------------------
-Existing boot loaders: OPTIONAL
-New boot loaders: OPTIONAL
+Existing boot loaders:
+ OPTIONAL
+New boot loaders:
+ OPTIONAL
If an initramfs is in use then, as with the dtb, it must be placed in
a region of memory where the kernel decompressor will not overwrite it
6. Calling the kernel image
---------------------------
-Existing boot loaders: MANDATORY
-New boot loaders: MANDATORY
+Existing boot loaders:
+ MANDATORY
+New boot loaders:
+ MANDATORY
There are two options for calling the kernel zImage. If the zImage
is stored in flash, and is linked correctly to be run from flash,
you many hours of debug.
- CPU register settings
- r0 = 0,
- r1 = machine type number discovered in (3) above.
- r2 = physical address of tagged list in system RAM, or
- physical address of device tree block (dtb) in system RAM
+
+ - r0 = 0,
+ - r1 = machine type number discovered in (3) above.
+ - r2 = physical address of tagged list in system RAM, or
+ physical address of device tree block (dtb) in system RAM
- CPU mode
+
All forms of interrupts must be disabled (IRQs and FIQs)
For CPUs which do not include the ARM virtualization extensions, the
entered in SVC mode.
- Caches, MMUs
+
The MMU must be off.
+
Instruction cache may be on or off.
+
Data cache must be off.
If the kernel is entered in HYP mode, the above requirements apply to
+=========================================================
Cluster-wide Power-up/power-down race avoidance algorithm
=========================================================
Each cluster and CPU is assigned a state, as follows:
- DOWN
- COMING_UP
- UP
- GOING_DOWN
+ - DOWN
+ - COMING_UP
+ - UP
+ - GOING_DOWN
+
+::
+---------> UP ----------+
| v
+--------- DOWN <--------+
-DOWN: The CPU or cluster is not coherent, and is either powered off or
+DOWN:
+ The CPU or cluster is not coherent, and is either powered off or
suspended, or is ready to be powered off or suspended.
-COMING_UP: The CPU or cluster has committed to moving to the UP state.
+COMING_UP:
+ The CPU or cluster has committed to moving to the UP state.
It may be part way through the process of initialisation and
enabling coherency.
-UP: The CPU or cluster is active and coherent at the hardware
+UP:
+ The CPU or cluster is active and coherent at the hardware
level. A CPU in this state is not necessarily being used
actively by the kernel.
-GOING_DOWN: The CPU or cluster has committed to moving to the DOWN
+GOING_DOWN:
+ The CPU or cluster has committed to moving to the DOWN
state. It may be part way through the process of teardown and
coherency exit.
level states are described in the "Cluster state" section.
To help distinguish the CPU states from cluster states in this
-discussion, the state names are given a CPU_ prefix for the CPU states,
-and a CLUSTER_ or INBOUND_ prefix for the cluster states.
+discussion, the state names are given a `CPU_` prefix for the CPU states,
+and a `CLUSTER_` or `INBOUND_` prefix for the cluster states.
CPU state
The algorithm defines the following states for each CPU in the system:
- CPU_DOWN
- CPU_COMING_UP
- CPU_UP
- CPU_GOING_DOWN
+ - CPU_DOWN
+ - CPU_COMING_UP
+ - CPU_UP
+ - CPU_GOING_DOWN
+
+::
cluster setup and
CPU setup complete policy decision
CPU_DOWN:
-
A CPU reaches the CPU_DOWN state when it is ready for
power-down. On reaching this state, the CPU will typically
power itself down or suspend itself, via a WFI instruction or a
firmware call.
- Next state: CPU_COMING_UP
- Conditions: none
+ Next state:
+ CPU_COMING_UP
+ Conditions:
+ none
Trigger events:
-
a) an explicit hardware power-up operation, resulting
from a policy decision on another CPU;
CPU_COMING_UP:
-
A CPU cannot start participating in hardware coherency until the
cluster is set up and coherent. If the cluster is not ready,
then the CPU will wait in the CPU_COMING_UP state until the
cluster has been set up.
- Next state: CPU_UP
- Conditions: The CPU's parent cluster must be in CLUSTER_UP.
- Trigger events: Transition of the parent cluster to CLUSTER_UP.
+ Next state:
+ CPU_UP
+ Conditions:
+ The CPU's parent cluster must be in CLUSTER_UP.
+ Trigger events:
+ Transition of the parent cluster to CLUSTER_UP.
Refer to the "Cluster state" section for a description of the
CLUSTER_UP state.
The CPU remains in this state until an explicit policy decision
is made to shut down or suspend the CPU.
- Next state: CPU_GOING_DOWN
- Conditions: none
- Trigger events: explicit policy decision
+ Next state:
+ CPU_GOING_DOWN
+ Conditions:
+ none
+ Trigger events:
+ explicit policy decision
CPU_GOING_DOWN:
-
While in this state, the CPU exits coherency, including any
operations required to achieve this (such as cleaning data
caches).
- Next state: CPU_DOWN
- Conditions: local CPU teardown complete
- Trigger events: (spontaneous)
+ Next state:
+ CPU_DOWN
+ Conditions:
+ local CPU teardown complete
+ Trigger events:
+ (spontaneous)
Cluster state
reason, the cluster state is split into two parts:
"cluster" state: The global state of the cluster; or the state
- on the outbound side:
+ on the outbound side:
- CLUSTER_DOWN
- CLUSTER_UP
- CLUSTER_GOING_DOWN
+ - CLUSTER_DOWN
+ - CLUSTER_UP
+ - CLUSTER_GOING_DOWN
"inbound" state: The state of the cluster on the inbound side.
- INBOUND_NOT_COMING_UP
- INBOUND_COMING_UP
+ - INBOUND_NOT_COMING_UP
+ - INBOUND_COMING_UP
The different pairings of these states results in six possible
- states for the cluster as a whole:
+ states for the cluster as a whole::
CLUSTER_UP
+==========> INBOUND_NOT_COMING_UP -------------+
CLUSTER_DOWN/INBOUND_NOT_COMING_UP:
+ Next state:
+ CLUSTER_DOWN/INBOUND_COMING_UP (inbound)
+ Conditions:
+ none
- Next state: CLUSTER_DOWN/INBOUND_COMING_UP (inbound)
- Conditions: none
Trigger events:
-
a) an explicit hardware power-up operation, resulting
from a policy decision on another CPU;
setup to enable other CPUs in the cluster to enter coherency
safely.
- Next state: CLUSTER_UP/INBOUND_COMING_UP (inbound)
- Conditions: cluster-level setup and hardware coherency complete
- Trigger events: (spontaneous)
+ Next state:
+ CLUSTER_UP/INBOUND_COMING_UP (inbound)
+ Conditions:
+ cluster-level setup and hardware coherency complete
+ Trigger events:
+ (spontaneous)
CLUSTER_UP/INBOUND_COMING_UP:
CLUSTER_UP/INBOUND_NOT_COMING_UP. All other CPUs on the cluster
should consider treat these two states as equivalent.
- Next state: CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound)
- Conditions: none
- Trigger events: (spontaneous)
+ Next state:
+ CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound)
+ Conditions:
+ none
+ Trigger events:
+ (spontaneous)
CLUSTER_UP/INBOUND_NOT_COMING_UP:
The cluster will remain in this state until a policy decision is
made to power the cluster down.
- Next state: CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound)
- Conditions: none
- Trigger events: policy decision to power down the cluster
+ Next state:
+ CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound)
+ Conditions:
+ none
+ Trigger events:
+ policy decision to power down the cluster
CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP:
Next states:
CLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound)
- Conditions: cluster torn down and ready to power off
- Trigger events: (spontaneous)
+ Conditions:
+ cluster torn down and ready to power off
+ Trigger events:
+ (spontaneous)
CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound)
- Conditions: none
- Trigger events:
+ Conditions:
+ none
+ Trigger events:
a) an explicit hardware power-up operation,
resulting from a policy decision on another
CPU;
Next states:
CLUSTER_UP/INBOUND_COMING_UP (outbound)
- Conditions: cluster-level setup and hardware
+ Conditions:
+ cluster-level setup and hardware
coherency complete
- Trigger events: (spontaneous)
+
+ Trigger events:
+ (spontaneous)
CLUSTER_DOWN/INBOUND_COMING_UP (outbound)
- Conditions: cluster torn down and ready to power off
- Trigger events: (spontaneous)
+ Conditions:
+ cluster torn down and ready to power off
+
+ Trigger events:
+ (spontaneous)
Last man and First man selection
arch/arm/common/mcpm_entry.c (everything else):
__mcpm_cpu_going_down() signals the transition of a CPU to the
- CPU_GOING_DOWN state.
+ CPU_GOING_DOWN state.
__mcpm_cpu_down() signals the transition of a CPU to the CPU_DOWN
- state.
+ state.
A CPU transitions to CPU_COMING_UP and then to CPU_UP via the
- low-level power-up code in mcpm_head.S. This could
- involve CPU-specific setup code, but in the current
- implementation it does not.
+ low-level power-up code in mcpm_head.S. This could
+ involve CPU-specific setup code, but in the current
+ implementation it does not.
__mcpm_outbound_enter_critical() and __mcpm_outbound_leave_critical()
- handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN
- and from there to CLUSTER_DOWN or back to CLUSTER_UP (in
- the case of an aborted cluster power-down).
+ handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN
+ and from there to CLUSTER_DOWN or back to CLUSTER_UP (in
+ the case of an aborted cluster power-down).
- These functions are more complex than the __mcpm_cpu_*()
- functions due to the extra inter-CPU coordination which
- is needed for safe transitions at the cluster level.
+ These functions are more complex than the __mcpm_cpu_*()
+ functions due to the extra inter-CPU coordination which
+ is needed for safe transitions at the cluster level.
A cluster transitions from CLUSTER_DOWN back to CLUSTER_UP via
- the low-level power-up code in mcpm_head.S. This
- typically involves platform-specific setup code,
- provided by the platform-specific power_up_setup
- function registered via mcpm_sync_init.
+ the low-level power-up code in mcpm_head.S. This
+ typically involves platform-specific setup code,
+ provided by the platform-specific power_up_setup
+ function registered via mcpm_sync_init.
Deep topologies:
-Interface for registering and calling firmware-specific operations for ARM.
-----
+==========================================================================
+Interface for registering and calling firmware-specific operations for ARM
+==========================================================================
+
Written by Tomasz Figa <t.figa@samsung.com>
Some boards are running with secure firmware running in TrustZone secure
Firmware operations can be specified by filling in a struct firmware_ops
with appropriate callbacks and then registering it with register_firmware_ops()
-function.
+function::
void register_firmware_ops(const struct firmware_ops *ops)
There is a default, empty set of operations provided, so there is no need to
set anything if platform does not require firmware operations.
-To call a firmware operation, a helper macro is provided
+To call a firmware operation, a helper macro is provided::
#define call_firmware_op(op, ...) \
((firmware_ops->op) ? firmware_ops->op(__VA_ARGS__) : (-ENOSYS))
-ENOSYS to signal that given operation is not available (for example, to allow
fallback to legacy operation).
-Example of registering firmware operations:
+Example of registering firmware operations::
/* board file */
register_firmware_ops(&platformX_firmware_ops);
}
-Example of using a firmware operation:
+Example of using a firmware operation::
/* some platform code, e.g. SMP initialization */
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+ARM Architecture
+================
+
+.. toctree::
+ :maxdepth: 1
+
+ arm
+ booting
+ cluster-pm-race-avoidance
+ firmware
+ interrupts
+ kernel_mode_neon
+ kernel_user_helpers
+ memory
+ mem_alignment
+ tcm
+ setup
+ swp_emulation
+ uefi
+ vlocks
+ porting
+
+SoC-specific documents
+======================
+
+.. toctree::
+ :maxdepth: 1
+
+ ixp4xx
+
+ marvel
+ microchip
+
+ netwinder
+ nwfpe/index
+
+ keystone/overview
+ keystone/knav-qmss
+
+ omap/index
+
+ pxa/mfp
+
+
+ sa1100/index
+
+ stm32/stm32f746-overview
+ stm32/overview
+ stm32/stm32h743-overview
+ stm32/stm32f769-overview
+ stm32/stm32f429-overview
+ stm32/stm32mp157-overview
+
+ sunxi
+
+ samsung/index
+ samsung-s3c24xx/index
+
+ sunxi/clocks
+
+ spear/overview
+
+ sti/stih416-overview
+ sti/stih407-overview
+ sti/stih418-overview
+ sti/overview
+ sti/stih415-overview
+
+ vfp/release-notes
+
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
-2.5.2-rmk5
-----------
+==========
+Interrupts
+==========
-This is the first kernel that contains a major shake up of some of the
-major architecture-specific subsystems.
+2.5.2-rmk5:
+ This is the first kernel that contains a major shake up of some of the
+ major architecture-specific subsystems.
Firstly, it contains some pretty major changes to the way we handle the
MMU TLB. Each MMU TLB variant is now handled completely separately -
array (basically all machine types) will break, and this means every
machine type that we currently have.
-Lets take an example. On the Assabet with Neponset, we have:
+Lets take an example. On the Assabet with Neponset, we have::
GPIO25 IRR:2
SA1100 ------------> Neponset -----------> SA1111
SA11x0 IRQs are handled by two separate "chip" structures, one for
GPIO0-10, and another for all the rest. It is just a container for
the various operations (maybe this'll change to a better name).
-This structure has the following operations:
-
-struct irqchip {
- /*
- * Acknowledge the IRQ.
- * If this is a level-based IRQ, then it is expected to mask the IRQ
- * as well.
- */
- void (*ack)(unsigned int irq);
- /*
- * Mask the IRQ in hardware.
- */
- void (*mask)(unsigned int irq);
- /*
- * Unmask the IRQ in hardware.
- */
- void (*unmask)(unsigned int irq);
- /*
- * Re-run the IRQ
- */
- void (*rerun)(unsigned int irq);
- /*
- * Set the type of the IRQ.
- */
- int (*type)(unsigned int irq, unsigned int, type);
-};
-
-ack - required. May be the same function as mask for IRQs
+This structure has the following operations::
+
+ struct irqchip {
+ /*
+ * Acknowledge the IRQ.
+ * If this is a level-based IRQ, then it is expected to mask the IRQ
+ * as well.
+ */
+ void (*ack)(unsigned int irq);
+ /*
+ * Mask the IRQ in hardware.
+ */
+ void (*mask)(unsigned int irq);
+ /*
+ * Unmask the IRQ in hardware.
+ */
+ void (*unmask)(unsigned int irq);
+ /*
+ * Re-run the IRQ
+ */
+ void (*rerun)(unsigned int irq);
+ /*
+ * Set the type of the IRQ.
+ */
+ int (*type)(unsigned int irq, unsigned int, type);
+ };
+
+ack
+ - required. May be the same function as mask for IRQs
handled by do_level_IRQ.
-mask - required.
-unmask - required.
-rerun - optional. Not required if you're using do_level_IRQ for all
+mask
+ - required.
+unmask
+ - required.
+rerun
+ - optional. Not required if you're using do_level_IRQ for all
IRQs that use this 'irqchip'. Generally expected to re-trigger
the hardware IRQ if possible. If not, may call the handler
directly.
-type - optional. If you don't support changing the type of an IRQ,
+type
+ - optional. If you don't support changing the type of an IRQ,
it should be null so people can detect if they are unable to
set the IRQ type.
SMC9196 and USAR above.
So, what's changed?
+===================
1. Machine implementations must not write to the irqdesc array.
absolutely necessary.
set_irq_chip(irq,chip)
-
Set the mask/unmask methods for handling this IRQ
set_irq_handler(irq,handler)
-
Set the handler for this IRQ (level, edge, simple)
set_irq_chained_handler(irq,handler)
-
Set a "chained" handler for this IRQ - automatically
enables this IRQ (eg, Neponset and SA1111 handlers).
set_irq_flags(irq,flags)
-
Set the valid/probe/noautoenable flags.
set_irq_type(irq,type)
-
Set active the IRQ edge(s)/level. This replaces the
SA1111 INTPOL manipulation, and the set_GPIO_IRQ_edge()
function. Type should be one of IRQ_TYPE_xxx defined in
be re-checked for pending events. (see the Neponset IRQ handler for
details).
-7. fixup_irq() is gone, as is arch/arm/mach-*/include/mach/irq.h
+7. fixup_irq() is gone, as is `arch/arm/mach-*/include/mach/irq.h`
Please note that this will not solve all problems - some of them are
hardware based. Mixing level-based and edge-based IRQs on the same
parent signal (eg neponset) is one such area where a software based
solution can't provide the full answer to low IRQ latency.
-
-
--------------------------------------------------------------------------
+===========================================================
Release Notes for Linux on Intel's IXP4xx Network Processor
+===========================================================
Maintained by Deepak Saxena <dsaxena@plexity.net>
-------------------------------------------------------------------------
1. Overview
Intel's IXP4xx network processor is a highly integrated SOC that
-is targeted for network applications, though it has become popular
+is targeted for network applications, though it has become popular
in industrial control and other areas due to low cost and power
consumption. The IXP4xx family currently consists of several processors
that support different network offload functions such as encryption,
http://developer.intel.com/design/network/products/npfamily/ixp4xx.htm
-Intel also made the IXCP1100 CPU for sometime which is an IXP4xx
+Intel also made the IXCP1100 CPU for sometime which is an IXP4xx
stripped of much of the network intelligence.
2. Linux Support
- PCI interface
- Flash access (MTD/JFFS)
- I2C through GPIO on IXP42x
-- GPIO for input/output/interrupts
+- GPIO for input/output/interrupts
See arch/arm/mach-ixp4xx/include/mach/platform.h for access functions.
- Timers (watchdog, OS)
If you need to use any of the above, you need to download Intel's
software from:
- http://developer.intel.com/design/network/products/npfamily/ixp425.htm
+ http://developer.intel.com/design/network/products/npfamily/ixp425.htm
DO NOT POST QUESTIONS TO THE LINUX MAILING LISTS REGARDING THE PROPRIETARY
SOFTWARE.
There are several websites that provide directions/pointers on using
Intel's software:
- http://sourceforge.net/projects/ixp4xx-osdg/
- Open Source Developer's Guide for using uClinux and the Intel libraries
+ - http://sourceforge.net/projects/ixp4xx-osdg/
+ Open Source Developer's Guide for using uClinux and the Intel libraries
-http://gatewaymaker.sourceforge.net/
- Simple one page summary of building a gateway using an IXP425 and Linux
+ - http://gatewaymaker.sourceforge.net/
+ Simple one page summary of building a gateway using an IXP425 and Linux
-http://ixp425.sourceforge.net/
- ATM device driver for IXP425 that relies on Intel's libraries
+ - http://ixp425.sourceforge.net/
+ ATM device driver for IXP425 that relies on Intel's libraries
3. Known Issues/Limitations
can only expose 64MB of that memory to the PCI bus. This means that if
you are running with > 64MB, all PCI buffers outside of the accessible
range will be bounced using the routines in arch/arm/common/dmabounce.c.
-
+
3b. Limited outbound PCI window
IXP4xx provides two methods of accessing PCI memory space:
To access PCI via this space, we simply ioremap() the BAR
into the kernel and we can use the standard read[bwl]/write[bwl]
macros. This is the preffered method due to speed but it
- limits the system to just 64MB of PCI memory. This can be
+ limits the system to just 64MB of PCI memory. This can be
problamatic if using video cards and other memory-heavy devices.
-
-2) If > 64MB of memory space is required, the IXP4xx can be
- configured to use indirect registers to access PCI This allows
- for up to 128MB (0x48000000 to 0x4fffffff) of memory on the bus.
- The disadvantage of this is that every PCI access requires
- three local register accesses plus a spinlock, but in some
- cases the performance hit is acceptable. In addition, you cannot
+
+2) If > 64MB of memory space is required, the IXP4xx can be
+ configured to use indirect registers to access PCI This allows
+ for up to 128MB (0x48000000 to 0x4fffffff) of memory on the bus.
+ The disadvantage of this is that every PCI access requires
+ three local register accesses plus a spinlock, but in some
+ cases the performance hit is acceptable. In addition, you cannot
mmap() PCI devices in this case due to the indirect nature
of the PCI window.
3c. GPIO as Interrupts
-Currently the code only handles level-sensitive GPIO interrupts
+Currently the code only handles level-sensitive GPIO interrupts
4. Supported platforms
ADI Engineering Coyote Gateway Reference Platform
http://www.adiengineering.com/productsCoyote.html
- The ADI Coyote platform is reference design for those building
+ The ADI Coyote platform is reference design for those building
small residential/office gateways. One NPE is connected to a 10/100
interface, one to 4-port 10/100 switch, and the third to and ADSL
interface. In addition, it also supports to POTs interfaces connected
the expansion bus.
Intel IXDP425 Development Platform
-http://www.intel.com/design/network/products/npfamily/ixdpg425.htm
+http://www.intel.com/design/network/products/npfamily/ixdpg425.htm
- This is Intel's standard reference platform for the IXDP425 and is
+ This is Intel's standard reference platform for the IXDP425 and is
also known as the Richfield board. It contains 4 PCI slots, 16MB
of flash, two 10/100 ports and one ADSL port.
The following people have contributed patches/comments/etc:
-Lennerty Buytenhek
-Lutz Jaenicke
-Justin Mayfield
-Robert E. Ranslam
-[I know I've forgotten others, please email me to be added]
+- Lennerty Buytenhek
+- Lutz Jaenicke
+- Justin Mayfield
+- Robert E. Ranslam
+
+[I know I've forgotten others, please email me to be added]
-------------------------------------------------------------------------
+================
Kernel mode NEON
================
Therefore, the recommended and only supported way of using NEON/VFP in the
kernel is by adhering to the following rules:
+
* isolate the NEON code in a separate compilation unit and compile it with
'-march=armv7-a -mfpu=neon -mfloat-abi=softfp';
* issue the calls to kernel_neon_begin(), kernel_neon_end() as well as the calls
NEON intrinsics are also supported. However, as code using NEON intrinsics
relies on the GCC header <arm_neon.h>, (which #includes <stdint.h>), you should
observe the following in addition to the rules above:
+
* Compile the unit containing the NEON intrinsics with '-ffreestanding' so GCC
uses its builtin version of <stdint.h> (this is a C99 header which the kernel
does not supply);
+============================
Kernel-provided User Helpers
============================
Location: 0xffff0ffc
-Reference declaration:
+Reference declaration::
extern int32_t __kuser_helper_version;
running kernel. User space may read this to determine the availability
of a particular helper.
-Usage example:
+Usage example::
-#define __kuser_helper_version (*(int32_t *)0xffff0ffc)
+ #define __kuser_helper_version (*(int32_t *)0xffff0ffc)
-void check_kuser_version(void)
-{
+ void check_kuser_version(void)
+ {
if (__kuser_helper_version < 2) {
fprintf(stderr, "can't do atomic operations, kernel too old\n");
abort();
}
-}
+ }
Notes:
Location: 0xffff0fe0
-Reference prototype:
+Reference prototype::
void * __kuser_get_tls(void);
Get the TLS value as previously set via the __ARM_NR_set_tls syscall.
-Usage example:
+Usage example::
-typedef void * (__kuser_get_tls_t)(void);
-#define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0)
+ typedef void * (__kuser_get_tls_t)(void);
+ #define __kuser_get_tls (*(__kuser_get_tls_t *)0xffff0fe0)
-void foo()
-{
+ void foo()
+ {
void *tls = __kuser_get_tls();
printf("TLS = %p\n", tls);
-}
+ }
Notes:
Location: 0xffff0fc0
-Reference prototype:
+Reference prototype::
int __kuser_cmpxchg(int32_t oldval, int32_t newval, volatile int32_t *ptr);
Definition:
- Atomically store newval in *ptr only if *ptr is equal to oldval.
- Return zero if *ptr was changed or non-zero if no exchange happened.
- The C flag is also set if *ptr was changed to allow for assembly
+ Atomically store newval in `*ptr` only if `*ptr` is equal to oldval.
+ Return zero if `*ptr` was changed or non-zero if no exchange happened.
+ The C flag is also set if `*ptr` was changed to allow for assembly
optimization in the calling code.
-Usage example:
+Usage example::
-typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr);
-#define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0)
+ typedef int (__kuser_cmpxchg_t)(int oldval, int newval, volatile int *ptr);
+ #define __kuser_cmpxchg (*(__kuser_cmpxchg_t *)0xffff0fc0)
-int atomic_add(volatile int *ptr, int val)
-{
+ int atomic_add(volatile int *ptr, int val)
+ {
int old, new;
do {
} while(__kuser_cmpxchg(old, new, ptr));
return new;
-}
+ }
Notes:
Location: 0xffff0fa0
-Reference prototype:
+Reference prototype::
void __kuser_memory_barrier(void);
Apply any needed memory barrier to preserve consistency with data modified
manually and __kuser_cmpxchg usage.
-Usage example:
+Usage example::
-typedef void (__kuser_dmb_t)(void);
-#define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0)
+ typedef void (__kuser_dmb_t)(void);
+ #define __kuser_dmb (*(__kuser_dmb_t *)0xffff0fa0)
Notes:
Location: 0xffff0f60
-Reference prototype:
+Reference prototype::
int __kuser_cmpxchg64(const int64_t *oldval,
const int64_t *newval,
Definition:
- Atomically store the 64-bit value pointed by *newval in *ptr only if *ptr
- is equal to the 64-bit value pointed by *oldval. Return zero if *ptr was
+ Atomically store the 64-bit value pointed by `*newval` in `*ptr` only if `*ptr`
+ is equal to the 64-bit value pointed by `*oldval`. Return zero if `*ptr` was
changed or non-zero if no exchange happened.
- The C flag is also set if *ptr was changed to allow for assembly
+ The C flag is also set if `*ptr` was changed to allow for assembly
optimization in the calling code.
-Usage example:
+Usage example::
-typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval,
- const int64_t *newval,
- volatile int64_t *ptr);
-#define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60)
+ typedef int (__kuser_cmpxchg64_t)(const int64_t *oldval,
+ const int64_t *newval,
+ volatile int64_t *ptr);
+ #define __kuser_cmpxchg64 (*(__kuser_cmpxchg64_t *)0xffff0f60)
-int64_t atomic_add64(volatile int64_t *ptr, int64_t val)
-{
+ int64_t atomic_add64(volatile int64_t *ptr, int64_t val)
+ {
int64_t old, new;
do {
} while(__kuser_cmpxchg64(&old, &new, ptr));
return new;
-}
+ }
Notes:
-* Texas Instruments Keystone Navigator Queue Management SubSystem driver
+======================================================================
+Texas Instruments Keystone Navigator Queue Management SubSystem driver
+======================================================================
Driver source code path
drivers/soc/ti/knav_qmss.c
accumulator channels defined in DTS (example in DT documentation) to monitor
1 or 32 queues per channel. More description on the firmware is available in
CPPI/QMSS Low Level Driver document (docs/CPPI_QMSS_LLD_SDS.pdf) at
+
git://git.ti.com/keystone-rtos/qmss-lld.git
k2_qmss_pdsp_acc48_k2_le_1_0_0_9.bin firmware supports upto 48 accumulator
channels. This firmware is available under ti-keystone folder of
firmware.git at
+
git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
To use copy the firmware image to lib/firmware folder of the initramfs or
- TI Keystone Linux Overview
- --------------------------
+==========================
+TI Keystone Linux Overview
+==========================
Introduction
------------
Following SoCs & EVMs are currently supported:-
------------- K2HK SoC and EVM --------------------------------------------------
+K2HK SoC and EVM
+=================
a.k.a Keystone 2 Hawking/Kepler SoC
TCI6636K2H & TCI6636K2K: See documentation at
+
http://www.ti.com/product/tci6638k2k
http://www.ti.com/product/tci6638k2h
EVM:
-http://www.advantech.com/Support/TI-EVM/EVMK2HX_sd.aspx
+ http://www.advantech.com/Support/TI-EVM/EVMK2HX_sd.aspx
------------- K2E SoC and EVM ---------------------------------------------------
+K2E SoC and EVM
+===============
a.k.a Keystone 2 Edison SoC
-K2E - 66AK2E05: See documentation at
+
+K2E - 66AK2E05:
+
+See documentation at
+
http://www.ti.com/product/66AK2E05/technicaldocuments
EVM:
-https://www.einfochips.com/index.php/partnerships/texas-instruments/k2e-evm.html
+ https://www.einfochips.com/index.php/partnerships/texas-instruments/k2e-evm.html
------------- K2L SoC and EVM ---------------------------------------------------
+K2L SoC and EVM
+===============
a.k.a Keystone 2 Lamarr SoC
-K2L - TCI6630K2L: See documentation at
+
+K2L - TCI6630K2L:
+
+See documentation at
http://www.ti.com/product/TCI6630K2L/technicaldocuments
+
EVM:
-https://www.einfochips.com/index.php/partnerships/texas-instruments/k2l-evm.html
+ https://www.einfochips.com/index.php/partnerships/texas-instruments/k2l-evm.html
Configuration
-------------
All of the K2 SoCs/EVMs share a common defconfig, keystone_defconfig and same
image is used to boot on individual EVMs. The platform configuration is
-specified through DTS. Following are the DTS used:-
- K2HK EVM : k2hk-evm.dts
- K2E EVM : k2e-evm.dts
- K2L EVM : k2l-evm.dts
+specified through DTS. Following are the DTS used:
+
+ K2HK EVM:
+ k2hk-evm.dts
+ K2E EVM:
+ k2e-evm.dts
+ K2L EVM:
+ k2l-evm.dts
The device tree documentation for the keystone machines are located at
+
Documentation/devicetree/bindings/arm/keystone/keystone.txt
Document Author
---------------
Murali Karicheri <m-karicheri2@ti.com>
+
Copyright 2015 Texas Instruments
--- /dev/null
+================
+ARM Marvell SoCs
+================
+
+This document lists all the ARM Marvell SoCs that are currently
+supported in mainline by the Linux kernel. As the Marvell families of
+SoCs are large and complex, it is hard to understand where the support
+for a particular SoC is available in the Linux kernel. This document
+tries to help in understanding where those SoCs are supported, and to
+match them with their corresponding public datasheet, when available.
+
+Orion family
+------------
+
+ Flavors:
+ - 88F5082
+ - 88F5181
+ - 88F5181L
+ - 88F5182
+
+ - Datasheet: http://www.embeddedarm.com/documentation/third-party/MV88F5182-datasheet.pdf
+ - Programmer's User Guide: http://www.embeddedarm.com/documentation/third-party/MV88F5182-opensource-manual.pdf
+ - User Manual: http://www.embeddedarm.com/documentation/third-party/MV88F5182-usermanual.pdf
+ - 88F5281
+
+ - Datasheet: http://www.ocmodshop.com/images/reviews/networking/qnap_ts409u/marvel_88f5281_data_sheet.pdf
+ - 88F6183
+ Core:
+ Feroceon 88fr331 (88f51xx) or 88fr531-vd (88f52xx) ARMv5 compatible
+ Linux kernel mach directory:
+ arch/arm/mach-orion5x
+ Linux kernel plat directory:
+ arch/arm/plat-orion
+
+Kirkwood family
+---------------
+
+ Flavors:
+ - 88F6282 a.k.a Armada 300
+
+ - Product Brief : http://www.marvell.com/embedded-processors/armada-300/assets/armada_310.pdf
+ - 88F6283 a.k.a Armada 310
+
+ - Product Brief : http://www.marvell.com/embedded-processors/armada-300/assets/armada_310.pdf
+ - 88F6190
+
+ - Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6190-003_WEB.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F619x_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
+ - 88F6192
+
+ - Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6192-003_ver1.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F619x_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
+ - 88F6182
+ - 88F6180
+
+ - Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6180-003_ver1.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F6180_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
+ - 88F6281
+
+ - Product Brief : http://www.marvell.com/embedded-processors/kirkwood/assets/88F6281-004_ver1.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F6281_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
+ Homepage:
+ http://www.marvell.com/embedded-processors/kirkwood/
+ Core:
+ Feroceon 88fr131 ARMv5 compatible
+ Linux kernel mach directory:
+ arch/arm/mach-mvebu
+ Linux kernel plat directory:
+ none
+
+Discovery family
+----------------
+
+ Flavors:
+ - MV78100
+
+ - Product Brief : http://www.marvell.com/embedded-processors/discovery-innovation/assets/MV78100-003_WEB.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/discovery-innovation/assets/HW_MV78100_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/discovery-innovation/assets/FS_MV76100_78100_78200_OpenSource.pdf
+ - MV78200
+
+ - Product Brief : http://www.marvell.com/embedded-processors/discovery-innovation/assets/MV78200-002_WEB.pdf
+ - Hardware Spec : http://www.marvell.com/embedded-processors/discovery-innovation/assets/HW_MV78200_OpenSource.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/discovery-innovation/assets/FS_MV76100_78100_78200_OpenSource.pdf
+ - MV76100
+
+ Not supported by the Linux kernel.
+
+ Core:
+ Feroceon 88fr571-vd ARMv5 compatible
+
+ Linux kernel mach directory:
+ arch/arm/mach-mv78xx0
+ Linux kernel plat directory:
+ arch/arm/plat-orion
+
+EBU Armada family
+-----------------
+
+ Armada 370 Flavors:
+ - 88F6710
+ - 88F6707
+ - 88F6W11
+
+ - Product Brief: http://www.marvell.com/embedded-processors/armada-300/assets/Marvell_ARMADA_370_SoC.pdf
+ - Hardware Spec: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA370-datasheet.pdf
+ - Functional Spec: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA370-FunctionalSpec-datasheet.pdf
+
+ Core:
+ Sheeva ARMv7 compatible PJ4B
+
+ Armada 375 Flavors:
+ - 88F6720
+
+ - Product Brief: http://www.marvell.com/embedded-processors/armada-300/assets/ARMADA_375_SoC-01_product_brief.pdf
+
+ Core:
+ ARM Cortex-A9
+
+ Armada 38x Flavors:
+ - 88F6810 Armada 380
+ - 88F6820 Armada 385
+ - 88F6828 Armada 388
+
+ - Product infos: http://www.marvell.com/embedded-processors/armada-38x/
+ - Functional Spec: https://marvellcorp.wufoo.com/forms/marvell-armada-38x-functional-specifications/
+
+ Core:
+ ARM Cortex-A9
+
+ Armada 39x Flavors:
+ - 88F6920 Armada 390
+ - 88F6928 Armada 398
+
+ - Product infos: http://www.marvell.com/embedded-processors/armada-39x/
+
+ Core:
+ ARM Cortex-A9
+
+ Armada XP Flavors:
+ - MV78230
+ - MV78260
+ - MV78460
+
+ NOTE:
+ not to be confused with the non-SMP 78xx0 SoCs
+
+ Product Brief:
+ http://www.marvell.com/embedded-processors/armada-xp/assets/Marvell-ArmadaXP-SoC-product%20brief.pdf
+
+ Functional Spec:
+ http://www.marvell.com/embedded-processors/armada-xp/assets/ARMADA-XP-Functional-SpecDatasheet.pdf
+
+ - Hardware Specs:
+
+ - http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78230_OS.PDF
+ - http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78260_OS.PDF
+ - http://www.marvell.com/embedded-processors/armada-xp/assets/HW_MV78460_OS.PDF
+
+ Core:
+ Sheeva ARMv7 compatible Dual-core or Quad-core PJ4B-MP
+
+ Linux kernel mach directory:
+ arch/arm/mach-mvebu
+ Linux kernel plat directory:
+ none
+
+EBU Armada family ARMv8
+-----------------------
+
+ Armada 3710/3720 Flavors:
+ - 88F3710
+ - 88F3720
+
+ Core:
+ ARM Cortex A53 (ARMv8)
+
+ Homepage:
+ http://www.marvell.com/embedded-processors/armada-3700/
+
+ Product Brief:
+ http://www.marvell.com/embedded-processors/assets/PB-88F3700-FNL.pdf
+
+ Device tree files:
+ arch/arm64/boot/dts/marvell/armada-37*
+
+ Armada 7K Flavors:
+ - 88F7020 (AP806 Dual + one CP110)
+ - 88F7040 (AP806 Quad + one CP110)
+
+ Core: ARM Cortex A72
+
+ Homepage:
+ http://www.marvell.com/embedded-processors/armada-70xx/
+
+ Product Brief:
+ - http://www.marvell.com/embedded-processors/assets/Armada7020PB-Jan2016.pdf
+ - http://www.marvell.com/embedded-processors/assets/Armada7040PB-Jan2016.pdf
+
+ Device tree files:
+ arch/arm64/boot/dts/marvell/armada-70*
+
+ Armada 8K Flavors:
+ - 88F8020 (AP806 Dual + two CP110)
+ - 88F8040 (AP806 Quad + two CP110)
+ Core:
+ ARM Cortex A72
+
+ Homepage:
+ http://www.marvell.com/embedded-processors/armada-80xx/
+
+ Product Brief:
+ - http://www.marvell.com/embedded-processors/assets/Armada8020PB-Jan2016.pdf
+ - http://www.marvell.com/embedded-processors/assets/Armada8040PB-Jan2016.pdf
+
+ Device tree files:
+ arch/arm64/boot/dts/marvell/armada-80*
+
+Avanta family
+-------------
+
+ Flavors:
+ - 88F6510
+ - 88F6530P
+ - 88F6550
+ - 88F6560
+
+ Homepage:
+ http://www.marvell.com/broadband/
+
+ Product Brief:
+ http://www.marvell.com/broadband/assets/Marvell_Avanta_88F6510_305_060-001_product_brief.pdf
+
+ No public datasheet available.
+
+ Core:
+ ARMv5 compatible
+
+ Linux kernel mach directory:
+ no code in mainline yet, planned for the future
+ Linux kernel plat directory:
+ no code in mainline yet, planned for the future
+
+Storage family
+--------------
+
+ Armada SP:
+ - 88RC1580
+
+ Product infos:
+ http://www.marvell.com/storage/armada-sp/
+
+ Core:
+ Sheeva ARMv7 comatible Quad-core PJ4C
+
+ (not supported in upstream Linux kernel)
+
+Dove family (application processor)
+-----------------------------------
+
+ Flavors:
+ - 88AP510 a.k.a Armada 510
+
+ Product Brief:
+ http://www.marvell.com/application-processors/armada-500/assets/Marvell_Armada510_SoC.pdf
+
+ Hardware Spec:
+ http://www.marvell.com/application-processors/armada-500/assets/Armada-510-Hardware-Spec.pdf
+
+ Functional Spec:
+ http://www.marvell.com/application-processors/armada-500/assets/Armada-510-Functional-Spec.pdf
+
+ Homepage:
+ http://www.marvell.com/application-processors/armada-500/
+
+ Core:
+ ARMv7 compatible
+
+ Directory:
+ - arch/arm/mach-mvebu (DT enabled platforms)
+ - arch/arm/mach-dove (non-DT enabled platforms)
+
+PXA 2xx/3xx/93x/95x family
+--------------------------
+
+ Flavors:
+ - PXA21x, PXA25x, PXA26x
+ - Application processor only
+ - Core: ARMv5 XScale1 core
+ - PXA270, PXA271, PXA272
+ - Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_pb.pdf
+ - Design guide : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_design_guide.pdf
+ - Developers manual : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_dev_man.pdf
+ - Specification : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_emts.pdf
+ - Specification update : http://www.marvell.com/application-processors/pxa-family/assets/pxa_27x_spec_update.pdf
+ - Application processor only
+ - Core: ARMv5 XScale2 core
+ - PXA300, PXA310, PXA320
+ - PXA 300 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA300_PB_R4.pdf
+ - PXA 310 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA310_PB_R4.pdf
+ - PXA 320 Product Brief : http://www.marvell.com/application-processors/pxa-family/assets/PXA320_PB_R4.pdf
+ - Design guide : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Design_Guide.pdf
+ - Developers manual : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Developers_Manual.zip
+ - Specifications : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_EMTS.pdf
+ - Specification Update : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_Spec_Update.zip
+ - Reference Manual : http://www.marvell.com/application-processors/pxa-family/assets/PXA3xx_TavorP_BootROM_Ref_Manual.pdf
+ - Application processor only
+ - Core: ARMv5 XScale3 core
+ - PXA930, PXA935
+ - Application processor with Communication processor
+ - Core: ARMv5 XScale3 core
+ - PXA955
+ - Application processor with Communication processor
+ - Core: ARMv7 compatible Sheeva PJ4 core
+
+ Comments:
+
+ * This line of SoCs originates from the XScale family developed by
+ Intel and acquired by Marvell in ~2006. The PXA21x, PXA25x,
+ PXA26x, PXA27x, PXA3xx and PXA93x were developed by Intel, while
+ the later PXA95x were developed by Marvell.
+
+ * Due to their XScale origin, these SoCs have virtually nothing in
+ common with the other (Kirkwood, Dove, etc.) families of Marvell
+ SoCs, except with the MMP/MMP2 family of SoCs.
+
+ Linux kernel mach directory:
+ arch/arm/mach-pxa
+ Linux kernel plat directory:
+ arch/arm/plat-pxa
+
+MMP/MMP2/MMP3 family (communication processor)
+----------------------------------------------
+
+ Flavors:
+ - PXA168, a.k.a Armada 168
+ - Homepage : http://www.marvell.com/application-processors/armada-100/armada-168.jsp
+ - Product brief : http://www.marvell.com/application-processors/armada-100/assets/pxa_168_pb.pdf
+ - Hardware manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_datasheet.pdf
+ - Software manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_software_manual.pdf
+ - Specification update : http://www.marvell.com/application-processors/armada-100/assets/ARMADA16x_Spec_update.pdf
+ - Boot ROM manual : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_ref_manual.pdf
+ - App node package : http://www.marvell.com/application-processors/armada-100/assets/armada_16x_app_note_package.pdf
+ - Application processor only
+ - Core: ARMv5 compatible Marvell PJ1 88sv331 (Mohawk)
+ - PXA910/PXA920
+ - Homepage : http://www.marvell.com/communication-processors/pxa910/
+ - Product Brief : http://www.marvell.com/communication-processors/pxa910/assets/Marvell_PXA910_Platform-001_PB_final.pdf
+ - Application processor with Communication processor
+ - Core: ARMv5 compatible Marvell PJ1 88sv331 (Mohawk)
+ - PXA688, a.k.a. MMP2, a.k.a Armada 610
+ - Product Brief : http://www.marvell.com/application-processors/armada-600/assets/armada610_pb.pdf
+ - Application processor only
+ - Core: ARMv7 compatible Sheeva PJ4 88sv581x core
+ - PXA2128, a.k.a. MMP3 (OLPC XO4, Linux support not upstream)
+ - Product Brief : http://www.marvell.com/application-processors/armada/pxa2128/assets/Marvell-ARMADA-PXA2128-SoC-PB.pdf
+ - Application processor only
+ - Core: Dual-core ARMv7 compatible Sheeva PJ4C core
+ - PXA960/PXA968/PXA978 (Linux support not upstream)
+ - Application processor with Communication Processor
+ - Core: ARMv7 compatible Sheeva PJ4 core
+ - PXA986/PXA988 (Linux support not upstream)
+ - Application processor with Communication Processor
+ - Core: Dual-core ARMv7 compatible Sheeva PJ4B-MP core
+ - PXA1088/PXA1920 (Linux support not upstream)
+ - Application processor with Communication Processor
+ - Core: quad-core ARMv7 Cortex-A7
+ - PXA1908/PXA1928/PXA1936
+ - Application processor with Communication Processor
+ - Core: multi-core ARMv8 Cortex-A53
+
+ Comments:
+
+ * This line of SoCs originates from the XScale family developed by
+ Intel and acquired by Marvell in ~2006. All the processors of
+ this MMP/MMP2 family were developed by Marvell.
+
+ * Due to their XScale origin, these SoCs have virtually nothing in
+ common with the other (Kirkwood, Dove, etc.) families of Marvell
+ SoCs, except with the PXA family of SoCs listed above.
+
+ Linux kernel mach directory:
+ arch/arm/mach-mmp
+ Linux kernel plat directory:
+ arch/arm/plat-pxa
+
+Berlin family (Multimedia Solutions)
+-------------------------------------
+
+ - Flavors:
+ - 88DE3010, Armada 1000 (no Linux support)
+ - Core: Marvell PJ1 (ARMv5TE), Dual-core
+ - Product Brief: http://www.marvell.com.cn/digital-entertainment/assets/armada_1000_pb.pdf
+ - 88DE3005, Armada 1500 Mini
+ - Design name: BG2CD
+ - Core: ARM Cortex-A9, PL310 L2CC
+ - 88DE3006, Armada 1500 Mini Plus
+ - Design name: BG2CDP
+ - Core: Dual Core ARM Cortex-A7
+ - 88DE3100, Armada 1500
+ - Design name: BG2
+ - Core: Marvell PJ4B-MP (ARMv7), Tauros3 L2CC
+ - 88DE3114, Armada 1500 Pro
+ - Design name: BG2Q
+ - Core: Quad Core ARM Cortex-A9, PL310 L2CC
+ - 88DE3214, Armada 1500 Pro 4K
+ - Design name: BG3
+ - Core: ARM Cortex-A15, CA15 integrated L2CC
+ - 88DE3218, ARMADA 1500 Ultra
+ - Core: ARM Cortex-A53
+
+ Homepage: https://www.synaptics.com/products/multimedia-solutions
+ Directory: arch/arm/mach-berlin
+
+ Comments:
+
+ * This line of SoCs is based on Marvell Sheeva or ARM Cortex CPUs
+ with Synopsys DesignWare (IRQ, GPIO, Timers, ...) and PXA IP (SDHCI, USB, ETH, ...).
+
+ * The Berlin family was acquired by Synaptics from Marvell in 2017.
+
+CPU Cores
+---------
+
+The XScale cores were designed by Intel, and shipped by Marvell in the older
+PXA processors. Feroceon is a Marvell designed core that developed in-house,
+and that evolved into Sheeva. The XScale and Feroceon cores were phased out
+over time and replaced with Sheeva cores in later products, which subsequently
+got replaced with licensed ARM Cortex-A cores.
+
+ XScale 1
+ CPUID 0x69052xxx
+ ARMv5, iWMMXt
+ XScale 2
+ CPUID 0x69054xxx
+ ARMv5, iWMMXt
+ XScale 3
+ CPUID 0x69056xxx or 0x69056xxx
+ ARMv5, iWMMXt
+ Feroceon-1850 88fr331 "Mohawk"
+ CPUID 0x5615331x or 0x41xx926x
+ ARMv5TE, single issue
+ Feroceon-2850 88fr531-vd "Jolteon"
+ CPUID 0x5605531x or 0x41xx926x
+ ARMv5TE, VFP, dual-issue
+ Feroceon 88fr571-vd "Jolteon"
+ CPUID 0x5615571x
+ ARMv5TE, VFP, dual-issue
+ Feroceon 88fr131 "Mohawk-D"
+ CPUID 0x5625131x
+ ARMv5TE, single-issue in-order
+ Sheeva PJ1 88sv331 "Mohawk"
+ CPUID 0x561584xx
+ ARMv5, single-issue iWMMXt v2
+ Sheeva PJ4 88sv581x "Flareon"
+ CPUID 0x560f581x
+ ARMv7, idivt, optional iWMMXt v2
+ Sheeva PJ4B 88sv581x
+ CPUID 0x561f581x
+ ARMv7, idivt, optional iWMMXt v2
+ Sheeva PJ4B-MP / PJ4C
+ CPUID 0x562f584x
+ ARMv7, idivt/idiva, LPAE, optional iWMMXt v2 and/or NEON
+
+Long-term plans
+---------------
+
+ * Unify the mach-dove/, mach-mv78xx0/, mach-orion5x/ into the
+ mach-mvebu/ to support all SoCs from the Marvell EBU (Engineering
+ Business Unit) in a single mach-<foo> directory. The plat-orion/
+ would therefore disappear.
+
+ * Unify the mach-mmp/ and mach-pxa/ into the same mach-pxa
+ directory. The plat-pxa/ would therefore disappear.
+
+Credits
+-------
+
+- Maen Suleiman <maen@marvell.com>
+- Lior Amsalem <alior@marvell.com>
+- Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+- Andrew Lunn <andrew@lunn.ch>
+- Nicolas Pitre <nico@fluxnic.net>
+- Eric Miao <eric.y.miao@gmail.com>
+================
+Memory alignment
+================
+
Too many problems popped up because of unnoticed misaligned memory access in
kernel code lately. Therefore the alignment fixup is now unconditionally
configured in for SA11x0 based targets. According to Alan Cox, this is a
To change the alignment trap behavior, simply echo a number into
/proc/cpu/alignment. The number is made up from various bits:
+=== ========================================================
bit behavior when set
---- -----------------
-
+=== ========================================================
0 A user process performing an unaligned memory access
will cause the kernel to print a message indicating
process name, pid, pc, instruction, address, and the
2 The kernel will send a SIGBUS signal to the user process
performing the unaligned access.
+=== ========================================================
Note that not all combinations are supported - only values 0 through 5.
(6 and 7 don't make sense).
For example, the following will turn on the warnings, but without
-fixing up or sending SIGBUS signals:
+fixing up or sending SIGBUS signals::
echo 1 > /proc/cpu/alignment
- Kernel Memory Layout on ARM Linux
+=================================
+Kernel Memory Layout on ARM Linux
+=================================
Russell King <rmk@arm.linux.org.uk>
+
November 17, 2005 (2.6.15)
This document describes the virtual memory layout which the Linux
certain regions of VM space for use for new facilities; therefore
this document may reserve more VM space over time.
+=============== =============== ===============================================
Start End Use
---------------------------------------------------------------------------
+=============== =============== ===============================================
ffff8000 ffffffff copy_user_page / clear_user_page use.
For SA11xx and Xscale, this is used to
setup a minicache mapping.
place their vector page here. NULL pointer
dereferences by both the kernel and user
space are also caught via this mapping.
+=============== =============== ===============================================
Please note that mappings which collide with the above areas may result
in a non-bootable kernel, or may cause the kernel to (eventually) panic
+=============================
ARM Microchip SoCs (aka AT91)
=============================
Flavors:
* ARM 920 based SoC
- at91rm9200
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-1768-32-bit-ARM920T-Embedded-Microprocessor-AT91RM9200_Datasheet.pdf
* ARM 926 based SoCs
- at91sam9260
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-6221-32-bit-ARM926EJ-S-Embedded-Microprocessor-SAM9260_Datasheet.pdf
- at91sam9xe
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-6254-32-bit-ARM926EJ-S-Embedded-Microprocessor-SAM9XE_Datasheet.pdf
- at91sam9261
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-6062-ARM926EJ-S-Microprocessor-SAM9261_Datasheet.pdf
- at91sam9263
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-6249-32-bit-ARM926EJ-S-Embedded-Microprocessor-SAM9263_Datasheet.pdf
- at91sam9rl
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/doc6289.pdf
- at91sam9g20
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/DS60001516A.pdf
- at91sam9g45 family
- at91sam9g46
- at91sam9m10
- at91sam9m11 (device superset)
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-6437-32-bit-ARM926-Embedded-Microprocessor-SAM9M11_Datasheet.pdf
- at91sam9x5 family (aka "The 5 series")
- at91sam9g35
- at91sam9x25
- at91sam9x35
- + Datasheet (can be considered as covering the whole family)
+
+ * Datasheet (can be considered as covering the whole family)
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-11055-32-bit-ARM926EJ-S-Microcontroller-SAM9X35_Datasheet.pdf
- at91sam9n12
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/DS60001517A.pdf
* ARM Cortex-A5 based SoCs
- sama5d3 family
+
- sama5d31
- sama5d33
- sama5d34
- sama5d35
- sama5d36 (device superset)
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-11121-32-bit-Cortex-A5-Microcontroller-SAMA5D3_Datasheet.pdf
* ARM Cortex-A5 + NEON based SoCs
- sama5d4 family
+
- sama5d41
- sama5d42
- sama5d43
- sama5d44 (device superset)
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/60001525A.pdf
- sama5d2 family
+
- sama5d21
- sama5d22
- sama5d23
- sama5d26
- sama5d27 (device superset)
- sama5d28 (device superset + environmental monitors)
- + Datasheet
+
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/DS60001476B.pdf
* ARM Cortex-M7 MCUs
- sams70 family
+
- sams70j19
- sams70j20
- sams70j21
- sams70q21
- samv70 family
+
- samv70j19
- samv70j20
- samv70n19
- samv70q20
- samv71 family
+
- samv71j19
- samv71j20
- samv71j21
- samv71q20
- samv71q21
- + Datasheet
+ * Datasheet
+
http://ww1.microchip.com/downloads/en/DeviceDoc/60001527A.pdf
This statement will be removed by AT91 MAINTAINERS when appropriate.
Naming conventions and best practice:
+
- SoCs Device Tree Source Include files are named after the official name of
the product (at91sam9g20.dtsi or sama5d33.dtsi for instance).
- Device Tree Source Include files (.dtsi) are used to collect common nodes that can be
--- /dev/null
+================================
+NetWinder specific documentation
+================================
+
+The NetWinder is a small low-power computer, primarily designed
+to run Linux. It is based around the StrongARM RISC processor,
+DC21285 PCI bridge, with PC-type hardware glued around it.
+
+Port usage
+==========
+
+======= ====== ===============================
+Min Max Description
+======= ====== ===============================
+0x0000 0x000f DMA1
+0x0020 0x0021 PIC1
+0x0060 0x006f Keyboard
+0x0070 0x007f RTC
+0x0080 0x0087 DMA1
+0x0088 0x008f DMA2
+0x00a0 0x00a3 PIC2
+0x00c0 0x00df DMA2
+0x0180 0x0187 IRDA
+0x01f0 0x01f6 ide0
+0x0201 Game port
+0x0203 RWA010 configuration read
+0x0220 ? SoundBlaster
+0x0250 ? WaveArtist
+0x0279 RWA010 configuration index
+0x02f8 0x02ff Serial ttyS1
+0x0300 0x031f Ether10
+0x0338 GPIO1
+0x033a GPIO2
+0x0370 0x0371 W83977F configuration registers
+0x0388 ? AdLib
+0x03c0 0x03df VGA
+0x03f6 ide0
+0x03f8 0x03ff Serial ttyS0
+0x0400 0x0408 DC21143
+0x0480 0x0487 DMA1
+0x0488 0x048f DMA2
+0x0a79 RWA010 configuration write
+0xe800 0xe80f ide0/ide1 BM DMA
+======= ====== ===============================
+
+
+Interrupt usage
+===============
+
+======= ======= ========================
+IRQ type Description
+======= ======= ========================
+ 0 ISA 100Hz timer
+ 1 ISA Keyboard
+ 2 ISA cascade
+ 3 ISA Serial ttyS1
+ 4 ISA Serial ttyS0
+ 5 ISA PS/2 mouse
+ 6 ISA IRDA
+ 7 ISA Printer
+ 8 ISA RTC alarm
+ 9 ISA
+10 ISA GP10 (Orange reset button)
+11 ISA
+12 ISA WaveArtist
+13 ISA
+14 ISA hda1
+15 ISA
+======= ======= ========================
+
+DMA usage
+=========
+
+======= ======= ===========
+DMA type Description
+======= ======= ===========
+ 0 ISA IRDA
+ 1 ISA
+ 2 ISA cascade
+ 3 ISA WaveArtist
+ 4 ISA
+ 5 ISA
+ 6 ISA
+ 7 ISA WaveArtist
+======= ======= ===========
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===================================
+NetWinder's floating point emulator
+===================================
+
+.. toctree::
+ :maxdepth: 1
+
+ nwfpe
+ netwinder-fpe
+ notes
+ todo
+=============
+Current State
+=============
+
The following describes the current state of the NetWinder's floating point
emulator.
In the following nomenclature is used to describe the floating point
instructions. It follows the conventions in the ARM manual.
-<S|D|E> = <single|double|extended>, no default
-{P|M|Z} = {round to +infinity,round to -infinity,round to zero},
- default = round to nearest
+::
+
+ <S|D|E> = <single|double|extended>, no default
+ {P|M|Z} = {round to +infinity,round to -infinity,round to zero},
+ default = round to nearest
Note: items enclosed in {} are optional.
<LFM|SFM>{cond}<FD,EA> Fd, <count>, [Rn]{!}
These instructions are fully implemented. They store/load three words
-for each floating point register into the memory location given in the
+for each floating point register into the memory location given in the
instruction. The format in memory is unlikely to be compatible with
other implementations, in particular the actual hardware. Specific
-mention of this is made in the ARM manuals.
+mention of this is made in the ARM manuals.
Floating Point Coprocessor Register Transfer Instructions (CPRT)
----------------------------------------------------------------
POL{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - polar angle (arctan2)
LOG{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base 10
-LGN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base e
+LGN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base e
EXP{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - exponent
SIN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - sine
COS{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - cosine
These are not implemented. They are not currently issued by the compiler,
and are handled by routines in libc. These are not implemented by the FPA11
-hardware, but are handled by the floating point support code. They should
+hardware, but are handled by the floating point support code. They should
be implemented in future versions.
Signalling:
The kernel provided with this distribution (vmlinux-nwfpe-0.93) contains
a fix for this problem and also incorporates the current version of the
emulator directly. It is possible to run with no floating point module
-loaded with this kernel. It is provided as a demonstration of the
+loaded with this kernel. It is provided as a demonstration of the
technology and for those who want to do floating point work that depends
on signals. It is not strictly necessary to use the module.
-A module (either the one provided by Russell King, or the one in this
+A module (either the one provided by Russell King, or the one in this
distribution) can be loaded to replace the functionality of the emulator
built into the kernel.
+Notes
+=====
+
There seems to be a problem with exp(double) and our emulator. I haven't
been able to track it down yet. This does not occur with the emulator
supplied by Russell King.
-This directory contains the version 0.92 test release of the NetWinder
+Introduction
+============
+
+This directory contains the version 0.92 test release of the NetWinder
Floating Point Emulator.
The majority of the code was written by me, Scott Bambrough It is
port of SoftFloat version 1 by Neil Carson for NetBSD/arm32.
The file README.FPE contains a description of what has been implemented
-so far in the emulator. The file TODO contains a information on what
+so far in the emulator. The file TODO contains a information on what
remains to be done, and other ideas for the emulator.
Bug reports, comments, suggestions should be directed to me at
The NetWinder Floating Point Emulator is free software. Everything Rebel.com
has written is provided under the GNU GPL. See the file COPYING for copying
-conditions. Excluded from the above is the SoftFloat code. John Hauser's
+conditions. Excluded from the above is the SoftFloat code. John Hauser's
legal notice for SoftFloat is included below.
-------------------------------------------------------------------------------
+
SoftFloat Legal Notice
SoftFloat was written by John R. Hauser. This work was made possible in
TODO LIST
----------
+=========
-POW{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - power
-RPW{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - reverse power
-POL{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - polar angle (arctan2)
+::
-LOG{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base 10
-LGN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base e
-EXP{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - exponent
-SIN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - sine
-COS{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - cosine
-TAN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - tangent
-ASN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arcsine
-ACS{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arccosine
-ATN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arctangent
+ POW{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - power
+ RPW{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - reverse power
+ POL{cond}<S|D|E>{P,M,Z} Fd, Fn, <Fm,#value> - polar angle (arctan2)
+
+ LOG{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base 10
+ LGN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - logarithm to base e
+ EXP{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - exponent
+ SIN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - sine
+ COS{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - cosine
+ TAN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - tangent
+ ASN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arcsine
+ ACS{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arccosine
+ ATN{cond}<S|D|E>{P,M,Z} Fd, <Fm,#value> - arctangent
These are not implemented. They are not currently issued by the compiler,
and are handled by routines in libc. These are not implemented by the FPA11
-hardware, but are handled by the floating point support code. They should
+hardware, but are handled by the floating point support code. They should
be implemented in future versions.
There are a couple of ways to approach the implementation of these. One
-method would be to use accurate table methods for these routines. I have
+method would be to use accurate table methods for these routines. I have
a couple of papers by S. Gal from IBM's research labs in Haifa, Israel that
seem to promise extreme accuracy (in the order of 99.8%) and reasonable speed.
These methods are used in GLIBC for some of the transcendental functions.
Another approach, which I know little about is CORDIC. This stands for
-Coordinate Rotation Digital Computer, and is a method of computing
+Coordinate Rotation Digital Computer, and is a method of computing
transcendental functions using mostly shifts and adds and a few
multiplications and divisions. The ARM excels at shifts and adds,
-so such a method could be promising, but requires more research to
+so such a method could be promising, but requires more research to
determine if it is feasible.
Rounding Methods
+----------------
The IEEE standard defines 4 rounding modes. Round to nearest is the
default, but rounding to + or - infinity or round to zero are also allowed.
the rounding mode one must specify it with each instruction.
This has made porting some benchmarks difficult. It is possible to
-introduce such a capability into the emulator. The FPCR contains
-bits describing the rounding mode. The emulator could be altered to
+introduce such a capability into the emulator. The FPCR contains
+bits describing the rounding mode. The emulator could be altered to
examine a flag, which if set forced it to ignore the rounding mode in
the instruction, and use the mode specified in the bits in the FPCR.
supervisor only instructions. If anyone has any ideas or comments I
would like to hear them.
-[NOTE: pulled out from some docs on ARM floating point, specifically
+NOTE:
+ pulled out from some docs on ARM floating point, specifically
for the Acorn FPE, but not limited to it:
The floating point control register (FPCR) may only be present in some
Hence, the answer is yes, you could do this, but then you will run a high
risk of becoming isolated if and when hardware FP emulation comes out
- -- Russell].
+
+ -- Russell.
+=========================
OMAP2/3 Display Subsystem
--------------------------
+=========================
This is an almost total rewrite of the OMAP FB driver in drivers/video/omap
(let's call it DSS1). The main differences between DSS1 and DSS2 are DSI,
default_color default background color (RGB24)
/sys/devices/platform/omapdss/display? directory:
+
+=============== =============================================================
ctrl_name Controller name
mirror 0=off, 1=on
update_mode 0=off, 1=auto, 2=manual
panel_name
tear_elim Tearing elimination 0=off, 1=on
output_type Output type (video encoder only): "composite" or "svideo"
+=============== =============================================================
There are also some debugfs files at <debugfs>/omapdss/ which show information
about clocks and registers.
Examples
--------
-The following definitions have been made for the examples below:
+The following definitions have been made for the examples below::
-ovl0=/sys/devices/platform/omapdss/overlay0
-ovl1=/sys/devices/platform/omapdss/overlay1
-ovl2=/sys/devices/platform/omapdss/overlay2
+ ovl0=/sys/devices/platform/omapdss/overlay0
+ ovl1=/sys/devices/platform/omapdss/overlay1
+ ovl2=/sys/devices/platform/omapdss/overlay2
-mgr0=/sys/devices/platform/omapdss/manager0
-mgr1=/sys/devices/platform/omapdss/manager1
+ mgr0=/sys/devices/platform/omapdss/manager0
+ mgr1=/sys/devices/platform/omapdss/manager1
-lcd=/sys/devices/platform/omapdss/display0
-dvi=/sys/devices/platform/omapdss/display1
-tv=/sys/devices/platform/omapdss/display2
+ lcd=/sys/devices/platform/omapdss/display0
+ dvi=/sys/devices/platform/omapdss/display1
+ tv=/sys/devices/platform/omapdss/display2
-fb0=/sys/class/graphics/fb0
-fb1=/sys/class/graphics/fb1
-fb2=/sys/class/graphics/fb2
+ fb0=/sys/class/graphics/fb0
+ fb1=/sys/class/graphics/fb1
+ fb2=/sys/class/graphics/fb2
Default setup on OMAP3 SDP
--------------------------
Here's the default setup on OMAP3 SDP board. All planes go to LCD. DVI
and TV-out are not in use. The columns from left to right are:
framebuffers, overlays, overlay managers, displays. Framebuffers are
-handled by omapfb, and the rest by the DSS.
+handled by omapfb, and the rest by the DSS::
-FB0 --- GFX -\ DVI
-FB1 --- VID1 --+- LCD ---- LCD
-FB2 --- VID2 -/ TV ----- TV
+ FB0 --- GFX -\ DVI
+ FB1 --- VID1 --+- LCD ---- LCD
+ FB2 --- VID2 -/ TV ----- TV
Example: Switch from LCD to DVI
-----------------------
+-------------------------------
+
+::
-w=`cat $dvi/timings | cut -d "," -f 2 | cut -d "/" -f 1`
-h=`cat $dvi/timings | cut -d "," -f 3 | cut -d "/" -f 1`
+ w=`cat $dvi/timings | cut -d "," -f 2 | cut -d "/" -f 1`
+ h=`cat $dvi/timings | cut -d "," -f 3 | cut -d "/" -f 1`
-echo "0" > $lcd/enabled
-echo "" > $mgr0/display
-fbset -fb /dev/fb0 -xres $w -yres $h -vxres $w -vyres $h
-# at this point you have to switch the dvi/lcd dip-switch from the omap board
-echo "dvi" > $mgr0/display
-echo "1" > $dvi/enabled
+ echo "0" > $lcd/enabled
+ echo "" > $mgr0/display
+ fbset -fb /dev/fb0 -xres $w -yres $h -vxres $w -vyres $h
+ # at this point you have to switch the dvi/lcd dip-switch from the omap board
+ echo "dvi" > $mgr0/display
+ echo "1" > $dvi/enabled
-After this the configuration looks like:
+After this the configuration looks like:::
-FB0 --- GFX -\ -- DVI
-FB1 --- VID1 --+- LCD -/ LCD
-FB2 --- VID2 -/ TV ----- TV
+ FB0 --- GFX -\ -- DVI
+ FB1 --- VID1 --+- LCD -/ LCD
+ FB2 --- VID2 -/ TV ----- TV
Example: Clone GFX overlay to LCD and TV
--------------------------------
+----------------------------------------
+
+::
-w=`cat $tv/timings | cut -d "," -f 2 | cut -d "/" -f 1`
-h=`cat $tv/timings | cut -d "," -f 3 | cut -d "/" -f 1`
+ w=`cat $tv/timings | cut -d "," -f 2 | cut -d "/" -f 1`
+ h=`cat $tv/timings | cut -d "," -f 3 | cut -d "/" -f 1`
-echo "0" > $ovl0/enabled
-echo "0" > $ovl1/enabled
+ echo "0" > $ovl0/enabled
+ echo "0" > $ovl1/enabled
-echo "" > $fb1/overlays
-echo "0,1" > $fb0/overlays
+ echo "" > $fb1/overlays
+ echo "0,1" > $fb0/overlays
-echo "$w,$h" > $ovl1/output_size
-echo "tv" > $ovl1/manager
+ echo "$w,$h" > $ovl1/output_size
+ echo "tv" > $ovl1/manager
-echo "1" > $ovl0/enabled
-echo "1" > $ovl1/enabled
+ echo "1" > $ovl0/enabled
+ echo "1" > $ovl1/enabled
-echo "1" > $tv/enabled
+ echo "1" > $tv/enabled
-After this the configuration looks like (only relevant parts shown):
+After this the configuration looks like (only relevant parts shown)::
-FB0 +-- GFX ---- LCD ---- LCD
- \- VID1 ---- TV ---- TV
+ FB0 +-- GFX ---- LCD ---- LCD
+ \- VID1 ---- TV ---- TV
Misc notes
----------
DSS locking
Error checking
+
- Lots of checks are missing or implemented just as BUG()
System DMA update for DSI
+
- Can be used for RGB16 and RGB24P modes. Probably not for RGB24U (how
to skip the empty byte?)
OMAP1 support
-- Not sure if needed
+- Not sure if needed
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=======
+TI OMAP
+=======
+
+.. toctree::
+ :maxdepth: 1
+
+ omap
+ omap_pm
+ dss
+============
+OMAP history
+============
+
This file contains documentation for running mainline
kernel on omaps.
+====== ======================================================
KERNEL NEW DEPENDENCIES
+====== ======================================================
v4.3+ Update is needed for custom .config files to make sure
CONFIG_REGULATOR_PBIAS is enabled for MMC1 to work
properly.
v4.18+ Update is needed for custom .config files to make sure
CONFIG_MMC_SDHCI_OMAP is enabled for all MMC instances
to work in DRA7 and K2G based boards.
+====== ======================================================
-
+=====================
The OMAP PM interface
=====================
This document proposes the OMAP PM interface, including the following
five power management functions for driver code:
-1. Set the maximum MPU wakeup latency:
+1. Set the maximum MPU wakeup latency::
+
(*pdata->set_max_mpu_wakeup_lat)(struct device *dev, unsigned long t)
-2. Set the maximum device wakeup latency:
+2. Set the maximum device wakeup latency::
+
(*pdata->set_max_dev_wakeup_lat)(struct device *dev, unsigned long t)
-3. Set the maximum system DMA transfer start latency (CORE pwrdm):
+3. Set the maximum system DMA transfer start latency (CORE pwrdm)::
+
(*pdata->set_max_sdma_lat)(struct device *dev, long t)
-4. Set the minimum bus throughput needed by a device:
+4. Set the minimum bus throughput needed by a device::
+
(*pdata->set_min_bus_tput)(struct device *dev, u8 agent_id, unsigned long r)
-5. Return the number of times the device has lost context
+5. Return the number of times the device has lost context::
+
(*pdata->get_dev_context_loss_count)(struct device *dev)
As the 'pdata' in the above examples indicates, these functions are
exposed to drivers through function pointers in driver .platform_data
-structures. The function pointers are initialized by the board-*.c
+structures. The function pointers are initialized by the `board-*.c`
files to point to the corresponding OMAP PM functions:
-.set_max_dev_wakeup_lat will point to
-omap_pm_set_max_dev_wakeup_lat(), etc. Other architectures which do
-not support these functions should leave these function pointers set
-to NULL. Drivers should use the following idiom:
+
+- set_max_dev_wakeup_lat will point to
+ omap_pm_set_max_dev_wakeup_lat(), etc. Other architectures which do
+ not support these functions should leave these function pointers set
+ to NULL. Drivers should use the following idiom::
if (pdata->set_max_dev_wakeup_lat)
(*pdata->set_max_dev_wakeup_lat)(dev, t);
set_max_mpu_wakeup_lat() function to constrain the MPU wakeup
latency, and the set_max_dev_wakeup_lat() function to constrain the
device wakeup latency (from clk_enable() to accessibility). For
-example,
+example::
/* Limit MPU wakeup latency */
if (pdata->set_max_mpu_wakeup_lat)
frequency) into the form that the underlying power management
implementation needs:
-6. (*pdata->dsp_get_opp_table)(void)
+6. `(*pdata->dsp_get_opp_table)(void)`
-7. (*pdata->dsp_set_min_opp)(u8 opp_id)
+7. `(*pdata->dsp_set_min_opp)(u8 opp_id)`
-8. (*pdata->dsp_get_opp)(void)
+8. `(*pdata->dsp_get_opp)(void)`
-9. (*pdata->cpu_get_freq_table)(void)
+9. `(*pdata->cpu_get_freq_table)(void)`
-10. (*pdata->cpu_set_freq)(unsigned long f)
+10. `(*pdata->cpu_set_freq)(unsigned long f)`
-11. (*pdata->cpu_get_freq)(void)
+11. `(*pdata->cpu_get_freq)(void)`
Customizing OPP for platform
============================
and the registration of OPP table should take place automatically.
However, in special cases, the default OPP table may need to be
tweaked, for e.g.:
+
* enable default OPPs which are disabled by default, but which
could be enabled on a platform
* Disable an unsupported OPP on the platform
* Define and add a custom opp table entry
-in these cases, the board file needs to do additional steps as follows:
-arch/arm/mach-omapx/board-xyz.c
+ in these cases, the board file needs to do additional steps as follows:
+
+arch/arm/mach-omapx/board-xyz.c::
+
#include "pm.h"
....
static void __init omap_xyz_init_irq(void)
/* Do customization to the defaults */
....
}
-NOTE: omapx_opp_init will be omap3_opp_init or as required
-based on the omap family.
+
+NOTE:
+ omapx_opp_init will be omap3_opp_init or as required
+ based on the omap family.
+=======
+Porting
+=======
+
Taken from list archive at http://lists.arm.linux.org.uk/pipermail/linux-arm-kernel/2001-July/004064.html
Initial definitions
Virtual address for the kernel data segment. Must not be defined
when using the decompressor.
-VMALLOC_START
-VMALLOC_END
+VMALLOC_START / VMALLOC_END
Virtual addresses bounding the vmalloc() area. There must not be
any static mappings in this area; vmalloc will overwrite them.
The addresses must also be in the kernel segment (see above).
----------------------------
BOOT_MEM(pram,pio,vio)
- `pram' specifies the physical start address of RAM. Must always
+ `pram` specifies the physical start address of RAM. Must always
be present, and should be the same as PHYS_OFFSET.
- `pio' is the physical address of an 8MB region containing IO for
+ `pio` is the physical address of an 8MB region containing IO for
use with the debugging macros in arch/arm/kernel/debug-armv.S.
- `vio' is the virtual address of the 8MB debugging region.
+ `vio` is the virtual address of the 8MB debugging region.
It is expected that the debugging region will be re-initialised
by the architecture specific code later in the code (via the
INITIRQ(func)
Machine specific function to initialise interrupts.
-
- MFP Configuration for PXA2xx/PXA3xx Processors
+==============================================
+MFP Configuration for PXA2xx/PXA3xx Processors
+==============================================
Eric Miao <eric.miao@marvell.com>
later PXA series processors. This document describes the existing MFP API,
and how board/platform driver authors could make use of it.
- Basic Concept
-===============
+Basic Concept
+=============
Unlike the GPIO alternate function settings on PXA25x and PXA27x, a new MFP
mechanism is introduced from PXA3xx to completely move the pin-mux functions
out of the GPIO controller. In addition to pin-mux configurations, the MFP
also controls the low power state, driving strength, pull-up/down and event
detection of each pin. Below is a diagram of internal connections between
-the MFP logic and the remaining SoC peripherals:
+the MFP logic and the remaining SoC peripherals::
+--------+
| |--(GPIO19)--+
mean it is a GPIO signal, and by MFP<xxx> or pin xxx, we mean a physical
pad (or ball).
- MFP API Usage
-===============
+MFP API Usage
+=============
For board code writers, here are some guidelines:
PXA310 supporting some additional ones), thus the difference is actually
covered in a single mfp-pxa300.h.
-2. prepare an array for the initial pin configurations, e.g.:
+2. prepare an array for the initial pin configurations, e.g.::
- static unsigned long mainstone_pin_config[] __initdata = {
+ static unsigned long mainstone_pin_config[] __initdata = {
/* Chip Select */
GPIO15_nCS_1,
/* GPIO */
GPIO1_GPIO | WAKEUP_ON_EDGE_BOTH,
- };
+ };
a) once the pin configurations are passed to pxa{2xx,3xx}_mfp_config(),
and written to the actual registers, they are useless and may discard,
d) although PXA3xx MFP supports edge detection on each pin, the
internal logic will only wakeup the system when those specific bits
in ADxER registers are set, which can be well mapped to the
- corresponding peripheral, thus set_irq_wake() can be called with
+ corresponding peripheral, thus set_irq_wake() can be called with
the peripheral IRQ to enable the wakeup.
- MFP on PXA3xx
-===============
+MFP on PXA3xx
+=============
Every external I/O pad on PXA3xx (excluding those for special purpose) has
one MFP logic associated, and is controlled by one MFP register (MFPR).
-The MFPR has the following bit definitions (for PXA300/PXA310/PXA320):
+The MFPR has the following bit definitions (for PXA300/PXA310/PXA320)::
31 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
+-------------------------+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
0b006 - slow 10mA
0b007 - fast 10mA
- MFP Design for PXA2xx/PXA3xx
-==============================
+MFP Design for PXA2xx/PXA3xx
+============================
Due to the difference of pin-mux handling between PXA2xx and PXA3xx, a unified
MFP API is introduced to cover both series of processors.
the actual API invoked to convert these definitions into register settings and
make them effective there-after.
- Files Involved
- --------------
+Files Involved
+--------------
- arch/arm/mach-pxa/include/mach/mfp.h
-
+
for
1. Unified pin definitions - enum constants for all configurable pins
2. processor-neutral bit definitions for a possible MFP configuration
for implementation of the pin configuration to take effect for the actual
processor.
- Pin Configuration
- -----------------
+Pin Configuration
+-----------------
The following comments are copied from mfp.h (see the actual source code
- for most updated info)
-
- /*
- * a possible MFP configuration is represented by a 32-bit integer
- *
- * bit 0.. 9 - MFP Pin Number (1024 Pins Maximum)
- * bit 10..12 - Alternate Function Selection
- * bit 13..15 - Drive Strength
- * bit 16..18 - Low Power Mode State
- * bit 19..20 - Low Power Mode Edge Detection
- * bit 21..22 - Run Mode Pull State
- *
- * to facilitate the definition, the following macros are provided
- *
- * MFP_CFG_DEFAULT - default MFP configuration value, with
- * alternate function = 0,
- * drive strength = fast 3mA (MFP_DS03X)
- * low power mode = default
- * edge detection = none
- *
- * MFP_CFG - default MFPR value with alternate function
- * MFP_CFG_DRV - default MFPR value with alternate function and
- * pin drive strength
- * MFP_CFG_LPM - default MFPR value with alternate function and
- * low power mode
- * MFP_CFG_X - default MFPR value with alternate function,
- * pin drive strength and low power mode
- */
-
- Examples of pin configurations are:
-
- #define GPIO94_SSP3_RXD MFP_CFG_X(GPIO94, AF1, DS08X, FLOAT)
+ for most updated info)::
+
+ /*
+ * a possible MFP configuration is represented by a 32-bit integer
+ *
+ * bit 0.. 9 - MFP Pin Number (1024 Pins Maximum)
+ * bit 10..12 - Alternate Function Selection
+ * bit 13..15 - Drive Strength
+ * bit 16..18 - Low Power Mode State
+ * bit 19..20 - Low Power Mode Edge Detection
+ * bit 21..22 - Run Mode Pull State
+ *
+ * to facilitate the definition, the following macros are provided
+ *
+ * MFP_CFG_DEFAULT - default MFP configuration value, with
+ * alternate function = 0,
+ * drive strength = fast 3mA (MFP_DS03X)
+ * low power mode = default
+ * edge detection = none
+ *
+ * MFP_CFG - default MFPR value with alternate function
+ * MFP_CFG_DRV - default MFPR value with alternate function and
+ * pin drive strength
+ * MFP_CFG_LPM - default MFPR value with alternate function and
+ * low power mode
+ * MFP_CFG_X - default MFPR value with alternate function,
+ * pin drive strength and low power mode
+ */
+
+ Examples of pin configurations are::
+
+ #define GPIO94_SSP3_RXD MFP_CFG_X(GPIO94, AF1, DS08X, FLOAT)
which reads GPIO94 can be configured as SSP3_RXD, with alternate function
selection of 1, driving strength of 0b101, and a float state in low power
do so, simply because this default setting is usually carefully encoded,
and is supposed to work in most cases.
- Register Settings
- -----------------
+Register Settings
+-----------------
Register settings on PXA3xx for a pin configuration is actually very
straight-forward, most bits can be converted directly into MFPR value
+===============================
ADS Bitsy Single Board Computer
+===============================
+
(It is different from Bitsy(iPAQ) of Compaq)
For more details, contact Applied Data Systems or see
Linux can be used with the ADS BootLoader that ships with the
newer rev boards. See their documentation on how to load Linux.
-Supported peripherals:
+Supported peripherals
+=====================
+
- SA1100 LCD frame buffer (8/16bpp...sort of)
- SA1111 USB Master
- SA1100 serial port
- serial ports (ttyS[0-2])
- ttyS0 is default for serial console
-To do:
+To do
+=====
+
- everything else! :-)
-Notes:
+Notes
+=====
- The flash on board is divided into 3 partitions.
You should be careful to use flash on board.
+============================================
The Intel Assabet (SA-1110 evaluation) board
============================================
Building the kernel
-------------------
-To build the kernel with current defaults:
+To build the kernel with current defaults::
make assabet_config
make oldconfig
you need to have RedBoot installed in your flash memory. A known to work
precompiled RedBoot binary is available from the following location:
-ftp://ftp.netwinder.org/users/n/nico/
-ftp://ftp.arm.linux.org.uk/pub/linux/arm/people/nico/
-ftp://ftp.handhelds.org/pub/linux/arm/sa-1100-patches/
+- ftp://ftp.netwinder.org/users/n/nico/
+- ftp://ftp.arm.linux.org.uk/pub/linux/arm/people/nico/
+- ftp://ftp.handhelds.org/pub/linux/arm/sa-1100-patches/
Look for redboot-assabet*.tgz. Some installation infos are provided in
redboot-assabet*.txt.
file transfers. You must insert it before RedBoot runs since it can't detect
it dynamically.
-To initialize the flash directory:
+To initialize the flash directory::
fis init -f
To initialize the non-volatile settings, like whether you want to use BOOTP or
-a static IP address, etc, use this command:
+a static IP address, etc, use this command::
fconfig -i
---------------------------------
First, the kernel image must be loaded into RAM. If you have the zImage file
-available on a TFTP server:
+available on a TFTP server::
load zImage -r -b 0x100000
-If you rather want to use Y-Modem upload over the serial port:
+If you rather want to use Y-Modem upload over the serial port::
load -m ymodem -r -b 0x100000
-To write it to flash:
+To write it to flash::
fis create "Linux kernel" -b 0x100000 -l 0xc0000
------------------
The kernel still requires a filesystem to boot. A ramdisk image can be loaded
-as follows:
+as follows::
load ramdisk_image.gz -r -b 0x800000
Again, Y-Modem upload can be used instead of TFTP by replacing the file name
by '-y ymodem'.
-Now the kernel can be retrieved from flash like this:
+Now the kernel can be retrieved from flash like this::
fis load "Linux kernel"
-or loaded as described previously. To boot the kernel:
+or loaded as described previously. To boot the kernel::
exec -b 0x100000 -l 0xc0000
For instance, a sample JFFS2 image can be retrieved from the same FTP sites
mentioned below for the precompiled RedBoot image.
-To load this file:
+To load this file::
load sample_img.jffs2 -r -b 0x100000
-The result should look like:
+The result should look like::
-RedBoot> load sample_img.jffs2 -r -b 0x100000
-Raw file loaded 0x00100000-0x00377424
+ RedBoot> load sample_img.jffs2 -r -b 0x100000
+ Raw file loaded 0x00100000-0x00377424
-Now we must know the size of the unallocated flash:
+Now we must know the size of the unallocated flash::
fis free
-Result:
+Result::
-RedBoot> fis free
- 0x500E0000 .. 0x503C0000
+ RedBoot> fis free
+ 0x500E0000 .. 0x503C0000
The values above may be different depending on the size of the filesystem and
the type of flash. See their usage below as an example and take care of
substituting yours appropriately.
-We must determine some values:
+We must determine some values::
-size of unallocated flash: 0x503c0000 - 0x500e0000 = 0x2e0000
-size of the filesystem image: 0x00377424 - 0x00100000 = 0x277424
+ size of unallocated flash: 0x503c0000 - 0x500e0000 = 0x2e0000
+ size of the filesystem image: 0x00377424 - 0x00100000 = 0x277424
We want to fit the filesystem image of course, but we also want to give it all
-the remaining flash space as well. To write it:
+the remaining flash space as well. To write it::
fis unlock -f 0x500E0000 -l 0x2e0000
fis erase -f 0x500E0000 -l 0x2e0000
Now the filesystem is associated to a MTD "partition" once Linux has discovered
what they are in the boot process. From Redboot, the 'fis list' command
-displays them:
-
-RedBoot> fis list
-Name FLASH addr Mem addr Length Entry point
-RedBoot 0x50000000 0x50000000 0x00020000 0x00000000
-RedBoot config 0x503C0000 0x503C0000 0x00020000 0x00000000
-FIS directory 0x503E0000 0x503E0000 0x00020000 0x00000000
-Linux kernel 0x50020000 0x00100000 0x000C0000 0x00000000
-JFFS2 0x500E0000 0x500E0000 0x002E0000 0x00000000
-
-However Linux should display something like:
-
-SA1100 flash: probing 32-bit flash bus
-SA1100 flash: Found 2 x16 devices at 0x0 in 32-bit mode
-Using RedBoot partition definition
-Creating 5 MTD partitions on "SA1100 flash":
-0x00000000-0x00020000 : "RedBoot"
-0x00020000-0x000e0000 : "Linux kernel"
-0x000e0000-0x003c0000 : "JFFS2"
-0x003c0000-0x003e0000 : "RedBoot config"
-0x003e0000-0x00400000 : "FIS directory"
+displays them::
+
+ RedBoot> fis list
+ Name FLASH addr Mem addr Length Entry point
+ RedBoot 0x50000000 0x50000000 0x00020000 0x00000000
+ RedBoot config 0x503C0000 0x503C0000 0x00020000 0x00000000
+ FIS directory 0x503E0000 0x503E0000 0x00020000 0x00000000
+ Linux kernel 0x50020000 0x00100000 0x000C0000 0x00000000
+ JFFS2 0x500E0000 0x500E0000 0x002E0000 0x00000000
+
+However Linux should display something like::
+
+ SA1100 flash: probing 32-bit flash bus
+ SA1100 flash: Found 2 x16 devices at 0x0 in 32-bit mode
+ Using RedBoot partition definition
+ Creating 5 MTD partitions on "SA1100 flash":
+ 0x00000000-0x00020000 : "RedBoot"
+ 0x00020000-0x000e0000 : "Linux kernel"
+ 0x000e0000-0x003c0000 : "JFFS2"
+ 0x003c0000-0x003e0000 : "RedBoot config"
+ 0x003e0000-0x00400000 : "FIS directory"
What's important here is the position of the partition we are interested in,
which is the third one. Within Linux, this correspond to /dev/mtdblock2.
Therefore to boot Linux with the kernel and its root filesystem in flash, we
-need this RedBoot command:
+need this RedBoot command::
fis load "Linux kernel"
exec -b 0x100000 -l 0xc0000 -c "root=/dev/mtdblock2"
process using RedBoot's scripting capability.
For example, I use this to boot Linux with both the kernel and the ramdisk
-images retrieved from a TFTP server on the network:
-
-RedBoot> fconfig
-Run script at boot: false true
-Boot script:
-Enter script, terminate with empty line
->> load zImage -r -b 0x100000
->> load ramdisk_ks.gz -r -b 0x800000
->> exec -b 0x100000 -l 0xc0000
->>
-Boot script timeout (1000ms resolution): 3
-Use BOOTP for network configuration: true
-GDB connection port: 9000
-Network debug at boot time: false
-Update RedBoot non-volatile configuration - are you sure (y/n)? y
+images retrieved from a TFTP server on the network::
+
+ RedBoot> fconfig
+ Run script at boot: false true
+ Boot script:
+ Enter script, terminate with empty line
+ >> load zImage -r -b 0x100000
+ >> load ramdisk_ks.gz -r -b 0x800000
+ >> exec -b 0x100000 -l 0xc0000
+ >>
+ Boot script timeout (1000ms resolution): 3
+ Use BOOTP for network configuration: true
+ GDB connection port: 9000
+ Network debug at boot time: false
+ Update RedBoot non-volatile configuration - are you sure (y/n)? y
Then, rebooting the Assabet is just a matter of waiting for the login prompt.
Nicolas Pitre
nico@fluxnic.net
+
June 12, 2001
Assabet:
Serial ports:
Radio: TX, RX, CTS, DSR, DCD, RI
- PM: Not tested.
- COM: TX, RX, CTS, DSR, DCD, RTS, DTR, PM
- PM: Not tested.
- I2C: Implemented, not fully tested.
- L3: Fully tested, pass.
- PM: Not tested.
+ - PM: Not tested.
+ - COM: TX, RX, CTS, DSR, DCD, RTS, DTR, PM
+ - PM: Not tested.
+ - I2C: Implemented, not fully tested.
+ - L3: Fully tested, pass.
+ - PM: Not tested.
Video:
- LCD: Fully tested. PM
- (LCD doesn't like being blanked with
- neponset connected)
- Video out: Not fully
+ - LCD: Fully tested. PM
+
+ (LCD doesn't like being blanked with neponset connected)
+
+ - Video out: Not fully
Audio:
UDA1341:
- Playback: Fully tested, pass.
- Record: Implemented, not tested.
- PM: Not tested.
+ - Playback: Fully tested, pass.
+ - Record: Implemented, not tested.
+ - PM: Not tested.
UCB1200:
- Audio play: Implemented, not heavily tested.
- Audio rec: Implemented, not heavily tested.
- Telco audio play: Implemented, not heavily tested.
- Telco audio rec: Implemented, not heavily tested.
- POTS control: No
- Touchscreen: Yes
- PM: Not tested.
+ - Audio play: Implemented, not heavily tested.
+ - Audio rec: Implemented, not heavily tested.
+ - Telco audio play: Implemented, not heavily tested.
+ - Telco audio rec: Implemented, not heavily tested.
+ - POTS control: No
+ - Touchscreen: Yes
+ - PM: Not tested.
Other:
- PCMCIA:
- LPE: Fully tested, pass.
- USB: No
- IRDA:
- SIR: Fully tested, pass.
- FIR: Fully tested, pass.
- PM: Not tested.
+ - PCMCIA:
+ - LPE: Fully tested, pass.
+ - USB: No
+ - IRDA:
+ - SIR: Fully tested, pass.
+ - FIR: Fully tested, pass.
+ - PM: Not tested.
Neponset:
Serial ports:
- COM1,2: TX, RX, CTS, DSR, DCD, RTS, DTR
- PM: Not tested.
- USB: Implemented, not heavily tested.
- PCMCIA: Implemented, not heavily tested.
- PM: Not tested.
- CF: Implemented, not heavily tested.
- PM: Not tested.
+ - COM1,2: TX, RX, CTS, DSR, DCD, RTS, DTR
+ - PM: Not tested.
+ - USB: Implemented, not heavily tested.
+ - PCMCIA: Implemented, not heavily tested.
+ - CF: Implemented, not heavily tested.
+ - PM: Not tested.
More stuff can be found in the -np (Nicolas Pitre's) tree.
-
-Brutus is an evaluation platform for the SA1100 manufactured by Intel.
+======
+Brutus
+======
+
+Brutus is an evaluation platform for the SA1100 manufactured by Intel.
For more details, see:
http://developer.intel.com
-To compile for Brutus, you must issue the following commands:
+To compile for Brutus, you must issue the following commands::
make brutus_config
make config
entry.
But prior to execute the kernel, a ramdisk image must also be loaded in
-memory. Use memory address 0xd8000000 for this. Note that the file
+memory. Use memory address 0xd8000000 for this. Note that the file
containing the (compressed) ramdisk image must not exceed 4 MB.
Typically, you'll need angelboot to load the kernel.
-The following angelboot.opt file should be used:
-
------ begin angelboot.opt -----
-base 0xc0008000
-entry 0xc0008000
-r0 0x00000000
-r1 0x00000010
-device /dev/ttyS0
-options "9600 8N1"
-baud 115200
-otherfile ramdisk_img.gz
-otherbase 0xd8000000
------ end angelboot.opt -----
-
-Then load the kernel and ramdisk with:
+The following angelboot.opt file should be used::
+
+ base 0xc0008000
+ entry 0xc0008000
+ r0 0x00000000
+ r1 0x00000010
+ device /dev/ttyS0
+ options "9600 8N1"
+ baud 115200
+ otherfile ramdisk_img.gz
+ otherbase 0xd8000000
+
+Then load the kernel and ramdisk with::
angelboot -f angelboot.opt zImage
you may use minicom configured with /dev/ttyS1, 9600 baud, 8N1, no flow
control.
-Currently supported:
+Currently supported
+===================
+
- RS232 serial ports
- audio output
- LCD screen
- keyboard
-
-The actual Brutus support may not be complete without extra patches.
-If such patches exist, they should be found from
+
+The actual Brutus support may not be complete without extra patches.
+If such patches exist, they should be found from
ftp.netwinder.org/users/n/nico.
A full PCMCIA support is still missing, although it's possible to hack
Please send patches to nico@fluxnic.net
Have Fun !
-
+==============
+CerfBoard/Cube
+==============
+
*** The StrongARM version of the CerfBoard/Cube has been discontinued ***
The Intrinsyc CerfBoard is a StrongARM 1110-based computer on a board
This document describes the support in the Linux kernel for the
Intrinsyc CerfBoard.
-Supported in this version:
+Supported in this version
+=========================
+
- CompactFlash+ slot (select PCMCIA in General Setup and any options
that may be required)
- Onboard Crystal CS8900 Ethernet controller (Cerf CS8900A support in
In order to get this kernel onto your Cerf, you need a server that runs
both BOOTP and TFTP. Detailed instructions should have come with your
evaluation kit on how to use the bootloader. This series of commands
-will suffice:
+will suffice::
make ARCH=arm CROSS_COMPILE=arm-linux- cerfcube_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux- zImage
--- /dev/null
+========
+Freebird
+========
+
+Freebird-1.1 is produced by Legend(C), Inc.
+`http://web.archive.org/web/*/http://www.legend.com.cn`
+and software/linux maintained by Coventive(C), Inc.
+(http://www.coventive.com)
+
+Based on the Nicolas's strongarm kernel tree.
+
+Maintainer:
+
+Chester Kuo
+ - <chester@coventive.com>
+ - <chester@linux.org.tw>
+
+Author:
+
+- Tim wu <timwu@coventive.com>
+- CIH <cih@coventive.com>
+- Eric Peng <ericpeng@coventive.com>
+- Jeff Lee <jeff_lee@coventive.com>
+- Allen Cheng
+- Tony Liu <tonyliu@coventive.com>
+=============================================
ADS GraphicsClient Plus Single Board Computer
+=============================================
For more details, contact Applied Data Systems or see
http://www.applieddata.net/products.html
-The original Linux support for this product has been provided by
+The original Linux support for this product has been provided by
Nicolas Pitre <nico@fluxnic.net>. Continued development work by
Woojung Huh <whuh@applieddata.net>
Use 'make graphicsclient_config' before any 'make config'. This will set up
defaults for GraphicsClient Plus support.
-The kernel zImage is linked to be loaded and executed at 0xc0200000.
-Also the following registers should have the specified values upon entry:
+The kernel zImage is linked to be loaded and executed at 0xc0200000.
+Also the following registers should have the specified values upon entry::
r0 = 0
r1 = 29 (this is the GraphicsClient architecture number)
something en masse, you should probably get the newer board.
If using Angel on the older boards, here is a typical angel.opt option file
-if the kernel is loaded through the Angel Debug Monitor:
-
------ begin angelboot.opt -----
-base 0xc0200000
-entry 0xc0200000
-r0 0x00000000
-r1 0x0000001d
-device /dev/ttyS1
-options "38400 8N1"
-baud 115200
-#otherfile ramdisk.gz
-#otherbase 0xc0800000
-exec minicom
------ end angelboot.opt -----
+if the kernel is loaded through the Angel Debug Monitor::
+
+ base 0xc0200000
+ entry 0xc0200000
+ r0 0x00000000
+ r1 0x0000001d
+ device /dev/ttyS1
+ options "38400 8N1"
+ baud 115200
+ #otherfile ramdisk.gz
+ #otherbase 0xc0800000
+ exec minicom
Then the kernel (and ramdisk if otherfile/otherbase lines above are
-uncommented) would be loaded with:
+uncommented) would be loaded with::
angelboot -f angelboot.opt zImage
for r0/r1 register values before jumping into the kernel.
-Supported peripherals:
+Supported peripherals
+=====================
+
- SA1100 LCD frame buffer (8/16bpp...sort of)
- on-board SMC 92C96 ethernet NIC
- SA1100 serial port
See http://www.eurotech-inc.com/linux-sbc.asp for IOCTL documentation
and example user space code. ps/2 keybd is multiplexed through this driver
-To do:
+To do
+=====
+
- UCB1200 audio with new ucb_generic layer
- everything else! :-)
-Notes:
+Notes
+=====
- The flash on board is divided into 3 partitions. mtd0 is where
the ADS boot ROM and zImage is stored. It's been marked as
fixed soon.
Any contribution can be sent to nico@fluxnic.net and will be greatly welcome!
-
+========================================
ADS GraphicsMaster Single Board Computer
+========================================
For more details, contact Applied Data Systems or see
http://www.applieddata.net/products.html
Linux can be used with the ADS BootLoader that ships with the
newer rev boards. See their documentation on how to load Linux.
-Supported peripherals:
+Supported peripherals
+=====================
+
- SA1100 LCD frame buffer (8/16bpp...sort of)
- SA1111 USB Master
- on-board SMC 92C96 ethernet NIC
See http://www.eurotech-inc.com/linux-sbc.asp for IOCTL documentation
and example user space code. ps/2 keybd is multiplexed through this driver
-To do:
+To do
+=====
+
- everything else! :-)
-Notes:
+Notes
+=====
- The flash on board is divided into 3 partitions. mtd0 is where
the zImage is stored. It's been marked as read-only to keep you
+=======================
+Hoeft & Wessel Webpanel
+=======================
+
The HUW_WEBPANEL is a product of the german company Hoeft & Wessel AG
If you want more information, please visit
http://www.hoeft-wessel.de
-To build the kernel:
+To build the kernel::
+
make huw_webpanel_config
make oldconfig
[accept all defaults]
Christoph Schulz schu@hoeft-wessel.de
2000/12/18/
-
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+====================
+Intel StrongARM 1100
+====================
+
+.. toctree::
+ :maxdepth: 1
+
+ adsbitsy
+ assabet
+ brutus
+ cerf
+ freebird
+ graphicsclient
+ graphicsmaster
+ huw_webpanel
+ itsy
+ lart
+ nanoengine
+ pangolin
+ pleb
+ serial_uart
+ tifon
+ yopy
+====
+Itsy
+====
+
Itsy is a research project done by the Western Research Lab, and Systems
Research Center in Palo Alto, CA. The Itsy project is one of several
research projects at Compaq that are related to pocket computing.
http://www.hpl.hp.com/downloads/crl/itsy/
Notes on initial 2.4 Itsy support (8/27/2000) :
+
The port was done on an Itsy version 1.5 machine with a daughtercard with
64 Meg of DRAM and 32 Meg of Flash. The initial work includes support for
serial console (to see what you're doing). No other devices have been
to build the params-itsy program used to boot the kernel.
In order to install the port of 2.4 to the itsy, You will need to set the
-configuration parameters in the monitor as follows:
-Arg 1:0x08340000, Arg2: 0xC0000000, Arg3:18 (0x12), Arg4:0
+configuration parameters in the monitor as follows::
+
+ Arg 1:0x08340000, Arg2: 0xC0000000, Arg3:18 (0x12), Arg4:0
+
Make sure the start-routine address is set to 0x00060000.
Next, flash the params-itsy program to 0x00060000 ("p 1 0x00060000" in the
handhelds.org.
The serial connection we established was at:
- 8-bit data, no parity, 1 stop bit(s), 115200.00 b/s. in the monitor, in the
+
+8-bit data, no parity, 1 stop bit(s), 115200.00 b/s. in the monitor, in the
params-itsy program, and in the kernel itself. This can be changed, but
not easily. The monitor parameters are easily changed, the params program
setup is assembly outl's, and the kernel is a configuration item specific to
+====================================
Linux Advanced Radio Terminal (LART)
-------------------------------------
+====================================
The LART is a small (7.5 x 10cm) SA-1100 board, designed for embedded
applications. It has 32 MB DRAM, 4MB Flash ROM, double RS232 and all
+==========
nanoEngine
-----------
+==========
-"nanoEngine" is a SA1110 based single board computer from
+"nanoEngine" is a SA1110 based single board computer from
Bright Star Engineering Inc. See www.brightstareng.com/arm
for more info.
(Ref: Stuart Adams <sja@brightstareng.com>)
Also visit Larry Doolittle's "Linux for the nanoEngine" site:
http://www.brightstareng.com/arm/nanoeng.htm
-
+========
+Pangolin
+========
+
Pangolin is a StrongARM 1110-based evaluation platform produced
by Dialogue Technology (http://www.dialogue.com.tw/).
It has EISA slots for ease of configuration with SDRAM/Flash
memory card, USB/Serial/Audio card, Compact Flash card,
PCMCIA/IDE card and TFT-LCD card.
-To compile for Pangolin, you must issue the following commands:
+To compile for Pangolin, you must issue the following commands::
make pangolin_config
make oldconfig
make zImage
-Supported peripherals:
+Supported peripherals
+=====================
+
- SA1110 serial port (UART1/UART2/UART3)
- flash memory access
- compact flash driver
+====
+PLEB
+====
+
The PLEB project was started as a student initiative at the School of
Computer Science and Engineering, University of New South Wales to make a
pocket computer capable of running the Linux Kernel.
For more information, see:
http://www.cse.unsw.edu.au
-
-
--- /dev/null
+==================
+SA1100 serial port
+==================
+
+The SA1100 serial port had its major/minor numbers officially assigned::
+
+ > Date: Sun, 24 Sep 2000 21:40:27 -0700
+ > From: H. Peter Anvin <hpa@transmeta.com>
+ > To: Nicolas Pitre <nico@CAM.ORG>
+ > Cc: Device List Maintainer <device@lanana.org>
+ > Subject: Re: device
+ >
+ > Okay. Note that device numbers 204 and 205 are used for "low density
+ > serial devices", so you will have a range of minors on those majors (the
+ > tty device layer handles this just fine, so you don't have to worry about
+ > doing anything special.)
+ >
+ > So your assignments are:
+ >
+ > 204 char Low-density serial ports
+ > 5 = /dev/ttySA0 SA1100 builtin serial port 0
+ > 6 = /dev/ttySA1 SA1100 builtin serial port 1
+ > 7 = /dev/ttySA2 SA1100 builtin serial port 2
+ >
+ > 205 char Low-density serial ports (alternate device)
+ > 5 = /dev/cusa0 Callout device for ttySA0
+ > 6 = /dev/cusa1 Callout device for ttySA1
+ > 7 = /dev/cusa2 Callout device for ttySA2
+ >
+
+You must create those inodes in /dev on the root filesystem used
+by your SA1100-based device::
+
+ mknod ttySA0 c 204 5
+ mknod ttySA1 c 204 6
+ mknod ttySA2 c 204 7
+ mknod cusa0 c 205 5
+ mknod cusa1 c 205 6
+ mknod cusa2 c 205 7
+
+In addition to the creation of the appropriate device nodes above, you
+must ensure your user space applications make use of the correct device
+name. The classic example is the content of the /etc/inittab file where
+you might have a getty process started on ttyS0.
+
+In this case:
+
+- replace occurrences of ttyS0 with ttySA0, ttyS1 with ttySA1, etc.
+
+- don't forget to add 'ttySA0', 'console', or the appropriate tty name
+ in /etc/securetty for root to be allowed to login as well.
+=====
Tifon
------
+=====
More info has to come...
Contact: Peter Danielsson <peter.danielsson@era-t.ericsson.se>
-
-See http://www.yopydeveloper.org for more.
+====
+Yopy
+====
+See http://www.yopydeveloper.org for more.
- S3C24XX CPUfreq support
- =======================
+=======================
+S3C24XX CPUfreq support
+=======================
Introduction
------------
- Simtec Electronics EB2410ITX (BAST)
- ===================================
+===================================
+Simtec Electronics EB2410ITX (BAST)
+===================================
http://www.simtec.co.uk/products/EB2410ITX/
- S3C24XX GPIO Control
- ====================
+====================
+S3C24XX GPIO Control
+====================
Introduction
------------
of the s3c2410 GPIO system, please read the Samsung provided
data-sheet/users manual to find out the complete list.
- See Documentation/arm/Samsung/GPIO.txt for the core implementation.
+ See Documentation/arm/samsung/gpio.rst for the core implementation.
GPIOLIB
listed below will be removed (they may be marked as __deprecated
in the near future).
- The following functions now either have a s3c_ specific variant
+ The following functions now either have a `s3c_` specific variant
or are merged into gpiolib. See the definitions in
arch/arm/plat-samsung/include/plat/gpio-cfg.h:
- s3c2410_gpio_setpin() gpio_set_value() or gpio_direction_output()
- s3c2410_gpio_getpin() gpio_get_value() or gpio_direction_input()
- s3c2410_gpio_getirq() gpio_to_irq()
- s3c2410_gpio_cfgpin() s3c_gpio_cfgpin()
- s3c2410_gpio_getcfg() s3c_gpio_getcfg()
- s3c2410_gpio_pullup() s3c_gpio_setpull()
+ - s3c2410_gpio_setpin() gpio_set_value() or gpio_direction_output()
+ - s3c2410_gpio_getpin() gpio_get_value() or gpio_direction_input()
+ - s3c2410_gpio_getirq() gpio_to_irq()
+ - s3c2410_gpio_cfgpin() s3c_gpio_cfgpin()
+ - s3c2410_gpio_getcfg() s3c_gpio_getcfg()
+ - s3c2410_gpio_pullup() s3c_gpio_setpull()
GPIOLIB conversion
6) s3c2410_gpio_getirq() should be directly replaceable with the
gpio_to_irq() call.
-The s3c2410_gpio and gpio_ calls have always operated on the same gpio
+The s3c2410_gpio and `gpio_` calls have always operated on the same gpio
numberspace, so there is no problem with converting the gpio numbering
between the calls.
- HP IPAQ H1940
- =============
+=============
+HP IPAQ H1940
+=============
http://www.handhelds.org/projects/h1940.html
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================
+Samsung S3C24XX SoC Family
+==========================
+
+.. toctree::
+ :maxdepth: 1
+
+ h1940
+ gpio
+ cpufreq
+ suspend
+ usb-host
+ s3c2412
+ eb2410itx
+ nand
+ smdk2440
+ s3c2413
+ overview
- S3C24XX NAND Support
- ====================
+====================
+S3C24XX NAND Support
+====================
Introduction
------------
---------------
Ben Dooks, Copyright 2007 Simtec Electronics
-
- S3C24XX ARM Linux Overview
- ==========================
+==========================
+S3C24XX ARM Linux Overview
+==========================
controller. If there are any problems the latest linux-mtd
code can be found from http://www.linux-mtd.infradead.org/
- For more information see Documentation/arm/Samsung-S3C24XX/NAND.txt
+ For more information see Documentation/arm/samsung-s3c24xx/nand.rst
SD/MMC
As of v2.6.34, the move towards using gpiolib support is almost
complete, and very little of the old calls are left.
- See Documentation/arm/Samsung-S3C24XX/GPIO.txt for the S3C24XX specific
- support and Documentation/arm/Samsung/GPIO.txt for the core Samsung
+ See Documentation/arm/samsung-s3c24xx/gpio.rst for the S3C24XX specific
+ support and Documentation/arm/samsung/gpio.rst for the core Samsung
implementation.
kmalloc()s an area of memory, and copies the __initdata
and then sets the relevant device's platform data. Making
the function `__init` takes care of ensuring it is discarded
- with the rest of the initialisation code
+ with the rest of the initialisation code::
- static __init void s3c24xx_xxx_set_platdata(struct xxx_data *pd)
- {
- struct s3c2410_xxx_mach_info *npd;
+ static __init void s3c24xx_xxx_set_platdata(struct xxx_data *pd)
+ {
+ struct s3c2410_xxx_mach_info *npd;
npd = kmalloc(sizeof(struct s3c2410_xxx_mach_info), GFP_KERNEL);
if (npd) {
memcpy(npd, pd, sizeof(struct s3c2410_xxx_mach_info));
s3c_device_xxx.dev.platform_data = npd;
} else {
- printk(KERN_ERR "no memory for xxx platform data\n");
+ printk(KERN_ERR "no memory for xxx platform data\n");
}
}
- S3C2412 ARM Linux Overview
- ==========================
+==========================
+S3C2412 ARM Linux Overview
+==========================
Introduction
------------
- S3C2413 ARM Linux Overview
- ==========================
+==========================
+S3C2413 ARM Linux Overview
+==========================
Introduction
------------
Camera Interface
----------------
+----------------
This block is currently not supported.
- Samsung/Meritech SMDK2440
- =========================
+=========================
+Samsung/Meritech SMDK2440
+=========================
Introduction
------------
- S3C24XX Suspend Support
- =======================
+=======================
+S3C24XX Suspend Support
+=======================
Introduction
and will end up initialising all compiled machines' pm init!
The following is an example of code used for testing wakeup from
- an falling edge on IRQ_EINT0:
+ an falling edge on IRQ_EINT0::
-static irqreturn_t button_irq(int irq, void *pw)
-{
+ static irqreturn_t button_irq(int irq, void *pw)
+ {
return IRQ_HANDLED;
-}
+ }
-statuc void __init machine_init(void)
-{
+ statuc void __init machine_init(void)
+ {
...
request_irq(IRQ_EINT0, button_irq, IRQF_TRIGGER_FALLING,
enable_irq_wake(IRQ_EINT0);
s3c_pm_init();
-}
+ }
Debugging
---------------
Ben Dooks, Copyright 2004 Simtec Electronics
-
- S3C24XX USB Host support
- ========================
+========================
+S3C24XX USB Host support
+========================
Enable at least the following kernel options:
- menuconfig:
+ menuconfig::
Device Drivers --->
USB support --->
.config:
- CONFIG_USB
- CONFIG_USB_OHCI_HCD
+
+ - CONFIG_USB
+ - CONFIG_USB_OHCI_HCD
Once these options are configured, the standard set of USB device
The ports are numbered 0 and 1.
power_control:
-
Called to enable or disable the power on the port.
enable_oc:
-
Called to enable or disable the over-current monitoring.
This should claim or release the resources being used to
check the power condition on the port, such as an IRQ.
report_oc:
-
The OHCI driver fills this field in for the over-current code
to call when there is a change to the over-current state on
an port. The ports argument is a bitmask of 1 bit per port,
ensure this is called correctly.
port[x]:
-
This is struct describes each port, 0 or 1. The platform driver
should set the flags field of each port to S3C_HCDFLG_USED if
the port is enabled.
- Interface between kernel and boot loaders on Exynos boards
- ==========================================================
+==========================================================
+Interface between kernel and boot loaders on Exynos boards
+==========================================================
Author: Krzysztof Kozlowski
+
Date : 6 June 2015
The document tries to describe currently used interface between Linux kernel
1. Non-Secure mode
Address: sysram_ns_base_addr
+
+============= ============================================ ==================
Offset Value Purpose
-=============================================================================
+============= ============================================ ==================
0x08 exynos_cpu_resume_ns, mcpm_entry_point System suspend
0x0c 0x00000bad (Magic cookie) System suspend
0x1c exynos4_secondary_startup Secondary CPU boot
0x24 exynos_cpu_resume_ns AFTR
0x28 + 4*cpu 0x8 (Magic cookie, Exynos3250) AFTR
0x28 0x0 or last value during resume (Exynos542x) System suspend
+============= ============================================ ==================
2. Secure mode
Address: sysram_base_addr
+
+============= ============================================ ==================
Offset Value Purpose
-=============================================================================
+============= ============================================ ==================
0x00 exynos4_secondary_startup Secondary CPU boot
0x04 exynos4_secondary_startup (Exynos542x) Secondary CPU boot
4*cpu exynos4_secondary_startup (Exynos4412) Secondary CPU boot
0x20 exynos_cpu_resume (Exynos4210 r1.0) AFTR
0x24 0xfcba0d10 (Magic cookie, Exynos4210 r1.0) AFTR
+============= ============================================ ==================
Address: pmu_base_addr
+
+============= ============================================ ==================
Offset Value Purpose
-=============================================================================
+============= ============================================ ==================
0x0800 exynos_cpu_resume AFTR, suspend
0x0800 mcpm_entry_point (Exynos542x with MCPM) AFTR, suspend
0x0804 0xfcba0d10 (Magic cookie) AFTR
0x0814 exynos4_secondary_startup (Exynos4210 r1.1) Secondary CPU boot
0x0818 0xfcba0d10 (Magic cookie, Exynos4210 r1.1) AFTR
0x081C exynos_cpu_resume (Exynos4210 r1.1) AFTR
-
+============= ============================================ ==================
3. Other (regardless of secure/non-secure mode)
Address: pmu_base_addr
+
+============= =============================== ===============================
Offset Value Purpose
-=============================================================================
+============= =============================== ===============================
0x0908 Non-zero Secondary CPU boot up indicator
on Exynos3250 and Exynos542x
+============= =============================== ===============================
4. Glossary
- Samsung GPIO implementation
- ===========================
+===========================
+Samsung GPIO implementation
+===========================
Introduction
------------
S3C24XX (Legacy)
----------------
-See Documentation/arm/Samsung-S3C24XX/GPIO.txt for more information
+See Documentation/arm/samsung-s3c24xx/gpio.rst for more information
about these devices. Their implementation has been brought into line
with the core samsung implementation described in this document.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===========
+Samsung SoC
+===========
+
+.. toctree::
+ :maxdepth: 1
+
+ gpio
+ bootloader-interface
+ overview
- Samsung ARM Linux Overview
- ==========================
+==========================
+Samsung ARM Linux Overview
+==========================
Introduction
------------
The currently supported SoCs are:
- - S3C24XX: See Documentation/arm/Samsung-S3C24XX/Overview.txt for full list
+ - S3C24XX: See Documentation/arm/samsung-s3c24xx/overview.rst for full list
- S3C64XX: S3C6400 and S3C6410
- S5PC110 / S5PV210
There is still documentation in Documnetation/arm/Samsung-S3C24XX/ which
deals with the architecture and drivers specific to these devices.
- See Documentation/arm/Samsung-S3C24XX/Overview.txt for more information
+ See Documentation/arm/samsung-s3c24xx/overview.rst for more information
on the implementation details and specific support.
A number of configurations are supplied, as there is no current way of
unifying all the SoCs into one kernel.
- s5pc110_defconfig - S5PC110 specific default configuration
- s5pv210_defconfig - S5PV210 specific default configuration
+ s5pc110_defconfig
+ - S5PC110 specific default configuration
+ s5pv210_defconfig
+ - S5PV210 specific default configuration
Layout
+=============================================
Kernel initialisation parameters on ARM Linux
----------------------------------------------
+=============================================
The following document describes the kernel initialisation parameter
structure, otherwise known as 'struct param_struct' which is used
below:
page_size
-
This parameter must be set to the page size of the machine, and
will be checked by the kernel.
nr_pages
-
This is the total number of pages of memory in the system. If
the memory is banked, then this should contain the total number
of pages in the system.
include this information.
ramdisk_size
-
This is now obsolete, and should not be used.
flags
-
Various kernel flags, including:
- bit 0 - 1 = mount root read only
- bit 1 - unused
- bit 2 - 0 = load ramdisk
- bit 3 - 0 = prompt for ramdisk
- rootdev
+ ===== ========================
+ bit 0 1 = mount root read only
+ bit 1 unused
+ bit 2 0 = load ramdisk
+ bit 3 0 = prompt for ramdisk
+ ===== ========================
+ rootdev
major/minor number pair of device to mount as the root filesystem.
- video_num_cols
- video_num_rows
-
+ video_num_cols / video_num_rows
These two together describe the character size of the dummy console,
or VGA console character size. They should not be used for any other
purpose.
the equivalent character size of your fbcon display. This then allows
all the bootup messages to be displayed correctly.
- video_x
- video_y
-
+ video_x / video_y
This describes the character position of cursor on VGA console, and
is otherwise unused. (should not be used for other console types, and
should not be used for other purposes).
memc_control_reg
-
MEMC chip control register for Acorn Archimedes and Acorn A5000
based machines. May be used differently by different architectures.
sounddefault
-
Default sound setting on Acorn machines. May be used differently by
different architectures.
adfsdrives
-
Number of ADFS/MFM disks. May be used differently by different
architectures.
- bytes_per_char_h
- bytes_per_char_v
-
+ bytes_per_char_h / bytes_per_char_v
These are now obsolete, and should not be used.
pages_in_bank[4]
-
Number of pages in each bank of the systems memory (used for RiscPC).
This is intended to be used on systems where the physical memory
is non-contiguous from the processors point of view.
pages_in_vram
-
Number of pages in VRAM (used on Acorn RiscPC). This value may also
be used by loaders if the size of the video RAM can't be obtained
from the hardware.
- initrd_start
- initrd_size
-
+ initrd_start / initrd_size
This describes the kernel virtual start address and size of the
initial ramdisk.
rd_start
-
Start address in sectors of the ramdisk image on a floppy disk.
system_rev
-
system revision number.
- system_serial_low
- system_serial_high
-
+ system_serial_low / system_serial_high
system 64-bit serial number
mem_fclk_21285
-
The speed of the external oscillator to the 21285 (footbridge),
which control's the speed of the memory bus, timer & serial port.
Depending upon the speed of the cpu its value can be between
then a value of 50 Mhz is the default on 21285 architectures.
paths[8][128]
-
These are now obsolete, and should not be used.
commandline
-
Kernel command line parameters. Details can be found elsewhere.
- SPEAr ARM Linux Overview
- ==========================
+========================
+SPEAr ARM Linux Overview
+========================
Introduction
------------
Hierarchy in SPEAr is as follows:
SPEAr (Platform)
+
- SPEAr3XX (3XX SOC series, based on ARM9)
- SPEAr300 (SOC)
- SPEAr300 Evaluation Board
- SPEAr1340 (SOC)
- SPEAr1340 Evaluation Board
- Configuration
- -------------
+Configuration
+-------------
A generic configuration is provided for each machine, and can be used as the
- default by
+ default by::
+
make spear13xx_defconfig
make spear3xx_defconfig
make spear6xx_defconfig
- Layout
- ------
+Layout
+------
The common files for multiple machine families (SPEAr3xx, SPEAr6xx and
SPEAr13xx) are located in the platform code contained in arch/arm/plat-spear
support Flattened Device Tree.
- Document Author
- ---------------
+Document Author
+---------------
Viresh Kumar <vireshk@kernel.org>, (c) 2010-2012 ST Microelectronics
- STi ARM Linux Overview
- ==========================
+======================
+STi ARM Linux Overview
+======================
Introduction
------------
B2000 and B2020 Reference boards.
- configuration
- -------------
+configuration
+-------------
A generic configuration is provided for both STiH415/416, and can be used as the
- default by
+ default by::
+
make stih41x_defconfig
- Layout
- ------
+Layout
+------
+
All the files for multiple machine families (STiH415, STiH416, and STiG125)
are located in the platform code contained in arch/arm/mach-sti
Device Trees.
- Document Author
- ---------------
+Document Author
+---------------
Srinivas Kandagatla <srinivas.kandagatla@st.com>, (c) 2013 ST Microelectronics
- STiH407 Overview
- ================
+================
+STiH407 Overview
+================
Introduction
------------
- ARM Cortex-A9 1.5 GHz dual core CPU (28nm)
- SATA2, USB 3.0, PCIe, Gbit Ethernet
- Document Author
- ---------------
+Document Author
+---------------
Maxime Coquelin <maxime.coquelin@st.com>, (c) 2014 ST Microelectronics
- STiH415 Overview
- ================
+================
+STiH415 Overview
+================
Introduction
------------
The STiH415 is the next generation of HD, AVC set-top box processors
for satellite, cable, terrestrial and IP-STB markets.
- Features
+ Features:
+
- ARM Cortex-A9 1.0 GHz, dual-core CPU
- SATA2x2,USB 2.0x3, PCIe, Gbit Ethernet MACx2
- STiH416 Overview
- ================
+================
+STiH416 Overview
+================
Introduction
------------
- STiH418 Overview
- ================
+================
+STiH418 Overview
+================
Introduction
------------
- HEVC L5.1 Main 10
- VP9
- Document Author
- ---------------
+Document Author
+---------------
Maxime Coquelin <maxime.coquelin@st.com>, (c) 2015 ST Microelectronics
-:orphan:
-
========================
STM32 ARM Linux Overview
========================
-:orphan:
-
+==================
STM32F429 Overview
==================
.. _STM32F429: http://www.st.com/web/en/catalog/mmc/FM141/SC1169/SS1577/LN1806?ecmp=stm32f429-439_pron_pr-ces2014_nov2013
-:Authors:
-
-Maxime Coquelin <mcoquelin.stm32@gmail.com>
+:Authors: Maxime Coquelin <mcoquelin.stm32@gmail.com>
-:orphan:
-
+==================
STM32F746 Overview
==================
.. _STM32F746: http://www.st.com/content/st_com/en/products/microcontrollers/stm32-32-bit-arm-cortex-mcus/stm32f7-series/stm32f7x6/stm32f746ng.html
-:Authors:
-
-Alexandre Torgue <alexandre.torgue@st.com>
+:Authors: Alexandre Torgue <alexandre.torgue@st.com>
-:orphan:
-
+==================
STM32F769 Overview
==================
.. _STM32F769: http://www.st.com/content/st_com/en/products/microcontrollers/stm32-32-bit-arm-cortex-mcus/stm32-high-performance-mcus/stm32f7-series/stm32f7x9/stm32f769ni.html
-:Authors:
-
-Alexandre Torgue <alexandre.torgue@st.com>
+:Authors: Alexandre Torgue <alexandre.torgue@st.com>
-:orphan:
-
+==================
STM32H743 Overview
==================
.. _STM32H743: http://www.st.com/en/microcontrollers/stm32h7x3.html?querycriteria=productId=LN2033
-:Authors:
-
-Alexandre Torgue <alexandre.torgue@st.com>
+:Authors: Alexandre Torgue <alexandre.torgue@st.com>
-:orphan:
-
+===================
STM32MP157 Overview
===================
+==================
ARM Allwinner SoCs
==================
Linux kernel mach directory: arch/arm/mach-sunxi
Flavors:
+
* ARM926 based SoCs
- Allwinner F20 (sun3i)
- + Not Supported
+
+ * Not Supported
* ARM Cortex-A8 based SoCs
- Allwinner A10 (sun4i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A10/A10%20Datasheet%20-%20v1.21%20%282012-04-06%29.pdf
- + User Manual
+ * User Manual
+
http://dl.linux-sunxi.org/A10/A10%20User%20Manual%20-%20v1.20%20%282012-04-09%2c%20DECRYPTED%29.pdf
- Allwinner A10s (sun5i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A10s/A10s%20Datasheet%20-%20v1.20%20%282012-03-27%29.pdf
- Allwinner A13 / R8 (sun5i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A13/A13%20Datasheet%20-%20v1.12%20%282012-03-29%29.pdf
- + User Manual
+ * User Manual
+
http://dl.linux-sunxi.org/A13/A13%20User%20Manual%20-%20v1.2%20%282013-01-08%29.pdf
- Next Thing Co GR8 (sun5i)
* Single ARM Cortex-A7 based SoCs
- Allwinner V3s (sun8i)
- + Datasheet
+
+ * Datasheet
+
http://linux-sunxi.org/File:Allwinner_V3s_Datasheet_V1.0.pdf
* Dual ARM Cortex-A7 based SoCs
- Allwinner A20 (sun7i)
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A20/A20%20User%20Manual%202013-03-22.pdf
- Allwinner A23 (sun8i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A23/A23%20Datasheet%20V1.0%2020130830.pdf
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A23/A23%20User%20Manual%20V1.0%2020130830.pdf
* Quad ARM Cortex-A7 based SoCs
- Allwinner A31 (sun6i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A31/A3x_release_document/A31/IC/A31%20datasheet%20V1.3%2020131106.pdf
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A31/A3x_release_document/A31/IC/A31%20user%20manual%20V1.1%2020130630.pdf
- Allwinner A31s (sun6i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A31/A3x_release_document/A31s/IC/A31s%20datasheet%20V1.3%2020131106.pdf
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A31/A3x_release_document/A31s/IC/A31s%20User%20Manual%20%20V1.0%2020130322.pdf
- Allwinner A33 (sun8i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A33/A33%20Datasheet%20release%201.1.pdf
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A33/A33%20user%20manual%20release%201.1.pdf
- Allwinner H2+ (sun8i)
- + No document available now, but is known to be working properly with
+
+ * No document available now, but is known to be working properly with
H3 drivers and memory map.
- Allwinner H3 (sun8i)
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/H3/Allwinner_H3_Datasheet_V1.0.pdf
- Allwinner R40 (sun8i)
- + Datasheet
+
+ * Datasheet
+
https://github.com/tinalinux/docs/raw/r40-v1.y/R40_Datasheet_V1.0.pdf
- + User Manual
+
+ * User Manual
+
https://github.com/tinalinux/docs/raw/r40-v1.y/Allwinner_R40_User_Manual_V1.0.pdf
* Quad ARM Cortex-A15, Quad ARM Cortex-A7 based SoCs
- Allwinner A80
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A80/A80_Datasheet_Revision_1.0_0404.pdf
* Octa ARM Cortex-A7 based SoCs
- Allwinner A83T
- + Datasheet
+
+ * Datasheet
+
https://github.com/allwinner-zh/documents/raw/master/A83T/A83T_Datasheet_v1.3_20150510.pdf
- + User Manual
+
+ * User Manual
+
https://github.com/allwinner-zh/documents/raw/master/A83T/A83T_User_Manual_v1.5.1_20150513.pdf
* Quad ARM Cortex-A53 based SoCs
- Allwinner A64
- + Datasheet
+
+ * Datasheet
+
http://dl.linux-sunxi.org/A64/A64_Datasheet_V1.1.pdf
- + User Manual
+
+ * User Manual
+
http://dl.linux-sunxi.org/A64/Allwinner%20A64%20User%20Manual%20v1.0.pdf
+=======================================================
Frequently asked questions about the sunxi clock system
=======================================================
steps, one can gate it and keep the system running. Consider this
simplified suspend example:
- While the system is operational, you would see something like
+ While the system is operational, you would see something like::
24MHz 32kHz
|
[CPU]
When you are about to suspend, you switch the CPU Mux to the 32kHz
- oscillator:
+ oscillator::
24Mhz 32kHz
| |
|
[CPU]
- Finally you can gate the main oscillator
+ Finally you can gate the main oscillator::
32kHz
|
signalled to the triggering process.
/proc/cpu/swp_emulation holds some statistics/information, including the PID of
-the last process to trigger the emulation to be invocated. For example:
----
-Emulated SWP: 12
-Emulated SWPB: 0
-Aborted SWP{B}: 1
-Last process: 314
----
+the last process to trigger the emulation to be invocated. For example::
-NOTE: when accessing uncached shared regions, LDREX/STREX rely on an external
-transaction monitoring block called a global monitor to maintain update
-atomicity. If your system does not implement a global monitor, this option can
-cause programs that perform SWP operations to uncached memory to deadlock, as
-the STREX operation will always fail.
+ Emulated SWP: 12
+ Emulated SWPB: 0
+ Aborted SWP{B}: 1
+ Last process: 314
+
+NOTE:
+ when accessing uncached shared regions, LDREX/STREX rely on an external
+ transaction monitoring block called a global monitor to maintain update
+ atomicity. If your system does not implement a global monitor, this option can
+ cause programs that perform SWP operations to uncached memory to deadlock, as
+ the STREX operation will always fail.
+==================================================
ARM TCM (Tightly-Coupled Memory) handling in Linux
-----
+==================================================
+
Written by Linus Walleij <linus.walleij@stericsson.com>
Some ARM SoC:s have a so-called TCM (Tightly-Coupled Memory).
wasting space, there is also the __tcmlocalfunc prefix that
will make the call relative.
-Variables to go into dtcm can be tagged like this:
-int __tcmdata foo;
+Variables to go into dtcm can be tagged like this::
+
+ int __tcmdata foo;
+
+Constants can be tagged like this::
-Constants can be tagged like this:
-int __tcmconst foo;
+ int __tcmconst foo;
+
+To put assembler into TCM just use::
+
+ .section ".tcm.text" or .section ".tcm.data"
-To put assembler into TCM just use
-.section ".tcm.text" or .section ".tcm.data"
respectively.
-Example code:
+Example code::
-#include <asm/tcm.h>
-/* Uninitialized data */
-static u32 __tcmdata tcmvar;
-/* Initialized data */
-static u32 __tcmdata tcmassigned = 0x2BADBABEU;
-/* Constant */
-static const u32 __tcmconst tcmconst = 0xCAFEBABEU;
+ /* Uninitialized data */
+ static u32 __tcmdata tcmvar;
+ /* Initialized data */
+ static u32 __tcmdata tcmassigned = 0x2BADBABEU;
+ /* Constant */
+ static const u32 __tcmconst tcmconst = 0xCAFEBABEU;
-static void __tcmlocalfunc tcm_to_tcm(void)
-{
+ static void __tcmlocalfunc tcm_to_tcm(void)
+ {
int i;
for (i = 0; i < 100; i++)
tcmvar ++;
-}
+ }
-static void __tcmfunc hello_tcm(void)
-{
+ static void __tcmfunc hello_tcm(void)
+ {
/* Some abstract code that runs in ITCM */
int i;
for (i = 0; i < 100; i++) {
tcmvar ++;
}
tcm_to_tcm();
-}
+ }
-static void __init test_tcm(void)
-{
+ static void __init test_tcm(void)
+ {
u32 *tcmem;
int i;
printk("TCM tcmem[%d] = %08x\n", i, tcmem[i]);
tcm_free(tcmem, 20);
}
-}
+ }
+================================================
+The Unified Extensible Firmware Interface (UEFI)
+================================================
+
UEFI, the Unified Extensible Firmware Interface, is a specification
governing the behaviours of compatible firmware interfaces. It is
maintained by the UEFI Forum - http://www.uefi.org/.
=====================
Booting on a platform with firmware compliant with the UEFI specification
makes it possible for the kernel to support additional features:
+
- UEFI Runtime Services
- Retrieving various configuration information through the standardised
interface of UEFI configuration tables. (ACPI, SMBIOS, ...)
For actually enabling [U]EFI support, enable:
+
- CONFIG_EFI=y
- CONFIG_EFI_VARS=y or m
The stub populates the FDT /chosen node with (and the kernel scans for) the
following parameters:
-________________________________________________________________________________
-Name | Size | Description
-================================================================================
-linux,uefi-system-table | 64-bit | Physical address of the UEFI System Table.
---------------------------------------------------------------------------------
-linux,uefi-mmap-start | 64-bit | Physical address of the UEFI memory map,
- | | populated by the UEFI GetMemoryMap() call.
---------------------------------------------------------------------------------
-linux,uefi-mmap-size | 32-bit | Size in bytes of the UEFI memory map
- | | pointed to in previous entry.
---------------------------------------------------------------------------------
-linux,uefi-mmap-desc-size | 32-bit | Size in bytes of each entry in the UEFI
- | | memory map.
---------------------------------------------------------------------------------
-linux,uefi-mmap-desc-ver | 32-bit | Version of the mmap descriptor format.
---------------------------------------------------------------------------------
+
+========================== ====== ===========================================
+Name Size Description
+========================== ====== ===========================================
+linux,uefi-system-table 64-bit Physical address of the UEFI System Table.
+
+linux,uefi-mmap-start 64-bit Physical address of the UEFI memory map,
+ populated by the UEFI GetMemoryMap() call.
+
+linux,uefi-mmap-size 32-bit Size in bytes of the UEFI memory map
+ pointed to in previous entry.
+
+linux,uefi-mmap-desc-size 32-bit Size in bytes of each entry in the UEFI
+ memory map.
+
+linux,uefi-mmap-desc-ver 32-bit Version of the mmap descriptor format.
+========================== ====== ===========================================
+===============================================
Release notes for Linux Kernel VFP support code
------------------------------------------------
+===============================================
Date: 20 May 2004
+
Author: Russell King
This is the first release of the Linux Kernel VFP support code. It
+======================================
vlocks for Bare-Metal Mutual Exclusion
======================================
Algorithm
---------
-The easiest way to explain the vlocks algorithm is with some pseudo-code:
+The easiest way to explain the vlocks algorithm is with some pseudo-code::
int currently_voting[NR_CPUS] = { 0, };
number of CPUs.
vlocks can be cascaded in a voting hierarchy to permit better scaling
- if necessary, as in the following hypothetical example for 4096 CPUs:
+ if necessary, as in the following hypothetical example for 4096 CPUs::
/* first level: local election */
my_town = towns[(this_cpu >> 4) & 0xf];
reduces the number of round-trips required to external memory.
In the ARM implementation, this means that we can use a single load
- and comparison:
+ and comparison::
LDR Rt, [Rn]
CMP Rt, #0
- ...in place of code equivalent to:
+ ...in place of code equivalent to::
LDRB Rt, [Rn]
CMP Rt, #0
-:orphan:
-
==================
ARM64 Architecture
==================
+++ /dev/null
-Kernel driver lp855x
-====================
-
-Backlight driver for LP855x ICs
-
-Supported chips:
- Texas Instruments LP8550, LP8551, LP8552, LP8553, LP8555, LP8556 and
- LP8557
-
-Author: Milo(Woogyom) Kim <milo.kim@ti.com>
-
-Description
------------
-
-* Brightness control
-
-Brightness can be controlled by the pwm input or the i2c command.
-The lp855x driver supports both cases.
-
-* Device attributes
-
-1) bl_ctl_mode
-Backlight control mode.
-Value : pwm based or register based
-
-2) chip_id
-The lp855x chip id.
-Value : lp8550/lp8551/lp8552/lp8553/lp8555/lp8556/lp8557
-
-Platform data for lp855x
-------------------------
-
-For supporting platform specific data, the lp855x platform data can be used.
-
-* name : Backlight driver name. If it is not defined, default name is set.
-* device_control : Value of DEVICE CONTROL register.
-* initial_brightness : Initial value of backlight brightness.
-* period_ns : Platform specific PWM period value. unit is nano.
- Only valid when brightness is pwm input mode.
-* size_program : Total size of lp855x_rom_data.
-* rom_data : List of new eeprom/eprom registers.
-
-example 1) lp8552 platform data : i2c register mode with new eeprom data
-
-#define EEPROM_A5_ADDR 0xA5
-#define EEPROM_A5_VAL 0x4f /* EN_VSYNC=0 */
-
-static struct lp855x_rom_data lp8552_eeprom_arr[] = {
- {EEPROM_A5_ADDR, EEPROM_A5_VAL},
-};
-
-static struct lp855x_platform_data lp8552_pdata = {
- .name = "lcd-bl",
- .device_control = I2C_CONFIG(LP8552),
- .initial_brightness = INITIAL_BRT,
- .size_program = ARRAY_SIZE(lp8552_eeprom_arr),
- .rom_data = lp8552_eeprom_arr,
-};
-
-example 2) lp8556 platform data : pwm input mode with default rom data
-
-static struct lp855x_platform_data lp8556_pdata = {
- .device_control = PWM_CONFIG(LP8556),
- .initial_brightness = INITIAL_BRT,
- .period_ns = 1000000,
-};
+==========================
BFQ (Budget Fair Queueing)
==========================
BFQ is a proportional-share I/O scheduler, with some extra
low-latency capabilities. In addition to cgroups support (blkio or io
controllers), BFQ's main features are:
+
- BFQ guarantees a high system and application responsiveness, and a
low latency for time-sensitive applications, such as audio or video
players;
BFQ works for multi-queue devices too.
-The table of contents follow. Impatients can just jump to Section 3.
+.. The table of contents follow. Impatients can just jump to Section 3.
-CONTENTS
+.. CONTENTS
-1. When may BFQ be useful?
- 1-1 Personal systems
- 1-2 Server systems
-2. How does BFQ work?
-3. What are BFQ's tunables and how to properly configure BFQ?
-4. BFQ group scheduling
- 4-1 Service guarantees provided
- 4-2 Interface
+ 1. When may BFQ be useful?
+ 1-1 Personal systems
+ 1-2 Server systems
+ 2. How does BFQ work?
+ 3. What are BFQ's tunables and how to properly configure BFQ?
+ 4. BFQ group scheduling
+ 4-1 Service guarantees provided
+ 4-2 Interface
1. When may BFQ be useful?
==========================
--------------------
Low latency for interactive applications
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Regardless of the actual background workload, BFQ guarantees that, for
interactive tasks, the storage device is virtually as responsive as if
it was idle. For example, even if one or more of the following
background workloads are being executed:
+
- one or more large files are being read, written or copied,
- a tree of source files is being compiled,
- one or more virtual machines are performing I/O,
- a software update is in progress,
- indexing daemons are scanning filesystems and updating their
databases,
+
starting an application or loading a file from within an application
takes about the same time as if the storage device was idle. As a
comparison, with CFQ, NOOP or DEADLINE, and in the same conditions,
until the background workload terminates (also on SSDs).
Low latency for soft real-time applications
-
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Also soft real-time applications, such as audio and video
players/streamers, enjoy a low latency and a low drop rate, regardless
of the background I/O workload. As a consequence, these applications
do not suffer from almost any glitch due to the background workload.
Higher speed for code-development tasks
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If some additional workload happens to be executed in parallel, then
BFQ executes the I/O-related components of typical code-development
NOOP or DEADLINE.
High throughput
+^^^^^^^^^^^^^^^
On hard disks, BFQ achieves up to 30% higher throughput than CFQ, and
up to 150% higher throughput than DEADLINE and NOOP, with all the
instead, about the same throughput as the other schedulers.
Strong fairness, bandwidth and delay guarantees
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
BFQ distributes the device throughput, and not just the device time,
among I/O-bound applications in proportion their weights, with any
properties as above. In particular, regardless of whether additional,
possibly heavy workloads are being served, BFQ guarantees:
-. audio and video-streaming with zero or very low jitter and drop
+* audio and video-streaming with zero or very low jitter and drop
rate;
-. fast retrieval of WEB pages and embedded objects;
+* fast retrieval of WEB pages and embedded objects;
-. real-time recording of data in live-dumping applications (e.g.,
+* real-time recording of data in live-dumping applications (e.g.,
packet logging);
-. responsiveness in local and remote access to a server.
+* responsiveness in local and remote access to a server.
2. How does BFQ work?
plus a lot of code, are borrowed from CFQ.
- Each process doing I/O on a device is associated with a weight and a
- (bfq_)queue.
+ `(bfq_)queue`.
- BFQ grants exclusive access to the device, for a while, to one queue
(process) at a time, and implements this service model by
created, and kept up-to-date by bfq, depends on whether
CONFIG_BFQ_CGROUP_DEBUG is set. If it is set, then bfq creates all
the stat files documented in
-Documentation/cgroup-v1/blkio-controller.rst. If, instead,
-CONFIG_BFQ_CGROUP_DEBUG is not set, then bfq creates only the files
-blkio.bfq.io_service_bytes
-blkio.bfq.io_service_bytes_recursive
-blkio.bfq.io_serviced
-blkio.bfq.io_serviced_recursive
+Documentation/admin-guide/cgroup-v1/blkio-controller.rst. If, instead,
+CONFIG_BFQ_CGROUP_DEBUG is not set, then bfq creates only the files::
+
+ blkio.bfq.io_service_bytes
+ blkio.bfq.io_service_bytes_recursive
+ blkio.bfq.io_serviced
+ blkio.bfq.io_serviced_recursive
The value of CONFIG_BFQ_CGROUP_DEBUG greatly influences the maximum
throughput sustainable with bfq, because updating the blkio.bfq.*
applications. Unset this tunable if you need/want to control weights.
-[1] P. Valente, A. Avanzini, "Evolution of the BFQ Storage I/O
+[1]
+ P. Valente, A. Avanzini, "Evolution of the BFQ Storage I/O
Scheduler", Proceedings of the First Workshop on Mobile System
Technologies (MST-2015), May 2015.
+
http://algogroup.unimore.it/people/paolo/disk_sched/mst-2015.pdf
-[2] P. Valente and M. Andreolini, "Improving Application
+[2]
+ P. Valente and M. Andreolini, "Improving Application
Responsiveness with the BFQ Disk I/O Scheduler", Proceedings of
the 5th Annual International Systems and Storage Conference
(SYSTOR '12), June 2012.
+
Slightly extended version:
- http://algogroup.unimore.it/people/paolo/disk_sched/bfq-v1-suite-
- results.pdf
-[3] https://github.com/Algodev-github/S
+ http://algogroup.unimore.it/people/paolo/disk_sched/bfq-v1-suite-results.pdf
+
+[3]
+ https://github.com/Algodev-github/S
- Notes on the Generic Block Layer Rewrite in Linux 2.5
- =====================================================
+=====================================================
+Notes on the Generic Block Layer Rewrite in Linux 2.5
+=====================================================
+
+.. note::
+
+ It seems that there are lot of outdated stuff here. This seems
+ to be written somewhat as a task list. Yet, eventually, something
+ here might still be useful.
Notes Written on Jan 15, 2002:
- Jens Axboe <jens.axboe@oracle.com>
- Suparna Bhattacharya <suparna@in.ibm.com>
+
+ - Jens Axboe <jens.axboe@oracle.com>
+ - Suparna Bhattacharya <suparna@in.ibm.com>
Last Updated May 2, 2002
+
September 2003: Updated I/O Scheduler portions
- Nick Piggin <npiggin@kernel.dk>
+ - Nick Piggin <npiggin@kernel.dk>
-Introduction:
+Introduction
+============
These are some notes describing some aspects of the 2.5 block layer in the
context of the bio rewrite. The idea is to bring out some of the key
Please mail corrections & suggestions to suparna@in.ibm.com.
-Credits:
----------
+Credits
+=======
2.5 bio rewrite:
- Jens Axboe <jens.axboe@oracle.com>
+ - Jens Axboe <jens.axboe@oracle.com>
Many aspects of the generic block layer redesign were driven by and evolved
over discussions, prior patches and the collective experience of several
The following people helped with review comments and inputs for this
document:
- Christoph Hellwig <hch@infradead.org>
- Arjan van de Ven <arjanv@redhat.com>
- Randy Dunlap <rdunlap@xenotime.net>
- Andre Hedrick <andre@linux-ide.org>
+
+ - Christoph Hellwig <hch@infradead.org>
+ - Arjan van de Ven <arjanv@redhat.com>
+ - Randy Dunlap <rdunlap@xenotime.net>
+ - Andre Hedrick <andre@linux-ide.org>
The following people helped with fixes/contributions to the bio patches
while it was still work-in-progress:
- David S. Miller <davem@redhat.com>
+ - David S. Miller <davem@redhat.com>
-Description of Contents:
-------------------------
-1. Scope for tuning of logic to various needs
- 1.1 Tuning based on device or low level driver capabilities
+.. Description of Contents:
+
+ 1. Scope for tuning of logic to various needs
+ 1.1 Tuning based on device or low level driver capabilities
- Per-queue parameters
- Highmem I/O support
- I/O scheduler modularization
- 1.2 Tuning based on high level requirements/capabilities
+ 1.2 Tuning based on high level requirements/capabilities
1.2.1 Request Priority/Latency
- 1.3 Direct access/bypass to lower layers for diagnostics and special
- device operations
+ 1.3 Direct access/bypass to lower layers for diagnostics and special
+ device operations
1.3.1 Pre-built commands
-2. New flexible and generic but minimalist i/o structure or descriptor
- (instead of using buffer heads at the i/o layer)
- 2.1 Requirements/Goals addressed
- 2.2 The bio struct in detail (multi-page io unit)
- 2.3 Changes in the request structure
-3. Using bios
- 3.1 Setup/teardown (allocation, splitting)
- 3.2 Generic bio helper routines
- 3.2.1 Traversing segments and completion units in a request
- 3.2.2 Setting up DMA scatterlists
- 3.2.3 I/O completion
- 3.2.4 Implications for drivers that do not interpret bios (don't handle
- multiple segments)
- 3.3 I/O submission
-4. The I/O scheduler
-5. Scalability related changes
- 5.1 Granular locking: Removal of io_request_lock
- 5.2 Prepare for transition to 64 bit sector_t
-6. Other Changes/Implications
- 6.1 Partition re-mapping handled by the generic block layer
-7. A few tips on migration of older drivers
-8. A list of prior/related/impacted patches/ideas
-9. Other References/Discussion Threads
+ 2. New flexible and generic but minimalist i/o structure or descriptor
+ (instead of using buffer heads at the i/o layer)
+ 2.1 Requirements/Goals addressed
+ 2.2 The bio struct in detail (multi-page io unit)
+ 2.3 Changes in the request structure
+ 3. Using bios
+ 3.1 Setup/teardown (allocation, splitting)
+ 3.2 Generic bio helper routines
+ 3.2.1 Traversing segments and completion units in a request
+ 3.2.2 Setting up DMA scatterlists
+ 3.2.3 I/O completion
+ 3.2.4 Implications for drivers that do not interpret bios (don't handle
+ multiple segments)
+ 3.3 I/O submission
+ 4. The I/O scheduler
+ 5. Scalability related changes
+ 5.1 Granular locking: Removal of io_request_lock
+ 5.2 Prepare for transition to 64 bit sector_t
+ 6. Other Changes/Implications
+ 6.1 Partition re-mapping handled by the generic block layer
+ 7. A few tips on migration of older drivers
+ 8. A list of prior/related/impacted patches/ideas
+ 9. Other References/Discussion Threads
----------------------------------------------------------------------------
Bio Notes
---------
+=========
Let us discuss the changes in the context of how some overall goals for the
block layer are addressed.
1. Scope for tuning the generic logic to satisfy various requirements
+=====================================================================
The block layer design supports adaptable abstractions to handle common
processing with the ability to tune the logic to an appropriate extent
capabilities.
1.1 Tuning based on low level device / driver capabilities
+----------------------------------------------------------
Sophisticated devices with large built-in caches, intelligent i/o scheduling
optimizations, high memory DMA support, etc may find some of the
Sets two variables that limit the size of the request.
- The request queue's max_sectors, which is a soft size in
- units of 512 byte sectors, and could be dynamically varied
- by the core kernel.
+ units of 512 byte sectors, and could be dynamically varied
+ by the core kernel.
- The request queue's max_hw_sectors, which is a hard limit
- and reflects the maximum size request a driver can handle
- in units of 512 byte sectors.
+ and reflects the maximum size request a driver can handle
+ in units of 512 byte sectors.
The default for both max_sectors and max_hw_sectors is
255. The upper limit of max_sectors is 1024.
New queue flags:
- QUEUE_FLAG_CLUSTER (see 3.2.2)
- QUEUE_FLAG_QUEUED (see 3.2.4)
+ - QUEUE_FLAG_CLUSTER (see 3.2.2)
+ - QUEUE_FLAG_QUEUED (see 3.2.4)
ii. High-mem i/o capabilities are now considered the default
See section 4. The I/O scheduler for details.
1.2 Tuning Based on High level code capabilities
+------------------------------------------------
i. Application capabilities for raw i/o
other upper level mechanism to communicate such settings to block.
1.2.1 Request Priority/Latency
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Todo/Under discussion:
-Arjan's proposed request priority scheme allows higher levels some broad
+Todo/Under discussion::
+
+ Arjan's proposed request priority scheme allows higher levels some broad
control (high/med/low) over the priority of an i/o request vs other pending
requests in the queue. For example it allows reads for bringing in an
executable page on demand to be given a higher priority over pending write
1.3 Direct Access to Low level Device/Driver Capabilities (Bypass mode)
- (e.g Diagnostics, Systems Management)
+-----------------------------------------------------------------------
+
+(e.g Diagnostics, Systems Management)
There are situations where high-level code needs to have direct access to
the low level device capabilities or requires the ability to issue commands
request->buffer, request->sector and request->nr_sectors or
request->current_nr_sectors fields itself rather than using the block layer
end_request or end_that_request_first completion interfaces.
-(See 2.3 or Documentation/block/request.txt for a brief explanation of
+(See 2.3 or Documentation/block/request.rst for a brief explanation of
the request structure fields)
-[TBD: end_that_request_last should be usable even in this case;
-Perhaps an end_that_direct_request_first routine could be implemented to make
-handling direct requests easier for such drivers; Also for drivers that
-expect bios, a helper function could be provided for setting up a bio
-corresponding to a data buffer]
-
-<JENS: I dont understand the above, why is end_that_request_first() not
-usable? Or _last for that matter. I must be missing something>
-<SUP: What I meant here was that if the request doesn't have a bio, then
- end_that_request_first doesn't modify nr_sectors or current_nr_sectors,
- and hence can't be used for advancing request state settings on the
- completion of partial transfers. The driver has to modify these fields
- directly by hand.
- This is because end_that_request_first only iterates over the bio list,
- and always returns 0 if there are none associated with the request.
- _last works OK in this case, and is not a problem, as I mentioned earlier
->
+::
+
+ [TBD: end_that_request_last should be usable even in this case;
+ Perhaps an end_that_direct_request_first routine could be implemented to make
+ handling direct requests easier for such drivers; Also for drivers that
+ expect bios, a helper function could be provided for setting up a bio
+ corresponding to a data buffer]
+
+ <JENS: I dont understand the above, why is end_that_request_first() not
+ usable? Or _last for that matter. I must be missing something>
+
+ <SUP: What I meant here was that if the request doesn't have a bio, then
+ end_that_request_first doesn't modify nr_sectors or current_nr_sectors,
+ and hence can't be used for advancing request state settings on the
+ completion of partial transfers. The driver has to modify these fields
+ directly by hand.
+ This is because end_that_request_first only iterates over the bio list,
+ and always returns 0 if there are none associated with the request.
+ _last works OK in this case, and is not a problem, as I mentioned earlier
+ >
1.3.1 Pre-built Commands
+^^^^^^^^^^^^^^^^^^^^^^^^
A request can be created with a pre-built custom command to be sent directly
to the device. The cmd block in the request structure has room for filling
the pre-builder hook can be invoked there.
-2. Flexible and generic but minimalist i/o structure/descriptor.
+2. Flexible and generic but minimalist i/o structure/descriptor
+===============================================================
2.1 Reason for a new structure and requirements addressed
+---------------------------------------------------------
Prior to 2.5, buffer heads were used as the unit of i/o at the generic block
layer, and the low level request structure was associated with a chain of
The following were some of the goals and expectations considered in the
redesign of the block i/o data structure in 2.5.
-i. Should be appropriate as a descriptor for both raw and buffered i/o -
+1. Should be appropriate as a descriptor for both raw and buffered i/o -
avoid cache related fields which are irrelevant in the direct/page i/o path,
or filesystem block size alignment restrictions which may not be relevant
for raw i/o.
-ii. Ability to represent high-memory buffers (which do not have a virtual
+2. Ability to represent high-memory buffers (which do not have a virtual
address mapping in kernel address space).
-iii.Ability to represent large i/os w/o unnecessarily breaking them up (i.e
+3. Ability to represent large i/os w/o unnecessarily breaking them up (i.e
greater than PAGE_SIZE chunks in one shot)
-iv. At the same time, ability to retain independent identity of i/os from
+4. At the same time, ability to retain independent identity of i/os from
different sources or i/o units requiring individual completion (e.g. for
latency reasons)
-v. Ability to represent an i/o involving multiple physical memory segments
+5. Ability to represent an i/o involving multiple physical memory segments
(including non-page aligned page fragments, as specified via readv/writev)
without unnecessarily breaking it up, if the underlying device is capable of
handling it.
-vi. Preferably should be based on a memory descriptor structure that can be
+6. Preferably should be based on a memory descriptor structure that can be
passed around different types of subsystems or layers, maybe even
networking, without duplication or extra copies of data/descriptor fields
themselves in the process
-vii.Ability to handle the possibility of splits/merges as the structure passes
+7. Ability to handle the possibility of splits/merges as the structure passes
through layered drivers (lvm, md, evms), with minimal overhead.
The solution was to define a new structure (bio) for the block layer,
mapped to bio structures.
2.2 The bio struct
+------------------
The bio structure uses a vector representation pointing to an array of tuples
of <page, offset, len> to describe the i/o buffer, and has various other
Notice that this representation means that a bio has no virtual address
mapping at all (unlike buffer heads).
-struct bio_vec {
+::
+
+ struct bio_vec {
struct page *bv_page;
unsigned short bv_len;
unsigned short bv_offset;
-};
+ };
-/*
- * main unit of I/O for the block layer and lower layers (ie drivers)
- */
-struct bio {
+ /*
+ * main unit of I/O for the block layer and lower layers (ie drivers)
+ */
+ struct bio {
struct bio *bi_next; /* request queue link */
struct block_device *bi_bdev; /* target device */
unsigned long bi_flags; /* status, command, etc */
bio_end_io_t *bi_end_io; /* bi_end_io (bio) */
atomic_t bi_cnt; /* pin count: free when it hits zero */
void *bi_private;
-};
+ };
With this multipage bio design:
- Splitting of an i/o request across multiple devices (as in the case of
lvm or raid) is achieved by cloning the bio (where the clone points to
the same bi_io_vec array, but with the index and size accordingly modified)
-- A linked list of bios is used as before for unrelated merges (*) - this
+- A linked list of bios is used as before for unrelated merges [#]_ - this
avoids reallocs and makes independent completions easier to handle.
- Code that traverses the req list can find all the segments of a bio
by using rq_for_each_segment. This handles the fact that a request
field to keep track of the next bio_vec entry to process.
(e.g a 1MB bio_vec needs to be handled in max 128kB chunks for IDE)
[TBD: Should preferably also have a bi_voffset and bi_vlen to avoid modifying
- bi_offset an len fields]
+ bi_offset an len fields]
+
+.. [#]
-(*) unrelated merges -- a request ends up containing two or more bios that
- didn't originate from the same place.
+ unrelated merges -- a request ends up containing two or more bios that
+ didn't originate from the same place.
bi_end_io() i/o callback gets called on i/o completion of the entire bio.
right now). The intent however is to enable clustering of pages etc to
become possible. The pagebuf abstraction layer from SGI also uses multi-page
bios, but that is currently not included in the stock development kernels.
-The same is true of Andrew Morton's work-in-progress multipage bio writeout
+The same is true of Andrew Morton's work-in-progress multipage bio writeout
and readahead patches.
2.3 Changes in the Request Structure
+------------------------------------
The request structure is the structure that gets passed down to low level
drivers. The block layer make_request function builds up a request structure,
Only some relevant fields (mainly those which changed or may be referred
to in some of the discussion here) are listed below, not necessarily in
the order in which they occur in the structure (see include/linux/blkdev.h)
-Refer to Documentation/block/request.txt for details about all the request
+Refer to Documentation/block/request.rst for details about all the request
structure fields and a quick reference about the layers which are
-supposed to use or modify those fields.
+supposed to use or modify those fields::
-struct request {
+ struct request {
struct list_head queuelist; /* Not meant to be directly accessed by
the driver.
Used by q->elv_next_request_fn
.
struct bio *bio, *biotail; /* bio list instead of bh */
struct request_list *rl;
-}
-
+ }
+
See the req_ops and req_flag_bits definitions for an explanation of the various
flags available. Some bits are used by the block layer or i/o scheduler.
-
+
The behaviour of the various sector counts are almost the same as before,
except that since we have multi-segment bios, current_nr_sectors refers
to the numbers of sectors in the current segment being processed which could
functions which the driver uses. (Section 1.3)
3. Using bios
+=============
3.1 Setup/Teardown
+------------------
There are routines for managing the allocation, and reference counting, and
freeing of bios (bio_alloc, bio_get, bio_put).
The caller of bio_alloc is expected to taken certain steps to avoid
deadlocks, e.g. avoid trying to allocate more memory from the pool while
already holding memory obtained from the pool.
-[TBD: This is a potential issue, though a rare possibility
- in the bounce bio allocation that happens in the current code, since
- it ends up allocating a second bio from the same pool while
- holding the original bio ]
+
+::
+
+ [TBD: This is a potential issue, though a rare possibility
+ in the bounce bio allocation that happens in the current code, since
+ it ends up allocating a second bio from the same pool while
+ holding the original bio ]
Memory allocated from the pool should be released back within a limited
amount of time (in the case of bio, that would be after the i/o is completed).
in lvm or md.
3.2 Generic bio helper Routines
+-------------------------------
3.2.1 Traversing segments and completion units in a request
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The macro rq_for_each_segment() should be used for traversing the bios
in the request list (drivers should avoid directly trying to do it
themselves). Using these helpers should also make it easier to cope
with block changes in the future.
+::
+
struct req_iterator iter;
rq_for_each_segment(bio_vec, rq, iter)
/* bio_vec is now current segment */
need to be reorganized to support multi-segment bios.
3.2.2 Setting up DMA scatterlists
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The blk_rq_map_sg() helper routine would be used for setting up scatter
gather lists from a request, so a driver need not do it on its own.
a driver could handle)
3.2.3 I/O completion
+^^^^^^^^^^^^^^^^^^^^
The existing generic block layer helper routines end_request,
end_that_request_first and end_that_request_last can be used for i/o
bio support, end_that_request_first requires an additional argument indicating
the number of sectors completed.
-3.2.4 Implications for drivers that do not interpret bios (don't handle
- multiple segments)
+3.2.4 Implications for drivers that do not interpret bios
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+(don't handle multiple segments)
Drivers that do not interpret bios e.g those which do not handle multiple
segments and do not support i/o into high memory addresses (require bounce
direct access requests which only specify rq->buffer without a valid rq->bio)
3.3 I/O Submission
+------------------
The routine submit_bio() is used to submit a single io. Higher level i/o
routines make use of this:
(a) Buffered i/o:
+
The routine submit_bh() invokes submit_bio() on a bio corresponding to the
bh, allocating the bio if required. ll_rw_block() uses submit_bh() as before.
(b) Kiobuf i/o (for raw/direct i/o):
+
The ll_rw_kio() routine breaks up the kiobuf into page sized chunks and
maps the array to one or more multi-page bios, issuing submit_bio() to
perform the i/o on each of these.
(c) Page i/o:
+
Todo/Under discussion:
Andrew Morton's multi-page bio patches attempt to issue multi-page
abstraction, but intended to be as lightweight as possible).
(d) Direct access i/o:
+
Direct access requests that do not contain bios would be submitted differently
as discussed earlier in section 1.3.
4. The I/O scheduler
+====================
+
I/O scheduler, a.k.a. elevator, is implemented in two layers. Generic dispatch
queue and specific I/O schedulers. Unless stated otherwise, elevator is used
to refer to both parts and I/O scheduler to specific I/O schedulers.
-Block layer implements generic dispatch queue in block/*.c.
+Block layer implements generic dispatch queue in `block/*.c`.
The generic dispatch queue is responsible for requeueing, handling non-fs
requests and all other subtleties.
keeping work.
4.1. I/O scheduler API
+----------------------
The functions an elevator may implement are: (* are mandatory)
+
+=============================== ================================================
elevator_merge_fn called to query requests for merge with a bio
elevator_merge_req_fn called when two requests get merged. the one
elevator_init_fn*
elevator_exit_fn Allocate and free any elevator specific storage
for a queue.
+=============================== ================================================
4.2 Request flows seen by I/O schedulers
+----------------------------------------
+
All requests seen by I/O schedulers strictly follow one of the following three
flows.
-> put_req_fn
4.3 I/O scheduler implementation
+--------------------------------
+
The generic i/o scheduler algorithm attempts to sort/merge/batch requests for
optimal disk scan and request servicing performance (based on generic
principles and device capabilities), optimized for:
+
i. improved throughput
ii. improved latency
iii. better utilization of h/w & CPU time
a big request from the broken up pieces coming by.
4.4 I/O contexts
+----------------
+
I/O contexts provide a dynamically allocated per process data area. They may
be used in I/O schedulers, and in the block layer (could be used for IO statis,
-priorities for example). See *io_context in block/ll_rw_blk.c, and as-iosched.c
+priorities for example). See `*io_context` in block/ll_rw_blk.c, and as-iosched.c
for an example of usage in an i/o scheduler.
5. Scalability related changes
+==============================
5.1 Granular Locking: io_request_lock replaced by a per-queue lock
+------------------------------------------------------------------
The global io_request_lock has been removed as of 2.5, to avoid
the scalability bottleneck it was causing, and has been replaced by more
should still be SMP safe. Drivers are free to drop the queue
lock themselves, if required. Drivers that explicitly used the
io_request_lock for serialization need to be modified accordingly.
-Usually it's as easy as adding a global lock:
+Usually it's as easy as adding a global lock::
static DEFINE_SPINLOCK(my_driver_lock);
and passing the address to that lock to blk_init_queue().
5.2 64 bit sector numbers (sector_t prepares for 64 bit support)
+----------------------------------------------------------------
The sector number used in the bio structure has been changed to sector_t,
which could be defined as 64 bit in preparation for 64 bit sector support.
6. Other Changes/Implications
+=============================
6.1 Partition re-mapping handled by the generic block layer
+-----------------------------------------------------------
In 2.5 some of the gendisk/partition related code has been reorganized.
Now the generic block layer performs partition-remapping early and thus
7. A Few Tips on Migration of older drivers
+===========================================
Old-style drivers that just use CURRENT and ignores clustered requests,
may not need much change. The generic layer will automatically handle
Drivers no longer have to map a {partition, sector offset} into the
correct absolute location anymore, this is done by the block layer, so
-where a driver received a request ala this before:
+where a driver received a request ala this before::
rq->rq_dev = mk_kdev(3, 5); /* /dev/hda5 */
rq->sector = 0; /* first sector on hda5 */
- it will now see
+it will now see::
rq->rq_dev = mk_kdev(3, 0); /* /dev/hda */
rq->sector = 123128; /* offset from start of disk */
8. Prior/Related/Impacted patches
+=================================
8.1. Earlier kiobuf patches (sct/axboe/chait/hch/mkp)
+-----------------------------------------------------
+
- orig kiobuf & raw i/o patches (now in 2.4 tree)
- direct kiobuf based i/o to devices (no intermediate bh's)
- page i/o using kiobuf
- kiobuf splitting for lvm (mkp)
- elevator support for kiobuf request merging (axboe)
+
8.2. Zero-copy networking (Dave Miller)
+---------------------------------------
+
8.3. SGI XFS - pagebuf patches - use of kiobufs
+-----------------------------------------------
8.4. Multi-page pioent patch for bio (Christoph Hellwig)
+--------------------------------------------------------
8.5. Direct i/o implementation (Andrea Arcangeli) since 2.4.10-pre11
+--------------------------------------------------------------------
8.6. Async i/o implementation patch (Ben LaHaise)
+-------------------------------------------------
8.7. EVMS layering design (IBM EVMS team)
-8.8. Larger page cache size patch (Ben LaHaise) and
- Large page size (Daniel Phillips)
+-----------------------------------------
+8.8. Larger page cache size patch (Ben LaHaise) and Large page size (Daniel Phillips)
+-------------------------------------------------------------------------------------
+
=> larger contiguous physical memory buffers
+
8.9. VM reservations patch (Ben LaHaise)
+----------------------------------------
8.10. Write clustering patches ? (Marcelo/Quintela/Riel ?)
+----------------------------------------------------------
8.11. Block device in page cache patch (Andrea Archangeli) - now in 2.4.10+
-8.12. Multiple block-size transfers for faster raw i/o (Shailabh Nagar,
- Badari)
+---------------------------------------------------------------------------
+8.12. Multiple block-size transfers for faster raw i/o (Shailabh Nagar, Badari)
+-------------------------------------------------------------------------------
8.13 Priority based i/o scheduler - prepatches (Arjan van de Ven)
+------------------------------------------------------------------
8.14 IDE Taskfile i/o patch (Andre Hedrick)
+--------------------------------------------
8.15 Multi-page writeout and readahead patches (Andrew Morton)
+---------------------------------------------------------------
8.16 Direct i/o patches for 2.5 using kvec and bio (Badari Pulavarthy)
+-----------------------------------------------------------------------
-9. Other References:
+9. Other References
+===================
-9.1 The Splice I/O Model - Larry McVoy (and subsequent discussions on lkml,
-and Linus' comments - Jan 2001)
-9.2 Discussions about kiobuf and bh design on lkml between sct, linus, alan
-et al - Feb-March 2001 (many of the initial thoughts that led to bio were
-brought up in this discussion thread)
-9.3 Discussions on mempool on lkml - Dec 2001.
+9.1 The Splice I/O Model
+------------------------
+
+Larry McVoy (and subsequent discussions on lkml, and Linus' comments - Jan 2001
+9.2 Discussions about kiobuf and bh design
+------------------------------------------
+
+On lkml between sct, linus, alan et al - Feb-March 2001 (many of the
+initial thoughts that led to bio were brought up in this discussion thread)
+
+9.3 Discussions on mempool on lkml - Dec 2001.
+----------------------------------------------
-
-Immutable biovecs and biovec iterators:
-=======================================
+======================================
+Immutable biovecs and biovec iterators
+======================================
Kent Overstreet <kmo@daterainc.com>
Usage of helpers:
=================
-* The following helpers whose names have the suffix of "_all" can only be used
-on non-BIO_CLONED bio. They are usually used by filesystem code. Drivers
-shouldn't use them because the bio may have been split before it reached the
-driver.
+* The following helpers whose names have the suffix of `_all` can only be used
+ on non-BIO_CLONED bio. They are usually used by filesystem code. Drivers
+ shouldn't use them because the bio may have been split before it reached the
+ driver.
+
+::
bio_for_each_segment_all()
bio_first_bvec_all()
bio_last_bvec_all()
* The following helpers iterate over single-page segment. The passed 'struct
-bio_vec' will contain a single-page IO vector during the iteration
+ bio_vec' will contain a single-page IO vector during the iteration::
bio_for_each_segment()
bio_for_each_segment_all()
* The following helpers iterate over multi-page bvec. The passed 'struct
-bio_vec' will contain a multi-page IO vector during the iteration
+ bio_vec' will contain a multi-page IO vector during the iteration::
bio_for_each_bvec()
rq_for_each_bvec()
--- /dev/null
+===============================
+Generic Block Device Capability
+===============================
+
+This file documents the sysfs file block/<disk>/capability
+
+capability is a hex word indicating which capabilities a specific disk
+supports. For more information on bits not listed here, see
+include/linux/genhd.h
+
+GENHD_FL_MEDIA_CHANGE_NOTIFY
+----------------------------
+
+Value: 4
+
+When this bit is set, the disk supports Asynchronous Notification
+of media change events. These events will be broadcast to user
+space via kernel uevent.
+++ /dev/null
-Generic Block Device Capability
-===============================================================================
-This file documents the sysfs file block/<disk>/capability
-
-capability is a hex word indicating which capabilities a specific disk
-supports. For more information on bits not listed here, see
-include/linux/genhd.h
-
-Capability Value
--------------------------------------------------------------------------------
-GENHD_FL_MEDIA_CHANGE_NOTIFY 4
- When this bit is set, the disk supports Asynchronous Notification
- of media change events. These events will be broadcast to user
- space via kernel uevent.
-
+==============================================
Embedded device command line partition parsing
-=====================================================================
+==============================================
The "blkdevparts" command line option adds support for reading the
block device partition table from the kernel command line.
<size>
partition size, in bytes, such as: 512, 1m, 1G.
size may contain an optional suffix of (upper or lower case):
+
K, M, G, T, P, E.
+
"-" is used to denote all remaining space.
<offset>
partition start address, in bytes.
offset may contain an optional suffix of (upper or lower case):
+
K, M, G, T, P, E.
(part-name)
User space application can access partition by partition name.
Example:
+
eMMC disk names are "mmcblk0" and "mmcblk0boot0".
- bootargs:
+ bootargs::
+
'blkdevparts=mmcblk0:1G(data0),1G(data1),-;mmcblk0boot0:1m(boot),-(kernel)'
- dmesg:
+ dmesg::
+
mmcblk0: p1(data0) p2(data1) p3()
mmcblk0boot0: p1(boot) p2(kernel)
-----------------------------------------------------------------------
-1. INTRODUCTION
+==============
+Data Integrity
+==============
+
+1. Introduction
+===============
Modern filesystems feature checksumming of data and metadata to
protect against data corruption. However, the detection of the
allows not only corruption prevention but also isolation of the point
of failure.
-----------------------------------------------------------------------
-2. THE DATA INTEGRITY EXTENSIONS
+2. The Data Integrity Extensions
+================================
As written, the protocol extensions only protect the path between
controller and storage device. However, many controllers actually
bodies (T10, T13), Oracle and its partners are trying to standardize
them within the Storage Networking Industry Association.
-----------------------------------------------------------------------
-3. KERNEL CHANGES
+3. Kernel Changes
+=================
The data integrity framework in Linux enables protection information
to be pinned to I/Os and sent to/received from controllers that
interface for this is being worked on.
-----------------------------------------------------------------------
-4. BLOCK LAYER IMPLEMENTATION DETAILS
+4. Block Layer Implementation Details
+=====================================
-4.1 BIO
+4.1 Bio
+-------
The data integrity patches add a new field to struct bio when
CONFIG_BLK_DEV_INTEGRITY is enabled. bio_integrity(bio) returns a
bio_free() will automatically free the bip.
-4.2 BLOCK DEVICE
+4.2 Block Device
+----------------
Because the format of the protection data is tied to the physical
disk, each block device has been extended with a block integrity
will require extra work due to the application tag.
-----------------------------------------------------------------------
-5.0 BLOCK LAYER INTEGRITY API
+5.0 Block Layer Integrity API
+=============================
-5.1 NORMAL FILESYSTEM
+5.1 Normal Filesystem
+---------------------
The normal filesystem is unaware that the underlying block device
is capable of sending/receiving integrity metadata. The IMD will
in case of a WRITE. A READ request will cause the I/O integrity
to be verified upon completion.
- IMD generation and verification can be toggled using the
+ IMD generation and verification can be toggled using the::
/sys/block/<bdev>/integrity/write_generate
- and
+ and::
/sys/block/<bdev>/integrity/read_verify
flags.
-5.2 INTEGRITY-AWARE FILESYSTEM
+5.2 Integrity-Aware Filesystem
+------------------------------
A filesystem that is integrity-aware can prepare I/Os with IMD
attached. It can also use the application tag space if this is
supported by the block device.
- bool bio_integrity_prep(bio);
+ `bool bio_integrity_prep(bio);`
To generate IMD for WRITE and to set up buffers for READ, the
filesystem must call bio_integrity_prep(bio).
Complete bio with error if prepare failed for some reson.
-5.3 PASSING EXISTING INTEGRITY METADATA
+5.3 Passing Existing Integrity Metadata
+---------------------------------------
Filesystems that either generate their own integrity metadata or
are capable of transferring IMD from user space can use the
following calls:
- struct bip * bio_integrity_alloc(bio, gfp_mask, nr_pages);
+ `struct bip * bio_integrity_alloc(bio, gfp_mask, nr_pages);`
Allocates the bio integrity payload and hangs it off of the bio.
nr_pages indicate how many pages of protection data need to be
The integrity payload will be freed at bio_free() time.
- int bio_integrity_add_page(bio, page, len, offset);
+ `int bio_integrity_add_page(bio, page, len, offset);`
Attaches a page containing integrity metadata to an existing
bio. The bio must have an existing bip,
integrity upon completion.
-5.4 REGISTERING A BLOCK DEVICE AS CAPABLE OF EXCHANGING INTEGRITY
- METADATA
+5.4 Registering A Block Device As Capable Of Exchanging Integrity Metadata
+--------------------------------------------------------------------------
To enable integrity exchange on a block device the gendisk must be
registered as capable:
- int blk_integrity_register(gendisk, blk_integrity);
+ `int blk_integrity_register(gendisk, blk_integrity);`
The blk_integrity struct is a template and should contain the
- following:
+ following::
static struct blk_integrity my_profile = {
.name = "STANDARDSBODY-TYPE-VARIANT-CSUM",
.generate_fn = my_generate_fn,
- .verify_fn = my_verify_fn,
+ .verify_fn = my_verify_fn,
.tuple_size = sizeof(struct my_tuple_size),
.tag_size = <tag bytes per hw sector>,
};
0 depending on the value of the Control Mode Page ATO bit.
----------------------------------------------------------------------
+
2007-12-24 Martin K. Petersen <martin.petersen@oracle.com>
+==============================
Deadline IO scheduler tunables
==============================
Selecting IO schedulers
-----------------------
-Refer to Documentation/block/switching-sched.txt for information on
+Refer to Documentation/block/switching-sched.rst for information on
selecting an io scheduler on a per-device basis.
-
-********************************************************************************
-
+------------------------------------------------------------------------------
read_expire (in ms)
------------
+-----------------------
The goal of the deadline io scheduler is to attempt to guarantee a start
service time for a request. As we focus mainly on read latencies, this is
write_expire (in ms)
------------
+-----------------------
Similar to read_expire mentioned above, but for writes.
fifo_batch (number of requests)
-----------
+------------------------------------
-Requests are grouped into ``batches'' of a particular data direction (read or
+Requests are grouped into ``batches`` of a particular data direction (read or
write) which are serviced in increasing sector order. To limit extra seeking,
deadline expiries are only checked between batches. fifo_batch controls the
maximum number of requests per batch.
writes_starved (number of dispatches)
---------------
+--------------------------------------
When we have to move requests from the io scheduler queue to the block
device dispatch queue, we always give a preference to reads. However, we
front_merges (bool)
-------------
+----------------------
Sometimes it happens that a request enters the io scheduler that is contiguous
with a request that is already on the queue. Either it fits in the back of that
Nov 11 2002, Jens Axboe <jens.axboe@oracle.com>
-
-
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=====
+Block
+=====
+
+.. toctree::
+ :maxdepth: 1
+
+ bfq-iosched
+ biodoc
+ biovecs
+ capability
+ cmdline-partition
+ data-integrity
+ deadline-iosched
+ ioprio
+ kyber-iosched
+ null_blk
+ pr
+ queue-sysfs
+ request
+ stat
+ switching-sched
+ writeback_cache_control
+===================
Block io priorities
===================
Tools
-----
-See below for a sample ionice tool. Usage:
+See below for a sample ionice tool. Usage::
-# ionice -c<class> -n<level> -p<pid>
+
# ionice -c<class> -n<level> -p<pid>
If pid isn't given, the current process is assumed. IO priority settings
are inherited on fork, so you can use ionice to start the process at a given
-level:
+level::
-# ionice -c2 -n0 /bin/ls
+ # ionice -c2 -n0 /bin/ls
will run ls at the best-effort scheduling class at the highest priority.
-For a running process, you can give the pid instead:
+For a running process, you can give the pid instead::
-# ionice -c1 -n2 -p100
+ # ionice -c1 -n2 -p100
will change pid 100 to run at the realtime scheduling class, at priority 2.
----> snip ionice.c tool <---
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <errno.h>
-#include <getopt.h>
-#include <unistd.h>
-#include <sys/ptrace.h>
-#include <asm/unistd.h>
-
-extern int sys_ioprio_set(int, int, int);
-extern int sys_ioprio_get(int, int);
-
-#if defined(__i386__)
-#define __NR_ioprio_set 289
-#define __NR_ioprio_get 290
-#elif defined(__ppc__)
-#define __NR_ioprio_set 273
-#define __NR_ioprio_get 274
-#elif defined(__x86_64__)
-#define __NR_ioprio_set 251
-#define __NR_ioprio_get 252
-#elif defined(__ia64__)
-#define __NR_ioprio_set 1274
-#define __NR_ioprio_get 1275
-#else
-#error "Unsupported arch"
-#endif
-
-static inline int ioprio_set(int which, int who, int ioprio)
-{
+ionice.c tool::
+
+ #include <stdio.h>
+ #include <stdlib.h>
+ #include <errno.h>
+ #include <getopt.h>
+ #include <unistd.h>
+ #include <sys/ptrace.h>
+
#include <asm/unistd.h>
+
+ extern int sys_ioprio_set(int, int, int);
+ extern int sys_ioprio_get(int, int);
+
+ #if defined(__i386__)
+ #define __NR_ioprio_set 289
+ #define __NR_ioprio_get 290
+ #elif defined(__ppc__)
+ #define __NR_ioprio_set 273
+ #define __NR_ioprio_get 274
+ #elif defined(__x86_64__)
+ #define __NR_ioprio_set 251
+ #define __NR_ioprio_get 252
+ #elif defined(__ia64__)
+ #define __NR_ioprio_set 1274
+ #define __NR_ioprio_get 1275
+ #else
+ #error "Unsupported arch"
+ #endif
+
+ static inline int ioprio_set(int which, int who, int ioprio)
+ {
return syscall(__NR_ioprio_set, which, who, ioprio);
-}
+ }
-static inline int ioprio_get(int which, int who)
-{
+ static inline int ioprio_get(int which, int who)
+ {
return syscall(__NR_ioprio_get, which, who);
-}
+ }
-enum {
+ enum {
IOPRIO_CLASS_NONE,
IOPRIO_CLASS_RT,
IOPRIO_CLASS_BE,
IOPRIO_CLASS_IDLE,
-};
+ };
-enum {
+ enum {
IOPRIO_WHO_PROCESS = 1,
IOPRIO_WHO_PGRP,
IOPRIO_WHO_USER,
-};
+ };
-#define IOPRIO_CLASS_SHIFT 13
+ #define IOPRIO_CLASS_SHIFT 13
-const char *to_prio[] = { "none", "realtime", "best-effort", "idle", };
+ const char *to_prio[] = { "none", "realtime", "best-effort", "idle", };
-int main(int argc, char *argv[])
-{
+ int main(int argc, char *argv[])
+ {
int ioprio = 4, set = 0, ioprio_class = IOPRIO_CLASS_BE;
int c, pid = 0;
}
return 0;
-}
-
----> snip ionice.c tool <---
+ }
March 11 2005, Jens Axboe <jens.axboe@oracle.com>
+============================
Kyber I/O scheduler tunables
-===========================
+============================
The only two tunables for the Kyber scheduler are the target latencies for
reads and synchronous writes. Kyber will throttle requests in order to meet
+========================
Null block device driver
-================================================================================
+========================
-I. Overview
+1. Overview
+===========
The null block device (/dev/nullb*) is used for benchmarking the various
block-layer implementations. It emulates a block device of X gigabytes in size.
The following instances are possible:
Single-queue block-layer
+
- Request-based.
- Single submission queue per device.
- Implements IO scheduling algorithms (CFQ, Deadline, noop).
+
Multi-queue block-layer
+
- Request-based.
- Configurable submission queues per device.
+
No block-layer (Known as bio-based)
+
- Bio-based. IO requests are submitted directly to the device driver.
- Directly accepts bio data structure and returns them.
All of them have a completion queue for each core in the system.
-II. Module parameters applicable for all instances:
+2. Module parameters applicable for all instances
+=================================================
queue_mode=[0-2]: Default: 2-Multi-queue
Selects which block-layer the module should instantiate with.
- 0: Bio-based.
- 1: Single-queue.
- 2: Multi-queue.
+ = ============
+ 0 Bio-based
+ 1 Single-queue
+ 2 Multi-queue
+ = ============
home_node=[0--nr_nodes]: Default: NUMA_NO_NODE
Selects what CPU node the data structures are allocated from.
irqmode=[0-2]: Default: 1-Soft-irq
The completion mode used for completing IOs to the block-layer.
- 0: None.
- 1: Soft-irq. Uses IPI to complete IOs across CPU nodes. Simulates the overhead
+ = ===========================================================================
+ 0 None.
+ 1 Soft-irq. Uses IPI to complete IOs across CPU nodes. Simulates the overhead
when IOs are issued from another CPU node than the home the device is
connected to.
- 2: Timer: Waits a specific period (completion_nsec) for each IO before
+ 2 Timer: Waits a specific period (completion_nsec) for each IO before
completion.
+ = ===========================================================================
completion_nsec=[ns]: Default: 10,000ns
Combined with irqmode=2 (timer). The time each completion event must wait.
III: Multi-queue specific parameters
use_per_node_hctx=[0/1]: Default: 0
- 0: The number of submit queues are set to the value of the submit_queues
+
+ = =====================================================================
+ 0 The number of submit queues are set to the value of the submit_queues
parameter.
- 1: The multi-queue block layer is instantiated with a hardware dispatch
+ 1 The multi-queue block layer is instantiated with a hardware dispatch
queue for each CPU node in the system.
+ = =====================================================================
no_sched=[0/1]: Default: 0
- 0: nullb* use default blk-mq io scheduler.
- 1: nullb* doesn't use io scheduler.
+
+ = ======================================
+ 0 nullb* use default blk-mq io scheduler
+ 1 nullb* doesn't use io scheduler
+ = ======================================
blocking=[0/1]: Default: 0
- 0: Register as a non-blocking blk-mq driver device.
- 1: Register as a blocking blk-mq driver device, null_blk will set
+
+ = ===============================================================
+ 0 Register as a non-blocking blk-mq driver device.
+ 1 Register as a blocking blk-mq driver device, null_blk will set
the BLK_MQ_F_BLOCKING flag, indicating that it sometimes/always
needs to block in its ->queue_rq() function.
+ = ===============================================================
shared_tags=[0/1]: Default: 0
- 0: Tag set is not shared.
- 1: Tag set shared between devices for blk-mq. Only makes sense with
+
+ = ================================================================
+ 0 Tag set is not shared.
+ 1 Tag set shared between devices for blk-mq. Only makes sense with
nr_devices > 1, otherwise there's no tag set to share.
+ = ================================================================
zoned=[0/1]: Default: 0
- 0: Block device is exposed as a random-access block device.
- 1: Block device is exposed as a host-managed zoned block device. Requires
+
+ = ======================================================================
+ 0 Block device is exposed as a random-access block device.
+ 1 Block device is exposed as a host-managed zoned block device. Requires
CONFIG_BLK_DEV_ZONED.
+ = ======================================================================
zone_size=[MB]: Default: 256
Per zone size when exposed as a zoned block device. Must be a power of two.
-
+===============================================
Block layer support for Persistent Reservations
===============================================
--------------------------------------------------
- PR_WRITE_EXCLUSIVE
-
Only the initiator that owns the reservation can write to the
device. Any initiator can read from the device.
- PR_EXCLUSIVE_ACCESS
-
Only the initiator that owns the reservation can access the
device.
- PR_WRITE_EXCLUSIVE_REG_ONLY
-
Only initiators with a registered key can write to the device,
Any initiator can read from the device.
- PR_EXCLUSIVE_ACCESS_REG_ONLY
-
Only initiators with a registered key can access the device.
- PR_WRITE_EXCLUSIVE_ALL_REGS
All initiators with a registered key are considered reservation
holders.
Please reference the SPC spec on the meaning of a reservation
- holder if you want to use this type.
+ holder if you want to use this type.
- PR_EXCLUSIVE_ACCESS_ALL_REGS
-
Only initiators with a registered key can access the device.
All initiators with a registered key are considered reservation
holders.
Please reference the SPC spec on the meaning of a reservation
- holder if you want to use this type.
+ holder if you want to use this type.
The following ioctl are supported:
----------------------------------
1. IOC_PR_REGISTER
+^^^^^^^^^^^^^^^^^^
This ioctl command registers a new reservation if the new_key argument
is non-null. If no existing reservation exists old_key must be zero,
2. IOC_PR_RESERVE
+^^^^^^^^^^^^^^^^^
This ioctl command reserves the device and thus restricts access for other
devices based on the type argument. The key argument must be the existing
3. IOC_PR_RELEASE
+^^^^^^^^^^^^^^^^^
This ioctl command releases the reservation specified by key and flags
and thus removes any access restriction implied by it.
4. IOC_PR_PREEMPT
+^^^^^^^^^^^^^^^^^
This ioctl command releases the existing reservation referred to by
old_key and replaces it with a new reservation of type for the
5. IOC_PR_PREEMPT_ABORT
+^^^^^^^^^^^^^^^^^^^^^^^
This ioctl command works like IOC_PR_PREEMPT except that it also aborts
any outstanding command sent over a connection identified by old_key.
6. IOC_PR_CLEAR
+^^^^^^^^^^^^^^^
This ioctl command unregisters both key and any other reservation key
registered with the device and drops any existing reservation.
All the ioctls have a flag field. Currently only one flag is supported:
- PR_FL_IGNORE_KEY
-
Ignore the existing reservation key. This is commonly supported for
IOC_PR_REGISTER, and some implementation may support the flag for
IOC_PR_RESERVE.
+=================
Queue sysfs files
=================
read-write.
add_random (RW)
-----------------
+---------------
This file allows to turn off the disk entropy contribution. Default
value of this file is '1'(on).
if true, '0' if not.
discard_granularity (RO)
------------------------
+------------------------
This shows the size of internal allocation of the device in bytes, if
reported by the device. A value of '0' means device does not support
the discard functionality.
discard_max_hw_bytes (RO)
-----------------------
+-------------------------
Devices that support discard functionality may have internal limits on
the number of bytes that can be trimmed or unmapped in a single operation.
The discard_max_bytes parameter is set by the device driver to the maximum
-
+============================
struct request documentation
+============================
Jens Axboe <jens.axboe@oracle.com> 27/05/02
-1.0
-Index
-2.0 Struct request members classification
+.. FIXME:
+ No idea about what does mean - seems just some noise, so comment it
+
+ 1.0
+ Index
+
+ 2.0 Struct request members classification
+
+ 2.1 struct request members explanation
- 2.1 struct request members explanation
+ 3.0
+
+
+ 2.0
-3.0
-2.0
Short explanation of request members
+====================================
Classification flags:
+ = ====================
D driver member
B block layer member
I I/O scheduler member
+ = ====================
Unless an entry contains a D classification, a device driver must not access
this member. Some members may contain D classifications, but should only be
<linux/blkdev.h>
-2.1
+=============================== ======= =======================================
Member Flag Comment
------- ---- -------
-
+=============================== ======= =======================================
struct list_head queuelist BI Organization on various internal
queues
-void *elevator_private I I/O scheduler private data
+``void *elevator_private`` I I/O scheduler private data
unsigned char cmd[16] D Driver can use this for setting up
a cdb before execution, see
int tag DB TCQ tag, if assigned
-void *special D Free to be used by driver
+``void *special`` D Free to be used by driver
-char *buffer D Map of first segment, also see
+``char *buffer`` D Map of first segment, also see
section on bouncing SECTION
-struct completion *waiting D Can be used by driver to get signalled
+``struct completion *waiting`` D Can be used by driver to get signalled
on request completion
-struct bio *bio DBI First bio in request
+``struct bio *bio`` DBI First bio in request
-struct bio *biotail DBI Last bio in request
+``struct bio *biotail`` DBI Last bio in request
-struct request_queue *q DB Request queue this request belongs to
+``struct request_queue *q`` DB Request queue this request belongs to
-struct request_list *rl B Request list this request came from
+``struct request_list *rl`` B Request list this request came from
+=============================== ======= =======================================
+===============================================
Block layer statistics in /sys/block/<dev>/stat
===============================================
The stat file provides several statistics about the state of block
device <dev>.
-Q. Why are there multiple statistics in a single file? Doesn't sysfs
+Q.
+ Why are there multiple statistics in a single file? Doesn't sysfs
normally contain a single value per file?
-A. By having a single file, the kernel can guarantee that the statistics
+
+A.
+ By having a single file, the kernel can guarantee that the statistics
represent a consistent snapshot of the state of the device. If the
statistics were exported as multiple files containing one statistic
each, it would be impossible to guarantee that a set of readings
values separated by whitespace. The fields are summarized in the
following table, and described in more detail below.
+
+=============== ============= =================================================
Name units description
----- ----- -----------
+=============== ============= =================================================
read I/Os requests number of read I/Os processed
read merges requests number of read I/Os merged with in-queue I/O
read sectors sectors number of sectors read
discard merges requests number of discard I/Os merged with in-queue I/O
discard sectors sectors number of sectors discarded
discard ticks milliseconds total wait time for discard requests
+=============== ============= =================================================
read I/Os, write I/Os, discard I/0s
===================================
+===================
+Switching Scheduler
+===================
+
To choose IO schedulers at boot time, use the argument 'elevator=deadline'.
'noop' and 'cfq' (the default) are also available. IO schedulers are assigned
globally at boot time only presently.
Each io queue has a set of io scheduler tunables associated with it. These
tunables control how the io scheduler works. You can find these entries
-in:
+in::
-/sys/block/<device>/queue/iosched
+ /sys/block/<device>/queue/iosched
assuming that you have sysfs mounted on /sys. If you don't have sysfs mounted,
-you can do so by typing:
+you can do so by typing::
-# mount none /sys -t sysfs
+ # mount none /sys -t sysfs
It is possible to change the IO scheduler for a given block device on
the fly to select one of mq-deadline, none, bfq, or kyber schedulers -
which can improve that device's throughput.
-To set a specific scheduler, simply do this:
+To set a specific scheduler, simply do this::
-echo SCHEDNAME > /sys/block/DEV/queue/scheduler
+ echo SCHEDNAME > /sys/block/DEV/queue/scheduler
where SCHEDNAME is the name of a defined IO scheduler, and DEV is the
device name (hda, hdb, sga, or whatever you happen to have).
The list of defined schedulers can be found by simply doing
a "cat /sys/block/DEV/queue/scheduler" - the list of valid names
-will be displayed, with the currently selected scheduler in brackets:
+will be displayed, with the currently selected scheduler in brackets::
-# cat /sys/block/sda/queue/scheduler
-[mq-deadline] kyber bfq none
-# echo none >/sys/block/sda/queue/scheduler
-# cat /sys/block/sda/queue/scheduler
-[none] mq-deadline kyber bfq
+ # cat /sys/block/sda/queue/scheduler
+ [mq-deadline] kyber bfq none
+ # echo none >/sys/block/sda/queue/scheduler
+ # cat /sys/block/sda/queue/scheduler
+ [none] mq-deadline kyber bfq
-
+==========================================
Explicit volatile write back cache control
-=====================================
+==========================================
Introduction
------------
Forced Unit Access
------------------
+------------------
The REQ_FUA flag can be OR ed into the r/w flags of a bio submitted from the
filesystem and will make sure that I/O completion for this request is only
Implementation details for request_fn based block drivers
---------------------------------------------------------------
+---------------------------------------------------------
For devices that do not support volatile write caches there is no driver
support required, the block layer completes empty REQ_PREFLUSH requests before
entering the driver and strips off the REQ_PREFLUSH and REQ_FUA bits from
requests that have a payload. For devices with volatile write caches the
driver needs to tell the block layer that it supports flushing caches by
-doing:
+doing::
blk_queue_write_cache(sdkp->disk->queue, true, false);
REQ_PREFLUSH requests with a payload are automatically turned into a sequence
of an empty REQ_OP_FLUSH request followed by the actual write by the block
layer. For devices that also support the FUA bit the block layer needs
-to be told to pass through the REQ_FUA bit using:
+to be told to pass through the REQ_FUA bit using::
blk_queue_write_cache(sdkp->disk->queue, true, true);
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
=====
cdrom
memory-hotplug
protection-keys
../RCU/index
+ gcc-plugins
Interfaces for kernel debugging
For printing kernel pointers which should be hidden from unprivileged
users. The behaviour of %pK depends on the kptr_restrict sysctl - see
-Documentation/sysctl/kernel.txt for more details.
+Documentation/admin-guide/sysctl/kernel.rst for more details.
Unmodified Addresses
--------------------
- Data cells of ocotp:
Detailed bindings are described in bindings/nvmem/nvmem.txt
+Watchdog bindings based on SCU Message Protocol
+------------------------------------------------------------
+
+Required properties:
+- compatible: should be:
+ "fsl,imx8qxp-sc-wdt"
+ followed by "fsl,imx-sc-wdt";
+Optional properties:
+- timeout-sec: contains the watchdog timeout in seconds.
+
Example (imx8qxp):
-------------
aliases {
rtc: rtc {
compatible = "fsl,imx8qxp-sc-rtc";
};
+
+ watchdog {
+ compatible = "fsl,imx8qxp-sc-wdt", "fsl,imx-sc-wdt";
+ timeout-sec = <60>;
+ };
};
};
- "mediatek,mt7622-audsys", "syscon"
- "mediatek,mt7623-audsys", "mediatek,mt2701-audsys", "syscon"
- "mediatek,mt8183-audiosys", "syscon"
+ - "mediatek,mt8516-audsys", "syscon"
- #clock-cells: Must be 1
The AUDSYS controller uses the common clk binding from
--- /dev/null
+ML-AHB interconnect bindings
+
+These bindings describe the STM32 SoCs ML-AHB interconnect bus which connects
+a Cortex-M subsystem with dedicated memories.
+The MCU SRAM and RETRAM memory parts can be accessed through different addresses
+(see "RAM aliases" in [1]) using different buses (see [2]) : balancing the
+Cortex-M firmware accesses among those ports allows to tune the system
+performance.
+
+[1]: https://www.st.com/resource/en/reference_manual/dm00327659.pdf
+[2]: https://wiki.st.com/stm32mpu/wiki/STM32MP15_RAM_mapping
+
+Required properties:
+- compatible: should be "simple-bus"
+- dma-ranges: describes memory addresses translation between the local CPU and
+ the remote Cortex-M processor. Each memory region, is declared with
+ 3 parameters:
+ - param 1: device base address (Cortex-M processor address)
+ - param 2: physical base address (local CPU address)
+ - param 3: size of the memory region.
+
+The Cortex-M remote processor accessed via the mlahb interconnect is described
+by a child node.
+
+Example:
+mlahb {
+ compatible = "simple-bus";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ dma-ranges = <0x00000000 0x38000000 0x10000>,
+ <0x10000000 0x10000000 0x60000>,
+ <0x30000000 0x30000000 0x60000>;
+
+ m4_rproc: m4@10000000 {
+ ...
+ };
+};
};
The format and meaning of the "xen,uefi-*" parameters are similar to those in
-Documentation/arm/uefi.txt, which are provided by the regular UEFI stub. However
+Documentation/arm/uefi.rst, which are provided by the regular UEFI stub. However
they differ because they are provided by the Xen hypervisor, together with a set
of UEFI runtime services implemented via hypercalls, see
http://xenbits.xen.org/docs/unstable/hypercall/x86_64/include,public,platform.h.html.
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/phy/allwinner,sun4i-a10-ccu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Allwinner Clock Control Unit Device Tree Bindings
+
+maintainers:
+ - Chen-Yu Tsai <wens@csie.org>
+ - Maxime Ripard <maxime.ripard@bootlin.com>
+
+properties:
+ "#clock-cells":
+ const: 1
+
+ "#reset-cells":
+ const: 1
+
+ compatible:
+ enum:
+ - allwinner,sun4i-a10-ccu
+ - allwinner,sun5i-a10s-ccu
+ - allwinner,sun5i-a13-ccu
+ - allwinner,sun6i-a31-ccu
+ - allwinner,sun7i-a20-ccu
+ - allwinner,sun8i-a23-ccu
+ - allwinner,sun8i-a33-ccu
+ - allwinner,sun8i-a83t-ccu
+ - allwinner,sun8i-a83t-r-ccu
+ - allwinner,sun8i-h3-ccu
+ - allwinner,sun8i-h3-r-ccu
+ - allwinner,sun8i-r40-ccu
+ - allwinner,sun8i-v3s-ccu
+ - allwinner,sun9i-a80-ccu
+ - allwinner,sun50i-a64-ccu
+ - allwinner,sun50i-a64-r-ccu
+ - allwinner,sun50i-h5-ccu
+ - allwinner,sun50i-h6-ccu
+ - allwinner,sun50i-h6-r-ccu
+ - allwinner,suniv-f1c100s-ccu
+ - nextthing,gr8-ccu
+
+ reg:
+ maxItems: 1
+
+ clocks:
+ minItems: 2
+ maxItems: 4
+ items:
+ - description: High Frequency Oscillator (usually at 24MHz)
+ - description: Low Frequency Oscillator (usually at 32kHz)
+ - description: Internal Oscillator
+ - description: Peripherals PLL
+
+ clock-names:
+ minItems: 2
+ maxItems: 4
+ items:
+ - const: hosc
+ - const: losc
+ - const: iosc
+ - const: pll-periph
+
+required:
+ - "#clock-cells"
+ - "#reset-cells"
+ - compatible
+ - reg
+ - clocks
+ - clock-names
+
+if:
+ properties:
+ compatible:
+ enum:
+ - allwinner,sun8i-a83t-r-ccu
+ - allwinner,sun8i-h3-r-ccu
+ - allwinner,sun50i-a64-r-ccu
+ - allwinner,sun50i-h6-r-ccu
+
+then:
+ properties:
+ clocks:
+ minItems: 4
+ maxItems: 4
+
+ clock-names:
+ minItems: 4
+ maxItems: 4
+
+else:
+ if:
+ properties:
+ compatible:
+ const: allwinner,sun50i-h6-ccu
+
+ then:
+ properties:
+ clocks:
+ minItems: 3
+ maxItems: 3
+
+ clock-names:
+ minItems: 3
+ maxItems: 3
+
+ else:
+ properties:
+ clocks:
+ minItems: 2
+ maxItems: 2
+
+ clock-names:
+ minItems: 2
+ maxItems: 2
+
+additionalProperties: false
+
+examples:
+ - |
+ ccu: clock@1c20000 {
+ compatible = "allwinner,sun8i-h3-ccu";
+ reg = <0x01c20000 0x400>;
+ clocks = <&osc24M>, <&osc32k>;
+ clock-names = "hosc", "losc";
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ };
+
+ - |
+ r_ccu: clock@1f01400 {
+ compatible = "allwinner,sun50i-a64-r-ccu";
+ reg = <0x01f01400 0x100>;
+ clocks = <&osc24M>, <&osc32k>, <&iosc>, <&ccu 11>;
+ clock-names = "hosc", "losc", "iosc", "pll-periph";
+ #clock-cells = <1>;
+ #reset-cells = <1>;
+ };
+
+...
"amlogic,gxl-clkc" for GXL and GXM SoC,
"amlogic,axg-clkc" for AXG SoC.
"amlogic,g12a-clkc" for G12A SoC.
+ "amlogic,g12b-clkc" for G12B SoC.
- clocks : list of clock phandle, one for each entry clock-names.
- clock-names : should contain the following:
* "xtal": the platform xtal
Required properties:
- compatible : shall be one of the following:
"atmel,at91sam9x5-sckc",
- "atmel,sama5d3-sckc" or
- "atmel,sama5d4-sckc":
+ "atmel,sama5d3-sckc",
+ "atmel,sama5d4-sckc" or
+ "microchip,sam9x60-sckc":
at91 SCKC (Slow Clock Controller)
-- #clock-cells : shall be 0.
+- #clock-cells : shall be 1 for "microchip,sam9x60-sckc" otherwise shall be 0.
- clocks : shall be the input parent clock phandle for the clock.
Optional properties:
--- /dev/null
+Gated Clock Controller Bindings for MIPS based BCM63XX SoCs
+
+Required properties:
+- compatible: must be one of:
+ "brcm,bcm3368-clocks"
+ "brcm,bcm6328-clocks"
+ "brcm,bcm6358-clocks"
+ "brcm,bcm6362-clocks"
+ "brcm,bcm6368-clocks"
+ "brcm,bcm63268-clocks"
+
+- reg: Address and length of the register set
+- #clock-cells: must be <1>
+
+
+Example:
+
+clkctl: clock-controller@10000004 {
+ compatible = "brcm,bcm6328-clocks";
+ reg = <0x10000004 0x4>;
+ #clock-cells = <1>;
+};
input audio clocks from host system.
- ln-psia1-mclk, ln-psia2-mclk : Optional input audio clocks from
external connector.
+ - ln-spdif-mclk : Optional input audio clock from SPDIF.
- ln-spdif-clkout : Optional input audio clock from SPDIF.
- ln-adat-mclk : Optional input audio clock from ADAT.
- ln-pmic-32k : On board fixed clock.
"marvell,dove-core-clock" - for Dove SoC core clocks
"marvell,kirkwood-core-clock" - for Kirkwood SoC (except mv88f6180)
"marvell,mv88f6180-core-clock" - for Kirkwood MV88f6180 SoC
+ "marvell,mv98dx1135-core-clock" - for Kirkwood 98dx1135 SoC
"marvell,mv88f5181-core-clock" - for Orion MV88F5181 SoC
"marvell,mv88f5182-core-clock" - for Orion MV88F5182 SoC
"marvell,mv88f5281-core-clock" - for Orion MV88F5281 SoC
--------------------------------------------------
Required properties :
-- compatible : shall contain "qcom,sdm845-gpucc"
+- compatible : shall contain "qcom,sdm845-gpucc" or "qcom,msm8998-gpucc"
- reg : shall contain base register location and length
- #clock-cells : from common clock binding, shall contain 1
- #reset-cells : from common reset binding, shall contain 1
- #power-domain-cells : from generic power domain binding, shall contain 1
- clocks : shall contain the XO clock
+ shall contain the gpll0 out main clock (msm8998)
- clock-names : shall be "xo"
+ shall be "gpll0" (msm8998)
Example:
gpucc: clock-controller@5090000 {
- external (optional) RGMII_REFCLK
- clock-names: Must be:
clock-names = "mclk", "rtc", "jtag", "rgmii_ref_ext";
+ - #power-domain-cells: Must be 0
Examples
--------
clocks = <&ext_mclk>, <&ext_rtc_clk>,
<&ext_jtag_clk>, <&ext_rgmii_ref>;
clock-names = "mclk", "rtc", "jtag", "rgmii_ref_ext";
+ #power-domain-cells = <0>;
};
- Other nodes can use the clocks provided by SYSCTRL as in:
interrupts = <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>;
reg-shift = <2>;
reg-io-width = <4>;
- clocks = <&sysctrl R9A06G032_CLK_UART0>;
- clock-names = "baudclk";
+ clocks = <&sysctrl R9A06G032_CLK_UART0>, <&sysctrl R9A06G032_HCLK_UART0>;
+ clock-names = "baudclk", "apb_pclk";
+ power-domains = <&sysctrl>;
};
--- /dev/null
+Binding for Silicon Labs Si5341 and Si5340 programmable i2c clock generator.
+
+Reference
+[1] Si5341 Data Sheet
+ https://www.silabs.com/documents/public/data-sheets/Si5341-40-D-DataSheet.pdf
+[2] Si5341 Reference Manual
+ https://www.silabs.com/documents/public/reference-manuals/Si5341-40-D-RM.pdf
+
+The Si5341 and Si5340 are programmable i2c clock generators with up to 10 output
+clocks. The chip contains a PLL that sources 5 (or 4) multisynth clocks, which
+in turn can be directed to any of the 10 (or 4) outputs through a divider.
+The internal structure of the clock generators can be found in [2].
+
+The driver can be used in "as is" mode, reading the current settings from the
+chip at boot, in case you have a (pre-)programmed device. If the PLL is not
+configured when the driver probes, it assumes the driver must fully initialize
+it.
+
+The device type, speed grade and revision are determined runtime by probing.
+
+The driver currently only supports XTAL input mode, and does not support any
+fancy input configurations. They can still be programmed into the chip and
+the driver will leave them "as is".
+
+==I2C device node==
+
+Required properties:
+- compatible: shall be one of the following:
+ "silabs,si5340" - Si5340 A/B/C/D
+ "silabs,si5341" - Si5341 A/B/C/D
+- reg: i2c device address, usually 0x74
+- #clock-cells: from common clock binding; shall be set to 2.
+ The first value is "0" for outputs, "1" for synthesizers.
+ The second value is the output or synthesizer index.
+- clocks: from common clock binding; list of parent clock handles,
+ corresponding to inputs. Use a fixed clock for the "xtal" input.
+ At least one must be present.
+- clock-names: One of: "xtal", "in0", "in1", "in2"
+- vdd-supply: Regulator node for VDD
+
+Optional properties:
+- vdda-supply: Regulator node for VDDA
+- vdds-supply: Regulator node for VDDS
+- silabs,pll-m-num, silabs,pll-m-den: Numerator and denominator for PLL
+ feedback divider. Must be such that the PLL output is in the valid range. For
+ example, to create 14GHz from a 48MHz xtal, use m-num=14000 and m-den=48. Only
+ the fraction matters, using 3500 and 12 will deliver the exact same result.
+ If these are not specified, and the PLL is not yet programmed when the driver
+ probes, the PLL will be set to 14GHz.
+- silabs,reprogram: When present, the driver will always assume the device must
+ be initialized, and always performs the soft-reset routine. Since this will
+ temporarily stop all output clocks, don't do this if the chip is generating
+ the CPU clock for example.
+- interrupts: Interrupt for INTRb pin.
+- #address-cells: shall be set to 1.
+- #size-cells: shall be set to 0.
+
+
+== Child nodes: Outputs ==
+
+The child nodes list the output clocks.
+
+Each of the clock outputs can be overwritten individually by using a child node.
+If a child node for a clock output is not set, the configuration remains
+unchanged.
+
+Required child node properties:
+- reg: number of clock output.
+
+Optional child node properties:
+- vdd-supply: Regulator node for VDD for this output. The driver selects default
+ values for common-mode and amplitude based on the voltage.
+- silabs,format: Output format, one of:
+ 1 = differential (defaults to LVDS levels)
+ 2 = low-power (defaults to HCSL levels)
+ 4 = LVCMOS
+- silabs,common-mode: Manually override output common mode, see [2] for values
+- silabs,amplitude: Manually override output amplitude, see [2] for values
+- silabs,synth-master: boolean. If present, this output is allowed to change the
+ multisynth frequency dynamically.
+- silabs,silabs,disable-high: boolean. If set, the clock output is driven HIGH
+ when disabled, otherwise it's driven LOW.
+
+==Example==
+
+/* 48MHz reference crystal */
+ref48: ref48M {
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <48000000>;
+};
+
+i2c-master-node {
+ /* Programmable clock (for logic) */
+ si5341: clock-generator@74 {
+ reg = <0x74>;
+ compatible = "silabs,si5341";
+ #clock-cells = <2>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ clocks = <&ref48>;
+ clock-names = "xtal";
+
+ silabs,pll-m-num = <14000>; /* PLL at 14.0 GHz */
+ silabs,pll-m-den = <48>;
+ silabs,reprogram; /* Chips are not programmed, always reset */
+
+ out@0 {
+ reg = <0>;
+ silabs,format = <1>; /* LVDS 3v3 */
+ silabs,common-mode = <3>;
+ silabs,amplitude = <3>;
+ silabs,synth-master;
+ };
+
+ /*
+ * Output 6 configuration:
+ * LVDS 1v8
+ */
+ out@6 {
+ reg = <6>;
+ silabs,format = <1>; /* LVDS 1v8 */
+ silabs,common-mode = <13>;
+ silabs,amplitude = <3>;
+ };
+
+ /*
+ * Output 8 configuration:
+ * HCSL 3v3
+ */
+ out@8 {
+ reg = <8>;
+ silabs,format = <2>;
+ silabs,common-mode = <11>;
+ silabs,amplitude = <3>;
+ };
+ };
+};
+
+some-video-node {
+ /* Standard clock bindings */
+ clock-names = "pixel";
+ clocks = <&si5341 0 7>; /* Output 7 */
+
+ /* Set output 7 to use syntesizer 3 as its parent */
+ assigned-clocks = <&si5341 0 7>, <&si5341 1 3>;
+ assigned-clock-parents = <&si5341 1 3>;
+ /* Set output 7 to 148.5 MHz using a synth frequency of 594 MHz */
+ assigned-clock-rates = <148500000>, <594000000>;
+};
+
+some-audio-node {
+ clock-names = "i2s-clk";
+ clocks = <&si5341 0 0>;
+ /*
+ * since output 0 is a synth-master, the synth will be automatically set
+ * to an appropriate frequency when the audio driver requests another
+ * frequency. We give control over synth 2 to this output here.
+ */
+ assigned-clocks = <&si5341 0 0>;
+ assigned-clock-parents = <&si5341 1 2>;
+};
+++ /dev/null
-Allwinner Clock Control Unit Binding
-------------------------------------
-
-Required properties :
-- compatible: must contain one of the following compatibles:
- - "allwinner,sun4i-a10-ccu"
- - "allwinner,sun5i-a10s-ccu"
- - "allwinner,sun5i-a13-ccu"
- - "allwinner,sun6i-a31-ccu"
- - "allwinner,sun7i-a20-ccu"
- - "allwinner,sun8i-a23-ccu"
- - "allwinner,sun8i-a33-ccu"
- - "allwinner,sun8i-a83t-ccu"
- - "allwinner,sun8i-a83t-r-ccu"
- - "allwinner,sun8i-h3-ccu"
- - "allwinner,sun8i-h3-r-ccu"
-+ - "allwinner,sun8i-r40-ccu"
- - "allwinner,sun8i-v3s-ccu"
- - "allwinner,sun9i-a80-ccu"
- - "allwinner,sun50i-a64-ccu"
- - "allwinner,sun50i-a64-r-ccu"
- - "allwinner,sun50i-h5-ccu"
- - "allwinner,sun50i-h6-ccu"
- - "allwinner,sun50i-h6-r-ccu"
- - "allwinner,suniv-f1c100s-ccu"
- - "nextthing,gr8-ccu"
-
-- reg: Must contain the registers base address and length
-- clocks: phandle to the oscillators feeding the CCU. Two are needed:
- - "hosc": the high frequency oscillator (usually at 24MHz)
- - "losc": the low frequency oscillator (usually at 32kHz)
- On the A83T, this is the internal 16MHz oscillator divided by 512
-- clock-names: Must contain the clock names described just above
-- #clock-cells : must contain 1
-- #reset-cells : must contain 1
-
-For the main CCU on H6, one more clock is needed:
-- "iosc": the SoC's internal frequency oscillator
-
-For the PRCM CCUs on A83T/H3/A64/H6, two more clocks are needed:
-- "pll-periph": the SoC's peripheral PLL from the main CCU
-- "iosc": the SoC's internal frequency oscillator
-
-Example for generic CCU:
-ccu: clock@1c20000 {
- compatible = "allwinner,sun8i-h3-ccu";
- reg = <0x01c20000 0x400>;
- clocks = <&osc24M>, <&osc32k>;
- clock-names = "hosc", "losc";
- #clock-cells = <1>;
- #reset-cells = <1>;
-};
-
-Example for PRCM CCU:
-r_ccu: clock@1f01400 {
- compatible = "allwinner,sun50i-a64-r-ccu";
- reg = <0x01f01400 0x100>;
- clocks = <&osc24M>, <&osc32k>, <&iosc>, <&ccu CLK_PLL_PERIPH0>;
- clock-names = "hosc", "losc", "iosc", "pll-periph";
- #clock-cells = <1>;
- #reset-cells = <1>;
-};
+++ /dev/null
-* Mediatek UART APDMA Controller
-
-Required properties:
-- compatible should contain:
- * "mediatek,mt2712-uart-dma" for MT2712 compatible APDMA
- * "mediatek,mt6577-uart-dma" for MT6577 and all of the above
-
-- reg: The base address of the APDMA register bank.
-
-- interrupts: A single interrupt specifier.
-
-- clocks : Must contain an entry for each entry in clock-names.
- See ../clocks/clock-bindings.txt for details.
-- clock-names: The APDMA clock for register accesses
-
-Examples:
-
- apdma: dma-controller@11000380 {
- compatible = "mediatek,mt2712-uart-dma";
- reg = <0 0x11000380 0 0x400>;
- interrupts = <GIC_SPI 63 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 64 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 65 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 66 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 67 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 68 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 69 IRQ_TYPE_LEVEL_LOW>,
- <GIC_SPI 70 IRQ_TYPE_LEVEL_LOW>;
- clocks = <&pericfg CLK_PERI_AP_DMA>;
- clock-names = "apdma";
- #dma-cells = <1>;
- };
-
- dma-channels: contains the total number of DMA channels supported by the DMAC
- dma-requests: contains the total number of DMA requests supported by the DMAC
- arm,pl330-broken-no-flushp: quirk for avoiding to execute DMAFLUSHP
+ - resets: contains an entry for each entry in reset-names.
+ See ../reset/reset.txt for details.
+ - reset-names: must contain at least "dma", and optional is "dma-ocp".
Example:
Required properties:
- compatible :
- "fsl,vf610-edma" for eDMA used similar to that on Vybrid vf610 SoC
+ - "fsl,imx7ulp-edma" for eDMA2 used similar to that on i.mx7ulp
- reg : Specifies base physical address(s) and size of the eDMA registers.
The 1st region is eDMA control register's address and size.
The 2nd and the 3rd regions are programmable channel multiplexing
control register's address and size.
- interrupts : A list of interrupt-specifiers, one for each entry in
- interrupt-names.
-- interrupt-names : Should contain:
- "edma-tx" - the transmission interrupt
- "edma-err" - the error interrupt
+ interrupt-names on vf610 similar SoC. But for i.mx7ulp per channel
+ per transmission interrupt, total 16 channel interrupt and 1
+ error interrupt(located in the last), no interrupt-names list on
+ i.mx7ulp for clean on dts.
- #dma-cells : Must be <2>.
The 1st cell specifies the DMAMUX(0 for DMAMUX0 and 1 for DMAMUX1).
Specific request source can only be multiplexed by specific channels
- clock-names : A list of channel group clock names. Should contain:
"dmamux0" - clock name of mux0 group
"dmamux1" - clock name of mux1 group
+ Note: No dmamux0 on i.mx7ulp, but another 'dma' clk added on i.mx7ulp.
- clocks : A list of phandle and clock-specifier pairs, one for each entry in
clock-names.
- big-endian: If present registers and hardware scatter/gather descriptors
of the eDMA are implemented in big endian mode, otherwise in little
mode.
+- interrupt-names : Should contain the below on vf610 similar SoC but not used
+ on i.mx7ulp similar SoC:
+ "edma-tx" - the transmission interrupt
+ "edma-err" - the error interrupt
Examples:
clock-names = "dmamux0", "dmamux1";
clocks = <&clks VF610_CLK_DMAMUX0>,
<&clks VF610_CLK_DMAMUX1>;
-};
+}; /* vf610 */
+edma1: dma-controller@40080000 {
+ #dma-cells = <2>;
+ compatible = "fsl,imx7ulp-edma";
+ reg = <0x40080000 0x2000>,
+ <0x40210000 0x1000>;
+ dma-channels = <32>;
+ interrupts = <GIC_SPI 0 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 1 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 3 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 6 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 11 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 12 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 13 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>,
+ /* last is eDMA2-ERR interrupt */
+ <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
+ clock-names = "dma", "dmamux0";
+ clocks = <&pcc2 IMX7ULP_CLK_DMA1>,
+ <&pcc2 IMX7ULP_CLK_DMA_MUX1>;
+}; /* i.mx7ulp */
* DMA clients
DMA client drivers that uses the DMA function must use the format described
--- /dev/null
+* Mediatek UART APDMA Controller
+
+Required properties:
+- compatible should contain:
+ * "mediatek,mt2712-uart-dma" for MT2712 compatible APDMA
+ * "mediatek,mt6577-uart-dma" for MT6577 and all of the above
+
+- reg: The base address of the APDMA register bank.
+
+- interrupts: A single interrupt specifier.
+ One interrupt per dma-requests, or 8 if no dma-requests property is present
+
+- dma-requests: The number of DMA channels
+
+- clocks : Must contain an entry for each entry in clock-names.
+ See ../clocks/clock-bindings.txt for details.
+- clock-names: The APDMA clock for register accesses
+
+- mediatek,dma-33bits: Present if the DMA requires support
+
+Examples:
+
+ apdma: dma-controller@11000400 {
+ compatible = "mediatek,mt2712-uart-dma";
+ reg = <0 0x11000400 0 0x80>,
+ <0 0x11000480 0 0x80>,
+ <0 0x11000500 0 0x80>,
+ <0 0x11000580 0 0x80>,
+ <0 0x11000600 0 0x80>,
+ <0 0x11000680 0 0x80>,
+ <0 0x11000700 0 0x80>,
+ <0 0x11000780 0 0x80>,
+ <0 0x11000800 0 0x80>,
+ <0 0x11000880 0 0x80>,
+ <0 0x11000900 0 0x80>,
+ <0 0x11000980 0 0x80>;
+ interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 104 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 105 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 106 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 107 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 108 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 109 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 110 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 111 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 112 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 113 IRQ_TYPE_LEVEL_LOW>,
+ <GIC_SPI 114 IRQ_TYPE_LEVEL_LOW>;
+ dma-requests = <12>;
+ clocks = <&pericfg CLK_PERI_AP_DMA>;
+ clock-names = "apdma";
+ mediatek,dma-33bits;
+ #dma-cells = <1>;
+ };
};
------------------------------------------------------------------------------
-For A64 DMA controller:
+For A64 and H6 DMA controller:
Required properties:
-- compatible: "allwinner,sun50i-a64-dma"
+- compatible: Must be one of
+ "allwinner,sun50i-a64-dma"
+ "allwinner,sun50i-h6-dma"
- dma-channels: Number of DMA channels supported by the controller.
Refer to Documentation/devicetree/bindings/dma/dma.txt
+- clocks: In addition to parent AHB clock, it should also contain mbus
+ clock (H6 only)
+- clock-names: Should contain "bus" and "mbus" (H6 only)
- all properties above, i.e. reg, interrupts, clocks, resets and #dma-cells
Optional properties:
-OMAP4+ HwSpinlock Driver
-========================
+TI HwSpinlock for OMAP and K3 based SoCs
+=========================================
Required properties:
-- compatible: Should be "ti,omap4-hwspinlock" for
- OMAP44xx, OMAP54xx, AM33xx, AM43xx, DRA7xx SoCs
+- compatible: Should be one of the following,
+ "ti,omap4-hwspinlock" for
+ OMAP44xx, OMAP54xx, AM33xx, AM43xx, DRA7xx SoCs
+ "ti,am654-hwspinlock" for
+ K3 AM65x and J721E SoCs
- reg: Contains the hwspinlock module register address space
(base address and length)
- ti,hwmods: Name of the hwmod associated with the hwspinlock device
+ (for OMAP architecture based SoCs only)
- #hwlock-cells: Should be 1. The OMAP hwspinlock users will use a
0-indexed relative hwlock number as the argument
specifier value for requesting a specific hwspinlock
Example:
-/* OMAP4 */
+1. OMAP4 SoCs
hwspinlock: spinlock@4a0f6000 {
compatible = "ti,omap4-hwspinlock";
reg = <0x4a0f6000 0x1000>;
ti,hwmods = "spinlock";
#hwlock-cells = <1>;
};
+
+2. AM65x SoCs and J721E SoCs
+&cbass_main {
+ cbass_main_navss: interconnect0 {
+ hwspinlock: spinlock@30e00000 {
+ compatible = "ti,am654-hwspinlock";
+ reg = <0x00 0x30e00000 0x00 0x1000>;
+ #hwlock-cells = <1>;
+ };
+ };
+};
This document explains only the device tree data binding. For general
-information about PHY subsystem refer to Documentation/phy.txt
+information about PHY subsystem refer to Documentation/driver-api/phy/phy.rst
PHY device node
===============
};
This document explains the device tree binding. For general
-information about PHY subsystem refer to Documentation/phy.txt
+information about PHY subsystem refer to Documentation/driver-api/phy/phy.rst
-Qualcomm Technology Inc. ADSP Peripheral Image Loader
+Qualcomm Technology Inc. Hexagon v56 Peripheral Image Loader
This document defines the binding for a component that loads and boots firmware
-on the Qualcomm Technology Inc. ADSP Hexagon core.
+on the Qualcomm Technology Inc. Hexagon v56 core.
- compatible:
Usage: required
Value type: <string>
Definition: must be one of:
+ "qcom,qcs404-cdsp-pil",
"qcom,sdm845-adsp-pil"
- reg:
- clocks:
Usage: required
Value type: <prop-encoded-array>
- Definition: List of 8 phandle and clock specifier pairs for the adsp.
+ Definition: List of phandles and clock specifier pairs for the Hexagon,
+ per clock-names below.
- clock-names:
- Usage: required
+ Usage: required for SDM845 ADSP
Value type: <stringlist>
Definition: List of clock input name strings sorted in the same
order as the clocks property. Definition must have
"lpass_ahbm_aon_cbcr", "qdsp6ss_xo", "qdsp6ss_sleep"
and "qdsp6ss_core".
+- clock-names:
+ Usage: required for QCS404 CDSP
+ Value type: <stringlist>
+ Definition: List of clock input name strings sorted in the same
+ order as the clocks property. Definition must have
+ "xo", "sway", "tbu", "bimc", "ahb_aon", "q6ss_slave",
+ "q6ss_master", "q6_axim".
+
- power-domains:
Usage: required
Value type: <phandle>
- resets:
Usage: required
Value type: <phandle>
- Definition: reference to the list of 2 reset-controller for the adsp.
+ Definition: reference to the list of resets for the Hexagon.
- reset-names:
- Usage: required
+ Usage: required for SDM845 ADSP
Value type: <stringlist>
Definition: must be "pdc_sync" and "cc_lpass"
+- reset-names:
+ Usage: required for QCS404 CDSP
+ Value type: <stringlist>
+ Definition: must be "restart"
+
- qcom,halt-regs:
Usage: required
Value type: <prop-encoded-array>
Definition: a phandle reference to a syscon representing TCSR followed
- by the offset within syscon for lpass halt register.
+ by the offset within syscon for Hexagon halt register.
- memory-region:
Usage: required
Value type: <phandle>
- Definition: reference to the reserved-memory for the ADSP
+ Definition: reference to the reserved-memory for the firmware
- qcom,smem-states:
Usage: required
Value type: <phandle>
- Definition: reference to the smem state for requesting the ADSP to
+ Definition: reference to the smem state for requesting the Hexagon to
shut down
- qcom,smem-state-names:
= SUBNODES
The adsp node may have an subnode named "glink-edge" that describes the
-communication edge, channels and devices related to the ADSP.
+communication edge, channels and devices related to the Hexagon.
See ../soc/qcom/qcom,glink.txt for details on how to describe these.
= EXAMPLE
--- /dev/null
+STMicroelectronics STM32 Remoteproc
+-----------------------------------
+This document defines the binding for the remoteproc component that loads and
+boots firmwares on the ST32MP family chipset.
+
+Required properties:
+- compatible: Must be "st,stm32mp1-m4"
+- reg: Address ranges of the RETRAM and MCU SRAM memories used by the
+ remote processor.
+- resets: Reference to a reset controller asserting the remote processor.
+- st,syscfg-holdboot: Reference to the system configuration which holds the
+ remote processor reset hold boot
+ 1st cell: phandle of syscon block
+ 2nd cell: register offset containing the hold boot setting
+ 3rd cell: register bitmask for the hold boot field
+- st,syscfg-tz: Reference to the system configuration which holds the RCC trust
+ zone mode
+ 1st cell: phandle to syscon block
+ 2nd cell: register offset containing the RCC trust zone mode setting
+ 3rd cell: register bitmask for the RCC trust zone mode bit
+
+Optional properties:
+- interrupts: Should contain the watchdog interrupt
+- mboxes: This property is required only if the rpmsg/virtio functionality
+ is used. List of phandle and mailbox channel specifiers:
+ - a channel (a) used to communicate through virtqueues with the
+ remote proc.
+ Bi-directional channel:
+ - from local to remote = send message
+ - from remote to local = send message ack
+ - a channel (b) working the opposite direction of channel (a)
+ - a channel (c) used by the local proc to notify the remote proc
+ that it is about to be shut down.
+ Unidirectional channel:
+ - from local to remote, where ACK from the remote means
+ that it is ready for shutdown
+- mbox-names: This property is required if the mboxes property is used.
+ - must be "vq0" for channel (a)
+ - must be "vq1" for channel (b)
+ - must be "shutdown" for channel (c)
+- memory-region: List of phandles to the reserved memory regions associated with
+ the remoteproc device. This is variable and describes the
+ memories shared with the remote processor (eg: remoteproc
+ firmware and carveouts, rpmsg vrings, ...).
+ (see ../reserved-memory/reserved-memory.txt)
+- st,syscfg-pdds: Reference to the system configuration which holds the remote
+ processor deep sleep setting
+ 1st cell: phandle to syscon block
+ 2nd cell: register offset containing the deep sleep setting
+ 3rd cell: register bitmask for the deep sleep bit
+- st,auto-boot: If defined, when remoteproc is probed, it loads the default
+ firmware and starts the remote processor.
+
+Example:
+ m4_rproc: m4@10000000 {
+ compatible = "st,stm32mp1-m4";
+ reg = <0x10000000 0x40000>,
+ <0x30000000 0x40000>,
+ <0x38000000 0x10000>;
+ resets = <&rcc MCU_R>;
+ st,syscfg-holdboot = <&rcc 0x10C 0x1>;
+ st,syscfg-tz = <&rcc 0x000 0x1>;
+ };
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/allwinner,sun4i-a10-rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Allwinner A10 RTC Device Tree Bindings
+
+allOf:
+ - $ref: "rtc.yaml#"
+
+maintainers:
+ - Chen-Yu Tsai <wens@csie.org>
+ - Maxime Ripard <maxime.ripard@bootlin.com>
+
+properties:
+ compatible:
+ enum:
+ - allwinner,sun4i-a10-rtc
+ - allwinner,sun7i-a20-rtc
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+required:
+ - compatible
+ - reg
+ - interrupts
+
+additionalProperties: false
+
+examples:
+ - |
+ rtc: rtc@1c20d00 {
+ compatible = "allwinner,sun4i-a10-rtc";
+ reg = <0x01c20d00 0x20>;
+ interrupts = <24>;
+ };
+
+...
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/allwinner,sun6i-a31-rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Allwinner A31 RTC Device Tree Bindings
+
+maintainers:
+ - Chen-Yu Tsai <wens@csie.org>
+ - Maxime Ripard <maxime.ripard@bootlin.com>
+
+properties:
+ "#clock-cells":
+ const: 1
+
+ compatible:
+ oneOf:
+ - const: allwinner,sun6i-a31-rtc
+ - const: allwinner,sun8i-a23-rtc
+ - const: allwinner,sun8i-h3-rtc
+ - const: allwinner,sun8i-r40-rtc
+ - const: allwinner,sun8i-v3-rtc
+ - const: allwinner,sun50i-h5-rtc
+ - items:
+ - const: allwinner,sun50i-a64-rtc
+ - const: allwinner,sun8i-h3-rtc
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ minItems: 1
+ maxItems: 2
+ items:
+ - description: RTC Alarm 0
+ - description: RTC Alarm 1
+
+ clocks:
+ maxItems: 1
+
+ clock-output-names:
+ minItems: 1
+ maxItems: 3
+ description:
+ The RTC provides up to three clocks
+ - the Low Frequency Oscillator or LOSC, at index 0,
+ - the Low Frequency Oscillator External output (X32KFOUT in
+ the datasheet), at index 1,
+ - the Internal Oscillator, at index 2.
+
+allOf:
+ - $ref: "rtc.yaml#"
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: allwinner,sun6i-a31-rtc
+
+ then:
+ properties:
+ clock-output-names:
+ minItems: 1
+ maxItems: 1
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - allwinner,sun8i-a23-rtc
+ - allwinner,sun8i-r40-rtc
+ - allwinner,sun8i-v3-rtc
+
+ then:
+ properties:
+ clock-output-names:
+ minItems: 2
+ maxItems: 2
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ enum:
+ - allwinner,sun8i-h3-rtc
+ - allwinner,sun50i-h5-rtc
+
+ then:
+ properties:
+ clock-output-names:
+ minItems: 3
+ maxItems: 3
+
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: allwinner,sun8i-r40-rtc
+
+ then:
+ properties:
+ interrupts:
+ minItems: 1
+ maxItems: 1
+
+ else:
+ properties:
+ interrupts:
+ minItems: 2
+ maxItems: 2
+
+required:
+ - "#clock-cells"
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-output-names
+
+additionalProperties: false
+
+examples:
+ - |
+ rtc: rtc@1f00000 {
+ compatible = "allwinner,sun6i-a31-rtc";
+ reg = <0x01f00000 0x400>;
+ interrupts = <0 40 4>, <0 41 4>;
+ clock-output-names = "osc32k";
+ clocks = <&ext_osc32k>;
+ #clock-cells = <1>;
+ };
+
+...
-Generic device tree bindings for Real Time Clock devices
-========================================================
-
-This document describes generic bindings which can be used to describe Real Time
-Clock devices in a device tree.
-
-Required properties
--------------------
-
-- compatible : name of RTC device following generic names recommended practice.
-
-For other required properties e.g. to describe register sets,
-clocks, etc. check the binding documentation of the specific driver.
-
-Optional properties
--------------------
-
-- start-year : if provided, the default hardware range supported by the RTC is
- shifted so the first usable year is the specified one.
-
-The following properties may not be supported by all drivers. However, if a
-driver wants to support one of the below features, it should adapt the bindings
-below.
-- trickle-resistor-ohms : Selected resistor for trickle charger. Should be given
- if trickle charger should be enabled
-- trickle-diode-disable : Do not use internal trickle charger diode Should be
- given if internal trickle charger diode should be
- disabled
-- wakeup-source : Enables wake up of host system on alarm
-- quartz-load-femtofarads : The capacitive load of the quartz(x-tal),
- expressed in femto Farad (fF).
- The default value shall be listed (if optional),
- and likewise all valid values.
-
-Trivial RTCs
-------------
-
-This is a list of trivial RTC devices that have simple device tree
-bindings, consisting only of a compatible field, an address and
-possibly an interrupt line.
-
-
-Compatible Vendor / Chip
-========== =============
-abracon,abb5zes3 AB-RTCMC-32.768kHz-B5ZE-S3: Real Time Clock/Calendar Module with I2C Interface
-abracon,abeoz9 AB-RTCMC-32.768kHz-EOZ9: Real Time Clock/Calendar Module with I2C Interface
-dallas,ds1374 I2C, 32-Bit Binary Counter Watchdog RTC with Trickle Charger and Reset Input/Output
-dallas,ds1672 Dallas DS1672 Real-time Clock
-dallas,ds3232 Extremely Accurate I²C RTC with Integrated Crystal and SRAM
-epson,rx8010 I2C-BUS INTERFACE REAL TIME CLOCK MODULE
-epson,rx8571 I2C-BUS INTERFACE REAL TIME CLOCK MODULE with Battery Backed RAM
-epson,rx8581 I2C-BUS INTERFACE REAL TIME CLOCK MODULE
-emmicro,em3027 EM Microelectronic EM3027 Real-time Clock
-isil,isl1208 Intersil ISL1208 Low Power RTC with Battery Backed SRAM
-isil,isl1218 Intersil ISL1218 Low Power RTC with Battery Backed SRAM
-isil,isl12022 Intersil ISL12022 Real-time Clock
-microcrystal,rv3028 Real Time Clock Module with I2C-Bus
-microcrystal,rv3029 Real Time Clock Module with I2C-Bus
-microcrystal,rv8523 Real Time Clock
-nxp,pcf2127 Real-time clock
-nxp,pcf2129 Real-time clock
-nxp,pcf8563 Real-time clock/calendar
-pericom,pt7c4338 Real-time Clock Module
-ricoh,r2025sd I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-ricoh,r2221tl I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-ricoh,rs5c372a I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-ricoh,rs5c372b I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-ricoh,rv5c386 I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-ricoh,rv5c387a I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-sii,s35390a 2-wire CMOS real-time clock
-whwave,sd3078 I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
-xircom,x1205 Xircom X1205 I2C RTC
+This file has been moved to rtc.yaml.
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RTC Generic Binding
+
+maintainers:
+ - Alexandre Belloni <alexandre.belloni@bootlin.com>
+
+description: |
+ This document describes generic bindings which can be used to
+ describe Real Time Clock devices in a device tree.
+
+properties:
+ $nodename:
+ pattern: "^rtc(@.*|-[0-9a-f])*$"
+
+ quartz-load-femtofarads:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ The capacitive load of the quartz(x-tal), expressed in femto
+ Farad (fF). The default value shall be listed (if optional),
+ and likewise all valid values.
+
+ start-year:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ If provided, the default hardware range supported by the RTC is
+ shifted so the first usable year is the specified one.
+
+ trickle-diode-disable:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description:
+ Do not use internal trickle charger diode. Should be given if
+ internal trickle charger diode should be disabled.
+
+ trickle-resistor-ohms:
+ $ref: /schemas/types.yaml#/definitions/uint32
+ description:
+ Selected resistor for trickle charger. Should be given
+ if trickle charger should be enabled.
+
+ wakeup-source:
+ $ref: /schemas/types.yaml#/definitions/flag
+ description:
+ Enables wake up of host system on alarm.
+
+...
+++ /dev/null
-* sun6i Real Time Clock
-
-RTC controller for the Allwinner A31
-
-Required properties:
-- compatible : Should be one of the following combinations:
- - "allwinner,sun6i-a31-rtc"
- - "allwinner,sun8i-a23-rtc"
- - "allwinner,sun8i-h3-rtc"
- - "allwinner,sun8i-r40-rtc", "allwinner,sun8i-h3-rtc"
- - "allwinner,sun8i-v3-rtc"
- - "allwinner,sun50i-a64-rtc", "allwinner,sun8i-h3-rtc"
- - "allwinner,sun50i-h5-rtc"
-
- Where there are two or more compatible strings, this
- denotes the hardware covered by the most specific one
- is backward-compatible with the latter ones, and the
- implementation for the latter ones can be used, albeit
- with reduced functionality.
-
-- reg : physical base address of the controller and length of
- memory mapped region.
-- interrupts : IRQ lines for the RTC alarm 0 and alarm 1, in that order.
-
-Required properties for new device trees
-- clocks : phandle to the 32kHz external oscillator
-- clock-output-names : names of up to three clock outputs. See below.
-- #clock-cells : must be equal to 1.
-
-The RTC provides the following clocks at the given indices:
-- 0: LOSC
-- 1: LOSC external output, known as X32KFOUT in the datasheet.
- This clock is not available on the A31 and is deprecated for old
- device trees still using the "allwinner,sun6i-a31-rtc" compatible.
-- 2: InternalOSC, or internal RC oscillator (A64/H3/H5 only)
-
-Example:
-
-rtc: rtc@1f00000 {
- compatible = "allwinner,sun6i-a31-rtc";
- reg = <0x01f00000 0x400>;
- interrupts = <0 40 4>, <0 41 4>;
- clock-output-names = "osc32k";
- clocks = <&ext_osc32k>;
- #clock-cells = <1>;
-};
+++ /dev/null
-* sun4i/sun7i Real Time Clock
-
-RTC controller for the Allwinner A10/A20
-
-Required properties:
-- compatible : Should be "allwinner,sun4i-a10-rtc" or "allwinner,sun7i-a20-rtc"
-- reg: physical base address of the controller and length of memory mapped
- region.
-- interrupts: IRQ line for the RTC.
-
-Example:
-
-rtc: rtc@1c20d00 {
- compatible = "allwinner,sun4i-a10-rtc";
- reg = <0x01c20d00 0x20>;
- interrupts = <24>;
-};
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/rtc/trivial-rtc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Trivial RTCs
+
+maintainers:
+ - Alexandre Belloni <alexandre.belloni@bootlin.com>
+
+description: |
+ This is a list of trivial RTC devices that have simple device tree
+ bindings, consisting only of a compatible field, an address and
+ possibly an interrupt line.
+
+allOf:
+ - $ref: "rtc.yaml#"
+
+properties:
+ compatible:
+ enum:
+ # AB-RTCMC-32.768kHz-B5ZE-S3: Real Time Clock/Calendar Module with I2C Interface
+ - abracon,abb5zes3
+ # AB-RTCMC-32.768kHz-EOZ9: Real Time Clock/Calendar Module with I2C Interface
+ - abracon,abeoz9
+ # I2C, 32-Bit Binary Counter Watchdog RTC with Trickle Charger and Reset Input/Output
+ - dallas,ds1374
+ # Dallas DS1672 Real-time Clock
+ - dallas,ds1672
+ # Extremely Accurate I²C RTC with Integrated Crystal and SRAM
+ - dallas,ds3232
+ # I2C-BUS INTERFACE REAL TIME CLOCK MODULE
+ - epson,rx8010
+ # I2C-BUS INTERFACE REAL TIME CLOCK MODULE with Battery Backed RAM
+ - epson,rx8571
+ # I2C-BUS INTERFACE REAL TIME CLOCK MODULE
+ - epson,rx8581
+ # Intersil ISL1208 Low Power RTC with Battery Backed SRAM
+ - isil,isl1208
+ # Intersil ISL1218 Low Power RTC with Battery Backed SRAM
+ - isil,isl1218
+ # Intersil ISL12022 Real-time Clock
+ - isil,isl12022
+ # Real Time Clock Module with I2C-Bus
+ - microcrystal,rv3028
+ # Real Time Clock Module with I2C-Bus
+ - microcrystal,rv3029
+ # Real Time Clock
+ - microcrystal,rv8523
+ # Real-time clock
+ - nxp,pcf2127
+ # Real-time clock
+ - nxp,pcf2129
+ # Real-time clock/calendar
+ - nxp,pcf8563
+ # Real-time Clock Module
+ - pericom,pt7c4338
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,r2025sd
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,r2221tl
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,rs5c372a
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,rs5c372b
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,rv5c386
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - ricoh,rv5c387a
+ # 2-wire CMOS real-time clock
+ - sii,s35390a
+ # I2C bus SERIAL INTERFACE REAL-TIME CLOCK IC
+ - whwave,sd3078
+ # Xircom X1205 I2C RTC
+ - xircom,x1205
+
+ reg:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ start-year: true
+
+required:
+ - compatible
+ - reg
+
+additionalProperties: false
+
+...
Definition: should specify the IRQ used by the remote processor to
signal this processor about communication related events
+- qcom,remote-pid:
+ Usage: required for glink-smem
+ Value type: <u32>
+ Definition: specifies the identifier of the remote endpoint of this edge
+
- qcom,rpm-msg-ram:
Usage: required for glink-rpm
Value type: <prop-encoded-array>
Required properties:
- compatible: should be "samsung,s3c2410-ohci" for USB host controller
- - reg: address and lenght of the controller memory mapped region
+ - reg: address and length of the controller memory mapped region
- interrupts: interrupt number for the USB OHCI controller
- clocks: Should reference the bus and host clocks
- clock-names: Should contain two strings
--- /dev/null
+* virtio IOMMU PCI device
+
+When virtio-iommu uses the PCI transport, its programming interface is
+discovered dynamically by the PCI probing infrastructure. However the
+device tree statically describes the relation between IOMMU and DMA
+masters. Therefore, the PCI root complex that hosts the virtio-iommu
+contains a child node representing the IOMMU device explicitly.
+
+Required properties:
+
+- compatible: Should be "virtio,pci-iommu"
+- reg: PCI address of the IOMMU. As defined in the PCI Bus
+ Binding reference [1], the reg property is a five-cell
+ address encoded as (phys.hi phys.mid phys.lo size.hi
+ size.lo). phys.hi should contain the device's BDF as
+ 0b00000000 bbbbbbbb dddddfff 00000000. The other cells
+ should be zero.
+- #iommu-cells: Each platform DMA master managed by the IOMMU is assigned
+ an endpoint ID, described by the "iommus" property [2].
+ For virtio-iommu, #iommu-cells must be 1.
+
+Notes:
+
+- DMA from the IOMMU device isn't managed by another IOMMU. Therefore the
+ virtio-iommu node doesn't have an "iommus" property, and is omitted from
+ the iommu-map property of the root complex.
+
+Example:
+
+pcie@10000000 {
+ compatible = "pci-host-ecam-generic";
+ ...
+
+ /* The IOMMU programming interface uses slot 00:01.0 */
+ iommu0: iommu@0008 {
+ compatible = "virtio,pci-iommu";
+ reg = <0x00000800 0 0 0 0>;
+ #iommu-cells = <1>;
+ };
+
+ /*
+ * The IOMMU manages all functions in this PCI domain except
+ * itself. Omit BDF 00:01.0.
+ */
+ iommu-map = <0x0 &iommu0 0x0 0x8>
+ <0x9 &iommu0 0x9 0xfff7>;
+};
+
+pcie@20000000 {
+ compatible = "pci-host-ecam-generic";
+ ...
+ /*
+ * The IOMMU also manages all functions from this domain,
+ * with endpoint IDs 0x10000 - 0x1ffff
+ */
+ iommu-map = <0x0 &iommu0 0x10000 0x10000>;
+};
+
+ethernet@fe001000 {
+ ...
+ /* The IOMMU manages this platform device with endpoint ID 0x20000 */
+ iommus = <&iommu0 0x20000>;
+};
+
+[1] Documentation/devicetree/bindings/pci/pci.txt
+[2] Documentation/devicetree/bindings/iommu/iommu.txt
- reg: control registers base address and size including configuration space
- interrupts: interrupt generated by the device
+Required properties for virtio-iommu:
+
+- #iommu-cells: When the node corresponds to a virtio-iommu device, it is
+ linked to DMA masters using the "iommus" or "iommu-map"
+ properties [1][2]. #iommu-cells specifies the size of the
+ "iommus" property. For virtio-iommu #iommu-cells must be
+ 1, each cell describing a single endpoint ID.
+
+Optional properties:
+
+- iommus: If the device accesses memory through an IOMMU, it should
+ have an "iommus" property [1]. Since virtio-iommu itself
+ does not access memory through an IOMMU, the "virtio,mmio"
+ node cannot have both an "#iommu-cells" and an "iommus"
+ property.
+
Example:
virtio_block@3000 {
compatible = "virtio,mmio";
reg = <0x3000 0x100>;
interrupts = <41>;
+
+ /* Device has endpoint ID 23 */
+ iommus = <&viommu 23>
}
+
+ viommu: iommu@3100 {
+ compatible = "virtio,mmio";
+ reg = <0x3100 0x100>;
+ interrupts = <42>;
+
+ #iommu-cells = <1>
+ }
+
+[1] Documentation/devicetree/bindings/iommu/iommu.txt
+[2] Documentation/devicetree/bindings/pci/pci-iommu.txt
+++ /dev/null
-* Freescale i.MX System Controller Watchdog
-
-i.MX system controller watchdog is for i.MX SoCs with system controller inside,
-the watchdog is managed by system controller, users can ONLY communicate with
-system controller from secure mode for watchdog operations, so Linux i.MX system
-controller watchdog driver will call ARM SMC API and trap into ARM-Trusted-Firmware
-for watchdog operations, ARM-Trusted-Firmware is running at secure EL3 mode and
-it will request system controller to execute the watchdog operation passed from
-Linux kernel.
-
-Required properties:
-- compatible: Should be :
- "fsl,imx8qxp-sc-wdt"
- followed by "fsl,imx-sc-wdt";
-
-Optional properties:
-- timeout-sec : Contains the watchdog timeout in seconds.
-
-Examples:
-
-watchdog {
- compatible = "fsl,imx8qxp-sc-wdt", "fsl,imx-sc-wdt";
- timeout-sec = <60>;
-};
"allwinner,sun4i-a10-wdt"
"allwinner,sun6i-a31-wdt"
"allwinner,sun50i-a64-wdt","allwinner,sun6i-a31-wdt"
+ "allwinner,sun50i-h6-wdt","allwinner,sun6i-a31-wdt"
"allwinner,suniv-f1c100s-wdt", "allwinner,sun4i-a10-wdt"
- reg : Specifies base physical address and size of the registers.
of the kernel image. That entry point supports two calling
conventions. A summary of the interface is described here. A full
description of the boot requirements is documented in
- Documentation/arm/Booting
+ Documentation/arm/booting.rst
a) ATAGS interface. Minimal information is passed from firmware
to the kernel with a tagged list of predefined parameters.
b) Entry with a flattened device-tree block. Firmware loads the
physical address of the flattened device tree block (dtb) into r2,
r1 is not used, but it is considered good practice to use a valid
- machine number as described in Documentation/arm/Booting.
+ machine number as described in Documentation/arm/booting.rst.
r0 : 0
--- /dev/null
+====================
+Kernel driver lp855x
+====================
+
+Backlight driver for LP855x ICs
+
+Supported chips:
+
+ Texas Instruments LP8550, LP8551, LP8552, LP8553, LP8555, LP8556 and
+ LP8557
+
+Author: Milo(Woogyom) Kim <milo.kim@ti.com>
+
+Description
+-----------
+
+* Brightness control
+
+ Brightness can be controlled by the pwm input or the i2c command.
+ The lp855x driver supports both cases.
+
+* Device attributes
+
+ 1) bl_ctl_mode
+
+ Backlight control mode.
+
+ Value: pwm based or register based
+
+ 2) chip_id
+
+ The lp855x chip id.
+
+ Value: lp8550/lp8551/lp8552/lp8553/lp8555/lp8556/lp8557
+
+Platform data for lp855x
+------------------------
+
+For supporting platform specific data, the lp855x platform data can be used.
+
+* name:
+ Backlight driver name. If it is not defined, default name is set.
+* device_control:
+ Value of DEVICE CONTROL register.
+* initial_brightness:
+ Initial value of backlight brightness.
+* period_ns:
+ Platform specific PWM period value. unit is nano.
+ Only valid when brightness is pwm input mode.
+* size_program:
+ Total size of lp855x_rom_data.
+* rom_data:
+ List of new eeprom/eprom registers.
+
+Examples
+========
+
+1) lp8552 platform data: i2c register mode with new eeprom data::
+
+ #define EEPROM_A5_ADDR 0xA5
+ #define EEPROM_A5_VAL 0x4f /* EN_VSYNC=0 */
+
+ static struct lp855x_rom_data lp8552_eeprom_arr[] = {
+ {EEPROM_A5_ADDR, EEPROM_A5_VAL},
+ };
+
+ static struct lp855x_platform_data lp8552_pdata = {
+ .name = "lcd-bl",
+ .device_control = I2C_CONFIG(LP8552),
+ .initial_brightness = INITIAL_BRT,
+ .size_program = ARRAY_SIZE(lp8552_eeprom_arr),
+ .rom_data = lp8552_eeprom_arr,
+ };
+
+2) lp8556 platform data: pwm input mode with default rom data::
+
+ static struct lp855x_platform_data lp8556_pdata = {
+ .device_control = PWM_CONFIG(LP8556),
+ .initial_brightness = INITIAL_BRT,
+ .period_ns = 1000000,
+ };
-/*****************************************/
-Kernel Connector.
-/*****************************************/
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+Kernel Connector
+================
Kernel connector - new netlink based userspace <-> kernel space easy
to use communication module.
From the userspace point of view it's quite straightforward:
- socket();
- bind();
- send();
- recv();
+ - socket();
+ - bind();
+ - send();
+ - recv();
But if kernelspace wants to use the full power of such connections, the
driver writer must create special sockets, must know about struct sk_buff
handling, etc... The Connector driver allows any kernelspace agents to use
netlink based networking for inter-process communication in a significantly
-easier way:
+easier way::
-int cn_add_callback(struct cb_id *id, char *name, void (*callback) (struct cn_msg *, struct netlink_skb_parms *));
-void cn_netlink_send_multi(struct cn_msg *msg, u16 len, u32 portid, u32 __group, int gfp_mask);
-void cn_netlink_send(struct cn_msg *msg, u32 portid, u32 __group, int gfp_mask);
+ int cn_add_callback(struct cb_id *id, char *name, void (*callback) (struct cn_msg *, struct netlink_skb_parms *));
+ void cn_netlink_send_multi(struct cn_msg *msg, u16 len, u32 portid, u32 __group, int gfp_mask);
+ void cn_netlink_send(struct cn_msg *msg, u32 portid, u32 __group, int gfp_mask);
-struct cb_id
-{
+ struct cb_id
+ {
__u32 idx;
__u32 val;
-};
+ };
idx and val are unique identifiers which must be registered in the
-connector.h header for in-kernel usage. void (*callback) (void *) is a
+connector.h header for in-kernel usage. `void (*callback) (void *)` is a
callback function which will be called when a message with above idx.val
is received by the connector core. The argument for that function must
-be dereferenced to struct cn_msg *.
+be dereferenced to `struct cn_msg *`::
-struct cn_msg
-{
+ struct cn_msg
+ {
struct cb_id id;
__u32 seq;
__u32 ack;
- __u32 len; /* Length of the following data */
+ __u32 len; /* Length of the following data */
__u8 data[0];
-};
-
-/*****************************************/
-Connector interfaces.
-/*****************************************/
-
-int cn_add_callback(struct cb_id *id, char *name, void (*callback) (struct cn_msg *, struct netlink_skb_parms *));
-
- Registers new callback with connector core.
-
- struct cb_id *id - unique connector's user identifier.
- It must be registered in connector.h for legal in-kernel users.
- char *name - connector's callback symbolic name.
- void (*callback) (struct cn..) - connector's callback.
- cn_msg and the sender's credentials
-
-
-void cn_del_callback(struct cb_id *id);
-
- Unregisters new callback with connector core.
-
- struct cb_id *id - unique connector's user identifier.
-
-
-int cn_netlink_send_multi(struct cn_msg *msg, u16 len, u32 portid, u32 __groups, int gfp_mask);
-int cn_netlink_send(struct cn_msg *msg, u32 portid, u32 __groups, int gfp_mask);
+ };
- Sends message to the specified groups. It can be safely called from
- softirq context, but may silently fail under strong memory pressure.
- If there are no listeners for given group -ESRCH can be returned.
+Connector interfaces
+====================
- struct cn_msg * - message header(with attached data).
- u16 len - for *_multi multiple cn_msg messages can be sent
- u32 port - destination port.
- If non-zero the message will be sent to the
- given port, which should be set to the
- original sender.
- u32 __group - destination group.
- If port and __group is zero, then appropriate group will
- be searched through all registered connector users,
- and message will be delivered to the group which was
- created for user with the same ID as in msg.
- If __group is not zero, then message will be delivered
- to the specified group.
- int gfp_mask - GFP mask.
+ .. kernel-doc:: include/linux/connector.h
- Note: When registering new callback user, connector core assigns
- netlink group to the user which is equal to its id.idx.
+ Note:
+ When registering new callback user, connector core assigns
+ netlink group to the user which is equal to its id.idx.
-/*****************************************/
-Protocol description.
-/*****************************************/
+Protocol description
+====================
The current framework offers a transport layer with fixed headers. The
recommended protocol which uses such a header is as following:
As example of this usage can be found in the cn_test.c module which
uses the connector to request notification and to send messages.
-/*****************************************/
-Reliability.
-/*****************************************/
+Reliability
+===========
Netlink itself is not a reliable protocol. That means that messages can
be lost due to memory pressure or process' receiving queue overflowed,
cn_msg [main connector's message header] contains u32 seq and u32 ack
fields.
-/*****************************************/
-Userspace usage.
-/*****************************************/
+Userspace usage
+===============
2.6.14 has a new netlink socket implementation, which by default does not
allow people to send data to netlink groups other than 1.
So, if you wish to use a netlink socket (for example using connector)
with a different group number, the userspace application must subscribe to
-that group first. It can be achieved by the following pseudocode:
+that group first. It can be achieved by the following pseudocode::
-s = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_CONNECTOR);
+ s = socket(PF_NETLINK, SOCK_DGRAM, NETLINK_CONNECTOR);
-l_local.nl_family = AF_NETLINK;
-l_local.nl_groups = 12345;
-l_local.nl_pid = 0;
+ l_local.nl_family = AF_NETLINK;
+ l_local.nl_groups = 12345;
+ l_local.nl_pid = 0;
-if (bind(s, (struct sockaddr *)&l_local, sizeof(struct sockaddr_nl)) == -1) {
+ if (bind(s, (struct sockaddr *)&l_local, sizeof(struct sockaddr_nl)) == -1) {
perror("bind");
close(s);
return -1;
-}
+ }
-{
+ {
int on = l_local.nl_groups;
setsockopt(s, 270, 1, &on, sizeof(on));
-}
+ }
Where 270 above is SOL_NETLINK, and 1 is a NETLINK_ADD_MEMBERSHIP socket
option. To drop a multicast subscription, one should call the above socket
Additional 0xf numbers are allocated to be used by non-in-kernel users.
Due to this limitation, group 0xffffffff does not work now, so one can
-not use add/remove connector's group notifications, but as far as I know,
+not use add/remove connector's group notifications, but as far as I know,
only cn_test.c test module used it.
Some work in netlink area is still being done, so things can be changed in
2.6.15 timeframe, if it will happen, documentation will be updated for that
kernel.
-/*****************************************/
Code samples
-/*****************************************/
+============
Sample code for a connector test module and user space can be found
in samples/connector/. To build this code, enable CONFIG_CONNECTOR
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
Console Drivers
===============
occupied by the system driver. In the same token, if the modular driver is
released by the console, the system driver will take over.
-Modular drivers, from the programmer's point of view, have to call:
+Modular drivers, from the programmer's point of view, have to call::
do_take_over_console() - load and bind driver to console layer
give_up_console() - unload driver; it will only work if driver
is fully unbound
-In newer kernels, the following are also available:
+In newer kernels, the following are also available::
do_register_con_driver()
do_unregister_con_driver()
If sysfs is enabled, the contents of /sys/class/vtconsole can be
examined. This shows the console backends currently registered by the
-system which are named vtcon<n> where <n> is an integer from 0 to 15. Thus:
+system which are named vtcon<n> where <n> is an integer from 0 to 15.
+Thus::
ls /sys/class/vtconsole
. .. vtcon0 vtcon1
-Each directory in /sys/class/vtconsole has 3 files:
+Each directory in /sys/class/vtconsole has 3 files::
ls /sys/class/vtconsole/vtcon0
. .. bind name uevent
read, or acts to bind or unbind the driver to the virtual consoles
when written to. The possible values are:
- 0 - means the driver is not bound and if echo'ed, commands the driver
+ 0
+ - means the driver is not bound and if echo'ed, commands the driver
to unbind
- 1 - means the driver is bound and if echo'ed, commands the driver to
+ 1
+ - means the driver is bound and if echo'ed, commands the driver to
bind
- 2. name - read-only file. Shows the name of the driver in this format:
+ 2. name - read-only file. Shows the name of the driver in this format::
- cat /sys/class/vtconsole/vtcon0/name
- (S) VGA+
+ cat /sys/class/vtconsole/vtcon0/name
+ (S) VGA+
- '(S)' stands for a (S)ystem driver, i.e., it cannot be directly
- commanded to bind or unbind
+ '(S)' stands for a (S)ystem driver, i.e., it cannot be directly
+ commanded to bind or unbind
- 'VGA+' is the name of the driver
+ 'VGA+' is the name of the driver
- cat /sys/class/vtconsole/vtcon1/name
- (M) frame buffer device
+ cat /sys/class/vtconsole/vtcon1/name
+ (M) frame buffer device
- In this case, '(M)' stands for a (M)odular driver, one that can be
- directly commanded to bind or unbind.
+ In this case, '(M)' stands for a (M)odular driver, one that can be
+ directly commanded to bind or unbind.
3. uevent - ignore this file
hand, will bind the driver to the consoles that are currently occupied by a
system driver.
-NOTE1: Binding and unbinding must be selected in Kconfig. It's under:
+NOTE1:
+ Binding and unbinding must be selected in Kconfig. It's under::
-Device Drivers -> Character devices -> Support for binding and unbinding
-console drivers
+ Device Drivers ->
+ Character devices ->
+ Support for binding and unbinding console drivers
-NOTE2: If any of the virtual consoles are in KD_GRAPHICS mode, then binding or
-unbinding will not succeed. An example of an application that sets the console
-to KD_GRAPHICS is X.
+NOTE2:
+ If any of the virtual consoles are in KD_GRAPHICS mode, then binding or
+ unbinding will not succeed. An example of an application that sets the
+ console to KD_GRAPHICS is X.
How useful is this feature? This is very useful for console driver
developers. By unbinding the driver from the console layer, one can unload the
this possible. (NOTE NOTE NOTE: Please read fbcon.txt under Documentation/fb
for more details.)
-Notes for developers:
-=====================
+Notes for developers
+====================
-do_take_over_console() is now broken up into:
+do_take_over_console() is now broken up into::
do_register_con_driver()
do_bind_con_driver() - private function
be fully unbound for this call to succeed. con_is_bound() will check if the
driver is bound or not.
-Guidelines for console driver writers:
+Guidelines for console driver writers
=====================================
In order for binding to and unbinding from the console to properly work,
and unbinding them may cause problems. With minimal fixes, these drivers can
be made to work correctly.
-==========================
Antonino Daplas <adaplas@pol.net>
-
dmatest.timeout=2000 dmatest.iterations=1 dmatest.channel=dma0chan0 dmatest.run=1
-Example of multi-channel test usage:
+Example of multi-channel test usage (new in the 5.0 kernel)::
+
% modprobe dmatest
% echo 2000 > /sys/module/dmatest/parameters/timeout
% echo 1 > /sys/module/dmatest/parameters/iterations
% echo dma0chan2 > /sys/module/dmatest/parameters/channel
% echo 1 > /sys/module/dmatest/parameters/run
-Note: the channel parameter should always be the last parameter set prior to
-running the test (setting run=1), this is because upon setting the channel
-parameter, that specific channel is requested using the dmaengine and a thread
-is created with the existing parameters. This thread is set as pending
-and will be executed once run is set to 1. Any parameters set after the thread
-is created are not applied.
+.. note::
+ For all tests, starting in the 5.0 kernel, either single- or multi-channel,
+ the channel parameter(s) must be set after all other parameters. It is at
+ that time that the existing parameter values are acquired for use by the
+ thread(s). All other parameters are shared. Therefore, if changes are made
+ to any of the other parameters, and an additional channel specified, the
+ (shared) parameters used for all threads will use the new values.
+ After the channels are specified, each thread is set as pending. All threads
+ begin execution when the run parameter is set to 1.
.. hint::
- available channel list could be extracted by running the following command::
+ A list of available channels can be found by running the following command::
% ls -1 /sys/class/dma/
Channels can be freed by setting run to 0.
Example::
+
% echo dma0chan1 > /sys/module/dmatest/parameters/channel
dmatest: Added 1 threads using dma0chan1
% cat /sys/class/dma/dma0chan1/in_use
devm_clk_get()
devm_clk_get_optional()
devm_clk_put()
+ devm_clk_bulk_get()
+ devm_clk_bulk_get_all()
+ devm_clk_bulk_get_optional()
+ devm_get_clk_from_childl()
devm_clk_hw_register()
devm_of_clk_add_hw_provider()
devm_clk_hw_register_clkdev()
-:orphan:
-
============
Driver Model
============
Overview
-Please refer to `Documentation/driver-model/*.rst` for definitions of
+Please refer to `Documentation/driver-api/driver-model/*.rst` for definitions of
various driver types and concepts.
Most of the work of porting devices drivers to the new model happens
- initramfs buffer format
- -----------------------
+=======================
+initramfs buffer format
+=======================
- Al Viro, H. Peter Anvin
- Last revision: 2002-01-13
+Al Viro, H. Peter Anvin
+
+Last revision: 2002-01-13
Starting with kernel 2.5.x, the old "initial ramdisk" protocol is
getting {replaced/complemented} with the new "initial ramfs"
initramfs buffer is thus a single .cpio.gz file.
The full format of the initramfs buffer is defined by the following
-grammar, where:
+grammar, where::
+
* is used to indicate "0 or more occurrences of"
(|) indicates alternatives
+ indicates concatenation
full width of the field, for example, the integer 4780 is represented
by the ASCII string "000012ac"):
+============= ================== ==============================================
Field name Field size Meaning
+============= ================== ==============================================
c_magic 6 bytes The string "070701" or "070702"
c_ino 8 bytes File inode number
c_mode 8 bytes File mode and permissions
c_namesize 8 bytes Length of filename, including final \0
c_chksum 8 bytes Checksum of data field if c_magic is 070702;
otherwise zero
+============= ================== ==============================================
The c_mode field matches the contents of st_mode returned by stat(2)
on Linux, and encodes the file type and file permissions.
marker; the c_filesize for an end-of-archive marker must be zero.
-*** Handling of hard links
+Handling of hard links
+======================
When a nondirectory with c_nlink > 1 is seen, the (c_maj,c_min,c_ino)
tuple is looked up in a tuple buffer. If not found, it is entered in
+=======================
Early userspace support
=======================
the image from specifications.
CPIO ARCHIVE method
+-------------------
You can create a cpio archive that contains the early userspace image.
Your cpio archive should be specified in CONFIG_INITRAMFS_SOURCE and it
combination with a cpio archive.
IMAGE BUILDING method
+---------------------
The kernel build process can also build an early userspace image from
source parts rather than supplying a cpio archive. This method provides
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+Early Userspace
+===============
+
+.. toctree::
+ :maxdepth: 1
+
+ early_userspace_support
+ buffer-format
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
====
EDID
(driver_data).
driver a generic driver, such as described in
- Documentation/driver-model/driver.rst. Only .name,
+ Documentation/driver-api/driver-model/driver.rst. Only .name,
.probe and .remove members are mandatory.
=============== ====================================================
flags are EISA_CONFIG_ENABLED and EISA_CONFIG_FORCED.
res set of four 256 bytes I/O regions allocated to this device
dma_mask DMA mask set from the parent device.
-dev generic device (see Documentation/driver-model/device.rst)
+dev generic device (see Documentation/driver-api/driver-model/device.rst)
======== ============================================================
You can get the 'struct eisa_device' from 'struct device' using the
will pass the struct gpio_chip* for the chip to all IRQ callbacks, so the
callbacks need to embed the gpio_chip in its state container and obtain a
pointer to the container using container_of().
- (See Documentation/driver-model/design-patterns.rst)
+ (See Documentation/driver-api/driver-model/design-patterns.rst)
- gpiochip_irqchip_add_nested(): adds a nested cascaded irqchip to a gpiochip,
as discussed above regarding different types of cascaded irqchips. The
.. toctree::
:maxdepth: 2
+ driver-model/index
basics
infrastructure
+ early-userspace/index
pm/index
clk
device-io
i2c
ipmb
i3c/index
+ interconnect
hsi
edac
scsi
mtdnand
miscellaneous
mei/index
+ mtd/index
+ mmc/index
+ nvdimm/index
w1
- rapidio
+ rapidio/index
s390-drivers
vme
80211/index
firmware/index
pinctl
gpio/index
+ md/index
misc_devices
+ nfc/index
dmaengine/index
slimbus
soundwire/index
fpga/index
acpi/index
+ backlight/lp855x-driver.rst
+ bt8xxgpio
+ connector
+ console
+ dcdbas
+ dell_rbu
+ edid
+ eisa
+ isa
+ isapnp
generic-counter
+ lightnvm-pblk
+ memory-devices/index
+ men-chameleon-bus
+ ntb
+ nvmem
+ parport-lowlevel
+ pps
+ ptp
+ phy/index
+ pti_intel_mid
+ pwm
+ rfkill
+ serial/index
+ sgi-ioc4
+ sm501
+ smsc_ece1099
+ switchtec
+ sync_file
+ vfio-mediated-device
+ vfio
+ xilinx/index
+ xillybus
+ zorro
.. only:: subproject and html
.. SPDX-License-Identifier: GPL-2.0
-:orphan:
-
=====================================
GENERIC SYSTEM INTERCONNECT SUBSYSTEM
=====================================
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+====
+RAID
+====
+
+.. toctree::
+ :maxdepth: 1
+
+ md-cluster
+ raid5-cache
+ raid5-ppl
+==========
+MD Cluster
+==========
+
The cluster MD is a shared-device RAID for a cluster, it supports
two levels: raid1 and raid10 (limited support).
1. On-disk format
+=================
Separate write-intent-bitmaps are used for each cluster node.
The bitmaps record all writes that may have been started on that node,
-and may not yet have finished. The on-disk layout is:
+and may not yet have finished. The on-disk layout is::
-0 4k 8k 12k
--------------------------------------------------------------------
-| idle | md super | bm super [0] + bits |
-| bm bits[0, contd] | bm super[1] + bits | bm bits[1, contd] |
-| bm super[2] + bits | bm bits [2, contd] | bm super[3] + bits |
-| bm bits [3, contd] | | |
+ 0 4k 8k 12k
+ -------------------------------------------------------------------
+ | idle | md super | bm super [0] + bits |
+ | bm bits[0, contd] | bm super[1] + bits | bm bits[1, contd] |
+ | bm super[2] + bits | bm bits [2, contd] | bm super[3] + bits |
+ | bm bits [3, contd] | | |
During "normal" functioning we assume the filesystem ensures that only
one node writes to any given block at a time, so a write request will
2. DLM Locks for management
+===========================
There are three groups of locks for managing the device:
2.1 Bitmap lock resource (bm_lockres)
+-------------------------------------
The bm_lockres protects individual node bitmaps. They are named in
the form bitmap000 for node 1, bitmap001 for node 2 and so on. When a
joins the cluster.
2.2 Message passing locks
+-------------------------
Each node has to communicate with other nodes when starting or ending
resync, and for metadata superblock updates. This communication is
with the Lock Value Block (LVB) of one of the "message" lock.
2.3 new-device management
+-------------------------
A single lock: "no-new-dev" is used to co-ordinate the addition of
new devices - this must be synchronized across the array.
Normally all nodes hold a concurrent-read lock on this device.
3. Communication
+================
Messages can be broadcast to all nodes, and the sender waits for all
other nodes to acknowledge the message before proceeding. Only one
message can be processed at a time.
3.1 Message Types
+-----------------
There are six types of messages which are passed:
- 3.1.1 METADATA_UPDATED: informs other nodes that the metadata has
+3.1.1 METADATA_UPDATED
+^^^^^^^^^^^^^^^^^^^^^^
+
+ informs other nodes that the metadata has
been updated, and the node must re-read the md superblock. This is
performed synchronously. It is primarily used to signal device
failure.
- 3.1.2 RESYNCING: informs other nodes that a resync is initiated or
+3.1.2 RESYNCING
+^^^^^^^^^^^^^^^
+ informs other nodes that a resync is initiated or
ended so that each node may suspend or resume the region. Each
RESYNCING message identifies a range of the devices that the
sending node is about to resync. This overrides any previous
notification from that node: only one ranged can be resynced at a
time per-node.
- 3.1.3 NEWDISK: informs other nodes that a device is being added to
+3.1.3 NEWDISK
+^^^^^^^^^^^^^
+
+ informs other nodes that a device is being added to
the array. Message contains an identifier for that device. See
below for further details.
- 3.1.4 REMOVE: A failed or spare device is being removed from the
+3.1.4 REMOVE
+^^^^^^^^^^^^
+
+ A failed or spare device is being removed from the
array. The slot-number of the device is included in the message.
- 3.1.5 RE_ADD: A failed device is being re-activated - the assumption
+ 3.1.5 RE_ADD:
+
+ A failed device is being re-activated - the assumption
is that it has been determined to be working again.
- 3.1.6 BITMAP_NEEDS_SYNC: if a node is stopped locally but the bitmap
+ 3.1.6 BITMAP_NEEDS_SYNC:
+
+ If a node is stopped locally but the bitmap
isn't clean, then another node is informed to take the ownership of
resync.
3.2 Communication mechanism
+---------------------------
The DLM LVB is used to communicate within nodes of the cluster. There
are three resources used for the purpose:
- 3.2.1 token: The resource which protects the entire communication
+3.2.1 token
+^^^^^^^^^^^
+ The resource which protects the entire communication
system. The node having the token resource is allowed to
communicate.
- 3.2.2 message: The lock resource which carries the data to
- communicate.
+3.2.2 message
+^^^^^^^^^^^^^
+ The lock resource which carries the data to communicate.
- 3.2.3 ack: The resource, acquiring which means the message has been
+3.2.3 ack
+^^^^^^^^^
+
+ The resource, acquiring which means the message has been
acknowledged by all nodes in the cluster. The BAST of the resource
is used to inform the receiving node that a node wants to
communicate.
The algorithm is:
- 1. receive status - all nodes have concurrent-reader lock on "ack".
+ 1. receive status - all nodes have concurrent-reader lock on "ack"::
+
+ sender receiver receiver
+ "ack":CR "ack":CR "ack":CR
- sender receiver receiver
- "ack":CR "ack":CR "ack":CR
+ 2. sender get EX on "token",
+ sender get EX on "message"::
- 2. sender get EX on "token"
- sender get EX on "message"
- sender receiver receiver
- "token":EX "ack":CR "ack":CR
- "message":EX
- "ack":CR
+ sender receiver receiver
+ "token":EX "ack":CR "ack":CR
+ "message":EX
+ "ack":CR
Sender checks that it still needs to send a message. Messages
received or other events that happened while waiting for the
"token" may have made this message inappropriate or redundant.
- 3. sender writes LVB.
+ 3. sender writes LVB
+
sender down-convert "message" from EX to CW
+
sender try to get EX of "ack"
- [ wait until all receivers have *processed* the "message" ]
- [ triggered by bast of "ack" ]
- receiver get CR on "message"
- receiver read LVB
- receiver processes the message
- [ wait finish ]
- receiver releases "ack"
- receiver tries to get PR on "message"
+ ::
+
+ [ wait until all receivers have *processed* the "message" ]
- sender receiver receiver
- "token":EX "message":CR "message":CR
- "message":CW
- "ack":EX
+ [ triggered by bast of "ack" ]
+ receiver get CR on "message"
+ receiver read LVB
+ receiver processes the message
+ [ wait finish ]
+ receiver releases "ack"
+ receiver tries to get PR on "message"
+
+ sender receiver receiver
+ "token":EX "message":CR "message":CR
+ "message":CW
+ "ack":EX
4. triggered by grant of EX on "ack" (indicating all receivers
have processed message)
+
sender down-converts "ack" from EX to CR
+
sender releases "message"
+
sender releases "token"
- receiver upconvert to PR on "message"
- receiver get CR of "ack"
- receiver release "message"
- sender receiver receiver
- "ack":CR "ack":CR "ack":CR
+ ::
+
+ receiver upconvert to PR on "message"
+ receiver get CR of "ack"
+ receiver release "message"
+
+ sender receiver receiver
+ "ack":CR "ack":CR "ack":CR
4. Handling Failures
+====================
4.1 Node Failure
+----------------
When a node fails, the DLM informs the cluster with the slot
number. The node starts a cluster recovery thread. The cluster
- cleans the bitmap of the failed node
- releases bitmap<number> lock of the failed node
- initiates resync of the bitmap on the current node
- md_check_recovery is invoked within recover_bitmaps,
- then md_check_recovery -> metadata_update_start/finish,
- it will lock the communication by lock_comm.
- Which means when one node is resyncing it blocks all
- other nodes from writing anywhere on the array.
+ md_check_recovery is invoked within recover_bitmaps,
+ then md_check_recovery -> metadata_update_start/finish,
+ it will lock the communication by lock_comm.
+ Which means when one node is resyncing it blocks all
+ other nodes from writing anywhere on the array.
The resync process is the regular md resync. However, in a clustered
environment when a resync is performed, it needs to tell other nodes
particular I/O range should be suspended or not.
4.2 Device Failure
+==================
Device failures are handled and communicated with the metadata update
routine. When a node detects a device failure it does not allow
acknowledged by all other nodes.
5. Adding a new Device
+----------------------
For adding a new device, it is necessary that all nodes "see" the new
device to be added. For this, the following algorithm is used:
- 1. Node 1 issues mdadm --manage /dev/mdX --add /dev/sdYY which issues
+ 1. Node 1 issues mdadm --manage /dev/mdX --add /dev/sdYY which issues
ioctl(ADD_NEW_DISK with disc.state set to MD_DISK_CLUSTER_ADD)
- 2. Node 1 sends a NEWDISK message with uuid and slot number
- 3. Other nodes issue kobject_uevent_env with uuid and slot number
+ 2. Node 1 sends a NEWDISK message with uuid and slot number
+ 3. Other nodes issue kobject_uevent_env with uuid and slot number
(Steps 4,5 could be a udev rule)
- 4. In userspace, the node searches for the disk, perhaps
+ 4. In userspace, the node searches for the disk, perhaps
using blkid -t SUB_UUID=""
- 5. Other nodes issue either of the following depending on whether
+ 5. Other nodes issue either of the following depending on whether
the disk was found:
ioctl(ADD_NEW_DISK with disc.state set to MD_DISK_CANDIDATE and
- disc.number set to slot number)
+ disc.number set to slot number)
ioctl(CLUSTERED_DISK_NACK)
- 6. Other nodes drop lock on "no-new-devs" (CR) if device is found
- 7. Node 1 attempts EX lock on "no-new-dev"
- 8. If node 1 gets the lock, it sends METADATA_UPDATED after
+ 6. Other nodes drop lock on "no-new-devs" (CR) if device is found
+ 7. Node 1 attempts EX lock on "no-new-dev"
+ 8. If node 1 gets the lock, it sends METADATA_UPDATED after
unmarking the disk as SpareLocal
- 9. If not (get "no-new-dev" lock), it fails the operation and sends
+ 9. If not (get "no-new-dev" lock), it fails the operation and sends
METADATA_UPDATED.
10. Other nodes get the information whether a disk is added or not
by the following METADATA_UPDATED.
-6. Module interface.
+6. Module interface
+===================
There are 17 call-backs which the md core can make to the cluster
module. Understanding these can give a good overview of the whole
process.
6.1 join(nodes) and leave()
+---------------------------
These are called when an array is started with a clustered bitmap,
and when the array is stopped. join() ensures the cluster is
Only the first 'nodes' nodes in the cluster can use the array.
6.2 slot_number()
+-----------------
Reports the slot number advised by the cluster infrastructure.
Range is from 0 to nodes-1.
6.3 resync_info_update()
+------------------------
This updates the resync range that is stored in the bitmap lock.
The starting point is updated as the resync progresses. The
It does *not* send a RESYNCING message.
6.4 resync_start(), resync_finish()
+-----------------------------------
These are called when resync/recovery/reshape starts or stops.
They update the resyncing range in the bitmap lock and also
resync_finish() also sends a BITMAP_NEEDS_SYNC message which
allows some other node to take over.
-6.5 metadata_update_start(), metadata_update_finish(),
- metadata_update_cancel().
+6.5 metadata_update_start(), metadata_update_finish(), metadata_update_cancel()
+-------------------------------------------------------------------------------
metadata_update_start is used to get exclusive access to
the metadata. If a change is still needed once that access is
can be used to release the lock.
6.6 area_resyncing()
+--------------------
This combines two elements of functionality.
a node failure.
6.7 add_new_disk_start(), add_new_disk_finish(), new_disk_ack()
+---------------------------------------------------------------
These are used to manage the new-disk protocol described above.
When a new device is added, add_new_disk_start() is called before
new_disk_ack() is called.
6.8 remove_disk()
+-----------------
This is called when a spare or failed device is removed from
the array. It causes a REMOVE message to be send to other nodes.
6.9 gather_bitmaps()
+--------------------
This sends a RE_ADD message to all other nodes and then
gathers bitmap information from all bitmaps. This combined
bitmap is then used to recovery the re-added device.
6.10 lock_all_bitmaps() and unlock_all_bitmaps()
+------------------------------------------------
These are called when change bitmap to none. If a node plans
to clear the cluster raid's bitmap, it need to make sure no other
accordingly.
7. Unsupported features
+=======================
There are somethings which are not supported by cluster MD yet.
-RAID5 cache
+================
+RAID 4/5/6 cache
+================
Raid 4/5/6 could include an extra disk for data cache besides normal RAID
disks. The role of RAID disks isn't changed with the cache disk. The cache disk
since 4.4) or write-back mode (supported since 4.10). mdadm (supported since
3.4) has a new option '--write-journal' to create array with cache. Please
refer to mdadm manual for details. By default (RAID array starts), the cache is
-in write-through mode. A user can switch it to write-back mode by:
+in write-through mode. A user can switch it to write-back mode by::
-echo "write-back" > /sys/block/md0/md/journal_mode
+ echo "write-back" > /sys/block/md0/md/journal_mode
-And switch it back to write-through mode by:
+And switch it back to write-through mode by::
-echo "write-through" > /sys/block/md0/md/journal_mode
+ echo "write-through" > /sys/block/md0/md/journal_mode
In both modes, all writes to the array will hit cache disk first. This means
the cache disk must be fast and sustainable.
--------------------------------------
-write-through mode:
+write-through mode
+==================
This mode mainly fixes the 'write hole' issue. For RAID 4/5/6 array, an unclean
shutdown can cause data in some stripes to not be in consistent state, eg, data
In write-through mode, the cache disk isn't required to be big. Several
hundreds megabytes are enough.
---------------------------------------
-write-back mode:
+write-back mode
+===============
write-back mode fixes the 'write hole' issue too, since all write data is
cached on cache disk. But the main goal of 'write-back' cache is to speed up
In write-back mode, MD also caches data in memory. The memory cache includes
the same data stored on cache disk, so a power loss doesn't cause data loss.
The memory cache size has performance impact for the array. It's recommended
-the size is big. A user can configure the size by:
+the size is big. A user can configure the size by::
-echo "2048" > /sys/block/md0/md/stripe_cache_size
+ echo "2048" > /sys/block/md0/md/stripe_cache_size
Too small cache disk will make the write aggregation less efficient in this
mode depending on the workloads. It's recommended to use a cache disk with at
least several gigabytes size in write-back mode.
---------------------------------------
-The implementation:
+The implementation
+==================
The write-through and write-back cache use the same disk format. The cache disk
is organized as a simple write log. The log consists of 'meta data' and 'data'
+==================
Partial Parity Log
+==================
Partial Parity Log (PPL) is a feature available for RAID5 arrays. The issue
addressed by PPL is that after a dirty shutdown, parity of a particular stripe
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+Memory Controller drivers
+=========================
+
+.. toctree::
+ :maxdepth: 1
+
+ ti-emif
+ ti-gpmc
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
-TI EMIF SDRAM Controller Driver:
+.. SPDX-License-Identifier: GPL-2.0
+
+===============================
+TI EMIF SDRAM Controller Driver
+===============================
Author
-========
+======
Aneesh V <aneesh@ti.com>
Location
-============
+========
driver/memory/emif.c
Supported SoCs:
-===================
+===============
TI OMAP44xx
TI OMAP54xx
Menuconfig option:
-==========================
+==================
Device Drivers
Memory devices
Texas Instruments EMIF driver
parameters and other settings during frequency, voltage and
temperature changes
-Platform Data (see include/linux/platform_data/emif_plat.h):
-=====================================================================
+Platform Data (see include/linux/platform_data/emif_plat.h)
+===========================================================
DDR device details and other board dependent and SoC dependent
information can be passed through platform data (struct emif_platform_data)
+
- DDR device details: 'struct ddr_device_info'
- Device AC timings: 'struct lpddr2_timings' and 'struct lpddr2_min_tck'
- Custom configurations: customizable policy options through
- IP revision
- PHY type
-Interface to the external world:
-================================
+Interface to the external world
+===============================
EMIF driver registers notifiers for voltage and frequency changes
affecting EMIF and takes appropriate actions when these are invoked.
+
- freq_pre_notify_handling()
- freq_post_notify_handling()
- volt_notify_handling()
Debugfs
-========
+=======
The driver creates two debugfs entries per device.
+
- regcache_dump : dump of register values calculated and saved for all
frequencies used so far.
- mr4 : last polled value of MR4 register in the LPDDR2 device. MR4
-GPMC (General Purpose Memory Controller):
-=========================================
+.. SPDX-License-Identifier: GPL-2.0
+
+========================================
+GPMC (General Purpose Memory Controller)
+========================================
GPMC is an unified memory controller dedicated to interfacing external
memory devices like
+
* Asynchronous SRAM like memories and application specific integrated
circuit devices.
* Asynchronous, synchronous, and page mode burst NOR flash devices
in mainline having custom timing routine) and by simulation.
gpmc timing dependency on peripheral timings:
+
[<gpmc_timing>: <peripheral timing1>, <peripheral timing2> ...]
1. common
-cs_on: t_ceasu
-adv_on: t_avdasu, t_ceavd
+
+cs_on:
+ t_ceasu
+adv_on:
+ t_avdasu, t_ceavd
2. sync common
-sync_clk: clk
-page_burst_access: t_bacc
-clk_activation: t_ces, t_avds
+
+sync_clk:
+ clk
+page_burst_access:
+ t_bacc
+clk_activation:
+ t_ces, t_avds
3. read async muxed
-adv_rd_off: t_avdp_r
-oe_on: t_oeasu, t_aavdh
-access: t_iaa, t_oe, t_ce, t_aa
-rd_cycle: t_rd_cycle, t_cez_r, t_oez
+
+adv_rd_off:
+ t_avdp_r
+oe_on:
+ t_oeasu, t_aavdh
+access:
+ t_iaa, t_oe, t_ce, t_aa
+rd_cycle:
+ t_rd_cycle, t_cez_r, t_oez
4. read async non-muxed
-adv_rd_off: t_avdp_r
-oe_on: t_oeasu
-access: t_iaa, t_oe, t_ce, t_aa
-rd_cycle: t_rd_cycle, t_cez_r, t_oez
+
+adv_rd_off:
+ t_avdp_r
+oe_on:
+ t_oeasu
+access:
+ t_iaa, t_oe, t_ce, t_aa
+rd_cycle:
+ t_rd_cycle, t_cez_r, t_oez
5. read sync muxed
-adv_rd_off: t_avdp_r, t_avdh
-oe_on: t_oeasu, t_ach, cyc_aavdh_oe
-access: t_iaa, cyc_iaa, cyc_oe
-rd_cycle: t_cez_r, t_oez, t_ce_rdyz
+
+adv_rd_off:
+ t_avdp_r, t_avdh
+oe_on:
+ t_oeasu, t_ach, cyc_aavdh_oe
+access:
+ t_iaa, cyc_iaa, cyc_oe
+rd_cycle:
+ t_cez_r, t_oez, t_ce_rdyz
6. read sync non-muxed
-adv_rd_off: t_avdp_r
-oe_on: t_oeasu
-access: t_iaa, cyc_iaa, cyc_oe
-rd_cycle: t_cez_r, t_oez, t_ce_rdyz
+
+adv_rd_off:
+ t_avdp_r
+oe_on:
+ t_oeasu
+access:
+ t_iaa, cyc_iaa, cyc_oe
+rd_cycle:
+ t_cez_r, t_oez, t_ce_rdyz
7. write async muxed
-adv_wr_off: t_avdp_w
-we_on, wr_data_mux_bus: t_weasu, t_aavdh, cyc_aavhd_we
-we_off: t_wpl
-cs_wr_off: t_wph
-wr_cycle: t_cez_w, t_wr_cycle
+
+adv_wr_off:
+ t_avdp_w
+we_on, wr_data_mux_bus:
+ t_weasu, t_aavdh, cyc_aavhd_we
+we_off:
+ t_wpl
+cs_wr_off:
+ t_wph
+wr_cycle:
+ t_cez_w, t_wr_cycle
8. write async non-muxed
-adv_wr_off: t_avdp_w
-we_on, wr_data_mux_bus: t_weasu
-we_off: t_wpl
-cs_wr_off: t_wph
-wr_cycle: t_cez_w, t_wr_cycle
+
+adv_wr_off:
+ t_avdp_w
+we_on, wr_data_mux_bus:
+ t_weasu
+we_off:
+ t_wpl
+cs_wr_off:
+ t_wph
+wr_cycle:
+ t_cez_w, t_wr_cycle
9. write sync muxed
-adv_wr_off: t_avdp_w, t_avdh
-we_on, wr_data_mux_bus: t_weasu, t_rdyo, t_aavdh, cyc_aavhd_we
-we_off: t_wpl, cyc_wpl
-cs_wr_off: t_wph
-wr_cycle: t_cez_w, t_ce_rdyz
+
+adv_wr_off:
+ t_avdp_w, t_avdh
+we_on, wr_data_mux_bus:
+ t_weasu, t_rdyo, t_aavdh, cyc_aavhd_we
+we_off:
+ t_wpl, cyc_wpl
+cs_wr_off:
+ t_wph
+wr_cycle:
+ t_cez_w, t_ce_rdyz
10. write sync non-muxed
-adv_wr_off: t_avdp_w
-we_on, wr_data_mux_bus: t_weasu, t_rdyo
-we_off: t_wpl, cyc_wpl
-cs_wr_off: t_wph
-wr_cycle: t_cez_w, t_ce_rdyz
-
-
-Note: Many of gpmc timings are dependent on other gpmc timings (a few
-gpmc timings purely dependent on other gpmc timings, a reason that
-some of the gpmc timings are missing above), and it will result in
-indirect dependency of peripheral timings to gpmc timings other than
-mentioned above, refer timing routine for more details. To know what
-these peripheral timings correspond to, please see explanations in
-struct gpmc_device_timings definition. And for gpmc timings refer
-IP details (link above).
+
+adv_wr_off:
+ t_avdp_w
+we_on, wr_data_mux_bus:
+ t_weasu, t_rdyo
+we_off:
+ t_wpl, cyc_wpl
+cs_wr_off:
+ t_wph
+wr_cycle:
+ t_cez_w, t_ce_rdyz
+
+
+Note:
+ Many of gpmc timings are dependent on other gpmc timings (a few
+ gpmc timings purely dependent on other gpmc timings, a reason that
+ some of the gpmc timings are missing above), and it will result in
+ indirect dependency of peripheral timings to gpmc timings other than
+ mentioned above, refer timing routine for more details. To know what
+ these peripheral timings correspond to, please see explanations in
+ struct gpmc_device_timings definition. And for gpmc timings refer
+ IP details (link above).
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+MMC/SD/SDIO card support
+========================
+
+.. toctree::
+ :maxdepth: 1
+
+ mmc-dev-attrs
+ mmc-dev-parts
+ mmc-async-req
+ mmc-tools
+========================
+MMC Asynchronous Request
+========================
+
Rationale
=========
How significant is the cache maintenance overhead?
+
It depends. Fast eMMC and multiple cache levels with speculative cache
pre-fetch makes the cache overhead relatively significant. If the DMA
preparations for the next request are done in parallel with the current
transfer, the DMA preparation overhead would not affect the MMC performance.
+
The intention of non-blocking (asynchronous) MMC requests is to minimize the
time between when an MMC request ends and another MMC request begins.
+
Using mmc_wait_for_req(), the MMC controller is idle while dma_map_sg and
dma_unmap_sg are processing. Using non-blocking MMC requests makes it
possible to prepare the caches for next job in parallel with an active
================
The mmc_blk_issue_rw_rq() in the MMC block driver is made non-blocking.
+
The increase in throughput is proportional to the time it takes to
prepare (major part of preparations are dma_map_sg() and dma_unmap_sg())
a request and how fast the memory is. The faster the MMC/SD is the
======================
There is one new public function mmc_start_req().
+
It starts a new MMC command request for a host. The function isn't
truly non-blocking. If there is an ongoing async request it waits
for completion of that request and starts the new one and returns. It
There are two optional members in the mmc_host_ops -- pre_req() and
post_req() -- that the host driver may implement in order to move work
to before and after the actual mmc_host_ops.request() function is called.
+
In the DMA case pre_req() may do dma_map_sg() and prepare the DMA
descriptor, and post_req() runs the dma_unmap_sg().
The first request in a series of requests can't be prepared in parallel
with the previous transfer, since there is no previous request.
+
The argument is_first_req in pre_req() indicates that there is no previous
request. The host driver may optimize for this scenario to minimize
the performance loss. A way to optimize for this is to split the current
request in two chunks, prepare the first chunk and start the request,
and finally prepare the second chunk and start the transfer.
-Pseudocode to handle is_first_req scenario with minimal prepare overhead:
-
-if (is_first_req && req->size > threshold)
- /* start MMC transfer for the complete transfer size */
- mmc_start_command(MMC_CMD_TRANSFER_FULL_SIZE);
-
- /*
- * Begin to prepare DMA while cmd is being processed by MMC.
- * The first chunk of the request should take the same time
- * to prepare as the "MMC process command time".
- * If prepare time exceeds MMC cmd time
- * the transfer is delayed, guesstimate max 4k as first chunk size.
- */
- prepare_1st_chunk_for_dma(req);
- /* flush pending desc to the DMAC (dmaengine.h) */
- dma_issue_pending(req->dma_desc);
-
- prepare_2nd_chunk_for_dma(req);
- /*
- * The second issue_pending should be called before MMC runs out
- * of the first chunk. If the MMC runs out of the first data chunk
- * before this call, the transfer is delayed.
- */
- dma_issue_pending(req->dma_desc);
+Pseudocode to handle is_first_req scenario with minimal prepare overhead::
+
+ if (is_first_req && req->size > threshold)
+ /* start MMC transfer for the complete transfer size */
+ mmc_start_command(MMC_CMD_TRANSFER_FULL_SIZE);
+
+ /*
+ * Begin to prepare DMA while cmd is being processed by MMC.
+ * The first chunk of the request should take the same time
+ * to prepare as the "MMC process command time".
+ * If prepare time exceeds MMC cmd time
+ * the transfer is delayed, guesstimate max 4k as first chunk size.
+ */
+ prepare_1st_chunk_for_dma(req);
+ /* flush pending desc to the DMAC (dmaengine.h) */
+ dma_issue_pending(req->dma_desc);
+
+ prepare_2nd_chunk_for_dma(req);
+ /*
+ * The second issue_pending should be called before MMC runs out
+ * of the first chunk. If the MMC runs out of the first data chunk
+ * before this call, the transfer is delayed.
+ */
+ dma_issue_pending(req->dma_desc);
+==================================
SD and MMC Block Device Attributes
==================================
The following attributes are read/write.
- force_ro Enforce read-only access even if write protect switch is off.
+ ======== ===============================================
+ force_ro Enforce read-only access even if write protect switch is off.
+ ======== ===============================================
SD and MMC Device Attributes
============================
All attributes are read-only.
+ ====================== ===============================================
cid Card Identification Register
csd Card Specific Data Register
scr SD Card Configuration Register (SD only)
date Manufacturing Date (from CID Register)
- fwrev Firmware/Product Revision (from CID Register) (SD and MMCv1 only)
- hwrev Hardware/Product Revision (from CID Register) (SD and MMCv1 only)
+ fwrev Firmware/Product Revision (from CID Register)
+ (SD and MMCv1 only)
+ hwrev Hardware/Product Revision (from CID Register)
+ (SD and MMCv1 only)
manfid Manufacturer ID (from CID Register)
name Product Name (from CID Register)
oemid OEM/Application ID (from CID Register)
- prv Product Revision (from CID Register) (SD and MMCv4 only)
+ prv Product Revision (from CID Register)
+ (SD and MMCv4 only)
serial Product Serial Number (from CID Register)
erase_size Erase group size
preferred_erase_size Preferred erase size
rel_sectors Reliable write sector count
ocr Operation Conditions Register
dsr Driver Stage Register
- cmdq_en Command Queue enabled: 1 => enabled, 0 => not enabled
+ cmdq_en Command Queue enabled:
+
+ 1 => enabled, 0 => not enabled
+ ====================== ===============================================
Note on Erase Size and Preferred Erase Size:
SD/MMC cards can erase an arbitrarily large area up to and
including the whole card. When erasing a large area it may
be desirable to do it in smaller chunks for three reasons:
- 1. A single erase command will make all other I/O on
+
+ 1. A single erase command will make all other I/O on
the card wait. This is not a problem if the whole card
is being erased, but erasing one partition will make
I/O for another partition on the same card wait for the
duration of the erase - which could be a several
minutes.
- 2. To be able to inform the user of erase progress.
- 3. The erase timeout becomes too large to be very
+ 2. To be able to inform the user of erase progress.
+ 3. The erase timeout becomes too large to be very
useful. Because the erase timeout contains a margin
which is multiplied by the size of the erase area,
the value can end up being several minutes for large
"preferred_erase_size" is in bytes.
Note on raw_rpmb_size_mult:
+
"raw_rpmb_size_mult" is a multiple of 128kB block.
+
RPMB size in byte is calculated by using the following equation:
- RPMB partition size = 128kB x raw_rpmb_size_mult
+
+ RPMB partition size = 128kB x raw_rpmb_size_mult
+============================
SD and MMC Device Partitions
============================
accidental bricking.
To enable write access to /dev/mmcblkXbootY, disable the forced read-only
-access with:
+access with::
-echo 0 > /sys/block/mmcblkXbootY/force_ro
+ echo 0 > /sys/block/mmcblkXbootY/force_ro
-To re-enable read-only access:
+To re-enable read-only access::
-echo 1 > /sys/block/mmcblkXbootY/force_ro
+ echo 1 > /sys/block/mmcblkXbootY/force_ro
The boot partitions can also be locked read only until the next power on,
-with:
+with::
-echo 1 > /sys/block/mmcblkXbootY/ro_lock_until_next_power_on
+ echo 1 > /sys/block/mmcblkXbootY/ro_lock_until_next_power_on
This is a feature of the card and not of the kernel. If the card does
not support boot partition locking, the file will not exist. If the
+======================
MMC tools introduction
======================
There is one MMC test tools called mmc-utils, which is maintained by Chris Ball,
you can find it at the below public git repository:
-http://git.kernel.org/cgit/linux/kernel/git/cjb/mmc-utils.git/
+
+ http://git.kernel.org/cgit/linux/kernel/git/cjb/mmc-utils.git/
Functions
=========
The mmc-utils tools can do the following:
+
- Print and parse extcsd data.
- Determine the eMMC writeprotect status.
- Set the eMMC writeprotect status.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==============================
+Memory Technology Device (MTD)
+==============================
+
+.. toctree::
+ :maxdepth: 1
+
+ intel-spi
+ nand_ecc
+ spi-nor
+==============================
Upgrading BIOS using intel-spi
-------------------------------
+==============================
Many Intel CPUs like Baytrail and Braswell include SPI serial flash host
controller which is used to hold BIOS and other platform specific data.
module parameter to modprobe).
4) Once the board is up and running again, find the right MTD partition
- (it is named as "BIOS"):
+ (it is named as "BIOS")::
- # cat /proc/mtd
- dev: size erasesize name
- mtd0: 00800000 00001000 "BIOS"
+ # cat /proc/mtd
+ dev: size erasesize name
+ mtd0: 00800000 00001000 "BIOS"
So here it will be /dev/mtd0 but it may vary.
- 5) Make backup of the existing image first:
+ 5) Make backup of the existing image first::
- # dd if=/dev/mtd0ro of=bios.bak
- 16384+0 records in
- 16384+0 records out
- 8388608 bytes (8.4 MB) copied, 10.0269 s, 837 kB/s
+ # dd if=/dev/mtd0ro of=bios.bak
+ 16384+0 records in
+ 16384+0 records out
+ 8388608 bytes (8.4 MB) copied, 10.0269 s, 837 kB/s
- 6) Verify the backup
+ 6) Verify the backup:
- # sha1sum /dev/mtd0ro bios.bak
- fdbb011920572ca6c991377c4b418a0502668b73 /dev/mtd0ro
- fdbb011920572ca6c991377c4b418a0502668b73 bios.bak
+ # sha1sum /dev/mtd0ro bios.bak
+ fdbb011920572ca6c991377c4b418a0502668b73 /dev/mtd0ro
+ fdbb011920572ca6c991377c4b418a0502668b73 bios.bak
The SHA1 sums must match. Otherwise do not continue any further!
7) Erase the SPI serial flash. After this step, do not reboot the
- board! Otherwise it will not start anymore.
+ board! Otherwise it will not start anymore::
- # flash_erase /dev/mtd0 0 0
- Erasing 4 Kibyte @ 7ff000 -- 100 % complete
+ # flash_erase /dev/mtd0 0 0
+ Erasing 4 Kibyte @ 7ff000 -- 100 % complete
8) Once completed without errors you can write the new BIOS image:
# dd if=MNW2MAX1.X64.0092.R01.1605221712.bin of=/dev/mtd0
9) Verify that the new content of the SPI serial flash matches the new
- BIOS image:
+ BIOS image::
- # sha1sum /dev/mtd0ro MNW2MAX1.X64.0092.R01.1605221712.bin
- 9b4df9e4be2057fceec3a5529ec3d950836c87a2 /dev/mtd0ro
- 9b4df9e4be2057fceec3a5529ec3d950836c87a2 MNW2MAX1.X64.0092.R01.1605221712.bin
+ # sha1sum /dev/mtd0ro MNW2MAX1.X64.0092.R01.1605221712.bin
+ 9b4df9e4be2057fceec3a5529ec3d950836c87a2 /dev/mtd0ro
+ 9b4df9e4be2057fceec3a5529ec3d950836c87a2 MNW2MAX1.X64.0092.R01.1605221712.bin
The SHA1 sums should match.
References
----------
-[1] https://firmware.intel.com/sites/default/files/MinnowBoard.MAX_.X64.92.R01.zip
+[1] https://firmware.intel.com/sites/default/files/MinnowBoard%2EMAX_%2EX64%2E92%2ER01%2Ezip
+
[2] http://www.linux-mtd.infradead.org/
+==========================
+NAND Error-correction Code
+==========================
+
Introduction
============
Back to ecc.
Let's give a small figure:
+========= ==== ==== ==== ==== ==== ==== ==== ==== === === === === ====
byte 0: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp0 rp2 rp4 ... rp14
byte 1: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp1 rp2 rp4 ... rp14
byte 2: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp0 rp3 rp4 ... rp14
byte 3: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp1 rp3 rp4 ... rp14
byte 4: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp0 rp2 rp5 ... rp14
-....
+...
byte 254: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp0 rp3 rp5 ... rp15
byte 255: bit7 bit6 bit5 bit4 bit3 bit2 bit1 bit0 rp1 rp3 rp5 ... rp15
cp1 cp0 cp1 cp0 cp1 cp0 cp1 cp0
cp3 cp3 cp2 cp2 cp3 cp3 cp2 cp2
cp5 cp5 cp5 cp5 cp4 cp4 cp4 cp4
+========= ==== ==== ==== ==== ==== ==== ==== ==== === === === === ====
This figure represents a sector of 256 bytes.
cp is my abbreviation for column parity, rp for row parity.
Let's start to explain column parity.
-cp0 is the parity that belongs to all bit0, bit2, bit4, bit6.
-so the sum of all bit0, bit2, bit4 and bit6 values + cp0 itself is even.
+
+- cp0 is the parity that belongs to all bit0, bit2, bit4, bit6.
+
+ so the sum of all bit0, bit2, bit4 and bit6 values + cp0 itself is even.
+
Similarly cp1 is the sum of all bit1, bit3, bit5 and bit7.
-cp2 is the parity over bit0, bit1, bit4 and bit5
-cp3 is the parity over bit2, bit3, bit6 and bit7.
-cp4 is the parity over bit0, bit1, bit2 and bit3.
-cp5 is the parity over bit4, bit5, bit6 and bit7.
+
+- cp2 is the parity over bit0, bit1, bit4 and bit5
+- cp3 is the parity over bit2, bit3, bit6 and bit7.
+- cp4 is the parity over bit0, bit1, bit2 and bit3.
+- cp5 is the parity over bit4, bit5, bit6 and bit7.
+
Note that each of cp0 .. cp5 is exactly one bit.
Row parity actually works almost the same.
-rp0 is the parity of all even bytes (0, 2, 4, 6, ... 252, 254)
-rp1 is the parity of all odd bytes (1, 3, 5, 7, ..., 253, 255)
-rp2 is the parity of all bytes 0, 1, 4, 5, 8, 9, ...
-(so handle two bytes, then skip 2 bytes).
-rp3 is covers the half rp2 does not cover (bytes 2, 3, 6, 7, 10, 11, ...)
-for rp4 the rule is cover 4 bytes, skip 4 bytes, cover 4 bytes, skip 4 etc.
-so rp4 calculates parity over bytes 0, 1, 2, 3, 8, 9, 10, 11, 16, ...)
-and rp5 covers the other half, so bytes 4, 5, 6, 7, 12, 13, 14, 15, 20, ..
+
+- rp0 is the parity of all even bytes (0, 2, 4, 6, ... 252, 254)
+- rp1 is the parity of all odd bytes (1, 3, 5, 7, ..., 253, 255)
+- rp2 is the parity of all bytes 0, 1, 4, 5, 8, 9, ...
+ (so handle two bytes, then skip 2 bytes).
+- rp3 is covers the half rp2 does not cover (bytes 2, 3, 6, 7, 10, 11, ...)
+- for rp4 the rule is cover 4 bytes, skip 4 bytes, cover 4 bytes, skip 4 etc.
+
+ so rp4 calculates parity over bytes 0, 1, 2, 3, 8, 9, 10, 11, 16, ...)
+- and rp5 covers the other half, so bytes 4, 5, 6, 7, 12, 13, 14, 15, 20, ..
+
The story now becomes quite boring. I guess you get the idea.
-rp6 covers 8 bytes then skips 8 etc
-rp7 skips 8 bytes then covers 8 etc
-rp8 covers 16 bytes then skips 16 etc
-rp9 skips 16 bytes then covers 16 etc
-rp10 covers 32 bytes then skips 32 etc
-rp11 skips 32 bytes then covers 32 etc
-rp12 covers 64 bytes then skips 64 etc
-rp13 skips 64 bytes then covers 64 etc
-rp14 covers 128 bytes then skips 128
-rp15 skips 128 bytes then covers 128
+
+- rp6 covers 8 bytes then skips 8 etc
+- rp7 skips 8 bytes then covers 8 etc
+- rp8 covers 16 bytes then skips 16 etc
+- rp9 skips 16 bytes then covers 16 etc
+- rp10 covers 32 bytes then skips 32 etc
+- rp11 skips 32 bytes then covers 32 etc
+- rp12 covers 64 bytes then skips 64 etc
+- rp13 skips 64 bytes then covers 64 etc
+- rp14 covers 128 bytes then skips 128
+- rp15 skips 128 bytes then covers 128
In the end the parity bits are grouped together in three bytes as
follows:
+
+===== ===== ===== ===== ===== ===== ===== ===== =====
ECC Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1 Bit 0
+===== ===== ===== ===== ===== ===== ===== ===== =====
ECC 0 rp07 rp06 rp05 rp04 rp03 rp02 rp01 rp00
ECC 1 rp15 rp14 rp13 rp12 rp11 rp10 rp09 rp08
ECC 2 cp5 cp4 cp3 cp2 cp1 cp0 1 1
+===== ===== ===== ===== ===== ===== ===== ===== =====
I detected after writing this that ST application note AN1823
(http://www.st.com/stonline/) gives a much
nicer picture.(but they use line parity as term where I use row parity)
Oh well, I'm graphically challenged, so suffer with me for a moment :-)
+
And I could not reuse the ST picture anyway for copyright reasons.
=========
Implementing the parity calculation is pretty simple.
-In C pseudocode:
-for (i = 0; i < 256; i++)
-{
+In C pseudocode::
+
+ for (i = 0; i < 256; i++)
+ {
if (i & 0x01)
rp1 = bit7 ^ bit6 ^ bit5 ^ bit4 ^ bit3 ^ bit2 ^ bit1 ^ bit0 ^ rp1;
else
cp3 = bit7 ^ bit6 ^ bit3 ^ bit2 ^ cp3
cp4 = bit3 ^ bit2 ^ bit1 ^ bit0 ^ cp4
cp5 = bit7 ^ bit6 ^ bit5 ^ bit4 ^ cp5
-}
+ }
Analysis 0
Attempt 1
=========
-const char parity[256] = {
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
- 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0
-};
-
-void ecc1(const unsigned char *buf, unsigned char *code)
-{
- int i;
- const unsigned char *bp = buf;
- unsigned char cur;
- unsigned char rp0, rp1, rp2, rp3, rp4, rp5, rp6, rp7;
- unsigned char rp8, rp9, rp10, rp11, rp12, rp13, rp14, rp15;
- unsigned char par;
-
- par = 0;
- rp0 = 0; rp1 = 0; rp2 = 0; rp3 = 0;
- rp4 = 0; rp5 = 0; rp6 = 0; rp7 = 0;
- rp8 = 0; rp9 = 0; rp10 = 0; rp11 = 0;
- rp12 = 0; rp13 = 0; rp14 = 0; rp15 = 0;
-
- for (i = 0; i < 256; i++)
- {
- cur = *bp++;
- par ^= cur;
- if (i & 0x01) rp1 ^= cur; else rp0 ^= cur;
- if (i & 0x02) rp3 ^= cur; else rp2 ^= cur;
- if (i & 0x04) rp5 ^= cur; else rp4 ^= cur;
- if (i & 0x08) rp7 ^= cur; else rp6 ^= cur;
- if (i & 0x10) rp9 ^= cur; else rp8 ^= cur;
- if (i & 0x20) rp11 ^= cur; else rp10 ^= cur;
- if (i & 0x40) rp13 ^= cur; else rp12 ^= cur;
- if (i & 0x80) rp15 ^= cur; else rp14 ^= cur;
- }
- code[0] =
- (parity[rp7] << 7) |
- (parity[rp6] << 6) |
- (parity[rp5] << 5) |
- (parity[rp4] << 4) |
- (parity[rp3] << 3) |
- (parity[rp2] << 2) |
- (parity[rp1] << 1) |
- (parity[rp0]);
- code[1] =
- (parity[rp15] << 7) |
- (parity[rp14] << 6) |
- (parity[rp13] << 5) |
- (parity[rp12] << 4) |
- (parity[rp11] << 3) |
- (parity[rp10] << 2) |
- (parity[rp9] << 1) |
- (parity[rp8]);
- code[2] =
- (parity[par & 0xf0] << 7) |
- (parity[par & 0x0f] << 6) |
- (parity[par & 0xcc] << 5) |
- (parity[par & 0x33] << 4) |
- (parity[par & 0xaa] << 3) |
- (parity[par & 0x55] << 2);
- code[0] = ~code[0];
- code[1] = ~code[1];
- code[2] = ~code[2];
-}
+::
+
+ const char parity[256] = {
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1,
+ 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0
+ };
+
+ void ecc1(const unsigned char *buf, unsigned char *code)
+ {
+ int i;
+ const unsigned char *bp = buf;
+ unsigned char cur;
+ unsigned char rp0, rp1, rp2, rp3, rp4, rp5, rp6, rp7;
+ unsigned char rp8, rp9, rp10, rp11, rp12, rp13, rp14, rp15;
+ unsigned char par;
+
+ par = 0;
+ rp0 = 0; rp1 = 0; rp2 = 0; rp3 = 0;
+ rp4 = 0; rp5 = 0; rp6 = 0; rp7 = 0;
+ rp8 = 0; rp9 = 0; rp10 = 0; rp11 = 0;
+ rp12 = 0; rp13 = 0; rp14 = 0; rp15 = 0;
+
+ for (i = 0; i < 256; i++)
+ {
+ cur = *bp++;
+ par ^= cur;
+ if (i & 0x01) rp1 ^= cur; else rp0 ^= cur;
+ if (i & 0x02) rp3 ^= cur; else rp2 ^= cur;
+ if (i & 0x04) rp5 ^= cur; else rp4 ^= cur;
+ if (i & 0x08) rp7 ^= cur; else rp6 ^= cur;
+ if (i & 0x10) rp9 ^= cur; else rp8 ^= cur;
+ if (i & 0x20) rp11 ^= cur; else rp10 ^= cur;
+ if (i & 0x40) rp13 ^= cur; else rp12 ^= cur;
+ if (i & 0x80) rp15 ^= cur; else rp14 ^= cur;
+ }
+ code[0] =
+ (parity[rp7] << 7) |
+ (parity[rp6] << 6) |
+ (parity[rp5] << 5) |
+ (parity[rp4] << 4) |
+ (parity[rp3] << 3) |
+ (parity[rp2] << 2) |
+ (parity[rp1] << 1) |
+ (parity[rp0]);
+ code[1] =
+ (parity[rp15] << 7) |
+ (parity[rp14] << 6) |
+ (parity[rp13] << 5) |
+ (parity[rp12] << 4) |
+ (parity[rp11] << 3) |
+ (parity[rp10] << 2) |
+ (parity[rp9] << 1) |
+ (parity[rp8]);
+ code[2] =
+ (parity[par & 0xf0] << 7) |
+ (parity[par & 0x0f] << 6) |
+ (parity[par & 0xcc] << 5) |
+ (parity[par & 0x33] << 4) |
+ (parity[par & 0xaa] << 3) |
+ (parity[par & 0x55] << 2);
+ code[0] = ~code[0];
+ code[1] = ~code[1];
+ code[2] = ~code[2];
+ }
Still pretty straightforward. The last three invert statements are there to
give a checksum of 0xff 0xff 0xff for an empty flash. In an empty flash
Attempt 2
=========
-extern const char parity[256];
-
-void ecc2(const unsigned char *buf, unsigned char *code)
-{
- int i;
- const unsigned long *bp = (unsigned long *)buf;
- unsigned long cur;
- unsigned long rp0, rp1, rp2, rp3, rp4, rp5, rp6, rp7;
- unsigned long rp8, rp9, rp10, rp11, rp12, rp13, rp14, rp15;
- unsigned long par;
-
- par = 0;
- rp0 = 0; rp1 = 0; rp2 = 0; rp3 = 0;
- rp4 = 0; rp5 = 0; rp6 = 0; rp7 = 0;
- rp8 = 0; rp9 = 0; rp10 = 0; rp11 = 0;
- rp12 = 0; rp13 = 0; rp14 = 0; rp15 = 0;
-
- for (i = 0; i < 64; i++)
- {
- cur = *bp++;
- par ^= cur;
- if (i & 0x01) rp5 ^= cur; else rp4 ^= cur;
- if (i & 0x02) rp7 ^= cur; else rp6 ^= cur;
- if (i & 0x04) rp9 ^= cur; else rp8 ^= cur;
- if (i & 0x08) rp11 ^= cur; else rp10 ^= cur;
- if (i & 0x10) rp13 ^= cur; else rp12 ^= cur;
- if (i & 0x20) rp15 ^= cur; else rp14 ^= cur;
- }
- /*
- we need to adapt the code generation for the fact that rp vars are now
- long; also the column parity calculation needs to be changed.
- we'll bring rp4 to 15 back to single byte entities by shifting and
- xoring
- */
- rp4 ^= (rp4 >> 16); rp4 ^= (rp4 >> 8); rp4 &= 0xff;
- rp5 ^= (rp5 >> 16); rp5 ^= (rp5 >> 8); rp5 &= 0xff;
- rp6 ^= (rp6 >> 16); rp6 ^= (rp6 >> 8); rp6 &= 0xff;
- rp7 ^= (rp7 >> 16); rp7 ^= (rp7 >> 8); rp7 &= 0xff;
- rp8 ^= (rp8 >> 16); rp8 ^= (rp8 >> 8); rp8 &= 0xff;
- rp9 ^= (rp9 >> 16); rp9 ^= (rp9 >> 8); rp9 &= 0xff;
- rp10 ^= (rp10 >> 16); rp10 ^= (rp10 >> 8); rp10 &= 0xff;
- rp11 ^= (rp11 >> 16); rp11 ^= (rp11 >> 8); rp11 &= 0xff;
- rp12 ^= (rp12 >> 16); rp12 ^= (rp12 >> 8); rp12 &= 0xff;
- rp13 ^= (rp13 >> 16); rp13 ^= (rp13 >> 8); rp13 &= 0xff;
- rp14 ^= (rp14 >> 16); rp14 ^= (rp14 >> 8); rp14 &= 0xff;
- rp15 ^= (rp15 >> 16); rp15 ^= (rp15 >> 8); rp15 &= 0xff;
- rp3 = (par >> 16); rp3 ^= (rp3 >> 8); rp3 &= 0xff;
- rp2 = par & 0xffff; rp2 ^= (rp2 >> 8); rp2 &= 0xff;
- par ^= (par >> 16);
- rp1 = (par >> 8); rp1 &= 0xff;
- rp0 = (par & 0xff);
- par ^= (par >> 8); par &= 0xff;
-
- code[0] =
- (parity[rp7] << 7) |
- (parity[rp6] << 6) |
- (parity[rp5] << 5) |
- (parity[rp4] << 4) |
- (parity[rp3] << 3) |
- (parity[rp2] << 2) |
- (parity[rp1] << 1) |
- (parity[rp0]);
- code[1] =
- (parity[rp15] << 7) |
- (parity[rp14] << 6) |
- (parity[rp13] << 5) |
- (parity[rp12] << 4) |
- (parity[rp11] << 3) |
- (parity[rp10] << 2) |
- (parity[rp9] << 1) |
- (parity[rp8]);
- code[2] =
- (parity[par & 0xf0] << 7) |
- (parity[par & 0x0f] << 6) |
- (parity[par & 0xcc] << 5) |
- (parity[par & 0x33] << 4) |
- (parity[par & 0xaa] << 3) |
- (parity[par & 0x55] << 2);
- code[0] = ~code[0];
- code[1] = ~code[1];
- code[2] = ~code[2];
-}
+::
+
+ extern const char parity[256];
+
+ void ecc2(const unsigned char *buf, unsigned char *code)
+ {
+ int i;
+ const unsigned long *bp = (unsigned long *)buf;
+ unsigned long cur;
+ unsigned long rp0, rp1, rp2, rp3, rp4, rp5, rp6, rp7;
+ unsigned long rp8, rp9, rp10, rp11, rp12, rp13, rp14, rp15;
+ unsigned long par;
+
+ par = 0;
+ rp0 = 0; rp1 = 0; rp2 = 0; rp3 = 0;
+ rp4 = 0; rp5 = 0; rp6 = 0; rp7 = 0;
+ rp8 = 0; rp9 = 0; rp10 = 0; rp11 = 0;
+ rp12 = 0; rp13 = 0; rp14 = 0; rp15 = 0;
+
+ for (i = 0; i < 64; i++)
+ {
+ cur = *bp++;
+ par ^= cur;
+ if (i & 0x01) rp5 ^= cur; else rp4 ^= cur;
+ if (i & 0x02) rp7 ^= cur; else rp6 ^= cur;
+ if (i & 0x04) rp9 ^= cur; else rp8 ^= cur;
+ if (i & 0x08) rp11 ^= cur; else rp10 ^= cur;
+ if (i & 0x10) rp13 ^= cur; else rp12 ^= cur;
+ if (i & 0x20) rp15 ^= cur; else rp14 ^= cur;
+ }
+ /*
+ we need to adapt the code generation for the fact that rp vars are now
+ long; also the column parity calculation needs to be changed.
+ we'll bring rp4 to 15 back to single byte entities by shifting and
+ xoring
+ */
+ rp4 ^= (rp4 >> 16); rp4 ^= (rp4 >> 8); rp4 &= 0xff;
+ rp5 ^= (rp5 >> 16); rp5 ^= (rp5 >> 8); rp5 &= 0xff;
+ rp6 ^= (rp6 >> 16); rp6 ^= (rp6 >> 8); rp6 &= 0xff;
+ rp7 ^= (rp7 >> 16); rp7 ^= (rp7 >> 8); rp7 &= 0xff;
+ rp8 ^= (rp8 >> 16); rp8 ^= (rp8 >> 8); rp8 &= 0xff;
+ rp9 ^= (rp9 >> 16); rp9 ^= (rp9 >> 8); rp9 &= 0xff;
+ rp10 ^= (rp10 >> 16); rp10 ^= (rp10 >> 8); rp10 &= 0xff;
+ rp11 ^= (rp11 >> 16); rp11 ^= (rp11 >> 8); rp11 &= 0xff;
+ rp12 ^= (rp12 >> 16); rp12 ^= (rp12 >> 8); rp12 &= 0xff;
+ rp13 ^= (rp13 >> 16); rp13 ^= (rp13 >> 8); rp13 &= 0xff;
+ rp14 ^= (rp14 >> 16); rp14 ^= (rp14 >> 8); rp14 &= 0xff;
+ rp15 ^= (rp15 >> 16); rp15 ^= (rp15 >> 8); rp15 &= 0xff;
+ rp3 = (par >> 16); rp3 ^= (rp3 >> 8); rp3 &= 0xff;
+ rp2 = par & 0xffff; rp2 ^= (rp2 >> 8); rp2 &= 0xff;
+ par ^= (par >> 16);
+ rp1 = (par >> 8); rp1 &= 0xff;
+ rp0 = (par & 0xff);
+ par ^= (par >> 8); par &= 0xff;
+
+ code[0] =
+ (parity[rp7] << 7) |
+ (parity[rp6] << 6) |
+ (parity[rp5] << 5) |
+ (parity[rp4] << 4) |
+ (parity[rp3] << 3) |
+ (parity[rp2] << 2) |
+ (parity[rp1] << 1) |
+ (parity[rp0]);
+ code[1] =
+ (parity[rp15] << 7) |
+ (parity[rp14] << 6) |
+ (parity[rp13] << 5) |
+ (parity[rp12] << 4) |
+ (parity[rp11] << 3) |
+ (parity[rp10] << 2) |
+ (parity[rp9] << 1) |
+ (parity[rp8]);
+ code[2] =
+ (parity[par & 0xf0] << 7) |
+ (parity[par & 0x0f] << 6) |
+ (parity[par & 0xcc] << 5) |
+ (parity[par & 0x33] << 4) |
+ (parity[par & 0xaa] << 3) |
+ (parity[par & 0x55] << 2);
+ code[0] = ~code[0];
+ code[1] = ~code[1];
+ code[2] = ~code[2];
+ }
The parity array is not shown any more. Note also that for these
examples I kinda deviated from my regular programming style by allowing
Attempt 3
=========
-Odd replaced:
- if (i & 0x01) rp5 ^= cur; else rp4 ^= cur;
- if (i & 0x02) rp7 ^= cur; else rp6 ^= cur;
- if (i & 0x04) rp9 ^= cur; else rp8 ^= cur;
- if (i & 0x08) rp11 ^= cur; else rp10 ^= cur;
- if (i & 0x10) rp13 ^= cur; else rp12 ^= cur;
- if (i & 0x20) rp15 ^= cur; else rp14 ^= cur;
-with
- if (i & 0x01) rp5 ^= cur;
- if (i & 0x02) rp7 ^= cur;
- if (i & 0x04) rp9 ^= cur;
- if (i & 0x08) rp11 ^= cur;
- if (i & 0x10) rp13 ^= cur;
- if (i & 0x20) rp15 ^= cur;
-
- and outside the loop added:
- rp4 = par ^ rp5;
- rp6 = par ^ rp7;
- rp8 = par ^ rp9;
- rp10 = par ^ rp11;
- rp12 = par ^ rp13;
- rp14 = par ^ rp15;
+Odd replaced::
+
+ if (i & 0x01) rp5 ^= cur; else rp4 ^= cur;
+ if (i & 0x02) rp7 ^= cur; else rp6 ^= cur;
+ if (i & 0x04) rp9 ^= cur; else rp8 ^= cur;
+ if (i & 0x08) rp11 ^= cur; else rp10 ^= cur;
+ if (i & 0x10) rp13 ^= cur; else rp12 ^= cur;
+ if (i & 0x20) rp15 ^= cur; else rp14 ^= cur;
+
+with::
+
+ if (i & 0x01) rp5 ^= cur;
+ if (i & 0x02) rp7 ^= cur;
+ if (i & 0x04) rp9 ^= cur;
+ if (i & 0x08) rp11 ^= cur;
+ if (i & 0x10) rp13 ^= cur;
+ if (i & 0x20) rp15 ^= cur;
+
+and outside the loop added::
+
+ rp4 = par ^ rp5;
+ rp6 = par ^ rp7;
+ rp8 = par ^ rp9;
+ rp10 = par ^ rp11;
+ rp12 = par ^ rp13;
+ rp14 = par ^ rp15;
And after that the code takes about 30% more time, although the number of
statements is reduced. This is also reflected in the assembly code.
=========
Unrolled the loop 1, 2, 3 and 4 times.
-For 4 the code starts with:
+For 4 the code starts with::
for (i = 0; i < 4; i++)
{
==========
Unrolling once gains about 15%
+
Unrolling twice keeps the gain at about 15%
+
Unrolling three times gives a gain of 30% compared to attempt 2.
+
Unrolling four times gives a marginal improvement compared to unrolling
three times.
Effectively so all odd digit rp assignments in the loop were removed.
This included the else clause of the if statements.
-Of course after the loop we need to correct things by adding code like:
+Of course after the loop we need to correct things by adding code like::
+
rp5 = par ^ rp4;
+
Also the initial assignments (rp5 = 0; etc) could be removed.
Along the line I also removed the initialisation of rp0/1/2/3.
Attempt 6
=========
-THe code within the for loop was changed to:
+THe code within the for loop was changed to::
for (i = 0; i < 4; i++)
{
definitely seemed to be the jackpot!
There is a little bit more room for improvement though. There are three
-places with statements:
-rp4 ^= cur; rp6 ^= cur;
+places with statements::
+
+ rp4 ^= cur; rp6 ^= cur;
+
It seems more efficient to also maintain a variable rp4_6 in the while
loop; This eliminates 3 statements per loop. Of course after the loop we
-need to correct by adding:
- rp4 ^= rp4_6;
- rp6 ^= rp4_6
+need to correct by adding::
+
+ rp4 ^= rp4_6;
+ rp6 ^= rp4_6
+
Furthermore there are 4 sequential assignments to rp8. This can be
encoded slightly more efficiently by saving tmppar before those 4 lines
and later do rp8 = rp8 ^ tmppar ^ notrp8;
Attempt 7
=========
-The new code now looks like:
+The new code now looks like::
for (i = 0; i < 4; i++)
{
further there is still room to optimize the generation of the ecc codes.
We can simply calculate the total parity. If this is 0 then rp4 = rp5
etc. If the parity is 1, then rp4 = !rp5;
+
But if rp4 = rp5 we do not need rp5 etc. We can just write the even bits
-in the result byte and then do something like
+in the result byte and then do something like::
+
code[0] |= (code[0] << 1);
+
Lets test this.
kind of other things, like having dedicated parity arrays to avoid the
shift after parity[rp7] << 7; No gain.
Change the lookup using the parity array by using shift operators (e.g.
-replace parity[rp7] << 7 with:
-rp7 ^= (rp7 << 4);
-rp7 ^= (rp7 << 2);
-rp7 ^= (rp7 << 1);
-rp7 &= 0x80;
+replace parity[rp7] << 7 with::
+
+ rp7 ^= (rp7 << 4);
+ rp7 ^= (rp7 << 2);
+ rp7 ^= (rp7 << 1);
+ rp7 &= 0x80;
+
No gain.
The only marginal change was inverting the parity bits, so we can remove
For correcting errors I again used the ST application note as a starter,
but I also peeked at the existing code.
+
The algorithm itself is pretty straightforward. Just xor the given and
the calculated ecc. If all bytes are 0 there is no problem. If 11 bits
are 1 we have one correctable bit error. If there is 1 bit 1, we have an
error in the given ecc code.
+
It proved to be fastest to do some table lookups. Performance gain
introduced by this is about a factor 2 on my system when a repair had to
be done, and 1% or so if no repair had to be done.
+
Code size increased from 330 bytes to 686 bytes for this function.
(gcc 4.2, -O3)
The gain when calculating the ecc is tremendous. Om my development hardware
a speedup of a factor of 18 for ecc calculation was achieved. On a test on an
embedded system with a MIPS core a factor 7 was obtained.
+
On a test with a Linksys NSLU2 (ARMv5TE processor) the speedup was a factor
5 (big endian mode, gcc 4.1.2, -O3)
+
For correction not much gain could be obtained (as bitflips are rare). Then
again there are also much less cycles spent there.
this is very tricky (at least for intel hw).
Author: Frans Meulenbroeks
+
Copyright (C) 2008 Koninklijke Philips Electronics NV.
- SPI NOR framework
- ============================================
+=================
+SPI NOR framework
+=================
Part I - Why do we need this framework?
---------------------------------------
With this new layer, the SPI NOR controller driver does not depend on the
m25p80 code anymore.
- Before this framework, the layer is like:
+Before this framework, the layer is like::
MTD
------------------------
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+Near Field Communication
+========================
+
+.. toctree::
+ :maxdepth: 1
+
+ nfc-hci
+ nfc-pn544
+========================
HCI backend for NFC Core
+========================
-Author: Eric Lapuyade, Samuel Ortiz
-Contact: eric.lapuyade@intel.com, samuel.ortiz@intel.com
+- Author: Eric Lapuyade, Samuel Ortiz
+- Contact: eric.lapuyade@intel.com, samuel.ortiz@intel.com
General
-------
and a translation will be forwarded to NFC Core as needed. There are hooks to
let the HCI driver handle proprietary events or override standard behavior.
HCI uses 2 execution contexts:
+
- one for executing commands : nfc_hci_msg_tx_work(). Only one command
-can be executing at any given moment.
+ can be executing at any given moment.
- one for dispatching received events and commands : nfc_hci_msg_rx_work().
-HCI Session initialization:
----------------------------
+HCI Session initialization
+--------------------------
The Session initialization is an HCI standard which must unfortunately
support proprietary gates. This is the reason why the driver will pass a list
--------------
A driver would normally register itself with HCI and provide the following
-entry points:
+entry points::
-struct nfc_hci_ops {
+ struct nfc_hci_ops {
int (*open)(struct nfc_hci_dev *hdev);
void (*close)(struct nfc_hci_dev *hdev);
int (*hci_ready) (struct nfc_hci_dev *hdev);
struct nfc_target *target);
int (*event_received)(struct nfc_hci_dev *hdev, u8 gate, u8 event,
struct sk_buff *skb);
-};
+ };
- open() and close() shall turn the hardware on and off.
- hci_ready() is an optional entry point that is called right after the hci
-session has been set up. The driver can use it to do additional initialization
-that must be performed using HCI commands.
+ session has been set up. The driver can use it to do additional initialization
+ that must be performed using HCI commands.
- xmit() shall simply write a frame to the physical link.
- start_poll() is an optional entrypoint that shall set the hardware in polling
-mode. This must be implemented only if the hardware uses proprietary gates or a
-mechanism slightly different from the HCI standard.
+ mode. This must be implemented only if the hardware uses proprietary gates or a
+ mechanism slightly different from the HCI standard.
- dep_link_up() is called after a p2p target has been detected, to finish
-the p2p connection setup with hardware parameters that need to be passed back
-to nfc core.
+ the p2p connection setup with hardware parameters that need to be passed back
+ to nfc core.
- dep_link_down() is called to bring the p2p link down.
- target_from_gate() is an optional entrypoint to return the nfc protocols
-corresponding to a proprietary gate.
+ corresponding to a proprietary gate.
- complete_target_discovered() is an optional entry point to let the driver
-perform additional proprietary processing necessary to auto activate the
-discovered target.
+ perform additional proprietary processing necessary to auto activate the
+ discovered target.
- im_transceive() must be implemented by the driver if proprietary HCI commands
-are required to send data to the tag. Some tag types will require custom
-commands, others can be written to using the standard HCI commands. The driver
-can check the tag type and either do proprietary processing, or return 1 to ask
-for standard processing. The data exchange command itself must be sent
-asynchronously.
+ are required to send data to the tag. Some tag types will require custom
+ commands, others can be written to using the standard HCI commands. The driver
+ can check the tag type and either do proprietary processing, or return 1 to ask
+ for standard processing. The data exchange command itself must be sent
+ asynchronously.
- tm_send() is called to send data in the case of a p2p connection
- check_presence() is an optional entry point that will be called regularly
-by the core to check that an activated tag is still in the field. If this is
-not implemented, the core will not be able to push tag_lost events to the user
-space
+ by the core to check that an activated tag is still in the field. If this is
+ not implemented, the core will not be able to push tag_lost events to the user
+ space
- event_received() is called to handle an event coming from the chip. Driver
-can handle the event or return 1 to let HCI attempt standard processing.
+ can handle the event or return 1 to let HCI attempt standard processing.
On the rx path, the driver is responsible to push incoming HCP frames to HCI
using nfc_hci_recv_frame(). HCI will take care of re-aggregation and handling
PHY Management
--------------
-The physical link (i2c, ...) management is defined by the following structure:
+The physical link (i2c, ...) management is defined by the following structure::
-struct nfc_phy_ops {
+ struct nfc_phy_ops {
int (*write)(void *dev_id, struct sk_buff *skb);
int (*enable)(void *dev_id);
void (*disable)(void *dev_id);
-};
-
-enable(): turn the phy on (power on), make it ready to transfer data
-disable(): turn the phy off
-write(): Send a data frame to the chip. Note that to enable higher
-layers such as an llc to store the frame for re-emission, this function must
-not alter the skb. It must also not return a positive result (return 0 for
-success, negative for failure).
+ };
+
+enable():
+ turn the phy on (power on), make it ready to transfer data
+disable():
+ turn the phy off
+write():
+ Send a data frame to the chip. Note that to enable higher
+ layers such as an llc to store the frame for re-emission, this
+ function must not alter the skb. It must also not return a positive
+ result (return 0 for success, negative for failure).
Data coming from the chip shall be sent directly to nfc_hci_recv_frame().
Communication between the CPU and the chip often requires some link layer
protocol. Those are isolated as modules managed by the HCI layer. There are
currently two modules : nop (raw transfert) and shdlc.
-A new llc must implement the following functions:
+A new llc must implement the following functions::
-struct nfc_llc_ops {
+ struct nfc_llc_ops {
void *(*init) (struct nfc_hci_dev *hdev, xmit_to_drv_t xmit_to_drv,
rcv_to_hci_t rcv_to_hci, int tx_headroom,
int tx_tailroom, int *rx_headroom, int *rx_tailroom,
int (*stop) (struct nfc_llc *llc);
void (*rcv_from_drv) (struct nfc_llc *llc, struct sk_buff *skb);
int (*xmit_from_hci) (struct nfc_llc *llc, struct sk_buff *skb);
-};
-
-- init() : allocate and init your private storage
-- deinit() : cleanup
-- start() : establish the logical connection
-- stop () : terminate the logical connection
-- rcv_from_drv() : handle data coming from the chip, going to HCI
-- xmit_from_hci() : handle data sent by HCI, going to the chip
+ };
+
+init():
+ allocate and init your private storage
+deinit():
+ cleanup
+start():
+ establish the logical connection
+stop ():
+ terminate the logical connection
+rcv_from_drv():
+ handle data coming from the chip, going to HCI
+xmit_from_hci():
+ handle data sent by HCI, going to the chip
The llc must be registered with nfc before it can be used. Do that by
-calling nfc_llc_register(const char *name, struct nfc_llc_ops *ops);
+calling::
+
+ nfc_llc_register(const char *name, struct nfc_llc_ops *ops);
Again, note that the llc does not handle the physical link. It is thus very
easy to mix any physical link with any llc for a given chip driver.
the current llc. In case of shdlc, the frame is queued in shdlc rx queue.
- SHDLC State Machine worker (SMW)
-Only when llc_shdlc is used: handles shdlc rx & tx queues.
-Dispatches HCI cmd responses.
+
+ Only when llc_shdlc is used: handles shdlc rx & tx queues.
+
+ Dispatches HCI cmd responses.
- HCI Tx Cmd worker (MSGTXWQ)
-Serializes execution of HCI commands. Completes execution in case of response
-timeout.
+
+ Serializes execution of HCI commands.
+
+ Completes execution in case of response timeout.
- HCI Rx worker (MSGRXWQ)
-Dispatches incoming HCI commands or events.
+
+ Dispatches incoming HCI commands or events.
- Syscall context from a userspace call (SYSCALL)
-Any entrypoint in HCI called from NFC Core
+
+ Any entrypoint in HCI called from NFC Core
Workflow executing an HCI command (using shdlc)
-----------------------------------------------
Executing an HCI command can easily be performed synchronously using the
-following API:
+following API::
-int nfc_hci_send_cmd (struct nfc_hci_dev *hdev, u8 gate, u8 cmd,
+ int nfc_hci_send_cmd (struct nfc_hci_dev *hdev, u8 gate, u8 cmd,
const u8 *param, size_t param_len, struct sk_buff **skb)
The API must be invoked from a context that can sleep. Most of the time, this
callback that was provided by nfc_hci_msg_tx_work() when it sent the command.
The completion callback will then wake the syscall context.
-It is also possible to execute the command asynchronously using this API:
+It is also possible to execute the command asynchronously using this API::
-static int nfc_hci_execute_cmd_async(struct nfc_hci_dev *hdev, u8 pipe, u8 cmd,
- const u8 *param, size_t param_len,
- data_exchange_cb_t cb, void *cb_context)
+ static int nfc_hci_execute_cmd_async(struct nfc_hci_dev *hdev, u8 pipe, u8 cmd,
+ const u8 *param, size_t param_len,
+ data_exchange_cb_t cb, void *cb_context)
The workflow is the same, except that the API call returns immediately, and
the callback will be called with the result from the SMW context.
Handling of these errors is done as follows:
- driver (pn544) fails to deliver an incoming frame: it stores the error such
-that any subsequent call to the driver will result in this error. Then it calls
-the standard nfc_shdlc_recv_frame() with a NULL argument to report the problem
-above. shdlc stores a EREMOTEIO sticky status, which will trigger SMW to
-report above in turn.
+ that any subsequent call to the driver will result in this error. Then it
+ calls the standard nfc_shdlc_recv_frame() with a NULL argument to report the
+ problem above. shdlc stores a EREMOTEIO sticky status, which will trigger
+ SMW to report above in turn.
- SMW is basically a background thread to handle incoming and outgoing shdlc
-frames. This thread will also check the shdlc sticky status and report to HCI
-when it discovers it is not able to run anymore because of an unrecoverable
-error that happened within shdlc or below. If the problem occurs during shdlc
-connection, the error is reported through the connect completion.
+ frames. This thread will also check the shdlc sticky status and report to HCI
+ when it discovers it is not able to run anymore because of an unrecoverable
+ error that happened within shdlc or below. If the problem occurs during shdlc
+ connection, the error is reported through the connect completion.
- HCI: if an internal HCI error happens (frame is lost), or HCI is reported an
-error from a lower layer, HCI will either complete the currently executing
-command with that error, or notify NFC Core directly if no command is executing.
+ error from a lower layer, HCI will either complete the currently executing
+ command with that error, or notify NFC Core directly if no command is
+ executing.
- NFC Core: when NFC Core is notified of an error from below and polling is
-active, it will send a tag discovered event with an empty tag list to the user
-space to let it know that the poll operation will never be able to detect a tag.
-If polling is not active and the error was sticky, lower levels will return it
-at next invocation.
+ active, it will send a tag discovered event with an empty tag list to the user
+ space to let it know that the poll operation will never be able to detect a
+ tag. If polling is not active and the error was sticky, lower levels will
+ return it at next invocation.
-Kernel driver for the NXP Semiconductors PN544 Near Field
-Communication chip
+============================================================================
+Kernel driver for the NXP Semiconductors PN544 Near Field Communication chip
+============================================================================
+
General
-------
+=============================
BTT - Block Translation Table
=============================
1. Introduction
----------------
+===============
Persistent memory based storage is able to perform IO at byte (or more
accurately, cache line) granularity. However, we often want to expose such
2. Static Layout
-----------------
+================
The underlying storage on which a BTT can be laid out is not limited in any way.
The BTT, however, splits the available space into chunks of up to 512 GiB,
Each arena follows the same layout for its metadata, and all references in an
arena are internal to it (with the exception of one field that points to the
-next arena). The following depicts the "On-disk" metadata layout:
-
-
- Backing Store +-------> Arena
-+---------------+ | +------------------+
-| | | | Arena info block |
-| Arena 0 +---+ | 4K |
-| 512G | +------------------+
-| | | |
-+---------------+ | |
-| | | |
-| Arena 1 | | Data Blocks |
-| 512G | | |
-| | | |
-+---------------+ | |
-| . | | |
-| . | | |
-| . | | |
-| | | |
-| | | |
-+---------------+ +------------------+
- | |
- | BTT Map |
- | |
- | |
- +------------------+
- | |
- | BTT Flog |
- | |
- +------------------+
- | Info block copy |
- | 4K |
- +------------------+
+next arena). The following depicts the "On-disk" metadata layout::
+
+
+ Backing Store +-------> Arena
+ +---------------+ | +------------------+
+ | | | | Arena info block |
+ | Arena 0 +---+ | 4K |
+ | 512G | +------------------+
+ | | | |
+ +---------------+ | |
+ | | | |
+ | Arena 1 | | Data Blocks |
+ | 512G | | |
+ | | | |
+ +---------------+ | |
+ | . | | |
+ | . | | |
+ | . | | |
+ | | | |
+ | | | |
+ +---------------+ +------------------+
+ | |
+ | BTT Map |
+ | |
+ | |
+ +------------------+
+ | |
+ | BTT Flog |
+ | |
+ +------------------+
+ | Info block copy |
+ | 4K |
+ +------------------+
3. Theory of Operation
-----------------------
+======================
a. The BTT Map
block. Each map entry is 32 bits. The two most significant bits are special
flags, and the remaining form the internal block number.
+======== =============================================================
Bit Description
-31 - 30 : Error and Zero flags - Used in the following way:
- Bit Description
- 31 30
- -----------------------------------------------------------------------
- 00 Initial state. Reads return zeroes; Premap = Postmap
- 01 Zero state: Reads return zeroes
- 10 Error state: Reads fail; Writes clear 'E' bit
- 11 Normal Block – has valid postmap
+======== =============================================================
+31 - 30 Error and Zero flags - Used in the following way::
+ == == ====================================================
+ 31 30 Description
+ == == ====================================================
+ 0 0 Initial state. Reads return zeroes; Premap = Postmap
+ 0 1 Zero state: Reads return zeroes
+ 1 0 Error state: Reads fail; Writes clear 'E' bit
+ 1 1 Normal Block – has valid postmap
+ == == ====================================================
-29 - 0 : Mappings to internal 'postmap' blocks
+29 - 0 Mappings to internal 'postmap' blocks
+======== =============================================================
Some of the terminology that will be subsequently used:
-External LBA : LBA as made visible to upper layers.
-ABA : Arena Block Address - Block offset/number within an arena
-Premap ABA : The block offset into an arena, which was decided upon by range
+============ ================================================================
+External LBA LBA as made visible to upper layers.
+ABA Arena Block Address - Block offset/number within an arena
+Premap ABA The block offset into an arena, which was decided upon by range
checking the External LBA
-Postmap ABA : The block number in the "Data Blocks" area obtained after
+Postmap ABA The block number in the "Data Blocks" area obtained after
indirection from the map
-nfree : The number of free blocks that are maintained at any given time.
+nfree The number of free blocks that are maintained at any given time.
This is the number of concurrent writes that can happen to the
arena.
+============ ================================================================
For example, after adding a BTT, we surface a disk of 1024G. We get a read for
maintained in the form of the BTT flog. 'Flog' is a combination of the words
"free list" and "log". The flog contains 'nfree' entries, and an entry contains:
-lba : The premap ABA that is being written to
-old_map : The old postmap ABA - after 'this' write completes, this will be a
+======== =====================================================================
+lba The premap ABA that is being written to
+old_map The old postmap ABA - after 'this' write completes, this will be a
free block.
-new_map : The new postmap ABA. The map will up updated to reflect this
+new_map The new postmap ABA. The map will up updated to reflect this
lba->postmap_aba mapping, but we log it here in case we have to
recover.
-seq : Sequence number to mark which of the 2 sections of this flog entry is
+seq Sequence number to mark which of the 2 sections of this flog entry is
valid/newest. It cycles between 01->10->11->01 (binary) under normal
operation, with 00 indicating an uninitialized state.
-lba' : alternate lba entry
-old_map': alternate old postmap entry
-new_map': alternate new postmap entry
-seq' : alternate sequence number.
+lba' alternate lba entry
+old_map' alternate old postmap entry
+new_map' alternate new postmap entry
+seq' alternate sequence number.
+======== =====================================================================
Each of the above fields is 32-bit, making one entry 32 bytes. Entries are also
padded to 64 bytes to avoid cache line sharing or aliasing. Flog updates are
While 'nfree' describes the number of concurrent IOs an arena can process
concurrently, 'nlanes' is the number of IOs the BTT device as a whole can
-process.
- nlanes = min(nfree, num_cpus)
+process::
+
+ nlanes = min(nfree, num_cpus)
+
A lane number is obtained at the start of any IO, and is used for indexing into
all the on-disk and in-memory data structures for the duration of the IO. If
there are more CPUs than the max number of available lanes, than lanes are
--------------------------------------
Consider a case where two writer threads are writing to the same LBA. There can
-be a race in the following sequence of steps:
+be a race in the following sequence of steps::
-free[lane] = map[premap_aba]
-map[premap_aba] = postmap_aba
+ free[lane] = map[premap_aba]
+ map[premap_aba] = postmap_aba
Both threads can update their respective free[lane] with the same old, freed
postmap_aba. This has made the layout inconsistent by losing a free entry, and
through all the entries, and for each lane, of the set of two possible
'sections', we always look at the most recent one only (based on the sequence
number). The reconstruction rules/steps are simple:
+
- Read map[log_entry.lba].
- If log_entry.new matches the map entry, then log_entry.old is free.
- If log_entry.new does not match the map entry, then log_entry.new is free.
1. Convert external LBA to Arena number + pre-map ABA
2. Get a lane (and take lane_lock)
3. Use lane to index into in-memory free list and obtain a new block, next flog
- index, next sequence number
+ index, next sequence number
4. Scan the RTT to check if free block is present, and spin/wait if it is.
5. Write data to this free block
6. Read map to get the existing post-map ABA entry for this pre-map ABA
An arena would be in an error state if any of the metadata is corrupted
irrecoverably, either due to a bug or a media error. The following conditions
indicate an error:
+
- Info block checksum does not match (and recovering from the copy also fails)
- All internal available blocks are not uniquely and entirely addressed by the
sum of mapped blocks and free blocks (from the BTT flog).
(pmem, or blk mode). The easiest way to set up such a namespace is using the
'ndctl' utility [1]:
-For example, the ndctl command line to setup a btt with a 4k sector size is:
+For example, the ndctl command line to setup a btt with a 4k sector size is::
ndctl create-namespace -f -e namespace0.0 -m sector -l 4k
See ndctl create-namespace --help for more options.
[1]: https://github.com/pmem/ndctl
-
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===================================
+Non-Volatile Memory Device (NVDIMM)
+===================================
+
+.. toctree::
+ :maxdepth: 1
+
+ nvdimm
+ btt
+ security
- LIBNVDIMM: Non-Volatile Devices
- libnvdimm - kernel / libndctl - userspace helper library
- linux-nvdimm@lists.01.org
- v13
+===============================
+LIBNVDIMM: Non-Volatile Devices
+===============================
+libnvdimm - kernel / libndctl - userspace helper library
+
+linux-nvdimm@lists.01.org
+
+Version 13
+
+.. contents:
Glossary
Overview
Glossary
---------
-
-PMEM: A system-physical-address range where writes are persistent. A
-block device composed of PMEM is capable of DAX. A PMEM address range
-may span an interleave of several DIMMs.
-
-BLK: A set of one or more programmable memory mapped apertures provided
-by a DIMM to access its media. This indirection precludes the
-performance benefit of interleaving, but enables DIMM-bounded failure
-modes.
-
-DPA: DIMM Physical Address, is a DIMM-relative offset. With one DIMM in
-the system there would be a 1:1 system-physical-address:DPA association.
-Once more DIMMs are added a memory controller interleave must be
-decoded to determine the DPA associated with a given
-system-physical-address. BLK capacity always has a 1:1 relationship
-with a single-DIMM's DPA range.
-
-DAX: File system extensions to bypass the page cache and block layer to
-mmap persistent memory, from a PMEM block device, directly into a
-process address space.
-
-DSM: Device Specific Method: ACPI method to to control specific
-device - in this case the firmware.
-
-DCR: NVDIMM Control Region Structure defined in ACPI 6 Section 5.2.25.5.
-It defines a vendor-id, device-id, and interface format for a given DIMM.
-
-BTT: Block Translation Table: Persistent memory is byte addressable.
-Existing software may have an expectation that the power-fail-atomicity
-of writes is at least one sector, 512 bytes. The BTT is an indirection
-table with atomic update semantics to front a PMEM/BLK block device
-driver and present arbitrary atomic sector sizes.
-
-LABEL: Metadata stored on a DIMM device that partitions and identifies
-(persistently names) storage between PMEM and BLK. It also partitions
-BLK storage to host BTTs with different parameters per BLK-partition.
-Note that traditional partition tables, GPT/MBR, are layered on top of a
-BLK or PMEM device.
+========
+
+PMEM:
+ A system-physical-address range where writes are persistent. A
+ block device composed of PMEM is capable of DAX. A PMEM address range
+ may span an interleave of several DIMMs.
+
+BLK:
+ A set of one or more programmable memory mapped apertures provided
+ by a DIMM to access its media. This indirection precludes the
+ performance benefit of interleaving, but enables DIMM-bounded failure
+ modes.
+
+DPA:
+ DIMM Physical Address, is a DIMM-relative offset. With one DIMM in
+ the system there would be a 1:1 system-physical-address:DPA association.
+ Once more DIMMs are added a memory controller interleave must be
+ decoded to determine the DPA associated with a given
+ system-physical-address. BLK capacity always has a 1:1 relationship
+ with a single-DIMM's DPA range.
+
+DAX:
+ File system extensions to bypass the page cache and block layer to
+ mmap persistent memory, from a PMEM block device, directly into a
+ process address space.
+
+DSM:
+ Device Specific Method: ACPI method to to control specific
+ device - in this case the firmware.
+
+DCR:
+ NVDIMM Control Region Structure defined in ACPI 6 Section 5.2.25.5.
+ It defines a vendor-id, device-id, and interface format for a given DIMM.
+
+BTT:
+ Block Translation Table: Persistent memory is byte addressable.
+ Existing software may have an expectation that the power-fail-atomicity
+ of writes is at least one sector, 512 bytes. The BTT is an indirection
+ table with atomic update semantics to front a PMEM/BLK block device
+ driver and present arbitrary atomic sector sizes.
+
+LABEL:
+ Metadata stored on a DIMM device that partitions and identifies
+ (persistently names) storage between PMEM and BLK. It also partitions
+ BLK storage to host BTTs with different parameters per BLK-partition.
+ Note that traditional partition tables, GPT/MBR, are layered on top of a
+ BLK or PMEM device.
Overview
---------
+========
The LIBNVDIMM subsystem provides support for three types of NVDIMMs, namely,
PMEM, BLK, and NVDIMM devices that can simultaneously support both PMEM
for exclusive access via one mode a time.
Supporting Documents
-ACPI 6: http://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
-NVDIMM Namespace: http://pmem.io/documents/NVDIMM_Namespace_Spec.pdf
-DSM Interface Example: http://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf
-Driver Writer's Guide: http://pmem.io/documents/NVDIMM_Driver_Writers_Guide.pdf
+--------------------
+
+ACPI 6:
+ http://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
+NVDIMM Namespace:
+ http://pmem.io/documents/NVDIMM_Namespace_Spec.pdf
+DSM Interface Example:
+ http://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf
+Driver Writer's Guide:
+ http://pmem.io/documents/NVDIMM_Driver_Writers_Guide.pdf
Git Trees
-LIBNVDIMM: https://git.kernel.org/cgit/linux/kernel/git/djbw/nvdimm.git
-LIBNDCTL: https://github.com/pmem/ndctl.git
-PMEM: https://github.com/01org/prd
+---------
+
+LIBNVDIMM:
+ https://git.kernel.org/cgit/linux/kernel/git/djbw/nvdimm.git
+LIBNDCTL:
+ https://github.com/pmem/ndctl.git
+PMEM:
+ https://github.com/01org/prd
LIBNVDIMM PMEM and BLK
-------------------
+======================
Prior to the arrival of the NFIT, non-volatile memory was described to a
system in various ad-hoc ways. Usually only the bare minimum was
device driver:
1. PMEM (nd_pmem.ko): Drives a system-physical-address range. This
- range is contiguous in system memory and may be interleaved (hardware
- memory controller striped) across multiple DIMMs. When interleaved the
- platform may optionally provide details of which DIMMs are participating
- in the interleave.
-
- Note that while LIBNVDIMM describes system-physical-address ranges that may
- alias with BLK access as ND_NAMESPACE_PMEM ranges and those without
- alias as ND_NAMESPACE_IO ranges, to the nd_pmem driver there is no
- distinction. The different device-types are an implementation detail
- that userspace can exploit to implement policies like "only interface
- with address ranges from certain DIMMs". It is worth noting that when
- aliasing is present and a DIMM lacks a label, then no block device can
- be created by default as userspace needs to do at least one allocation
- of DPA to the PMEM range. In contrast ND_NAMESPACE_IO ranges, once
- registered, can be immediately attached to nd_pmem.
+ range is contiguous in system memory and may be interleaved (hardware
+ memory controller striped) across multiple DIMMs. When interleaved the
+ platform may optionally provide details of which DIMMs are participating
+ in the interleave.
+
+ Note that while LIBNVDIMM describes system-physical-address ranges that may
+ alias with BLK access as ND_NAMESPACE_PMEM ranges and those without
+ alias as ND_NAMESPACE_IO ranges, to the nd_pmem driver there is no
+ distinction. The different device-types are an implementation detail
+ that userspace can exploit to implement policies like "only interface
+ with address ranges from certain DIMMs". It is worth noting that when
+ aliasing is present and a DIMM lacks a label, then no block device can
+ be created by default as userspace needs to do at least one allocation
+ of DPA to the PMEM range. In contrast ND_NAMESPACE_IO ranges, once
+ registered, can be immediately attached to nd_pmem.
2. BLK (nd_blk.ko): This driver performs I/O using a set of platform
- defined apertures. A set of apertures will access just one DIMM.
- Multiple windows (apertures) allow multiple concurrent accesses, much like
- tagged-command-queuing, and would likely be used by different threads or
- different CPUs.
+ defined apertures. A set of apertures will access just one DIMM.
+ Multiple windows (apertures) allow multiple concurrent accesses, much like
+ tagged-command-queuing, and would likely be used by different threads or
+ different CPUs.
+
+ The NFIT specification defines a standard format for a BLK-aperture, but
+ the spec also allows for vendor specific layouts, and non-NFIT BLK
+ implementations may have other designs for BLK I/O. For this reason
+ "nd_blk" calls back into platform-specific code to perform the I/O.
- The NFIT specification defines a standard format for a BLK-aperture, but
- the spec also allows for vendor specific layouts, and non-NFIT BLK
- implementations may have other designs for BLK I/O. For this reason
- "nd_blk" calls back into platform-specific code to perform the I/O.
- One such implementation is defined in the "Driver Writer's Guide" and "DSM
- Interface Example".
+ One such implementation is defined in the "Driver Writer's Guide" and "DSM
+ Interface Example".
Why BLK?
---------
+========
While PMEM provides direct byte-addressable CPU-load/store access to
NVDIMM storage, it does not provide the best system RAS (recovery,
to a corrupted address through an BLK-aperture causes that block window
to raise an error status in a register. The latter is more aligned with
the standard error model that host-bus-adapter attached disks present.
+
Also, if an administrator ever wants to replace a memory it is easier to
service a system at DIMM module boundaries. Compare this to PMEM where
data could be interleaved in an opaque hardware specific manner across
several DIMMs.
PMEM vs BLK
+-----------
+
BLK-apertures solve these RAS problems, but their presence is also the
major contributing factor to the complexity of the ND subsystem. They
complicate the implementation because PMEM and BLK alias in DPA space.
extents.
BLK-REGIONs, PMEM-REGIONs, Atomic Sectors, and DAX
---------------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
One of the few
reasons to allow multiple BLK namespaces per REGION is so that each
BLK-namespace can be configured with a BTT with unique atomic sector
sizes. While a PMEM device can host a BTT the LABEL specification does
not provide for a sector size to be specified for a PMEM namespace.
+
This is due to the expectation that the primary usage model for PMEM is
via DAX, and the BTT is incompatible with DAX. However, for the cases
where an application or filesystem still needs atomic sector update
Example NVDIMM Platform
------------------------
+=======================
For the remainder of this document the following diagram will be
-referenced for any example sysfs layouts.
-
-
- (a) (b) DIMM BLK-REGION
- +-------------------+--------+--------+--------+
-+------+ | pm0.0 | blk2.0 | pm1.0 | blk2.1 | 0 region2
-| imc0 +--+- - - region0- - - +--------+ +--------+
-+--+---+ | pm0.0 | blk3.0 | pm1.0 | blk3.1 | 1 region3
- | +-------------------+--------v v--------+
-+--+---+ | |
-| cpu0 | region1
-+--+---+ | |
- | +----------------------------^ ^--------+
-+--+---+ | blk4.0 | pm1.0 | blk4.0 | 2 region4
-| imc1 +--+----------------------------| +--------+
-+------+ | blk5.0 | pm1.0 | blk5.0 | 3 region5
- +----------------------------+--------+--------+
+referenced for any example sysfs layouts::
+
+
+ (a) (b) DIMM BLK-REGION
+ +-------------------+--------+--------+--------+
+ +------+ | pm0.0 | blk2.0 | pm1.0 | blk2.1 | 0 region2
+ | imc0 +--+- - - region0- - - +--------+ +--------+
+ +--+---+ | pm0.0 | blk3.0 | pm1.0 | blk3.1 | 1 region3
+ | +-------------------+--------v v--------+
+ +--+---+ | |
+ | cpu0 | region1
+ +--+---+ | |
+ | +----------------------------^ ^--------+
+ +--+---+ | blk4.0 | pm1.0 | blk4.0 | 2 region4
+ | imc1 +--+----------------------------| +--------+
+ +------+ | blk5.0 | pm1.0 | blk5.0 | 3 region5
+ +----------------------------+--------+--------+
In this platform we have four DIMMs and two memory controllers in one
socket. Each unique interface (BLK or PMEM) to DPA space is identified
by a region device with a dynamically assigned id (REGION0 - REGION5).
1. The first portion of DIMM0 and DIMM1 are interleaved as REGION0. A
- single PMEM namespace is created in the REGION0-SPA-range that spans most
- of DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that
- interleaved system-physical-address range is reclaimed as BLK-aperture
- accessed space starting at DPA-offset (a) into each DIMM. In that
- reclaimed space we create two BLK-aperture "namespaces" from REGION2 and
- REGION3 where "blk2.0" and "blk3.0" are just human readable names that
- could be set to any user-desired name in the LABEL.
+ single PMEM namespace is created in the REGION0-SPA-range that spans most
+ of DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that
+ interleaved system-physical-address range is reclaimed as BLK-aperture
+ accessed space starting at DPA-offset (a) into each DIMM. In that
+ reclaimed space we create two BLK-aperture "namespaces" from REGION2 and
+ REGION3 where "blk2.0" and "blk3.0" are just human readable names that
+ could be set to any user-desired name in the LABEL.
2. In the last portion of DIMM0 and DIMM1 we have an interleaved
- system-physical-address range, REGION1, that spans those two DIMMs as
- well as DIMM2 and DIMM3. Some of REGION1 is allocated to a PMEM namespace
- named "pm1.0", the rest is reclaimed in 4 BLK-aperture namespaces (for
- each DIMM in the interleave set), "blk2.1", "blk3.1", "blk4.0", and
- "blk5.0".
+ system-physical-address range, REGION1, that spans those two DIMMs as
+ well as DIMM2 and DIMM3. Some of REGION1 is allocated to a PMEM namespace
+ named "pm1.0", the rest is reclaimed in 4 BLK-aperture namespaces (for
+ each DIMM in the interleave set), "blk2.1", "blk3.1", "blk4.0", and
+ "blk5.0".
3. The portion of DIMM2 and DIMM3 that do not participate in the REGION1
- interleaved system-physical-address range (i.e. the DPA address past
- offset (b) are also included in the "blk4.0" and "blk5.0" namespaces.
- Note, that this example shows that BLK-aperture namespaces don't need to
- be contiguous in DPA-space.
+ interleaved system-physical-address range (i.e. the DPA address past
+ offset (b) are also included in the "blk4.0" and "blk5.0" namespaces.
+ Note, that this example shows that BLK-aperture namespaces don't need to
+ be contiguous in DPA-space.
This bus is provided by the kernel under the device
/sys/devices/platform/nfit_test.0 when CONFIG_NFIT_TEST is enabled and
LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API
-----------------------------------------------------
+========================================================
What follows is a description of the LIBNVDIMM sysfs layout and a
corresponding object hierarchy diagram as viewed through the LIBNDCTL
test.
LIBNDCTL: Context
+-----------------
+
Every API call in the LIBNDCTL library requires a context that holds the
logging parameters and other library instance state. The library is
based on the libabc template:
-https://git.kernel.org/cgit/linux/kernel/git/kay/libabc.git
+
+ https://git.kernel.org/cgit/linux/kernel/git/kay/libabc.git
LIBNDCTL: instantiate a new library context example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+::
struct ndctl_ctx *ctx;
return NULL;
LIBNVDIMM/LIBNDCTL: Bus
--------------------
+-----------------------
A bus has a 1:1 relationship with an NFIT. The current expectation for
ACPI based systems is that there is only ever one platform-global NFIT.
test.
LIBNVDIMM: control class device in /sys/class
+---------------------------------------------
This character device accepts DSM messages to be passed to DIMM
-identified by its NFIT handle.
+identified by its NFIT handle::
/sys/class/nd/ndctl0
|-- dev
LIBNVDIMM: bus
+--------------
+
+::
struct nvdimm_bus *nvdimm_bus_register(struct device *parent,
struct nvdimm_bus_descriptor *nfit_desc);
+::
+
/sys/devices/platform/nfit_test.0/ndbus0
|-- commands
|-- nd
`-- wait_probe
LIBNDCTL: bus enumeration example
-Find the bus handle that describes the bus from Example NVDIMM Platform
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Find the bus handle that describes the bus from Example NVDIMM Platform::
static struct ndctl_bus *get_bus_by_provider(struct ndctl_ctx *ctx,
const char *provider)
LIBNVDIMM/LIBNDCTL: DIMM (NMEM)
----------------------------
+-------------------------------
The DIMM device provides a character device for sending commands to
hardware, and it is a container for LABELs. If the DIMM is defined by
be physical DIMMs, so we use a more generic name.
LIBNVDIMM: DIMM (NMEM)
+^^^^^^^^^^^^^^^^^^^^^^
+
+::
struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
const struct attribute_group **groups, unsigned long flags,
unsigned long *dsm_mask);
+::
+
/sys/devices/platform/nfit_test.0/ndbus0
|-- nmem0
| |-- available_slots
LIBNDCTL: DIMM enumeration example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Note, in this example we are assuming NFIT-defined DIMMs which are
identified by an "nfit_handle" a 32-bit value where:
-Bit 3:0 DIMM number within the memory channel
-Bit 7:4 memory channel number
-Bit 11:8 memory controller ID
-Bit 15:12 socket ID (within scope of a Node controller if node controller is present)
-Bit 27:16 Node Controller ID
-Bit 31:28 Reserved
+
+ - Bit 3:0 DIMM number within the memory channel
+ - Bit 7:4 memory channel number
+ - Bit 11:8 memory controller ID
+ - Bit 15:12 socket ID (within scope of a Node controller if node
+ controller is present)
+ - Bit 27:16 Node Controller ID
+ - Bit 31:28 Reserved
+
+::
static struct ndctl_dimm *get_dimm_by_handle(struct ndctl_bus *bus,
unsigned int handle)
dimm = get_dimm_by_handle(bus, DIMM_HANDLE(0, 0, 0, 0, 0));
LIBNVDIMM/LIBNDCTL: Region
-----------------------
+--------------------------
A generic REGION device is registered for each PMEM range or BLK-aperture
set. Per the example there are 6 regions: 2 PMEM and 4 BLK-aperture
at the 'add' event, and finally, the optional "spa_index" is provided in
the case where the region is defined by a SPA.
-LIBNVDIMM: region
+LIBNVDIMM: region::
struct nd_region *nvdimm_pmem_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc);
struct nd_region *nvdimm_blk_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc);
+::
+
/sys/devices/platform/nfit_test.0/ndbus0
|-- region0
| |-- available_size
[..]
LIBNDCTL: region enumeration example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sample region retrieval routines based on NFIT-unique data like
"spa_index" (interleave set id) for PMEM and "nfit_handle" (dimm id) for
-BLK.
+BLK::
static struct ndctl_region *get_pmem_region_by_spa_index(struct ndctl_bus *bus,
unsigned int spa_index)
region-attributes for four reasons:
1. There are already more than two REGION and "namespace" types. For
- PMEM there are two subtypes. As mentioned previously we have PMEM where
- the constituent DIMM devices are known and anonymous PMEM. For BLK
- regions the NFIT specification already anticipates vendor specific
- implementations. The exact distinction of what a region contains is in
- the region-attributes not the region-name or the region-devtype.
+ PMEM there are two subtypes. As mentioned previously we have PMEM where
+ the constituent DIMM devices are known and anonymous PMEM. For BLK
+ regions the NFIT specification already anticipates vendor specific
+ implementations. The exact distinction of what a region contains is in
+ the region-attributes not the region-name or the region-devtype.
2. A region with zero child-namespaces is a possible configuration. For
- example, the NFIT allows for a DCR to be published without a
- corresponding BLK-aperture. This equates to a DIMM that can only accept
- control/configuration messages, but no i/o through a descendant block
- device. Again, this "type" is advertised in the attributes ('mappings'
- == 0) and the name does not tell you much.
+ example, the NFIT allows for a DCR to be published without a
+ corresponding BLK-aperture. This equates to a DIMM that can only accept
+ control/configuration messages, but no i/o through a descendant block
+ device. Again, this "type" is advertised in the attributes ('mappings'
+ == 0) and the name does not tell you much.
3. What if a third major interface type arises in the future? Outside
- of vendor specific implementations, it's not difficult to envision a
- third class of interface type beyond BLK and PMEM. With a generic name
- for the REGION level of the device-hierarchy old userspace
- implementations can still make sense of new kernel advertised
- region-types. Userspace can always rely on the generic region
- attributes like "mappings", "size", etc and the expected child devices
- named "namespace". This generic format of the device-model hierarchy
- allows the LIBNVDIMM and LIBNDCTL implementations to be more uniform and
- future-proof.
+ of vendor specific implementations, it's not difficult to envision a
+ third class of interface type beyond BLK and PMEM. With a generic name
+ for the REGION level of the device-hierarchy old userspace
+ implementations can still make sense of new kernel advertised
+ region-types. Userspace can always rely on the generic region
+ attributes like "mappings", "size", etc and the expected child devices
+ named "namespace". This generic format of the device-model hierarchy
+ allows the LIBNVDIMM and LIBNDCTL implementations to be more uniform and
+ future-proof.
4. There are more robust mechanisms for determining the major type of a
- region than a device name. See the next section, How Do I Determine the
- Major Type of a Region?
+ region than a device name. See the next section, How Do I Determine the
+ Major Type of a Region?
How Do I Determine the Major Type of a Region?
----------------------------------------------
looking at the kernel header (/usr/include/linux/ndctl.h) to decode the
"nstype" integer attribute, here are some other options.
- 1. module alias lookup:
+1. module alias lookup
+^^^^^^^^^^^^^^^^^^^^^^
The whole point of region/namespace device type differentiation is to
decide which block-device driver will attach to a given LIBNVDIMM namespace.
the resulting namespaces. The output from module resolution is more
accurate than a region-name or region-devtype.
- 2. udev:
+2. udev
+^^^^^^^
+
+ The kernel "devtype" is registered in the udev database::
- The kernel "devtype" is registered in the udev database
- # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region0
- P: /devices/platform/nfit_test.0/ndbus0/region0
- E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region0
- E: DEVTYPE=nd_pmem
- E: MODALIAS=nd:t2
- E: SUBSYSTEM=nd
+ # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region0
+ P: /devices/platform/nfit_test.0/ndbus0/region0
+ E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region0
+ E: DEVTYPE=nd_pmem
+ E: MODALIAS=nd:t2
+ E: SUBSYSTEM=nd
- # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region4
- P: /devices/platform/nfit_test.0/ndbus0/region4
- E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region4
- E: DEVTYPE=nd_blk
- E: MODALIAS=nd:t3
- E: SUBSYSTEM=nd
+ # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region4
+ P: /devices/platform/nfit_test.0/ndbus0/region4
+ E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region4
+ E: DEVTYPE=nd_blk
+ E: MODALIAS=nd:t3
+ E: SUBSYSTEM=nd
...and is available as a region attribute, but keep in mind that the
"devtype" does not indicate sub-type variations and scripts should
really be understanding the other attributes.
- 3. type specific attributes:
+3. type specific attributes
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
As it currently stands a BLK-aperture region will never have a
"nfit/spa_index" attribute, but neither will a non-NFIT PMEM region. A
LIBNVDIMM/LIBNDCTL: Namespace
--------------------------
+-----------------------------
A REGION, after resolving DPA aliasing and LABEL specified boundaries,
surfaces one or more "namespace" devices. The arrival of a "namespace"
and register a disk/block device.
LIBNVDIMM: namespace
+^^^^^^^^^^^^^^^^^^^^
+
Here is a sample layout from the three major types of NAMESPACE where
namespace0.0 represents DIMM-info-backed PMEM (note that it has a 'uuid'
attribute), namespace2.0 represents a BLK namespace (note it has a
'sector_size' attribute) that, and namespace6.0 represents an anonymous
PMEM namespace (note that has no 'uuid' attribute due to not support a
-LABEL).
+LABEL)::
/sys/devices/platform/nfit_test.0/ndbus0/region0/namespace0.0
|-- alt_name
`-- uevent
LIBNDCTL: namespace enumeration example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Namespaces are indexed relative to their parent region, example below.
These indexes are mostly static from boot to boot, but subsystem makes
no guarantees in this regard. For a static namespace identifier use its
'uuid' attribute.
-static struct ndctl_namespace *get_namespace_by_id(struct ndctl_region *region,
- unsigned int id)
-{
- struct ndctl_namespace *ndns;
+::
- ndctl_namespace_foreach(region, ndns)
- if (ndctl_namespace_get_id(ndns) == id)
- return ndns;
+ static struct ndctl_namespace
+ *get_namespace_by_id(struct ndctl_region *region, unsigned int id)
+ {
+ struct ndctl_namespace *ndns;
- return NULL;
-}
+ ndctl_namespace_foreach(region, ndns)
+ if (ndctl_namespace_get_id(ndns) == id)
+ return ndns;
+
+ return NULL;
+ }
LIBNDCTL: namespace creation example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
Idle namespaces are automatically created by the kernel if a given
region has enough available capacity to create a new namespace.
Namespace instantiation involves finding an idle namespace and
configuring it. For the most part the setting of namespace attributes
can occur in any order, the only constraint is that 'uuid' must be set
before 'size'. This enables the kernel to track DPA allocations
-internally with a static identifier.
+internally with a static identifier::
-static int configure_namespace(struct ndctl_region *region,
- struct ndctl_namespace *ndns,
- struct namespace_parameters *parameters)
-{
- char devname[50];
+ static int configure_namespace(struct ndctl_region *region,
+ struct ndctl_namespace *ndns,
+ struct namespace_parameters *parameters)
+ {
+ char devname[50];
- snprintf(devname, sizeof(devname), "namespace%d.%d",
- ndctl_region_get_id(region), paramaters->id);
+ snprintf(devname, sizeof(devname), "namespace%d.%d",
+ ndctl_region_get_id(region), paramaters->id);
- ndctl_namespace_set_alt_name(ndns, devname);
- /* 'uuid' must be set prior to setting size! */
- ndctl_namespace_set_uuid(ndns, paramaters->uuid);
- ndctl_namespace_set_size(ndns, paramaters->size);
- /* unlike pmem namespaces, blk namespaces have a sector size */
- if (parameters->lbasize)
- ndctl_namespace_set_sector_size(ndns, parameters->lbasize);
- ndctl_namespace_enable(ndns);
-}
+ ndctl_namespace_set_alt_name(ndns, devname);
+ /* 'uuid' must be set prior to setting size! */
+ ndctl_namespace_set_uuid(ndns, paramaters->uuid);
+ ndctl_namespace_set_size(ndns, paramaters->size);
+ /* unlike pmem namespaces, blk namespaces have a sector size */
+ if (parameters->lbasize)
+ ndctl_namespace_set_sector_size(ndns, parameters->lbasize);
+ ndctl_namespace_enable(ndns);
+ }
Why the Term "namespace"?
+^^^^^^^^^^^^^^^^^^^^^^^^^
1. Why not "volume" for instance? "volume" ran the risk of confusing
- ND (libnvdimm subsystem) to a volume manager like device-mapper.
+ ND (libnvdimm subsystem) to a volume manager like device-mapper.
2. The term originated to describe the sub-devices that can be created
- within a NVME controller (see the nvme specification:
- http://www.nvmexpress.org/specifications/), and NFIT namespaces are
- meant to parallel the capabilities and configurability of
- NVME-namespaces.
+ within a NVME controller (see the nvme specification:
+ http://www.nvmexpress.org/specifications/), and NFIT namespaces are
+ meant to parallel the capabilities and configurability of
+ NVME-namespaces.
LIBNVDIMM/LIBNDCTL: Block Translation Table "btt"
----------------------------------------------
+-------------------------------------------------
A BTT (design document: http://pmem.io/2014/09/23/btt.html) is a stacked
block device driver that fronts either the whole block device or a
partition of a block device emitted by either a PMEM or BLK NAMESPACE.
LIBNVDIMM: btt layout
+^^^^^^^^^^^^^^^^^^^^^
+
Every region will start out with at least one BTT device which is the
seed device. To activate it set the "namespace", "uuid", and
"sector_size" attributes and then bind the device to the nd_pmem or
-nd_blk driver depending on the region type.
+nd_blk driver depending on the region type::
/sys/devices/platform/nfit_test.1/ndbus0/region0/btt0/
|-- namespace
`-- uuid
LIBNDCTL: btt creation example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
Similar to namespaces an idle BTT device is automatically created per
region. Each time this "seed" btt device is configured and enabled a new
seed is created. Creating a BTT configuration involves two steps of
-finding and idle BTT and assigning it to consume a PMEM or BLK namespace.
+finding and idle BTT and assigning it to consume a PMEM or BLK namespace::
static struct ndctl_btt *get_idle_btt(struct ndctl_region *region)
{
------------------------
For the given example above, here is the view of the objects as seen by the
-LIBNDCTL API:
- +---+
- |CTX| +---------+ +--------------+ +---------------+
- +-+-+ +-> REGION0 +---> NAMESPACE0.0 +--> PMEM8 "pm0.0" |
- | | +---------+ +--------------+ +---------------+
-+-------+ | | +---------+ +--------------+ +---------------+
-| DIMM0 <-+ | +-> REGION1 +---> NAMESPACE1.0 +--> PMEM6 "pm1.0" |
-+-------+ | | | +---------+ +--------------+ +---------------+
-| DIMM1 <-+ +-v--+ | +---------+ +--------------+ +---------------+
-+-------+ +-+BUS0+---> REGION2 +-+-> NAMESPACE2.0 +--> ND6 "blk2.0" |
-| DIMM2 <-+ +----+ | +---------+ | +--------------+ +----------------------+
-+-------+ | | +-> NAMESPACE2.1 +--> ND5 "blk2.1" | BTT2 |
-| DIMM3 <-+ | +--------------+ +----------------------+
-+-------+ | +---------+ +--------------+ +---------------+
- +-> REGION3 +-+-> NAMESPACE3.0 +--> ND4 "blk3.0" |
- | +---------+ | +--------------+ +----------------------+
- | +-> NAMESPACE3.1 +--> ND3 "blk3.1" | BTT1 |
- | +--------------+ +----------------------+
- | +---------+ +--------------+ +---------------+
- +-> REGION4 +---> NAMESPACE4.0 +--> ND2 "blk4.0" |
- | +---------+ +--------------+ +---------------+
- | +---------+ +--------------+ +----------------------+
- +-> REGION5 +---> NAMESPACE5.0 +--> ND1 "blk5.0" | BTT0 |
- +---------+ +--------------+ +---------------+------+
-
-
+LIBNDCTL API::
+
+ +---+
+ |CTX| +---------+ +--------------+ +---------------+
+ +-+-+ +-> REGION0 +---> NAMESPACE0.0 +--> PMEM8 "pm0.0" |
+ | | +---------+ +--------------+ +---------------+
+ +-------+ | | +---------+ +--------------+ +---------------+
+ | DIMM0 <-+ | +-> REGION1 +---> NAMESPACE1.0 +--> PMEM6 "pm1.0" |
+ +-------+ | | | +---------+ +--------------+ +---------------+
+ | DIMM1 <-+ +-v--+ | +---------+ +--------------+ +---------------+
+ +-------+ +-+BUS0+---> REGION2 +-+-> NAMESPACE2.0 +--> ND6 "blk2.0" |
+ | DIMM2 <-+ +----+ | +---------+ | +--------------+ +----------------------+
+ +-------+ | | +-> NAMESPACE2.1 +--> ND5 "blk2.1" | BTT2 |
+ | DIMM3 <-+ | +--------------+ +----------------------+
+ +-------+ | +---------+ +--------------+ +---------------+
+ +-> REGION3 +-+-> NAMESPACE3.0 +--> ND4 "blk3.0" |
+ | +---------+ | +--------------+ +----------------------+
+ | +-> NAMESPACE3.1 +--> ND3 "blk3.1" | BTT1 |
+ | +--------------+ +----------------------+
+ | +---------+ +--------------+ +---------------+
+ +-> REGION4 +---> NAMESPACE4.0 +--> ND2 "blk4.0" |
+ | +---------+ +--------------+ +---------------+
+ | +---------+ +--------------+ +----------------------+
+ +-> REGION5 +---> NAMESPACE5.0 +--> ND1 "blk5.0" | BTT0 |
+ +---------+ +--------------+ +---------------+------+
-NVDIMM SECURITY
+===============
+NVDIMM Security
===============
1. Introduction
by the extended security status.
[1]: http://pmem.io/documents/NVDIMM_DSM_Interface-V1.8.pdf
+
[2]: http://www.t13.org/documents/UploadedDocuments/docs2006/e05179r4-ACS-SecurityClarifications.pdf
- NVMEM SUBSYSTEM
- Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+NVMEM Subsystem
+===============
+
+ Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
This document explains the NVMEM Framework along with the APIs provided,
and how to use it.
nvmem_unregister(nvmem) is used to unregister a previously registered provider.
-For example, a simple qfprom case:
+For example, a simple qfprom case::
-static struct nvmem_config econfig = {
+ static struct nvmem_config econfig = {
.name = "qfprom",
.owner = THIS_MODULE,
-};
+ };
-static int qfprom_probe(struct platform_device *pdev)
-{
+ static int qfprom_probe(struct platform_device *pdev)
+ {
...
econfig.dev = &pdev->dev;
nvmem = nvmem_register(&econfig);
...
-}
+ }
It is mandatory that the NVMEM provider has a regmap associated with its
struct device. Failure to do would return error code from nvmem_register().
Users of board files can define and register nvmem cells using the
-nvmem_cell_table struct:
+nvmem_cell_table struct::
-static struct nvmem_cell_info foo_nvmem_cells[] = {
+ static struct nvmem_cell_info foo_nvmem_cells[] = {
{
.name = "macaddr",
.offset = 0x7f00,
.bytes = ETH_ALEN,
}
-};
+ };
-static struct nvmem_cell_table foo_nvmem_cell_table = {
+ static struct nvmem_cell_table foo_nvmem_cell_table = {
.nvmem_name = "i2c-eeprom",
.cells = foo_nvmem_cells,
.ncells = ARRAY_SIZE(foo_nvmem_cells),
-};
+ };
-nvmem_add_cell_table(&foo_nvmem_cell_table);
+ nvmem_add_cell_table(&foo_nvmem_cell_table);
Additionally it is possible to create nvmem cell lookup entries and register
-them with the nvmem framework from machine code as shown in the example below:
+them with the nvmem framework from machine code as shown in the example below::
-static struct nvmem_cell_lookup foo_nvmem_lookup = {
+ static struct nvmem_cell_lookup foo_nvmem_lookup = {
.nvmem_name = "i2c-eeprom",
.cell_name = "macaddr",
.dev_id = "foo_mac.0",
.con_id = "mac-address",
-};
+ };
-nvmem_add_cell_lookups(&foo_nvmem_lookup, 1);
+ nvmem_add_cell_lookups(&foo_nvmem_lookup, 1);
NVMEM Consumers
+++++++++++++++
=================================
NVMEM cells are the data entries/fields in the NVMEM.
-The NVMEM framework provides 3 APIs to read/write NVMEM cells.
+The NVMEM framework provides 3 APIs to read/write NVMEM cells::
-struct nvmem_cell *nvmem_cell_get(struct device *dev, const char *name);
-struct nvmem_cell *devm_nvmem_cell_get(struct device *dev, const char *name);
+ struct nvmem_cell *nvmem_cell_get(struct device *dev, const char *name);
+ struct nvmem_cell *devm_nvmem_cell_get(struct device *dev, const char *name);
-void nvmem_cell_put(struct nvmem_cell *cell);
-void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
+ void nvmem_cell_put(struct nvmem_cell *cell);
+ void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
-void *nvmem_cell_read(struct nvmem_cell *cell, ssize_t *len);
-int nvmem_cell_write(struct nvmem_cell *cell, void *buf, ssize_t len);
+ void *nvmem_cell_read(struct nvmem_cell *cell, ssize_t *len);
+ int nvmem_cell_write(struct nvmem_cell *cell, void *buf, ssize_t len);
-*nvmem_cell_get() apis will get a reference to nvmem cell for a given id,
+`*nvmem_cell_get()` apis will get a reference to nvmem cell for a given id,
and nvmem_cell_read/write() can then read or write to the cell.
-Once the usage of the cell is finished the consumer should call *nvmem_cell_put()
-to free all the allocation memory for the cell.
+Once the usage of the cell is finished the consumer should call
+`*nvmem_cell_put()` to free all the allocation memory for the cell.
4. Direct NVMEM device based consumer APIs
==========================================
In some instances it is necessary to directly read/write the NVMEM.
-To facilitate such consumers NVMEM framework provides below apis.
+To facilitate such consumers NVMEM framework provides below apis::
-struct nvmem_device *nvmem_device_get(struct device *dev, const char *name);
-struct nvmem_device *devm_nvmem_device_get(struct device *dev,
+ struct nvmem_device *nvmem_device_get(struct device *dev, const char *name);
+ struct nvmem_device *devm_nvmem_device_get(struct device *dev,
const char *name);
-void nvmem_device_put(struct nvmem_device *nvmem);
-int nvmem_device_read(struct nvmem_device *nvmem, unsigned int offset,
+ void nvmem_device_put(struct nvmem_device *nvmem);
+ int nvmem_device_read(struct nvmem_device *nvmem, unsigned int offset,
size_t bytes, void *buf);
-int nvmem_device_write(struct nvmem_device *nvmem, unsigned int offset,
+ int nvmem_device_write(struct nvmem_device *nvmem, unsigned int offset,
size_t bytes, void *buf);
-int nvmem_device_cell_read(struct nvmem_device *nvmem,
+ int nvmem_device_cell_read(struct nvmem_device *nvmem,
struct nvmem_cell_info *info, void *buf);
-int nvmem_device_cell_write(struct nvmem_device *nvmem,
+ int nvmem_device_cell_write(struct nvmem_device *nvmem,
struct nvmem_cell_info *info, void *buf);
Before the consumers can read/write NVMEM directly, it should get hold
-of nvmem_controller from one of the *nvmem_device_get() api.
+of nvmem_controller from one of the `*nvmem_device_get()` api.
The difference between these apis and cell based apis is that these apis always
take nvmem_device as parameter.
When a consumer no longer needs the NVMEM, it has to release the reference
to the NVMEM it has obtained using the APIs mentioned in the above section.
-The NVMEM framework provides 2 APIs to release a reference to the NVMEM.
+The NVMEM framework provides 2 APIs to release a reference to the NVMEM::
-void nvmem_cell_put(struct nvmem_cell *cell);
-void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
-void nvmem_device_put(struct nvmem_device *nvmem);
-void devm_nvmem_device_put(struct device *dev, struct nvmem_device *nvmem);
+ void nvmem_cell_put(struct nvmem_cell *cell);
+ void devm_nvmem_cell_put(struct device *dev, struct nvmem_cell *cell);
+ void nvmem_device_put(struct nvmem_device *nvmem);
+ void devm_nvmem_device_put(struct device *dev, struct nvmem_device *nvmem);
Both these APIs are used to release a reference to the NVMEM and
devm_nvmem_cell_put and devm_nvmem_device_put destroys the devres associated
6. Userspace binary interface
==============================
-Userspace can read/write the raw NVMEM file located at
-/sys/bus/nvmem/devices/*/nvmem
+Userspace can read/write the raw NVMEM file located at::
+
+ /sys/bus/nvmem/devices/*/nvmem
-ex:
+ex::
-hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
+ hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
-0000000 0000 0000 0000 0000 0000 0000 0000 0000
-*
-00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
-0000000 0000 0000 0000 0000 0000 0000 0000 0000
-...
-*
-0001000
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ *
+ 00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ ...
+ *
+ 0001000
7. DeviceTree Binding
=====================
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=====================
+Generic PHY Framework
+=====================
+
+.. toctree::
+
+ phy
+ samsung-usb2
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
+
-.------------------------------------------------------------------------------+
-| Samsung USB 2.0 PHY adaptation layer |
-+-----------------------------------------------------------------------------+'
+====================================
+Samsung USB 2.0 PHY adaptation layer
+====================================
-| 1. Description
-+----------------
+1. Description
+--------------
The architecture of the USB 2.0 PHY module in Samsung SoCs is similar
among many SoCs. In spite of the similarities it proved difficult to
a compromise between having separate drivers and having a single driver
with added support for many special cases.
-| 2. Files description
-+----------------------
+2. Files description
+--------------------
- phy-samsung-usb2.c
This is the main file of the adaptation layer. This file contains
driver. In addition it should contain extern declarations for
structures that describe particular SoCs.
-| 3. Supporting SoCs
-+--------------------
+3. Supporting SoCs
+------------------
To support a new SoC a new file should be added to the drivers/phy
directory. Each SoC's configuration is stored in an instance of the
-struct samsung_usb2_phy_config.
+struct samsung_usb2_phy_config::
-struct samsung_usb2_phy_config {
+ struct samsung_usb2_phy_config {
const struct samsung_usb2_common_phy *phys;
int (*rate_to_clk)(unsigned long, u32 *);
unsigned int num_phys;
bool has_mode_switch;
-};
+ };
-The num_phys is the number of phys handled by the driver. *phys is an
+The num_phys is the number of phys handled by the driver. `*phys` is an
array that contains the configuration for each phy. The has_mode_switch
property is a boolean flag that determines whether the SoC has USB host
and device on a single pair of pins. If so, a special register has to
be modified to change the internal routing of these pins between a USB
device or host module.
-For example the configuration for Exynos 4210 is following:
+For example the configuration for Exynos 4210 is following::
-const struct samsung_usb2_phy_config exynos4210_usb2_phy_config = {
+ const struct samsung_usb2_phy_config exynos4210_usb2_phy_config = {
.has_mode_switch = 0,
.num_phys = EXYNOS4210_NUM_PHYS,
.phys = exynos4210_phys,
.rate_to_clk = exynos4210_rate_to_clk,
-}
+ }
+
+- `int (*rate_to_clk)(unsigned long, u32 *)`
-- int (*rate_to_clk)(unsigned long, u32 *)
The rate_to_clk callback is to convert the rate of the clock
used as the reference clock for the PHY module to the value
that should be written in the hardware register.
-The exynos4210_phys configuration array is as follows:
+The exynos4210_phys configuration array is as follows::
-static const struct samsung_usb2_common_phy exynos4210_phys[] = {
+ static const struct samsung_usb2_common_phy exynos4210_phys[] = {
{
.label = "device",
.id = EXYNOS4210_DEVICE,
.power_off = exynos4210_power_off,
},
{},
-};
+ };
+
+- `int (*power_on)(struct samsung_usb2_phy_instance *);`
+ `int (*power_off)(struct samsung_usb2_phy_instance *);`
-- int (*power_on)(struct samsung_usb2_phy_instance *);
-- int (*power_off)(struct samsung_usb2_phy_instance *);
These two callbacks are used to power on and power off the phy
by modifying appropriate registers.
Final change to the driver is adding appropriate compatible value to the
phy-samsung-usb2.c file. In case of Exynos 4210 the following lines were
-added to the struct of_device_id samsung_usb2_phy_of_match[] array:
+added to the struct of_device_id samsung_usb2_phy_of_match[] array::
-#ifdef CONFIG_PHY_EXYNOS4210_USB2
+ #ifdef CONFIG_PHY_EXYNOS4210_USB2
{
.compatible = "samsung,exynos4210-usb2-phy",
.data = &exynos4210_usb2_phy_config,
},
-#endif
+ #endif
To add further flexibility to the driver the Kconfig file enables to
include support for selected SoCs in the compiled driver. The Kconfig
-entry for Exynos 4210 is following:
+entry for Exynos 4210 is following::
-config PHY_EXYNOS4210_USB2
+ config PHY_EXYNOS4210_USB2
bool "Support for Exynos 4210"
depends on PHY_SAMSUNG_USB2
depends on CPU_EXYNOS4210
phys are available - device, host, HSCI0 and HSCI1.
The newly created file that supports the new SoC has to be also added to the
-Makefile. In case of Exynos 4210 the added line is following:
+Makefile. In case of Exynos 4210 the added line is following::
-obj-$(CONFIG_PHY_EXYNOS4210_USB2) += phy-exynos4210-usb2.o
+ obj-$(CONFIG_PHY_EXYNOS4210_USB2) += phy-exynos4210-usb2.o
After completing these steps the support for the new SoC should be ready.
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
======================
PPS - Pulse Per Second
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=============
+Intel MID PTI
+=============
+
+The Intel MID PTI project is HW implemented in Intel Atom
+system-on-a-chip designs based on the Parallel Trace
+Interface for MIPI P1149.7 cJTAG standard. The kernel solution
+for this platform involves the following files::
+
+ ./include/linux/pti.h
+ ./drivers/.../n_tracesink.h
+ ./drivers/.../n_tracerouter.c
+ ./drivers/.../n_tracesink.c
+ ./drivers/.../pti.c
+
+pti.c is the driver that enables various debugging features
+popular on platforms from certain mobile manufacturers.
+n_tracerouter.c and n_tracesink.c allow extra system information to
+be collected and routed to the pti driver, such as trace
+debugging data from a modem. Although n_tracerouter
+and n_tracesink are a part of the complete PTI solution,
+these two line disciplines can work separately from
+pti.c and route any data stream from one /dev/tty node
+to another /dev/tty node via kernel-space. This provides
+a stable, reliable connection that will not break unless
+the user-space application shuts down (plus avoids
+kernel->user->kernel context switch overheads of routing
+data).
+
+An example debugging usage for this driver system:
+
+ * Hook /dev/ttyPTI0 to syslogd. Opening this port will also start
+ a console device to further capture debugging messages to PTI.
+ * Hook /dev/ttyPTI1 to modem debugging data to write to PTI HW.
+ This is where n_tracerouter and n_tracesink are used.
+ * Hook /dev/pti to a user-level debugging application for writing
+ to PTI HW.
+ * `Use mipi_` Kernel Driver API in other device drivers for
+ debugging to PTI by first requesting a PTI write address via
+ mipi_request_masterchannel(1).
+
+Below is example pseudo-code on how a 'privileged' application
+can hook up n_tracerouter and n_tracesink to any tty on
+a system. 'Privileged' means the application has enough
+privileges to successfully manipulate the ldisc drivers
+but is not just blindly executing as 'root'. Keep in mind
+the use of ioctl(,TIOCSETD,) is not specific to the n_tracerouter
+and n_tracesink line discpline drivers but is a generic
+operation for a program to use a line discpline driver
+on a tty port other than the default n_tty::
+
+ /////////// To hook up n_tracerouter and n_tracesink /////////
+
+ // Note that n_tracerouter depends on n_tracesink.
+ #include <errno.h>
+ #define ONE_TTY "/dev/ttyOne"
+ #define TWO_TTY "/dev/ttyTwo"
+
+ // needed global to hand onto ldisc connection
+ static int g_fd_source = -1;
+ static int g_fd_sink = -1;
+
+ // these two vars used to grab LDISC values from loaded ldisc drivers
+ // in OS. Look at /proc/tty/ldiscs to get the right numbers from
+ // the ldiscs loaded in the system.
+ int source_ldisc_num, sink_ldisc_num = -1;
+ int retval;
+
+ g_fd_source = open(ONE_TTY, O_RDWR); // must be R/W
+ g_fd_sink = open(TWO_TTY, O_RDWR); // must be R/W
+
+ if (g_fd_source <= 0) || (g_fd_sink <= 0) {
+ // doubt you'll want to use these exact error lines of code
+ printf("Error on open(). errno: %d\n",errno);
+ return errno;
+ }
+
+ retval = ioctl(g_fd_sink, TIOCSETD, &sink_ldisc_num);
+ if (retval < 0) {
+ printf("Error on ioctl(). errno: %d\n", errno);
+ return errno;
+ }
+
+ retval = ioctl(g_fd_source, TIOCSETD, &source_ldisc_num);
+ if (retval < 0) {
+ printf("Error on ioctl(). errno: %d\n", errno);
+ return errno;
+ }
+
+ /////////// To disconnect n_tracerouter and n_tracesink ////////
+
+ // First make sure data through the ldiscs has stopped.
+
+ // Second, disconnect ldiscs. This provides a
+ // little cleaner shutdown on tty stack.
+ sink_ldisc_num = 0;
+ source_ldisc_num = 0;
+ ioctl(g_fd_uart, TIOCSETD, &sink_ldisc_num);
+ ioctl(g_fd_gadget, TIOCSETD, &source_ldisc_num);
+
+ // Three, program closes connection, and cleanup:
+ close(g_fd_uart);
+ close(g_fd_gadget);
+ g_fd_uart = g_fd_gadget = NULL;
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
===========================================
PTP hardware clock infrastructure for Linux
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===========================
+The Linux RapidIO Subsystem
+===========================
+
+.. toctree::
+ :maxdepth: 1
+
+ rapidio
+ sysfs
+
+ tsi721
+ mport_cdev
+ rio_cm
-RapidIO subsystem mport character device driver (rio_mport_cdev.c)
==================================================================
-
-Version History:
-----------------
- 1.0.0 - Initial driver release.
-
+RapidIO subsystem mport character device driver (rio_mport_cdev.c)
==================================================================
-I. Overview
+1. Overview
+===========
This device driver is the result of collaboration within the RapidIO.org
Software Task Group (STG) between Texas Instruments, Freescale,
following RapidIO bus and subsystem operations:
- Reads and writes from/to configuration registers of mport devices
- (RIO_MPORT_MAINT_READ_LOCAL/RIO_MPORT_MAINT_WRITE_LOCAL)
+ (RIO_MPORT_MAINT_READ_LOCAL/RIO_MPORT_MAINT_WRITE_LOCAL)
- Reads and writes from/to configuration registers of remote RapidIO devices.
This operations are defined as RapidIO Maintenance reads/writes in RIO spec.
- (RIO_MPORT_MAINT_READ_REMOTE/RIO_MPORT_MAINT_WRITE_REMOTE)
+ (RIO_MPORT_MAINT_READ_REMOTE/RIO_MPORT_MAINT_WRITE_REMOTE)
- Set RapidIO Destination ID for mport devices (RIO_MPORT_MAINT_HDID_SET)
- Set RapidIO Component Tag for mport devices (RIO_MPORT_MAINT_COMPTAG_SET)
- Query logical index of mport devices (RIO_MPORT_MAINT_PORT_IDX_GET)
- Query capabilities and RapidIO link configuration of mport devices
- (RIO_MPORT_GET_PROPERTIES)
+ (RIO_MPORT_GET_PROPERTIES)
- Enable/Disable reporting of RapidIO doorbell events to user-space applications
- (RIO_ENABLE_DOORBELL_RANGE/RIO_DISABLE_DOORBELL_RANGE)
+ (RIO_ENABLE_DOORBELL_RANGE/RIO_DISABLE_DOORBELL_RANGE)
- Enable/Disable reporting of RIO port-write events to user-space applications
- (RIO_ENABLE_PORTWRITE_RANGE/RIO_DISABLE_PORTWRITE_RANGE)
+ (RIO_ENABLE_PORTWRITE_RANGE/RIO_DISABLE_PORTWRITE_RANGE)
- Query/Control type of events reported through this driver: doorbells,
port-writes or both (RIO_SET_EVENT_MASK/RIO_GET_EVENT_MASK)
- Configure/Map mport's outbound requests window(s) for specific size,
RapidIO destination ID, hopcount and request type
- (RIO_MAP_OUTBOUND/RIO_UNMAP_OUTBOUND)
+ (RIO_MAP_OUTBOUND/RIO_UNMAP_OUTBOUND)
- Configure/Map mport's inbound requests window(s) for specific size,
RapidIO base address and local memory base address
- (RIO_MAP_INBOUND/RIO_UNMAP_INBOUND)
+ (RIO_MAP_INBOUND/RIO_UNMAP_INBOUND)
- Allocate/Free contiguous DMA coherent memory buffer for DMA data transfers
to/from remote RapidIO devices (RIO_ALLOC_DMA/RIO_FREE_DMA)
- Initiate DMA data transfers to/from remote RapidIO devices (RIO_TRANSFER).
Supports blocking, asynchronous and posted (a.k.a 'fire-and-forget') data
transfer modes.
- Check/Wait for completion of asynchronous DMA data transfer
- (RIO_WAIT_FOR_ASYNC)
+ (RIO_WAIT_FOR_ASYNC)
- Manage device objects supported by RapidIO subsystem (RIO_DEV_ADD/RIO_DEV_DEL).
This allows implementation of various RapidIO fabric enumeration algorithms
as user-space applications while using remaining functionality provided by
kernel RapidIO subsystem.
-II. Hardware Compatibility
+2. Hardware Compatibility
+=========================
This device driver uses standard interfaces defined by kernel RapidIO subsystem
and therefore it can be used with any mport device driver registered by RapidIO
specific DMA engine support and therefore DMA data transfers mport_cdev driver
are not available.
-III. Module parameters
+3. Module parameters
+====================
-- 'dma_timeout' - DMA transfer completion timeout (in msec, default value 3000).
+- 'dma_timeout'
+ - DMA transfer completion timeout (in msec, default value 3000).
This parameter set a maximum completion wait time for SYNC mode DMA
transfer requests and for RIO_WAIT_FOR_ASYNC ioctl requests.
-- 'dbg_level' - This parameter allows to control amount of debug information
+- 'dbg_level'
+ - This parameter allows to control amount of debug information
generated by this device driver. This parameter is formed by set of
bit masks that correspond to the specific functional blocks.
For mask definitions see 'drivers/rapidio/devices/rio_mport_cdev.c'
This parameter can be changed dynamically.
Use CONFIG_RAPIDIO_DEBUG=y to enable debug output at the top level.
-IV. Known problems
+4. Known problems
+=================
None.
-V. User-space Applications and API
+5. User-space Applications and API
+==================================
API library and applications that use this device driver are available from
RapidIO.org.
-VI. TODO List
+6. TODO List
+============
- Add support for sending/receiving "raw" RapidIO messaging packets.
- Add memory mapped DMA data transfers as an option when RapidIO-specific DMA
- The Linux RapidIO Subsystem
-
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+============
+Introduction
+============
The RapidIO standard is a packet-based fabric interconnect standard designed for
use in embedded systems. Development of the RapidIO standard is directed by the
information on its major components.
1 Overview
-----------
+==========
Because the RapidIO subsystem follows the Linux device model it is integrated
into the kernel similarly to other buses by defining RapidIO-specific device and
subsystem operations.
2. Core Components
-------------------
+==================
A typical RapidIO network is a combination of endpoints and switches.
Each of these components is represented in the subsystem by an associated data
in include/linux/rio.h file.
2.1 Master Port
+---------------
A master port (or mport) is a RapidIO interface controller that is local to the
processor executing the Linux code. A master port generates and receives RapidIO
implementations of RapidIO functions.
2.2 Device
+----------
A RapidIO device is any endpoint (other than mport) or switch in the network.
All devices are presented in the RapidIO subsystem by corresponding rio_dev data
(depending on number of available mports and networks).
2.3 Switch
+----------
A RapidIO switch is a special class of device that routes packets between its
ports towards their final destination. The packet destination port within a
implementation of common switch management routines.
2.4 Network
+-----------
A RapidIO network is a combination of interconnected endpoint and switch devices.
Each RapidIO network known to the system is represented by corresponding rio_net
master port that is used to communicate with devices within the network.
2.5 Device Drivers
+------------------
RapidIO device-specific drivers follow Linux Kernel Driver Model and are
intended to support specific RapidIO devices attached to the RapidIO network.
2.6 Subsystem Interfaces
+------------------------
RapidIO interconnect specification defines features that may be used to provide
one or more common service layers for all participating RapidIO devices. These
the same device without blocking attachment of a device-specific driver.
3. Subsystem Initialization
----------------------------
+===========================
In order to initialize the RapidIO subsystem, a platform must initialize and
register at least one master port within the RapidIO network. To register mport
modular component of the kernel (see details below).
4. Enumeration and Discovery
-----------------------------
+============================
4.1 Overview
------------
(b) If the RapidIO subsystem core is built as a loadable module, in addition
to the method shown above, the host destination ID(s) can be specified using
traditional methods of passing module parameter "hdid=" during its loading:
+
- from command line: "modprobe rapidio hdid=-1,7", or
- from modprobe configuration file using configuration command "options",
like in this example: "options rapidio hdid=-1,7". An example of modprobe
configuration file is provided in the section below.
- NOTES:
+NOTES:
(i) if "hdid=" parameter is omitted all available mport will be assigned
destination ID = -1;
+
(ii) the "hdid=" parameter in systems with multiple mports can have
destination ID assignments omitted from the end of list (default = -1).
This process can be automated by specifying pre- or post- dependencies in the
RapidIO-specific modprobe configuration file as shown in the example below.
- File /etc/modprobe.d/rapidio.conf:
- ----------------------------------
+File /etc/modprobe.d/rapidio.conf::
# Configure RapidIO subsystem modules
--------------------------
-NOTE: In the example above, one of "softdep" commands must be removed or
-commented out to keep required module loading sequence.
+NOTE:
+ In the example above, one of "softdep" commands must be removed or
+ commented out to keep required module loading sequence.
-A. References
--------------
+5. References
+=============
[1] RapidIO Trade Association. RapidIO Interconnect Specifications.
http://www.rapidio.org.
+
[2] Rapidio TA. Technology Comparisons.
http://www.rapidio.org/education/technology_comparisons/
+
[3] RapidIO support for Linux.
http://lwn.net/Articles/139118/
+
[4] Matt Porter. RapidIO for Linux. Ottawa Linux Symposium, 2005
http://www.kernel.org/doc/ols/2005/ols2005v2-pages-43-56.pdf
+==========================================================================
RapidIO subsystem Channelized Messaging character device driver (rio_cm.c)
==========================================================================
-Version History:
-----------------
- 1.0.0 - Initial driver release.
-
-==========================================================================
-I. Overview
+1. Overview
+===========
This device driver is the result of collaboration within the RapidIO.org
Software Task Group (STG) between Texas Instruments, Prodrive Technologies,
Following ioctl commands are available to user-space applications:
-- RIO_CM_MPORT_GET_LIST : Returns to caller list of local mport devices that
+- RIO_CM_MPORT_GET_LIST:
+ Returns to caller list of local mport devices that
support messaging operations (number of entries up to RIO_MAX_MPORTS).
Each list entry is combination of mport's index in the system and RapidIO
destination ID assigned to the port.
-- RIO_CM_EP_GET_LIST_SIZE : Returns number of messaging capable remote endpoints
+- RIO_CM_EP_GET_LIST_SIZE:
+ Returns number of messaging capable remote endpoints
in a RapidIO network associated with the specified mport device.
-- RIO_CM_EP_GET_LIST : Returns list of RapidIO destination IDs for messaging
+- RIO_CM_EP_GET_LIST:
+ Returns list of RapidIO destination IDs for messaging
capable remote endpoints (peers) available in a RapidIO network associated
with the specified mport device.
-- RIO_CM_CHAN_CREATE : Creates RapidIO message exchange channel data structure
+- RIO_CM_CHAN_CREATE:
+ Creates RapidIO message exchange channel data structure
with channel ID assigned automatically or as requested by a caller.
-- RIO_CM_CHAN_BIND : Binds the specified channel data structure to the specified
+- RIO_CM_CHAN_BIND:
+ Binds the specified channel data structure to the specified
mport device.
-- RIO_CM_CHAN_LISTEN : Enables listening for connection requests on the specified
+- RIO_CM_CHAN_LISTEN:
+ Enables listening for connection requests on the specified
channel.
-- RIO_CM_CHAN_ACCEPT : Accepts a connection request from peer on the specified
+- RIO_CM_CHAN_ACCEPT:
+ Accepts a connection request from peer on the specified
channel. If wait timeout for this request is specified by a caller it is
a blocking call. If timeout set to 0 this is non-blocking call - ioctl
handler checks for a pending connection request and if one is not available
exits with -EGAIN error status immediately.
-- RIO_CM_CHAN_CONNECT : Sends a connection request to a remote peer/channel.
-- RIO_CM_CHAN_SEND : Sends a data message through the specified channel.
+- RIO_CM_CHAN_CONNECT:
+ Sends a connection request to a remote peer/channel.
+- RIO_CM_CHAN_SEND:
+ Sends a data message through the specified channel.
The handler for this request assumes that message buffer specified by
a caller includes the reserved space for a packet header required by
this driver.
-- RIO_CM_CHAN_RECEIVE : Receives a data message through a connected channel.
+- RIO_CM_CHAN_RECEIVE:
+ Receives a data message through a connected channel.
If the channel does not have an incoming message ready to return this ioctl
handler will wait for new message until timeout specified by a caller
expires. If timeout value is set to 0, ioctl handler uses a default value
defined by MAX_SCHEDULE_TIMEOUT.
-- RIO_CM_CHAN_CLOSE : Closes a specified channel and frees associated buffers.
+- RIO_CM_CHAN_CLOSE:
+ Closes a specified channel and frees associated buffers.
If the specified channel is in the CONNECTED state, sends close notification
to the remote peer.
The ioctl command codes and corresponding data structures intended for use by
user-space applications are defined in 'include/uapi/linux/rio_cm_cdev.h'.
-II. Hardware Compatibility
+2. Hardware Compatibility
+=========================
This device driver uses standard interfaces defined by kernel RapidIO subsystem
and therefore it can be used with any mport device driver registered by RapidIO
subsystem with limitations set by available mport HW implementation of messaging
mailboxes.
-III. Module parameters
+3. Module parameters
+====================
-- 'dbg_level' - This parameter allows to control amount of debug information
+- 'dbg_level'
+ - This parameter allows to control amount of debug information
generated by this device driver. This parameter is formed by set of
bit masks that correspond to the specific functional block.
For mask definitions see 'drivers/rapidio/devices/rio_cm.c'
This parameter can be changed dynamically.
Use CONFIG_RAPIDIO_DEBUG=y to enable debug output at the top level.
-- 'cmbox' - Number of RapidIO mailbox to use (default value is 1).
+- 'cmbox'
+ - Number of RapidIO mailbox to use (default value is 1).
This parameter allows to set messaging mailbox number that will be used
within entire RapidIO network. It can be used when default mailbox is
used by other device drivers or is not supported by some nodes in the
RapidIO network.
-- 'chstart' - Start channel number for dynamic assignment. Default value - 256.
+- 'chstart'
+ - Start channel number for dynamic assignment. Default value - 256.
Allows to exclude channel numbers below this parameter from dynamic
allocation to avoid conflicts with software components that use
reserved predefined channel numbers.
-IV. Known problems
+4. Known problems
+=================
None.
-V. User-space Applications and API Library
+5. User-space Applications and API Library
+==========================================
Messaging API library and applications that use this device driver are available
from RapidIO.org.
-VI. TODO List
+6. TODO List
+============
- Add support for system notification messages (reserved channel 0).
+=============
+Sysfs entries
+=============
+
The RapidIO sysfs files have moved to:
Documentation/ABI/testing/sysfs-bus-rapidio and
Documentation/ABI/testing/sysfs-class-rapidio
+=========================================================================
RapidIO subsystem mport driver for IDT Tsi721 PCI Express-to-SRIO bridge.
=========================================================================
-I. Overview
+1. Overview
+===========
This driver implements all currently defined RapidIO mport callback functions.
It supports maintenance read and write operations, inbound and outbound RapidIO
fully compatible with RIONET driver (Ethernet over RapidIO messaging services).
1. Module parameters:
-- 'dbg_level' - This parameter allows to control amount of debug information
+
+- 'dbg_level'
+ - This parameter allows to control amount of debug information
generated by this device driver. This parameter is formed by set of
This parameter can be changed bit masks that correspond to the specific
functional block.
This parameter can be changed dynamically.
Use CONFIG_RAPIDIO_DEBUG=y to enable debug output at the top level.
-- 'dma_desc_per_channel' - This parameter defines number of hardware buffer
+- 'dma_desc_per_channel'
+ - This parameter defines number of hardware buffer
descriptors allocated for each registered Tsi721 DMA channel.
Its default value is 128.
-- 'dma_txqueue_sz' - DMA transactions queue size. Defines number of pending
+- 'dma_txqueue_sz'
+ - DMA transactions queue size. Defines number of pending
transaction requests that can be accepted by each DMA channel.
Default value is 16.
-- 'dma_sel' - DMA channel selection mask. Bitmask that defines which hardware
+- 'dma_sel'
+ - DMA channel selection mask. Bitmask that defines which hardware
DMA channels (0 ... 6) will be registered with DmaEngine core.
If bit is set to 1, the corresponding DMA channel will be registered.
DMA channels not selected by this mask will not be used by this device
driver. Default value is 0x7f (use all channels).
-- 'pcie_mrrs' - override value for PCIe Maximum Read Request Size (MRRS).
+- 'pcie_mrrs'
+ - override value for PCIe Maximum Read Request Size (MRRS).
This parameter gives an ability to override MRRS value set during PCIe
configuration process. Tsi721 supports read request sizes up to 4096B.
Value for this parameter must be set as defined by PCIe specification:
0 = 128B, 1 = 256B, 2 = 512B, 3 = 1024B, 4 = 2048B and 5 = 4096B.
Default value is '-1' (= keep platform setting).
-- 'mbox_sel' - RIO messaging MBOX selection mask. This is a bitmask that defines
+- 'mbox_sel'
+ - RIO messaging MBOX selection mask. This is a bitmask that defines
messaging MBOXes are managed by this device driver. Mask bits 0 - 3
correspond to MBOX0 - MBOX3. MBOX is under driver's control if the
corresponding bit is set to '1'. Default value is 0x0f (= all).
-II. Known problems
+2. Known problems
+=================
None.
-III. DMA Engine Support
+3. DMA Engine Support
+=====================
Tsi721 mport driver supports DMA data transfers between local system memory and
remote RapidIO devices. This functionality is implemented according to SLAVE
If Tsi721 mport driver have been built with RAPIDIO_DMA_ENGINE support included,
this driver will accept DMA-specific module parameter:
- "dma_desc_per_channel" - defines number of hardware buffer descriptors used by
+
+ "dma_desc_per_channel"
+ - defines number of hardware buffer descriptors used by
each BDMA channel of Tsi721 (by default - 128).
-IV. Version History
+4. Version History
- 1.1.0 - DMA operations re-worked to support data scatter/gather lists larger
+ ===== ====================================================================
+ 1.1.0 DMA operations re-worked to support data scatter/gather lists larger
than hardware buffer descriptors ring.
- 1.0.0 - Initial driver release.
+ 1.0.0 Initial driver release.
+ ===== ====================================================================
-V. License
------------------------------------------------
+5. License
+===========
Copyright(c) 2011 Integrated Device Technology, Inc. All rights reserved.
This call must not sleep
set_ldisc(port,termios)
- Notifier for discipline change. See Documentation/serial/tty.rst.
+ Notifier for discipline change. See Documentation/driver-api/serial/tty.rst.
Locking: caller holds tty_port->mutex
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
==========================
Support for Serial devices
NT EP BAR 2 will be dynamically configured as a Direct Window, and
the configuration file does not need to configure it explicitly.
-Please refer to Documentation/ntb.txt in Linux source tree for an overall
+Please refer to Documentation/driver-api/ntb.rst in Linux source tree for an overall
understanding of the Linux NTB stack. ntb_hw_switchtec works as an NTB
Hardware Driver in this stack.
References
==========
-1. See Documentation/vfio.txt for more information on VFIO.
+1. See Documentation/driver-api/vfio.rst for more information on VFIO.
2. struct mdev_driver in include/linux/mdev.h
3. struct mdev_parent_ops in include/linux/mdev.h
4. struct vfio_iommu_driver_ops in include/linux/vfio.h
-:orphan:
===========
Xilinx FPGA
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
===============
fault-injection
Assuming the VGA driver can be unloaded, one must first unbind the VGA driver
from the console layer before unloading the driver. The VGA driver cannot be
unloaded if it is still bound to the console layer. (See
-Documentation/console/console.txt for more information).
+Documentation/driver-api/console.rst for more information).
This is more complicated in the case of the framebuffer console (fbcon),
because fbcon is an intermediate layer between the console and the drivers::
fbcon.
So, how do we unbind fbcon from the console? Part of the answer is in
-Documentation/console/console.txt. To summarize:
+Documentation/driver-api/console.rst. To summarize:
Echo a value to the bind file that represents the framebuffer console
driver. So assuming vtcon1 represents fbcon, then::
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
============
Frame Buffer
==============
Switching modes is done using the vga=... boot parameter. Read
-Documentation/svga.txt for details.
+Documentation/admin-guide/svga.rst for details.
You should compile in both vgacon (for text mode) and vesafb (for
graphics mode). Which of them takes over the console depends on
-
+ struct coda_timespec {
+ int64_t tv_sec; /* seconds */
+ long tv_nsec; /* nanoseconds */
+ };
struct coda_vattr {
enum coda_vtype va_type; /* vnode type (for create) */
long va_fileid; /* file id */
u_quad_t va_size; /* file size in bytes */
long va_blocksize; /* blocksize preferred for i/o */
- struct timespec va_atime; /* time of last access */
- struct timespec va_mtime; /* time of last modification */
- struct timespec va_ctime; /* time file changed */
+ struct coda_timespec va_atime; /* time of last access */
+ struct coda_timespec va_mtime; /* time of last modification */
+ struct coda_timespec va_ctime; /* time file changed */
u_long va_gen; /* generation number of file */
u_long va_flags; /* flags defined for file */
dev_t va_rdev; /* device special file represents */
These filesystems may be used for inspiration:
- ext2: see Documentation/filesystems/ext2.txt
- ext4: see Documentation/filesystems/ext4/
-- xfs: see Documentation/filesystems/xfs.txt
+- xfs: see Documentation/admin-guide/xfs.rst
Handling Media Errors
A description of the process of mounting the root file system can be
found in:
- Documentation/early-userspace/README
+ Documentation/driver-api/early-userspace/early_userspace_support.rst
--
[mandatory]
d_alloc_root() is gone, along with a lot of bugs caused by code
-misusing it. Replacement: d_make_root(inode). The difference is,
-d_make_root() drops the reference to inode if dentry allocation fails.
+misusing it. Replacement: d_make_root(inode). On success d_make_root(inode)
+allocates and returns a new dentry instantiated with the passed in inode.
+On failure NULL is returned and the passed in inode is dropped so the reference
+to inode is consumed in all cases and failure handling need not do any cleanup
+for the inode. If d_make_root(inode) is passed a NULL inode it returns NULL
+and also requires no further error handling. Typical usage is:
+
+ inode = foofs_new_inode(....);
+ s->s_root = d_make_inode(inode);
+ if (!s->s_root)
+ /* Nothing needed for the inode cleanup */
+ return -ENOMEM;
+ ...
--
[mandatory]
"SwapPss" shows proportional swap share of this mapping. Unlike "Swap", this
does not take into account swapped out page of underlying shmem objects.
"Locked" indicates whether the mapping is locked in memory or not.
-"THPeligible" indicates whether the mapping is eligible for THP pages - 1 if
-true, 0 otherwise.
+"THPeligible" indicates whether the mapping is eligible for allocating THP
+pages - 1 if true, 0 otherwise. It just shows the current status.
"VmFlags" field deserves a separate description. This member represents the kernel
flags associated with the particular virtual memory area in two letter encoded
This chapter is heavily based on the documentation included in the pre 2.2
kernels, and became part of it in version 2.2.1 of the Linux kernel.
-Please see: Documentation/sysctl/ directory for descriptions of these
+Please see: Documentation/admin-guide/sysctl/ directory for descriptions of these
entries.
------------------------------------------------------------------------------
- The old initrd file was a gzipped filesystem image (in some file format,
such as ext2, that needed a driver built into the kernel), while the new
initramfs archive is a gzipped cpio archive (like tar only simpler,
- see cpio(1) and Documentation/early-userspace/buffer-format.txt). The
+ see cpio(1) and Documentation/driver-api/early-userspace/buffer-format.rst). The
kernel's cpio extraction code is not only extremely small, it's also
__init text and data that can be discarded during the boot process.
set permissions or create device nodes in the new archive. (Note that those
two example "file" entries expect to find files named "init.sh" and "busybox" in
a directory called "initramfs", under the linux-2.6.* directory. See
-Documentation/early-userspace/README for more details.)
+Documentation/driver-api/early-userspace/early_userspace_support.rst for more details.)
The kernel does not depend on external cpio tools. If you specify a
directory instead of a configuration file, the kernel's build infrastructure
a stat(2) operation.
More information can driver-model specific features can be found in
-Documentation/driver-model/.
+Documentation/driver-api/driver-model/.
TODO: Finish this section.
use at file creation time. When a task allocates a file in the file
system, the mount option memory policy will be applied with a NodeList,
if any, modified by the calling task's cpuset constraints
-[See Documentation/cgroup-v1/cpusets.rst] and any optional flags, listed
+[See Documentation/admin-guide/cgroup-v1/cpusets.rst] and any optional flags, listed
below. If the resulting NodeLists is the empty set, the effective memory
policy for the file will revert to "default" policy.
we need to translate them to the corresponding Linux GPIO descriptors.
There is a standard GPIO API for that and is documented in
-Documentation/gpio/.
+Documentation/admin-guide/gpio/.
In the above example we can get the corresponding two GPIO descriptors with
a code like this::
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
====
fpga
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
=============================
Human Interface Devices (HID)
console. Excessive logging can seriously affect system performance.
* Use devres functions whenever possible to allocate resources. For rationale
- and supported functions, please see Documentation/driver-model/devres.rst.
+ and supported functions, please see Documentation/driver-api/driver-model/devres.rst.
If a function is not supported by devres, consider using devm_add_action().
* If the driver has a detect function, make sure it is silent. Debug messages
::
+ int hwspin_lock_timeout_raw(struct hwspinlock *hwlock, unsigned int timeout);
+
+Lock a previously-assigned hwspinlock with a timeout limit (specified in
+msecs). If the hwspinlock is already taken, the function will busy loop
+waiting for it to be released, but give up when the timeout elapses.
+
+Caution: User must protect the routine of getting hardware lock with mutex
+or spinlock to avoid dead-lock, that will let user can do some time-consuming
+or sleepable operations under the hardware lock.
+
+Returns 0 when successful and an appropriate error code otherwise (most
+notably -ETIMEDOUT if the hwspinlock is still busy after timeout msecs).
+
+The function will never sleep.
+
+::
+
+ int hwspin_lock_timeout_in_atomic(struct hwspinlock *hwlock, unsigned int to);
+
+Lock a previously-assigned hwspinlock with a timeout limit (specified in
+msecs). If the hwspinlock is already taken, the function will busy loop
+waiting for it to be released, but give up when the timeout elapses.
+
+This function shall be called only from an atomic context and the timeout
+value shall not exceed a few msecs.
+
+Returns 0 when successful and an appropriate error code otherwise (most
+notably -ETIMEDOUT if the hwspinlock is still busy after timeout msecs).
+
+The function will never sleep.
+
+::
+
int hwspin_trylock(struct hwspinlock *hwlock);
::
+ int hwspin_trylock_raw(struct hwspinlock *hwlock);
+
+Attempt to lock a previously-assigned hwspinlock, but immediately fail if
+it is already taken.
+
+Caution: User must protect the routine of getting hardware lock with mutex
+or spinlock to avoid dead-lock, that will let user can do some time-consuming
+or sleepable operations under the hardware lock.
+
+Returns 0 on success and an appropriate error code otherwise (most
+notably -EBUSY if the hwspinlock was already taken).
+The function will never sleep.
+
+::
+
+ int hwspin_trylock_in_atomic(struct hwspinlock *hwlock);
+
+Attempt to lock a previously-assigned hwspinlock, but immediately fail if
+it is already taken.
+
+This function shall be called only from an atomic context.
+
+Returns 0 on success and an appropriate error code otherwise (most
+notably -EBUSY if the hwspinlock was already taken).
+The function will never sleep.
+
+::
+
void hwspin_unlock(struct hwspinlock *hwlock);
Unlock a previously-locked hwspinlock. Always succeed, and can be called
::
+ void hwspin_unlock_raw(struct hwspinlock *hwlock);
+
+Unlock a previously-locked hwspinlock.
+
+The caller should **never** unlock an hwspinlock which is already unlocked.
+Doing so is considered a bug (there is no protection against this).
+This function will never sleep.
+
+::
+
+ void hwspin_unlock_in_atomic(struct hwspinlock *hwlock);
+
+Unlock a previously-locked hwspinlock.
+
+The caller should **never** unlock an hwspinlock which is already unlocked.
+Doing so is considered a bug (there is no protection against this).
+This function will never sleep.
+
+::
+
int hwspin_lock_get_id(struct hwspinlock *hwlock);
Retrieve id number of a given hwspinlock. This is needed when an
- MEMORY ATTRIBUTE ALIASING ON IA-64
+==================================
+Memory Attribute Aliasing on IA-64
+==================================
- Bjorn Helgaas
- <bjorn.helgaas@hp.com>
- May 4, 2006
+Bjorn Helgaas <bjorn.helgaas@hp.com>
+May 4, 2006
-MEMORY ATTRIBUTES
+
+Memory Attributes
+=================
Itanium supports several attributes for virtual memory references.
The attribute is part of the virtual translation, i.e., it is
contained in the TLB entry. The ones of most interest to the Linux
kernel are:
- WB Write-back (cacheable)
+ == ======================
+ WB Write-back (cacheable)
UC Uncacheable
WC Write-coalescing
+ == ======================
System memory typically uses the WB attribute. The UC attribute is
used for memory-mapped I/O devices. The WC attribute is uncacheable
support either WB or UC access to main memory, while others support
only WB access.
-MEMORY MAP
+Memory Map
+==========
Platform firmware describes the physical memory map and the
supported attributes for each region. At boot-time, the kernel uses
The efi_memmap table is preserved unmodified because the original
boot-time information is required for kexec.
-KERNEL IDENTITY MAPPINGS
+Kernel Identify Mappings
+========================
Linux/ia64 identity mappings are done with large pages, currently
either 16MB or 64MB, referred to as "granules." Cacheable mappings
are only partially populated, or populated with a combination of UC
and WB regions.
-USER MAPPINGS
+User Mappings
+=============
User mappings are typically done with 16K or 64K pages. The smaller
page size allows more flexibility because only 16K or 64K has to be
homogeneous with respect to memory attributes.
-POTENTIAL ATTRIBUTE ALIASING CASES
+Potential Attribute Aliasing Cases
+==================================
There are several ways the kernel creates new mappings:
- mmap of /dev/mem
+mmap of /dev/mem
+----------------
This uses remap_pfn_range(), which creates user mappings. These
mappings may be either WB or UC. If the region being mapped
Since the EFI memory map does not describe MMIO on some
machines, this should use an uncacheable mapping as a fallback.
- mmap of /sys/class/pci_bus/.../legacy_mem
+mmap of /sys/class/pci_bus/.../legacy_mem
+-----------------------------------------
This is very similar to mmap of /dev/mem, except that legacy_mem
only allows mmap of the one megabyte "legacy MMIO" area for a
The /dev/mem mmap constraints apply.
- mmap of /proc/bus/pci/.../??.?
+mmap of /proc/bus/pci/.../??.?
+------------------------------
- This is an MMIO mmap of PCI functions, which additionally may or
+ This is an MMIO mmap of PCI functions, which additionally may or
may not be requested as using the WC attribute.
If WC is requested, and the region in kern_memmap is either WC
Otherwise, the user mapping must use the same attribute as the
kernel mapping.
- read/write of /dev/mem
+read/write of /dev/mem
+----------------------
This uses copy_from_user(), which implicitly uses a kernel
identity mapping. This is obviously safe for things in
eight-byte accesses, and the copy_from_user() path doesn't allow
any control over the access size, so this would be dangerous.
- ioremap()
+ioremap()
+---------
This returns a mapping for use inside the kernel.
Failing all of the above, we have to fall back to a UC mapping.
-PAST PROBLEM CASES
+Past Problem Cases
+==================
- mmap of various MMIO regions from /dev/mem by "X" on Intel platforms
+mmap of various MMIO regions from /dev/mem by "X" on Intel platforms
+--------------------------------------------------------------------
The EFI memory map may not report these MMIO regions.
succeed. It may create either WB or UC user mappings, depending
on whether the region is in kern_memmap or the EFI memory map.
- mmap of 0x0-0x9FFFF /dev/mem by "hwinfo" on HP sx1000 with VGA enabled
+mmap of 0x0-0x9FFFF /dev/mem by "hwinfo" on HP sx1000 with VGA enabled
+----------------------------------------------------------------------
The EFI memory map reports the following attributes:
+
+ =============== ======= ==================
0x00000-0x9FFFF WB only
0xA0000-0xBFFFF UC only (VGA frame buffer)
0xC0000-0xFFFFF WB only
+ =============== ======= ==================
This mmap is done with user pages, not kernel identity mappings,
so it is safe to use WB mappings.
never generate an uncacheable reference to the WB-only areas unless
the driver explicitly touches them.
- mmap of 0x0-0xFFFFF legacy_mem by "X"
+mmap of 0x0-0xFFFFF legacy_mem by "X"
+-------------------------------------
If the EFI memory map reports that the entire range supports the
same attributes, we can allow the mmap (and we will prefer WB if
that doesn't report the VGA frame buffer at all), we should fail the
mmap and force the user to map just the specific region of interest.
- mmap of 0xA0000-0xBFFFF legacy_mem by "X" on HP sx1000 with VGA disabled
+mmap of 0xA0000-0xBFFFF legacy_mem by "X" on HP sx1000 with VGA disabled
+------------------------------------------------------------------------
+
+ The EFI memory map reports the following attributes::
- The EFI memory map reports the following attributes:
0x00000-0xFFFFF WB only (no VGA MMIO hole)
This is a special case of the previous case, and the mmap should
fail for the same reason as above.
- read of /sys/devices/.../rom
+read of /sys/devices/.../rom
+----------------------------
For VGA devices, this may cause an ioremap() of 0xC0000. This
used to be done with a UC mapping, because the VGA frame buffer
We should use WB page table mappings to avoid covering the VGA
frame buffer.
-NOTES
+Notes
+=====
[1] SDM rev 2.2, vol 2, sec 4.4.1.
[2] SDM rev 2.2, vol 2, sec 4.4.6.
+==========================
EFI Real Time Clock driver
--------------------------------
+==========================
+
S. Eranian <eranian@hpl.hp.com>
+
March 2000
-I/ Introduction
+1. Introduction
+===============
This document describes the efirtc.c driver has provided for
-the IA-64 platform.
+the IA-64 platform.
The purpose of this driver is to supply an API for kernel and user applications
to get access to the Time Service offered by EFI version 0.92.
driver. We describe those calls as well the design of the driver in the
following sections.
-II/ Design Decisions
+2. Design Decisions
+===================
-The original ideas was to provide a very simple driver to get access to,
-at first, the time of day service. This is required in order to access, in a
-portable way, the CMOS clock. A program like /sbin/hwclock uses such a clock
+The original ideas was to provide a very simple driver to get access to,
+at first, the time of day service. This is required in order to access, in a
+portable way, the CMOS clock. A program like /sbin/hwclock uses such a clock
to initialize the system view of the time during boot.
Because we wanted to minimize the impact on existing user-level apps using
the CMOS clock, we decided to expose an API that was very similar to the one
-used today with the legacy RTC driver (driver/char/rtc.c). However, because
+used today with the legacy RTC driver (driver/char/rtc.c). However, because
EFI provides a simpler services, not all ioctl() are available. Also
-new ioctl()s have been introduced for things that EFI provides but not the
+new ioctl()s have been introduced for things that EFI provides but not the
legacy.
EFI uses a slightly different way of representing the time, noticeably
the reference date is different. Year is the using the full 4-digit format.
The Epoch is January 1st 1998. For backward compatibility reasons we don't
-expose this new way of representing time. Instead we use something very
+expose this new way of representing time. Instead we use something very
similar to the struct tm, i.e. struct rtc_time, as used by hwclock.
One of the reasons for doing it this way is to allow for EFI to still evolve
without necessarily impacting any of the user applications. The decoupling
enables flexibility and permits writing wrapper code is ncase things change.
The driver exposes two interfaces, one via the device file and a set of
-ioctl()s. The other is read-only via the /proc filesystem.
+ioctl()s. The other is read-only via the /proc filesystem.
As of today we don't offer a /proc/sys interface.
To allow for a uniform interface between the legacy RTC and EFI time service,
-we have created the include/linux/rtc.h header file to contain only the
-"public" API of the two drivers. The specifics of the legacy RTC are still
+we have created the include/linux/rtc.h header file to contain only the
+"public" API of the two drivers. The specifics of the legacy RTC are still
in include/linux/mc146818rtc.h.
-
-III/ Time of day service
+
+3. Time of day service
+======================
The part of the driver gives access to the time of day service of EFI.
Two ioctl()s, compatible with the legacy RTC calls:
- Read the CMOS clock: ioctl(d, RTC_RD_TIME, &rtc);
+ Read the CMOS clock::
+
+ ioctl(d, RTC_RD_TIME, &rtc);
+
+ Write the CMOS clock::
- Write the CMOS clock: ioctl(d, RTC_SET_TIME, &rtc);
+ ioctl(d, RTC_SET_TIME, &rtc);
The rtc is a pointer to a data structure defined in rtc.h which is close
-to a struct tm:
-
-struct rtc_time {
- int tm_sec;
- int tm_min;
- int tm_hour;
- int tm_mday;
- int tm_mon;
- int tm_year;
- int tm_wday;
- int tm_yday;
- int tm_isdst;
-};
+to a struct tm::
+
+ struct rtc_time {
+ int tm_sec;
+ int tm_min;
+ int tm_hour;
+ int tm_mday;
+ int tm_mon;
+ int tm_year;
+ int tm_wday;
+ int tm_yday;
+ int tm_isdst;
+ };
The driver takes care of converting back an forth between the EFI time and
this format.
Those two ioctl()s can be exercised with the hwclock command:
-For reading:
-# /sbin/hwclock --show
-Mon Mar 6 15:32:32 2000 -0.910248 seconds
+For reading::
-For setting:
-# /sbin/hwclock --systohc
+ # /sbin/hwclock --show
+ Mon Mar 6 15:32:32 2000 -0.910248 seconds
+
+For setting::
+
+ # /sbin/hwclock --systohc
Root privileges are required to be able to set the time of day.
-IV/ Wakeup Alarm service
+4. Wakeup Alarm service
+=======================
EFI provides an API by which one can program when a machine should wakeup,
i.e. reboot. This is very different from the alarm provided by the legacy
RTC which is some kind of interval timer alarm. For this reason we don't use
the same ioctl()s to get access to the service. Instead we have
-introduced 2 news ioctl()s to the interface of an RTC.
+introduced 2 news ioctl()s to the interface of an RTC.
We have added 2 new ioctl()s that are specific to the EFI driver:
- Read the current state of the alarm
- ioctl(d, RTC_WKLAM_RD, &wkt)
+ Read the current state of the alarm::
+
+ ioctl(d, RTC_WKLAM_RD, &wkt)
+
+ Set the alarm or change its status::
+
+ ioctl(d, RTC_WKALM_SET, &wkt)
- Set the alarm or change its status
- ioctl(d, RTC_WKALM_SET, &wkt)
+The wkt structure encapsulates a struct rtc_time + 2 extra fields to get
+status information::
-The wkt structure encapsulates a struct rtc_time + 2 extra fields to get
-status information:
-
-struct rtc_wkalrm {
+ struct rtc_wkalrm {
- unsigned char enabled; /* =1 if alarm is enabled */
- unsigned char pending; /* =1 if alarm is pending */
+ unsigned char enabled; /* =1 if alarm is enabled */
+ unsigned char pending; /* =1 if alarm is pending */
- struct rtc_time time;
-}
+ struct rtc_time time;
+ }
As of today, none of the existing user-level apps supports this feature.
-However writing such a program should be hard by simply using those two
-ioctl().
+However writing such a program should be hard by simply using those two
+ioctl().
Root privileges are required to be able to set the alarm.
-V/ References.
+5. References
+=============
Checkout the following Web site for more information on EFI:
-
+========================================
IPF Machine Check (MC) error inject tool
========================================
the pal_mc_error_inject PAL procedure. The following err.conf has been tested
on latest Montecito PAL.
-err.conf:
+err.conf::
-#This is configuration file for err_inject_tool.
-#The format of the each line is:
-#cpu, loop, interval, err_type_info, err_struct_info, err_data_buffer
-#where
-# cpu: logical cpu number the error will be inject in.
-# loop: times the error will be injected.
-# interval: In second. every so often one error is injected.
-# err_type_info, err_struct_info: PAL parameters.
-#
-#Note: All values are hex w/o or w/ 0x prefix.
+ #This is configuration file for err_inject_tool.
+ #The format of the each line is:
+ #cpu, loop, interval, err_type_info, err_struct_info, err_data_buffer
+ #where
+ # cpu: logical cpu number the error will be inject in.
+ # loop: times the error will be injected.
+ # interval: In second. every so often one error is injected.
+ # err_type_info, err_struct_info: PAL parameters.
+ #
+ #Note: All values are hex w/o or w/ 0x prefix.
-#On cpu2, inject only total 0x10 errors, interval 5 seconds
-#corrected, data cache, hier-2, physical addr(assigned by tool code).
-#working on Montecito latest PAL.
-2, 10, 5, 4101, 95
+ #On cpu2, inject only total 0x10 errors, interval 5 seconds
+ #corrected, data cache, hier-2, physical addr(assigned by tool code).
+ #working on Montecito latest PAL.
+ 2, 10, 5, 4101, 95
-#On cpu4, inject and consume total 0x10 errors, interval 5 seconds
-#corrected, data cache, hier-2, physical addr(assigned by tool code).
-#working on Montecito latest PAL.
-4, 10, 5, 4109, 95
+ #On cpu4, inject and consume total 0x10 errors, interval 5 seconds
+ #corrected, data cache, hier-2, physical addr(assigned by tool code).
+ #working on Montecito latest PAL.
+ 4, 10, 5, 4109, 95
-#On cpu15, inject and consume total 0x10 errors, interval 5 seconds
-#recoverable, DTR0, hier-2.
-#working on Montecito latest PAL.
-0xf, 0x10, 5, 4249, 15
+ #On cpu15, inject and consume total 0x10 errors, interval 5 seconds
+ #recoverable, DTR0, hier-2.
+ #working on Montecito latest PAL.
+ 0xf, 0x10, 5, 4249, 15
The sample application source code:
-err_injection_tool.c:
-
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
- * NON INFRINGEMENT. See the GNU General Public License for more
- * details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
- *
- * Copyright (C) 2006 Intel Co
- * Fenghua Yu <fenghua.yu@intel.com>
- *
- */
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <stdio.h>
-#include <sched.h>
-#include <unistd.h>
-#include <stdlib.h>
-#include <stdarg.h>
-#include <string.h>
-#include <errno.h>
-#include <time.h>
-#include <sys/ipc.h>
-#include <sys/sem.h>
-#include <sys/wait.h>
-#include <sys/mman.h>
-#include <sys/shm.h>
-
-#define MAX_FN_SIZE 256
-#define MAX_BUF_SIZE 256
-#define DATA_BUF_SIZE 256
-#define NR_CPUS 512
-#define MAX_TASK_NUM 2048
-#define MIN_INTERVAL 5 // seconds
-#define ERR_DATA_BUFFER_SIZE 3 // Three 8-byte.
-#define PARA_FIELD_NUM 5
-#define MASK_SIZE (NR_CPUS/64)
-#define PATH_FORMAT "/sys/devices/system/cpu/cpu%d/err_inject/"
-
-int sched_setaffinity(pid_t pid, unsigned int len, unsigned long *mask);
-
-int verbose;
-#define vbprintf if (verbose) printf
-
-int log_info(int cpu, const char *fmt, ...)
-{
+err_injection_tool.c::
+
+ /*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or
+ * NON INFRINGEMENT. See the GNU General Public License for more
+ * details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ * Copyright (C) 2006 Intel Co
+ * Fenghua Yu <fenghua.yu@intel.com>
+ *
+ */
+ #include <sys/types.h>
+ #include <sys/stat.h>
+ #include <fcntl.h>
+ #include <stdio.h>
+ #include <sched.h>
+ #include <unistd.h>
+ #include <stdlib.h>
+ #include <stdarg.h>
+ #include <string.h>
+ #include <errno.h>
+ #include <time.h>
+ #include <sys/ipc.h>
+ #include <sys/sem.h>
+ #include <sys/wait.h>
+ #include <sys/mman.h>
+ #include <sys/shm.h>
+
+ #define MAX_FN_SIZE 256
+ #define MAX_BUF_SIZE 256
+ #define DATA_BUF_SIZE 256
+ #define NR_CPUS 512
+ #define MAX_TASK_NUM 2048
+ #define MIN_INTERVAL 5 // seconds
+ #define ERR_DATA_BUFFER_SIZE 3 // Three 8-byte.
+ #define PARA_FIELD_NUM 5
+ #define MASK_SIZE (NR_CPUS/64)
+ #define PATH_FORMAT "/sys/devices/system/cpu/cpu%d/err_inject/"
+
+ int sched_setaffinity(pid_t pid, unsigned int len, unsigned long *mask);
+
+ int verbose;
+ #define vbprintf if (verbose) printf
+
+ int log_info(int cpu, const char *fmt, ...)
+ {
FILE *log;
char fn[MAX_FN_SIZE];
char buf[MAX_BUF_SIZE];
fclose(log);
return 0;
-}
+ }
-typedef unsigned long u64;
-typedef unsigned int u32;
+ typedef unsigned long u64;
+ typedef unsigned int u32;
-typedef union err_type_info_u {
+ typedef union err_type_info_u {
struct {
u64 mode : 3, /* 0-2 */
err_inj : 3, /* 3-5 */
reserved : 48; /* 16-63 */
} err_type_info_u;
u64 err_type_info;
-} err_type_info_t;
+ } err_type_info_t;
-typedef union err_struct_info_u {
+ typedef union err_struct_info_u {
struct {
u64 siv : 1, /* 0 */
c_t : 2, /* 1-2 */
u64 reserved;
} err_struct_info_bus_processor_interconnect;
u64 err_struct_info;
-} err_struct_info_t;
+ } err_struct_info_t;
-typedef union err_data_buffer_u {
+ typedef union err_data_buffer_u {
struct {
u64 trigger_addr; /* 0-63 */
u64 inj_addr; /* 64-127 */
u64 reserved; /* 0-63 */
} err_data_buffer_bus_processor_interconnect;
u64 err_data_buffer[ERR_DATA_BUFFER_SIZE];
-} err_data_buffer_t;
+ } err_data_buffer_t;
-typedef union capabilities_u {
+ typedef union capabilities_u {
struct {
u64 i : 1,
d : 1,
struct {
u64 reserved;
} capabilities_bus_processor_interconnect;
-} capabilities_t;
+ } capabilities_t;
-typedef struct resources_s {
+ typedef struct resources_s {
u64 ibr0 : 1,
ibr2 : 1,
ibr4 : 1,
dbr4 : 1,
dbr6 : 1,
reserved : 48;
-} resources_t;
+ } resources_t;
-long get_page_size(void)
-{
+ long get_page_size(void)
+ {
long page_size=sysconf(_SC_PAGESIZE);
return page_size;
-}
+ }
-#define PAGE_SIZE (get_page_size()==-1?0x4000:get_page_size())
-#define SHM_SIZE (2*PAGE_SIZE*NR_CPUS)
-#define SHM_VA 0x2000000100000000
+ #define PAGE_SIZE (get_page_size()==-1?0x4000:get_page_size())
+ #define SHM_SIZE (2*PAGE_SIZE*NR_CPUS)
+ #define SHM_VA 0x2000000100000000
-int shmid;
-void *shmaddr;
+ int shmid;
+ void *shmaddr;
-int create_shm(void)
-{
+ int create_shm(void)
+ {
key_t key;
char fn[MAX_FN_SIZE];
mlock(shmaddr, SHM_SIZE);
return 0;
-}
+ }
-int free_shm()
-{
+ int free_shm()
+ {
munlock(shmaddr, SHM_SIZE);
- shmdt(shmaddr);
+ shmdt(shmaddr);
semctl(shmid, 0, IPC_RMID);
return 0;
-}
+ }
-#ifdef _SEM_SEMUN_UNDEFINED
-union semun
-{
+ #ifdef _SEM_SEMUN_UNDEFINED
+ union semun
+ {
int val;
struct semid_ds *buf;
unsigned short int *array;
struct seminfo *__buf;
-};
-#endif
+ };
+ #endif
-u32 mode=1; /* 1: physical mode; 2: virtual mode. */
-int one_lock=1;
-key_t key[NR_CPUS];
-int semid[NR_CPUS];
+ u32 mode=1; /* 1: physical mode; 2: virtual mode. */
+ int one_lock=1;
+ key_t key[NR_CPUS];
+ int semid[NR_CPUS];
-int create_sem(int cpu)
-{
+ int create_sem(int cpu)
+ {
union semun arg;
char fn[MAX_FN_SIZE];
int sid;
}
return 0;
-}
+ }
-static int lock(int cpu)
-{
+ static int lock(int cpu)
+ {
struct sembuf lock;
lock.sem_num = cpu;
lock.sem_op = 1;
semop(semid[cpu], &lock, 1);
- return 0;
-}
+ return 0;
+ }
-static int unlock(int cpu)
-{
+ static int unlock(int cpu)
+ {
struct sembuf unlock;
unlock.sem_num = cpu;
unlock.sem_op = -1;
semop(semid[cpu], &unlock, 1);
- return 0;
-}
+ return 0;
+ }
-void free_sem(int cpu)
-{
+ void free_sem(int cpu)
+ {
semctl(semid[cpu], 0, IPC_RMID);
-}
+ }
-int wr_multi(char *fn, unsigned long *data, int size)
-{
+ int wr_multi(char *fn, unsigned long *data, int size)
+ {
int fd;
char buf[MAX_BUF_SIZE];
int ret;
ret=write(fd, buf, sizeof(buf));
close(fd);
return ret;
-}
+ }
-int wr(char *fn, unsigned long data)
-{
+ int wr(char *fn, unsigned long data)
+ {
return wr_multi(fn, &data, 1);
-}
+ }
-int rd(char *fn, unsigned long *data)
-{
+ int rd(char *fn, unsigned long *data)
+ {
int fd;
char buf[MAX_BUF_SIZE];
*data=strtoul(buf, NULL, 16);
close(fd);
return 0;
-}
+ }
-int rd_status(char *path, int *status)
-{
+ int rd_status(char *path, int *status)
+ {
char fn[MAX_FN_SIZE];
sprintf(fn, "%s/status", path);
if (rd(fn, (u64*)status)<0) {
}
return 0;
-}
+ }
-int rd_capabilities(char *path, u64 *capabilities)
-{
+ int rd_capabilities(char *path, u64 *capabilities)
+ {
char fn[MAX_FN_SIZE];
sprintf(fn, "%s/capabilities", path);
if (rd(fn, capabilities)<0) {
}
return 0;
-}
+ }
-int rd_all(char *path)
-{
+ int rd_all(char *path)
+ {
unsigned long err_type_info, err_struct_info, err_data_buffer;
int status;
unsigned long capabilities, resources;
printf("resources=%lx\n", resources);
return 0;
-}
+ }
-int query_capabilities(char *path, err_type_info_t err_type_info,
+ int query_capabilities(char *path, err_type_info_t err_type_info,
u64 *capabilities)
-{
+ {
char fn[MAX_FN_SIZE];
err_struct_info_t err_struct_info;
err_data_buffer_t err_data_buffer;
return -1;
return 0;
-}
+ }
-int query_all_capabilities()
-{
+ int query_all_capabilities()
+ {
int status;
err_type_info_t err_type_info;
int err_sev, err_struct, struct_hier;
}
return 0;
-}
+ }
-int err_inject(int cpu, char *path, err_type_info_t err_type_info,
+ int err_inject(int cpu, char *path, err_type_info_t err_type_info,
err_struct_info_t err_struct_info,
err_data_buffer_t err_data_buffer)
-{
+ {
int status;
char fn[MAX_FN_SIZE];
}
return status;
-}
+ }
-static int construct_data_buf(char *path, err_type_info_t err_type_info,
+ static int construct_data_buf(char *path, err_type_info_t err_type_info,
err_struct_info_t err_struct_info,
err_data_buffer_t *err_data_buffer,
void *va1)
-{
+ {
char fn[MAX_FN_SIZE];
u64 virt_addr=0, phys_addr=0;
}
return 0;
-}
+ }
-typedef struct {
+ typedef struct {
u64 cpu;
u64 loop;
u64 interval;
u64 err_type_info;
u64 err_struct_info;
u64 err_data_buffer[ERR_DATA_BUFFER_SIZE];
-} parameters_t;
+ } parameters_t;
-parameters_t line_para;
-int para;
+ parameters_t line_para;
+ int para;
-static int empty_data_buffer(u64 *err_data_buffer)
-{
+ static int empty_data_buffer(u64 *err_data_buffer)
+ {
int empty=1;
int i;
empty=0;
return empty;
-}
+ }
-int err_inj()
-{
+ int err_inj()
+ {
err_type_info_t err_type_info;
err_struct_info_t err_struct_info;
err_data_buffer_t err_data_buffer;
printf("All done.\n");
return 0;
-}
+ }
-void help()
-{
+ void help()
+ {
printf("err_inject_tool:\n");
printf("\t-q: query all capabilities. default: off\n");
printf("\t-m: procedure mode. 1: physical 2: virtual. default: 1\n");
printf("The tool will take err.conf file as ");
printf("input to inject single or multiple errors ");
printf("on one or multiple cpus in parallel.\n");
-}
+ }
-int main(int argc, char **argv)
-{
+ int main(int argc, char **argv)
+ {
char c;
int do_err_inj=0;
int do_query_all=0;
if (count!=PARA_FIELD_NUM+3) {
line_para.err_data_buffer[0]=-1,
line_para.err_data_buffer[1]=-1,
- line_para.err_data_buffer[2]=-1;
+ line_para.err_data_buffer[2]=-1;
count=sscanf(optarg, "%lx, %lx, %lx, %lx, %lx\n",
&line_para.cpu,
&line_para.loop,
help();
return 0;
-}
-
+ }
--*-Mode: outline-*-
-
- Light-weight System Calls for IA-64
- -----------------------------------
+===================================
+Light-weight System Calls for IA-64
+===================================
Started: 13-Jan-2003
+
Last update: 27-Sep-2003
David Mosberger-Tang
care (see below).
-* How to tell fsys-mode
+How to tell fsys-mode
+=====================
Linux operates in fsys-mode when (a) the privilege level is 0 (most
privileged) and (b) the stacks have NOT been switched to kernel memory
yet. For convenience, the header file <asm-ia64/ptrace.h> provides
-three macros:
+three macros::
user_mode(regs)
user_stack(task,regs)
user_stack() returns TRUE if the state pointed to by "regs" was
executing on the user-level stack(s). Finally, fsys_mode() returns
TRUE if the CPU state pointed to by "regs" was executing in fsys-mode.
-The fsys_mode() macro is equivalent to the expression:
+The fsys_mode() macro is equivalent to the expression::
!user_mode(regs) && user_stack(task,regs)
-* How to write an fsyscall handler
+How to write an fsyscall handler
+================================
The file arch/ia64/kernel/fsys.S contains a table of fsyscall-handlers
(fsyscall_table). This table contains one entry for each system call.
The entry and exit-state of an fsyscall handler is as follows:
-** Machine state on entry to fsyscall handler:
-
- - r10 = 0
- - r11 = saved ar.pfs (a user-level value)
- - r15 = system call number
- - r16 = "current" task pointer (in normal kernel-mode, this is in r13)
- - r32-r39 = system call arguments
- - b6 = return address (a user-level value)
- - ar.pfs = previous frame-state (a user-level value)
- - PSR.be = cleared to zero (i.e., little-endian byte order is in effect)
- - all other registers may contain values passed in from user-mode
-
-** Required machine state on exit to fsyscall handler:
-
- - r11 = saved ar.pfs (as passed into the fsyscall handler)
- - r15 = system call number (as passed into the fsyscall handler)
- - r32-r39 = system call arguments (as passed into the fsyscall handler)
- - b6 = return address (as passed into the fsyscall handler)
- - ar.pfs = previous frame-state (as passed into the fsyscall handler)
+Machine state on entry to fsyscall handler
+------------------------------------------
+
+ ========= ===============================================================
+ r10 0
+ r11 saved ar.pfs (a user-level value)
+ r15 system call number
+ r16 "current" task pointer (in normal kernel-mode, this is in r13)
+ r32-r39 system call arguments
+ b6 return address (a user-level value)
+ ar.pfs previous frame-state (a user-level value)
+ PSR.be cleared to zero (i.e., little-endian byte order is in effect)
+ - all other registers may contain values passed in from user-mode
+ ========= ===============================================================
+
+Required machine state on exit to fsyscall handler
+--------------------------------------------------
+
+ ========= ===========================================================
+ r11 saved ar.pfs (as passed into the fsyscall handler)
+ r15 system call number (as passed into the fsyscall handler)
+ r32-r39 system call arguments (as passed into the fsyscall handler)
+ b6 return address (as passed into the fsyscall handler)
+ ar.pfs previous frame-state (as passed into the fsyscall handler)
+ ========= ===========================================================
Fsyscall handlers can execute with very little overhead, but with that
speed comes a set of restrictions:
- o Fsyscall-handlers MUST check for any pending work in the flags
+ * Fsyscall-handlers MUST check for any pending work in the flags
member of the thread-info structure and if any of the
TIF_ALLWORK_MASK flags are set, the handler needs to fall back on
doing a full system call (by calling fsys_fallback_syscall).
- o Fsyscall-handlers MUST preserve incoming arguments (r32-r39, r11,
+ * Fsyscall-handlers MUST preserve incoming arguments (r32-r39, r11,
r15, b6, and ar.pfs) because they will be needed in case of a
system call restart. Of course, all "preserved" registers also
must be preserved, in accordance to the normal calling conventions.
- o Fsyscall-handlers MUST check argument registers for containing a
+ * Fsyscall-handlers MUST check argument registers for containing a
NaT value before using them in any way that could trigger a
NaT-consumption fault. If a system call argument is found to
contain a NaT value, an fsyscall-handler may return immediately
with r8=EINVAL, r10=-1.
- o Fsyscall-handlers MUST NOT use the "alloc" instruction or perform
+ * Fsyscall-handlers MUST NOT use the "alloc" instruction or perform
any other operation that would trigger mandatory RSE
(register-stack engine) traffic.
- o Fsyscall-handlers MUST NOT write to any stacked registers because
+ * Fsyscall-handlers MUST NOT write to any stacked registers because
it is not safe to assume that user-level called a handler with the
proper number of arguments.
- o Fsyscall-handlers need to be careful when accessing per-CPU variables:
+ * Fsyscall-handlers need to be careful when accessing per-CPU variables:
unless proper safe-guards are taken (e.g., interruptions are avoided),
execution may be pre-empted and resumed on another CPU at any given
time.
- o Fsyscall-handlers must be careful not to leak sensitive kernel'
+ * Fsyscall-handlers must be careful not to leak sensitive kernel'
information back to user-level. In particular, before returning to
user-level, care needs to be taken to clear any scratch registers
that could contain sensitive information (note that the current
task pointer is not considered sensitive: it's already exposed
through ar.k6).
- o Fsyscall-handlers MUST NOT access user-memory without first
+ * Fsyscall-handlers MUST NOT access user-memory without first
validating access-permission (this can be done typically via
probe.r.fault and/or probe.w.fault) and without guarding against
memory access exceptions (this can be done with the EX() macros
semantics), but there is also a lot of flexibility in handling more
complicated cases.
-* Signal handling
+Signal handling
+===============
The delivery of (asynchronous) signals must be delayed until fsys-mode
is exited. This is accomplished with the help of the lower-privilege
occur. The trap handler clears PSR.lp again and returns immediately.
The kernel exit path then checks for and delivers any pending signals.
-* PSR Handling
+PSR Handling
+============
The "epc" instruction doesn't change the contents of PSR at all. This
is in contrast to a regular interruption, which clears almost all
work as expected. The following discussion describes how each PSR bit
is handled.
+======= =======================================================================
PSR.be Cleared when entering fsys-mode. A srlz.d instruction is used
to ensure the CPU is in little-endian mode before the first
load/store instruction is executed. PSR.be is normally NOT
PSR.di Unchanged.
PSR.si Unchanged.
PSR.db Unchanged. The kernel prevents user-level from setting a hardware
- breakpoint that triggers at any privilege level other than 3 (user-mode).
+ breakpoint that triggers at any privilege level other than
+ 3 (user-mode).
PSR.lp Unchanged.
PSR.tb Lazy redirect. If a taken-branch trap occurs while in
fsys-mode, the trap-handler modifies the saved machine state
PSR.bn Unchanged. Note: fsys-mode handlers may clear the bit, if needed.
Doing so requires clearing PSR.i and PSR.ic as well.
PSR.ia Unchanged. Note: the ia64 linux kernel never sets this bit.
+======= =======================================================================
-* Using fast system calls
+Using fast system calls
+=======================
To use fast system calls, userspace applications need simply call
__kernel_syscall_via_epc(). For example
-- example fgettimeofday() call --
+
-- fgettimeofday.S --
-#include <asm/asmmacro.h>
+::
+
+ #include <asm/asmmacro.h>
-GLOBAL_ENTRY(fgettimeofday)
-.prologue
-.save ar.pfs, r11
-mov r11 = ar.pfs
-.body
+ GLOBAL_ENTRY(fgettimeofday)
+ .prologue
+ .save ar.pfs, r11
+ mov r11 = ar.pfs
+ .body
-mov r2 = 0xa000000000020660;; // gate address
- // found by inspection of System.map for the
+ mov r2 = 0xa000000000020660;; // gate address
+ // found by inspection of System.map for the
// __kernel_syscall_via_epc() function. See
// below for how to do this for real.
-mov b7 = r2
-mov r15 = 1087 // gettimeofday syscall
-;;
-br.call.sptk.many b6 = b7
-;;
+ mov b7 = r2
+ mov r15 = 1087 // gettimeofday syscall
+ ;;
+ br.call.sptk.many b6 = b7
+ ;;
-.restore sp
+ .restore sp
-mov ar.pfs = r11
-br.ret.sptk.many rp;; // return to caller
-END(fgettimeofday)
+ mov ar.pfs = r11
+ br.ret.sptk.many rp;; // return to caller
+ END(fgettimeofday)
-- end fgettimeofday.S --
In reality, getting the gate address is accomplished by two extra
values passed via the ELF auxiliary vector (include/asm-ia64/elf.h)
- o AT_SYSINFO : is the address of __kernel_syscall_via_epc()
- o AT_SYSINFO_EHDR : is the address of the kernel gate ELF DSO
+ * AT_SYSINFO : is the address of __kernel_syscall_via_epc()
+ * AT_SYSINFO_EHDR : is the address of the kernel gate ELF DSO
The ELF DSO is a pre-linked library that is mapped in by the kernel at
the gate page. It is a proper ELF shared object so, with a dynamic
- Linux kernel release 2.4.xx for the IA-64 Platform
+===========================================
+Linux kernel release for the IA-64 Platform
+===========================================
- These are the release notes for Linux version 2.4 for IA-64
+ These are the release notes for Linux since version 2.4 for IA-64
platform. This document provides information specific to IA-64
ONLY, to get additional information about the Linux kernel also
read the original Linux README provided with the kernel.
-INSTALLING the kernel:
+Installing the Kernel
+=====================
- IA-64 kernel installation is the same as the other platforms, see
original README for details.
-SOFTWARE REQUIREMENTS
+Software Requirements
+=====================
Compiling and running this kernel requires an IA-64 compliant GCC
compiler. And various software packages also compiled with an
IA-64 compliant GCC compiler.
-CONFIGURING the kernel:
+Configuring the Kernel
+======================
Configuration is the same, see original README for details.
-COMPILING the kernel:
+Compiling the Kernel:
- Compiling this kernel doesn't differ from other platform so read
the original README for details BUT make sure you have an IA-64
compliant GCC compiler.
-IA-64 SPECIFICS
+IA-64 Specifics
+===============
- General issues:
- o Hardly any performance tuning has been done. Obvious targets
+ * Hardly any performance tuning has been done. Obvious targets
include the library routines (IP checksum, etc.). Less
obvious targets include making sure we don't flush the TLB
needlessly, etc.
- o SMP locks cleanup/optimization
+ * SMP locks cleanup/optimization
- o IA32 support. Currently experimental. It mostly works.
+ * IA32 support. Currently experimental. It mostly works.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+==================
+IA-64 Architecture
+==================
+
+.. toctree::
+ :maxdepth: 1
+
+ ia64
+ aliasing
+ efirtc
+ err_inject
+ fsys
+ irq-redir
+ mca
+ serial
+ xen
+==============================
IRQ affinity on IA64 platforms
-------------------------------
- 07.01.2002, Erich Focht <efocht@ess.nec.de>
+==============================
+
+07.01.2002, Erich Focht <efocht@ess.nec.de>
By writing to /proc/irq/IRQ#/smp_affinity the interrupt routing can be
CPUs. Only the first non-zero bit is taken into account.
-Usage examples:
+Usage examples
+==============
The target CPU has to be specified as a hexadecimal CPU mask. The
first non-zero bit is the selected CPU. This format has been kept for
compatibility reasons with i386.
Set the delivery mode of interrupt 41 to fixed and route the
-interrupts to CPU #3 (logical CPU number) (2^3=0x08):
+interrupts to CPU #3 (logical CPU number) (2^3=0x08)::
+
echo "8" >/proc/irq/41/smp_affinity
Set the default route for IRQ number 41 to CPU 6 in lowest priority
-delivery mode (redirectable):
+delivery mode (redirectable)::
+
echo "r 40" >/proc/irq/41/smp_affinity
-The output of the command
+The output of the command::
+
cat /proc/irq/IRQ#/smp_affinity
+
gives the target CPU mask for the specified interrupt vector. If the CPU
mask is preceded by the character "r", the interrupt is redirectable
(i.e. lowest priority mode routing is used), otherwise its route is
-Initialization and default behavior:
+Initialization and default behavior
+===================================
If the platform features IRQ redirection (info provided by SAL) all
IO-SAPIC interrupts are initialized with CPU#0 as their default target
fixed SAPIC mode with hint). The XTP chipset registers are used as hints
for the IRQ routing. Currently in Linux XTP registers can have three
values:
+
- minimal for an idle task,
- normal if any other task runs,
- maximal if the CPU is going to be switched off.
+
The IRQ is routed to the CPU with lowest XTP register value, the
search begins at the default CPU. Therefore most of the interrupts
will be handled by CPU #0.
If the platform doesn't feature interrupt redirection IOSAPIC fixed
routing is used. The target CPUs are distributed in a round robin
manner. IRQs will be routed only to the selected target CPUs. Check
-with
+with::
+
cat /proc/interrupts
-Comments:
+Comments
+========
On large (multi-node) systems it is recommended to route the IRQs to
the node to which the corresponding device is connected.
is because usually the chipsets on each node redirect the interrupts
only to their own CPUs (as they cannot see the XTP registers on the
other nodes).
-
-An ad-hoc collection of notes on IA64 MCA and INIT processing. Feel
-free to update it with notes about any area that is not clear.
+=============================================================
+An ad-hoc collection of notes on IA64 MCA and INIT processing
+=============================================================
+
+Feel free to update it with notes about any area that is not clear.
---
own stack as running on that cpu. Then a recursive error gets a
trace of the failing handler's "task".
-[1] My (Keith Owens) original design called for ia64 to separate its
+[1]
+ My (Keith Owens) original design called for ia64 to separate its
struct task and the kernel stacks. Then the MCA/INIT data would be
chained stacks like i386 interrupt stacks. But that required
radical surgery on the rest of ia64, plus extra hard wired TLB
-SERIAL DEVICE NAMING
+==============
+Serial Devices
+==============
+
+Serial Device Naming
+====================
As of 2.6.10, serial devices on ia64 are named based on the
order of ACPI and PCI enumeration. The first device in the
(described in the ACPI namespace) plus an MP[2] (a PCI device) has
these ports:
- pre-2.6.10 pre-2.6.10
- MMIO (EFI console (EFI console
- address on builtin) on MP port) 2.6.10
- ========== ========== ========== ======
+ ========== ========== ============ ============ =======
+ Type MMIO pre-2.6.10 pre-2.6.10 2.6.10+
+ address
+ (EFI console (EFI console
+ on builtin) on MP port)
+ ========== ========== ============ ============ =======
builtin 0xff5e0000 ttyS0 ttyS1 ttyS0
MP UPS 0xf8031000 ttyS1 ttyS2 ttyS1
MP Console 0xf8030000 ttyS2 ttyS0 ttyS2
MP 2 0xf8030010 ttyS3 ttyS3 ttyS3
MP 3 0xf8030038 ttyS4 ttyS4 ttyS4
+ ========== ========== ============ ============ =======
-CONSOLE SELECTION
+Console Selection
+=================
EFI knows what your console devices are, but it doesn't tell the
kernel quite enough to actually locate them. The DIG64 HCDP
entries in /etc/inittab (for getty) and /etc/securetty (to allow
root login).
-EARLY SERIAL CONSOLE
+Early Serial Console
+====================
The kernel can't start using a serial console until it knows where
the device lives. Normally this happens when the driver enumerates
or if the EFI console path contains only a UART device and the
firmware supplies an HCDP.
-TROUBLESHOOTING SERIAL CONSOLE PROBLEMS
+Troubleshooting Serial Console Problems
+=======================================
No kernel output after elilo prints "Uncompressing Linux... done":
-[1] http://www.dig64.org/specifications/agreement
+[1]
+ http://www.dig64.org/specifications/agreement
The table was originally defined as the "HCDP" for "Headless
Console/Debug Port." The current version is the "PCDP" for
"Primary Console and Debug Port Devices."
-[2] The HP MP (management processor) is a PCI device that provides
+[2]
+ The HP MP (management processor) is a PCI device that provides
several UARTs. One of the UARTs is often used as a console; the
EFI Boot Manager identifies it as "Acpi(HWP0002,700)/Pci(...)/Uart".
The external connection is usually a 25-pin connector, and a
special dongle converts that to three 9-pin connectors, one of
which is labelled "Console."
-[3] EFI console devices are configured using the EFI Boot Manager
+[3]
+ EFI console devices are configured using the EFI Boot Manager
"Boot option maintenance" menu. You may have to interrupt the
boot sequence to use this menu, and you will have to reset the
box after changing console configuration.
--- /dev/null
+********************************************************
+Recipe for getting/building/running Xen/ia64 with pv_ops
+********************************************************
+This recipe describes how to get xen-ia64 source and build it,
+and run domU with pv_ops.
+
+Requirements
+============
+
+ - python
+ - mercurial
+ it (aka "hg") is an open-source source code
+ management software. See the below.
+ http://www.selenic.com/mercurial/wiki/
+ - git
+ - bridge-utils
+
+Getting and Building Xen and Dom0
+=================================
+
+ My environment is:
+
+ - Machine : Tiger4
+ - Domain0 OS : RHEL5
+ - DomainU OS : RHEL5
+
+ 1. Download source::
+
+ # hg clone http://xenbits.xensource.com/ext/ia64/xen-unstable.hg
+ # cd xen-unstable.hg
+ # hg clone http://xenbits.xensource.com/ext/ia64/linux-2.6.18-xen.hg
+
+ 2. # make world
+
+ 3. # make install-tools
+
+ 4. copy kernels and xen::
+
+ # cp xen/xen.gz /boot/efi/efi/redhat/
+ # cp build-linux-2.6.18-xen_ia64/vmlinux.gz \
+ /boot/efi/efi/redhat/vmlinuz-2.6.18.8-xen
+
+ 5. make initrd for Dom0/DomU::
+
+ # make -C linux-2.6.18-xen.hg ARCH=ia64 modules_install \
+ O=$(pwd)/build-linux-2.6.18-xen_ia64
+ # mkinitrd -f /boot/efi/efi/redhat/initrd-2.6.18.8-xen.img \
+ 2.6.18.8-xen --builtin mptspi --builtin mptbase \
+ --builtin mptscsih --builtin uhci-hcd --builtin ohci-hcd \
+ --builtin ehci-hcd
+
+Making a disk image for guest OS
+================================
+
+ 1. make file::
+
+ # dd if=/dev/zero of=/root/rhel5.img bs=1M seek=4096 count=0
+ # mke2fs -F -j /root/rhel5.img
+ # mount -o loop /root/rhel5.img /mnt
+ # cp -ax /{dev,var,etc,usr,bin,sbin,lib} /mnt
+ # mkdir /mnt/{root,proc,sys,home,tmp}
+
+ Note: You may miss some device files. If so, please create them
+ with mknod. Or you can use tar instead of cp.
+
+ 2. modify DomU's fstab::
+
+ # vi /mnt/etc/fstab
+ /dev/xvda1 / ext3 defaults 1 1
+ none /dev/pts devpts gid=5,mode=620 0 0
+ none /dev/shm tmpfs defaults 0 0
+ none /proc proc defaults 0 0
+ none /sys sysfs defaults 0 0
+
+ 3. modify inittab
+
+ set runlevel to 3 to avoid X trying to start::
+
+ # vi /mnt/etc/inittab
+ id:3:initdefault:
+
+ Start a getty on the hvc0 console::
+
+ X0:2345:respawn:/sbin/mingetty hvc0
+
+ tty1-6 mingetty can be commented out
+
+ 4. add hvc0 into /etc/securetty::
+
+ # vi /mnt/etc/securetty (add hvc0)
+
+ 5. umount::
+
+ # umount /mnt
+
+FYI, virt-manager can also make a disk image for guest OS.
+It's GUI tools and easy to make it.
+
+Boot Xen & Domain0
+==================
+
+ 1. replace elilo
+ elilo of RHEL5 can boot Xen and Dom0.
+ If you use old elilo (e.g RHEL4), please download from the below
+ http://elilo.sourceforge.net/cgi-bin/blosxom
+ and copy into /boot/efi/efi/redhat/::
+
+ # cp elilo-3.6-ia64.efi /boot/efi/efi/redhat/elilo.efi
+
+ 2. modify elilo.conf (like the below)::
+
+ # vi /boot/efi/efi/redhat/elilo.conf
+ prompt
+ timeout=20
+ default=xen
+ relocatable
+
+ image=vmlinuz-2.6.18.8-xen
+ label=xen
+ vmm=xen.gz
+ initrd=initrd-2.6.18.8-xen.img
+ read-only
+ append=" -- rhgb root=/dev/sda2"
+
+The append options before "--" are for xen hypervisor,
+the options after "--" are for dom0.
+
+FYI, your machine may need console options like
+"com1=19200,8n1 console=vga,com1". For example,
+append="com1=19200,8n1 console=vga,com1 -- rhgb console=tty0 \
+console=ttyS0 root=/dev/sda2"
+
+Getting and Building domU with pv_ops
+=====================================
+
+ 1. get pv_ops tree::
+
+ # git clone http://people.valinux.co.jp/~yamahata/xen-ia64/linux-2.6-xen-ia64.git/
+
+ 2. git branch (if necessary)::
+
+ # cd linux-2.6-xen-ia64/
+ # git checkout -b your_branch origin/xen-ia64-domu-minimal-2008may19
+
+ Note:
+ The current branch is xen-ia64-domu-minimal-2008may19.
+ But you would find the new branch. You can see with
+ "git branch -r" to get the branch lists.
+
+ http://people.valinux.co.jp/~yamahata/xen-ia64/for_eagl/linux-2.6-ia64-pv-ops.git/
+
+ is also available.
+
+ The tree is based on
+
+ git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6 test)
+
+ 3. copy .config for pv_ops of domU::
+
+ # cp arch/ia64/configs/xen_domu_wip_defconfig .config
+
+ 4. make kernel with pv_ops::
+
+ # make oldconfig
+ # make
+
+ 5. install the kernel and initrd::
+
+ # cp vmlinux.gz /boot/efi/efi/redhat/vmlinuz-2.6-pv_ops-xenU
+ # make modules_install
+ # mkinitrd -f /boot/efi/efi/redhat/initrd-2.6-pv_ops-xenU.img \
+ 2.6.26-rc3xen-ia64-08941-g1b12161 --builtin mptspi \
+ --builtin mptbase --builtin mptscsih --builtin uhci-hcd \
+ --builtin ohci-hcd --builtin ehci-hcd
+
+Boot DomainU with pv_ops
+========================
+
+ 1. make config of DomU::
+
+ # vi /etc/xen/rhel5
+ kernel = "/boot/efi/efi/redhat/vmlinuz-2.6-pv_ops-xenU"
+ ramdisk = "/boot/efi/efi/redhat/initrd-2.6-pv_ops-xenU.img"
+ vcpus = 1
+ memory = 512
+ name = "rhel5"
+ disk = [ 'file:/root/rhel5.img,xvda1,w' ]
+ root = "/dev/xvda1 ro"
+ extra= "rhgb console=hvc0"
+
+ 2. After boot xen and dom0, start xend::
+
+ # /etc/init.d/xend start
+
+ ( In the debugging case, `# XEND_DEBUG=1 xend trace_start` )
+
+ 3. start domU::
+
+ # xm create -c rhel5
+
+Reference
+=========
+- Wiki of Xen/IA64 upstream merge
+ http://wiki.xensource.com/xenwiki/XenIA64/UpstreamMerge
+
+Written by Akio Takebe <takebe_akio@jp.fujitsu.com> on 28 May 2008
+++ /dev/null
- Recipe for getting/building/running Xen/ia64 with pv_ops
- --------------------------------------------------------
-
-This recipe describes how to get xen-ia64 source and build it,
-and run domU with pv_ops.
-
-============
-Requirements
-============
-
- - python
- - mercurial
- it (aka "hg") is an open-source source code
- management software. See the below.
- http://www.selenic.com/mercurial/wiki/
- - git
- - bridge-utils
-
-=================================
-Getting and Building Xen and Dom0
-=================================
-
- My environment is;
- Machine : Tiger4
- Domain0 OS : RHEL5
- DomainU OS : RHEL5
-
- 1. Download source
- # hg clone http://xenbits.xensource.com/ext/ia64/xen-unstable.hg
- # cd xen-unstable.hg
- # hg clone http://xenbits.xensource.com/ext/ia64/linux-2.6.18-xen.hg
-
- 2. # make world
-
- 3. # make install-tools
-
- 4. copy kernels and xen
- # cp xen/xen.gz /boot/efi/efi/redhat/
- # cp build-linux-2.6.18-xen_ia64/vmlinux.gz \
- /boot/efi/efi/redhat/vmlinuz-2.6.18.8-xen
-
- 5. make initrd for Dom0/DomU
- # make -C linux-2.6.18-xen.hg ARCH=ia64 modules_install \
- O=$(pwd)/build-linux-2.6.18-xen_ia64
- # mkinitrd -f /boot/efi/efi/redhat/initrd-2.6.18.8-xen.img \
- 2.6.18.8-xen --builtin mptspi --builtin mptbase \
- --builtin mptscsih --builtin uhci-hcd --builtin ohci-hcd \
- --builtin ehci-hcd
-
-================================
-Making a disk image for guest OS
-================================
-
- 1. make file
- # dd if=/dev/zero of=/root/rhel5.img bs=1M seek=4096 count=0
- # mke2fs -F -j /root/rhel5.img
- # mount -o loop /root/rhel5.img /mnt
- # cp -ax /{dev,var,etc,usr,bin,sbin,lib} /mnt
- # mkdir /mnt/{root,proc,sys,home,tmp}
-
- Note: You may miss some device files. If so, please create them
- with mknod. Or you can use tar instead of cp.
-
- 2. modify DomU's fstab
- # vi /mnt/etc/fstab
- /dev/xvda1 / ext3 defaults 1 1
- none /dev/pts devpts gid=5,mode=620 0 0
- none /dev/shm tmpfs defaults 0 0
- none /proc proc defaults 0 0
- none /sys sysfs defaults 0 0
-
- 3. modify inittab
- set runlevel to 3 to avoid X trying to start
- # vi /mnt/etc/inittab
- id:3:initdefault:
- Start a getty on the hvc0 console
- X0:2345:respawn:/sbin/mingetty hvc0
- tty1-6 mingetty can be commented out
-
- 4. add hvc0 into /etc/securetty
- # vi /mnt/etc/securetty (add hvc0)
-
- 5. umount
- # umount /mnt
-
-FYI, virt-manager can also make a disk image for guest OS.
-It's GUI tools and easy to make it.
-
-==================
-Boot Xen & Domain0
-==================
-
- 1. replace elilo
- elilo of RHEL5 can boot Xen and Dom0.
- If you use old elilo (e.g RHEL4), please download from the below
- http://elilo.sourceforge.net/cgi-bin/blosxom
- and copy into /boot/efi/efi/redhat/
- # cp elilo-3.6-ia64.efi /boot/efi/efi/redhat/elilo.efi
-
- 2. modify elilo.conf (like the below)
- # vi /boot/efi/efi/redhat/elilo.conf
- prompt
- timeout=20
- default=xen
- relocatable
-
- image=vmlinuz-2.6.18.8-xen
- label=xen
- vmm=xen.gz
- initrd=initrd-2.6.18.8-xen.img
- read-only
- append=" -- rhgb root=/dev/sda2"
-
-The append options before "--" are for xen hypervisor,
-the options after "--" are for dom0.
-
-FYI, your machine may need console options like
-"com1=19200,8n1 console=vga,com1". For example,
-append="com1=19200,8n1 console=vga,com1 -- rhgb console=tty0 \
-console=ttyS0 root=/dev/sda2"
-
-=====================================
-Getting and Building domU with pv_ops
-=====================================
-
- 1. get pv_ops tree
- # git clone http://people.valinux.co.jp/~yamahata/xen-ia64/linux-2.6-xen-ia64.git/
-
- 2. git branch (if necessary)
- # cd linux-2.6-xen-ia64/
- # git checkout -b your_branch origin/xen-ia64-domu-minimal-2008may19
- (Note: The current branch is xen-ia64-domu-minimal-2008may19.
- But you would find the new branch. You can see with
- "git branch -r" to get the branch lists.
- http://people.valinux.co.jp/~yamahata/xen-ia64/for_eagl/linux-2.6-ia64-pv-ops.git/
- is also available. The tree is based on
- git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6 test)
-
-
- 3. copy .config for pv_ops of domU
- # cp arch/ia64/configs/xen_domu_wip_defconfig .config
-
- 4. make kernel with pv_ops
- # make oldconfig
- # make
-
- 5. install the kernel and initrd
- # cp vmlinux.gz /boot/efi/efi/redhat/vmlinuz-2.6-pv_ops-xenU
- # make modules_install
- # mkinitrd -f /boot/efi/efi/redhat/initrd-2.6-pv_ops-xenU.img \
- 2.6.26-rc3xen-ia64-08941-g1b12161 --builtin mptspi \
- --builtin mptbase --builtin mptscsih --builtin uhci-hcd \
- --builtin ohci-hcd --builtin ehci-hcd
-
-========================
-Boot DomainU with pv_ops
-========================
-
- 1. make config of DomU
- # vi /etc/xen/rhel5
- kernel = "/boot/efi/efi/redhat/vmlinuz-2.6-pv_ops-xenU"
- ramdisk = "/boot/efi/efi/redhat/initrd-2.6-pv_ops-xenU.img"
- vcpus = 1
- memory = 512
- name = "rhel5"
- disk = [ 'file:/root/rhel5.img,xvda1,w' ]
- root = "/dev/xvda1 ro"
- extra= "rhgb console=hvc0"
-
- 2. After boot xen and dom0, start xend
- # /etc/init.d/xend start
- ( In the debugging case, # XEND_DEBUG=1 xend trace_start )
-
- 3. start domU
- # xm create -c rhel5
-
-=========
-Reference
-=========
-- Wiki of Xen/IA64 upstream merge
- http://wiki.xensource.com/xenwiki/XenIA64/UpstreamMerge
-
-Written by Akio Takebe <takebe_akio@jp.fujitsu.com> on 28 May 2008
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
==================================
Integrated Drive Electronics (IDE)
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
==============
Industrial I/O
+
.. The Linux Kernel documentation master file, created by
sphinx-quickstart on Fri Feb 12 13:51:46 2016.
You can adapt this file completely to your liking, but it should at least
:maxdepth: 2
admin-guide/index
+ kbuild/index
Firmware-related documentation
------------------------------
:maxdepth: 2
userspace-api/index
+ ioctl/index
Introduction to kernel development
kernel-hacking/index
trace/index
maintainer/index
+ fault-injection/index
+ livepatch/index
+
Kernel API documentation
------------------------
driver-api/index
core-api/index
+ locking/index
+ accounting/index
+ block/index
+ cdrom/index
+ ide/index
+ fb/index
+ fpga/index
+ hid/index
+ iio/index
infiniband/index
+ leds/index
media/index
+ netlabel/index
networking/index
+ pcmcia/index
+ target/index
+ timers/index
+ watchdog/index
input/index
hwmon/index
gpu/index
usb/index
PCI/index
misc-devices/index
+ mic/index
+ scheduler/index
Architecture-specific documentation
-----------------------------------
:maxdepth: 2
sh/index
+ arm/index
+ arm64/index
+ ia64/index
+ m68k/index
+ riscv/index
+ s390/index
+ sh/index
+ sparc/index
x86/index
+ xtensa/index
Filesystem Documentation
------------------------
+=================================
(How to avoid) Botching up ioctls
=================================
--- /dev/null
+============================
+Summary of CDROM ioctl calls
+============================
+
+- Edward A. Falk <efalk@google.com>
+
+November, 2004
+
+This document attempts to describe the ioctl(2) calls supported by
+the CDROM layer. These are by-and-large implemented (as of Linux 2.6)
+in drivers/cdrom/cdrom.c and drivers/block/scsi_ioctl.c
+
+ioctl values are listed in <linux/cdrom.h>. As of this writing, they
+are as follows:
+
+ ====================== ===============================================
+ CDROMPAUSE Pause Audio Operation
+ CDROMRESUME Resume paused Audio Operation
+ CDROMPLAYMSF Play Audio MSF (struct cdrom_msf)
+ CDROMPLAYTRKIND Play Audio Track/index (struct cdrom_ti)
+ CDROMREADTOCHDR Read TOC header (struct cdrom_tochdr)
+ CDROMREADTOCENTRY Read TOC entry (struct cdrom_tocentry)
+ CDROMSTOP Stop the cdrom drive
+ CDROMSTART Start the cdrom drive
+ CDROMEJECT Ejects the cdrom media
+ CDROMVOLCTRL Control output volume (struct cdrom_volctrl)
+ CDROMSUBCHNL Read subchannel data (struct cdrom_subchnl)
+ CDROMREADMODE2 Read CDROM mode 2 data (2336 Bytes)
+ (struct cdrom_read)
+ CDROMREADMODE1 Read CDROM mode 1 data (2048 Bytes)
+ (struct cdrom_read)
+ CDROMREADAUDIO (struct cdrom_read_audio)
+ CDROMEJECT_SW enable(1)/disable(0) auto-ejecting
+ CDROMMULTISESSION Obtain the start-of-last-session
+ address of multi session disks
+ (struct cdrom_multisession)
+ CDROM_GET_MCN Obtain the "Universal Product Code"
+ if available (struct cdrom_mcn)
+ CDROM_GET_UPC Deprecated, use CDROM_GET_MCN instead.
+ CDROMRESET hard-reset the drive
+ CDROMVOLREAD Get the drive's volume setting
+ (struct cdrom_volctrl)
+ CDROMREADRAW read data in raw mode (2352 Bytes)
+ (struct cdrom_read)
+ CDROMREADCOOKED read data in cooked mode
+ CDROMSEEK seek msf address
+ CDROMPLAYBLK scsi-cd only, (struct cdrom_blk)
+ CDROMREADALL read all 2646 bytes
+ CDROMGETSPINDOWN return 4-bit spindown value
+ CDROMSETSPINDOWN set 4-bit spindown value
+ CDROMCLOSETRAY pendant of CDROMEJECT
+ CDROM_SET_OPTIONS Set behavior options
+ CDROM_CLEAR_OPTIONS Clear behavior options
+ CDROM_SELECT_SPEED Set the CD-ROM speed
+ CDROM_SELECT_DISC Select disc (for juke-boxes)
+ CDROM_MEDIA_CHANGED Check is media changed
+ CDROM_DRIVE_STATUS Get tray position, etc.
+ CDROM_DISC_STATUS Get disc type, etc.
+ CDROM_CHANGER_NSLOTS Get number of slots
+ CDROM_LOCKDOOR lock or unlock door
+ CDROM_DEBUG Turn debug messages on/off
+ CDROM_GET_CAPABILITY get capabilities
+ CDROMAUDIOBUFSIZ set the audio buffer size
+ DVD_READ_STRUCT Read structure
+ DVD_WRITE_STRUCT Write structure
+ DVD_AUTH Authentication
+ CDROM_SEND_PACKET send a packet to the drive
+ CDROM_NEXT_WRITABLE get next writable block
+ CDROM_LAST_WRITTEN get last block written on disc
+ ====================== ===============================================
+
+
+The information that follows was determined from reading kernel source
+code. It is likely that some corrections will be made over time.
+
+------------------------------------------------------------------------------
+
+General:
+
+ Unless otherwise specified, all ioctl calls return 0 on success
+ and -1 with errno set to an appropriate value on error. (Some
+ ioctls return non-negative data values.)
+
+ Unless otherwise specified, all ioctl calls return -1 and set
+ errno to EFAULT on a failed attempt to copy data to or from user
+ address space.
+
+ Individual drivers may return error codes not listed here.
+
+ Unless otherwise specified, all data structures and constants
+ are defined in <linux/cdrom.h>
+
+------------------------------------------------------------------------------
+
+
+CDROMPAUSE
+ Pause Audio Operation
+
+
+ usage::
+
+ ioctl(fd, CDROMPAUSE, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+
+CDROMRESUME
+ Resume paused Audio Operation
+
+
+ usage::
+
+ ioctl(fd, CDROMRESUME, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+
+CDROMPLAYMSF
+ Play Audio MSF
+
+ (struct cdrom_msf)
+
+
+ usage::
+
+ struct cdrom_msf msf;
+
+ ioctl(fd, CDROMPLAYMSF, &msf);
+
+ inputs:
+ cdrom_msf structure, describing a segment of music to play
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+ notes:
+ - MSF stands for minutes-seconds-frames
+ - LBA stands for logical block address
+ - Segment is described as start and end times, where each time
+ is described as minutes:seconds:frames.
+ A frame is 1/75 of a second.
+
+
+CDROMPLAYTRKIND
+ Play Audio Track/index
+
+ (struct cdrom_ti)
+
+
+ usage::
+
+ struct cdrom_ti ti;
+
+ ioctl(fd, CDROMPLAYTRKIND, &ti);
+
+ inputs:
+ cdrom_ti structure, describing a segment of music to play
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+ notes:
+ - Segment is described as start and end times, where each time
+ is described as a track and an index.
+
+
+
+CDROMREADTOCHDR
+ Read TOC header
+
+ (struct cdrom_tochdr)
+
+
+ usage::
+
+ cdrom_tochdr header;
+
+ ioctl(fd, CDROMREADTOCHDR, &header);
+
+ inputs:
+ cdrom_tochdr structure
+
+
+ outputs:
+ cdrom_tochdr structure
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+
+
+CDROMREADTOCENTRY
+ Read TOC entry
+
+ (struct cdrom_tocentry)
+
+
+ usage::
+
+ struct cdrom_tocentry entry;
+
+ ioctl(fd, CDROMREADTOCENTRY, &entry);
+
+ inputs:
+ cdrom_tocentry structure
+
+
+ outputs:
+ cdrom_tocentry structure
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+ - EINVAL entry.cdte_format not CDROM_MSF or CDROM_LBA
+ - EINVAL requested track out of bounds
+ - EIO I/O error reading TOC
+
+ notes:
+ - TOC stands for Table Of Contents
+ - MSF stands for minutes-seconds-frames
+ - LBA stands for logical block address
+
+
+
+CDROMSTOP
+ Stop the cdrom drive
+
+
+ usage::
+
+ ioctl(fd, CDROMSTOP, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+ notes:
+ - Exact interpretation of this ioctl depends on the device,
+ but most seem to spin the drive down.
+
+
+CDROMSTART
+ Start the cdrom drive
+
+
+ usage::
+
+ ioctl(fd, CDROMSTART, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+ notes:
+ - Exact interpretation of this ioctl depends on the device,
+ but most seem to spin the drive up and/or close the tray.
+ Other devices ignore the ioctl completely.
+
+
+CDROMEJECT
+ - Ejects the cdrom media
+
+
+ usage::
+
+ ioctl(fd, CDROMEJECT, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error returns:
+ - ENOSYS cd drive not capable of ejecting
+ - EBUSY other processes are accessing drive, or door is locked
+
+ notes:
+ - See CDROM_LOCKDOOR, below.
+
+
+
+
+CDROMCLOSETRAY
+ pendant of CDROMEJECT
+
+
+ usage::
+
+ ioctl(fd, CDROMCLOSETRAY, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error returns:
+ - ENOSYS cd drive not capable of closing the tray
+ - EBUSY other processes are accessing drive, or door is locked
+
+ notes:
+ - See CDROM_LOCKDOOR, below.
+
+
+
+
+CDROMVOLCTRL
+ Control output volume (struct cdrom_volctrl)
+
+
+ usage::
+
+ struct cdrom_volctrl volume;
+
+ ioctl(fd, CDROMVOLCTRL, &volume);
+
+ inputs:
+ cdrom_volctrl structure containing volumes for up to 4
+ channels.
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+
+
+CDROMVOLREAD
+ Get the drive's volume setting
+
+ (struct cdrom_volctrl)
+
+
+ usage::
+
+ struct cdrom_volctrl volume;
+
+ ioctl(fd, CDROMVOLREAD, &volume);
+
+ inputs:
+ none
+
+
+ outputs:
+ The current volume settings.
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+
+
+
+CDROMSUBCHNL
+ Read subchannel data
+
+ (struct cdrom_subchnl)
+
+
+ usage::
+
+ struct cdrom_subchnl q;
+
+ ioctl(fd, CDROMSUBCHNL, &q);
+
+ inputs:
+ cdrom_subchnl structure
+
+
+ outputs:
+ cdrom_subchnl structure
+
+
+ error return:
+ - ENOSYS cd drive not audio-capable.
+ - EINVAL format not CDROM_MSF or CDROM_LBA
+
+ notes:
+ - Format is converted to CDROM_MSF or CDROM_LBA
+ as per user request on return
+
+
+
+CDROMREADRAW
+ read data in raw mode (2352 Bytes)
+
+ (struct cdrom_read)
+
+ usage::
+
+ union {
+
+ struct cdrom_msf msf; /* input */
+ char buffer[CD_FRAMESIZE_RAW]; /* return */
+ } arg;
+ ioctl(fd, CDROMREADRAW, &arg);
+
+ inputs:
+ cdrom_msf structure indicating an address to read.
+
+ Only the start values are significant.
+
+ outputs:
+ Data written to address provided by user.
+
+
+ error return:
+ - EINVAL address less than 0, or msf less than 0:2:0
+ - ENOMEM out of memory
+
+ notes:
+ - As of 2.6.8.1, comments in <linux/cdrom.h> indicate that this
+ ioctl accepts a cdrom_read structure, but actual source code
+ reads a cdrom_msf structure and writes a buffer of data to
+ the same address.
+
+ - MSF values are converted to LBA values via this formula::
+
+ lba = (((m * CD_SECS) + s) * CD_FRAMES + f) - CD_MSF_OFFSET;
+
+
+
+
+CDROMREADMODE1
+ Read CDROM mode 1 data (2048 Bytes)
+
+ (struct cdrom_read)
+
+ notes:
+ Identical to CDROMREADRAW except that block size is
+ CD_FRAMESIZE (2048) bytes
+
+
+
+CDROMREADMODE2
+ Read CDROM mode 2 data (2336 Bytes)
+
+ (struct cdrom_read)
+
+ notes:
+ Identical to CDROMREADRAW except that block size is
+ CD_FRAMESIZE_RAW0 (2336) bytes
+
+
+
+CDROMREADAUDIO
+ (struct cdrom_read_audio)
+
+ usage::
+
+ struct cdrom_read_audio ra;
+
+ ioctl(fd, CDROMREADAUDIO, &ra);
+
+ inputs:
+ cdrom_read_audio structure containing read start
+ point and length
+
+ outputs:
+ audio data, returned to buffer indicated by ra
+
+
+ error return:
+ - EINVAL format not CDROM_MSF or CDROM_LBA
+ - EINVAL nframes not in range [1 75]
+ - ENXIO drive has no queue (probably means invalid fd)
+ - ENOMEM out of memory
+
+
+CDROMEJECT_SW
+ enable(1)/disable(0) auto-ejecting
+
+
+ usage::
+
+ int val;
+
+ ioctl(fd, CDROMEJECT_SW, val);
+
+ inputs:
+ Flag specifying auto-eject flag.
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS Drive is not capable of ejecting.
+ - EBUSY Door is locked
+
+
+
+
+CDROMMULTISESSION
+ Obtain the start-of-last-session address of multi session disks
+
+ (struct cdrom_multisession)
+
+ usage::
+
+ struct cdrom_multisession ms_info;
+
+ ioctl(fd, CDROMMULTISESSION, &ms_info);
+
+ inputs:
+ cdrom_multisession structure containing desired
+
+ format.
+
+ outputs:
+ cdrom_multisession structure is filled with last_session
+ information.
+
+ error return:
+ - EINVAL format not CDROM_MSF or CDROM_LBA
+
+
+CDROM_GET_MCN
+ Obtain the "Universal Product Code"
+ if available
+
+ (struct cdrom_mcn)
+
+
+ usage::
+
+ struct cdrom_mcn mcn;
+
+ ioctl(fd, CDROM_GET_MCN, &mcn);
+
+ inputs:
+ none
+
+
+ outputs:
+ Universal Product Code
+
+
+ error return:
+ - ENOSYS Drive is not capable of reading MCN data.
+
+ notes:
+ - Source code comments state::
+
+ The following function is implemented, although very few
+ audio discs give Universal Product Code information, which
+ should just be the Medium Catalog Number on the box. Note,
+ that the way the code is written on the CD is /not/ uniform
+ across all discs!
+
+
+
+
+CDROM_GET_UPC
+ CDROM_GET_MCN (deprecated)
+
+
+ Not implemented, as of 2.6.8.1
+
+
+
+CDROMRESET
+ hard-reset the drive
+
+
+ usage::
+
+ ioctl(fd, CDROMRESET, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ none
+
+
+ error return:
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - ENOSYS Drive is not capable of resetting.
+
+
+
+
+CDROMREADCOOKED
+ read data in cooked mode
+
+
+ usage::
+
+ u8 buffer[CD_FRAMESIZE]
+
+ ioctl(fd, CDROMREADCOOKED, buffer);
+
+ inputs:
+ none
+
+
+ outputs:
+ 2048 bytes of data, "cooked" mode.
+
+
+ notes:
+ Not implemented on all drives.
+
+
+
+
+
+CDROMREADALL
+ read all 2646 bytes
+
+
+ Same as CDROMREADCOOKED, but reads 2646 bytes.
+
+
+
+CDROMSEEK
+ seek msf address
+
+
+ usage::
+
+ struct cdrom_msf msf;
+
+ ioctl(fd, CDROMSEEK, &msf);
+
+ inputs:
+ MSF address to seek to.
+
+
+ outputs:
+ none
+
+
+
+
+CDROMPLAYBLK
+ scsi-cd only
+
+ (struct cdrom_blk)
+
+
+ usage::
+
+ struct cdrom_blk blk;
+
+ ioctl(fd, CDROMPLAYBLK, &blk);
+
+ inputs:
+ Region to play
+
+
+ outputs:
+ none
+
+
+
+
+CDROMGETSPINDOWN
+ usage::
+
+ char spindown;
+
+ ioctl(fd, CDROMGETSPINDOWN, &spindown);
+
+ inputs:
+ none
+
+
+ outputs:
+ The value of the current 4-bit spindown value.
+
+
+
+
+
+CDROMSETSPINDOWN
+ usage::
+
+ char spindown
+
+ ioctl(fd, CDROMSETSPINDOWN, &spindown);
+
+ inputs:
+ 4-bit value used to control spindown (TODO: more detail here)
+
+
+ outputs:
+ none
+
+
+
+
+
+
+CDROM_SET_OPTIONS
+ Set behavior options
+
+
+ usage::
+
+ int options;
+
+ ioctl(fd, CDROM_SET_OPTIONS, options);
+
+ inputs:
+ New values for drive options. The logical 'or' of:
+
+ ============== ==================================
+ CDO_AUTO_CLOSE close tray on first open(2)
+ CDO_AUTO_EJECT open tray on last release
+ CDO_USE_FFLAGS use O_NONBLOCK information on open
+ CDO_LOCK lock tray on open files
+ CDO_CHECK_TYPE check type on open for data
+ ============== ==================================
+
+ outputs:
+ Returns the resulting options settings in the
+ ioctl return value. Returns -1 on error.
+
+ error return:
+ - ENOSYS selected option(s) not supported by drive.
+
+
+
+
+CDROM_CLEAR_OPTIONS
+ Clear behavior options
+
+
+ Same as CDROM_SET_OPTIONS, except that selected options are
+ turned off.
+
+
+
+CDROM_SELECT_SPEED
+ Set the CD-ROM speed
+
+
+ usage::
+
+ int speed;
+
+ ioctl(fd, CDROM_SELECT_SPEED, speed);
+
+ inputs:
+ New drive speed.
+
+
+ outputs:
+ none
+
+
+ error return:
+ - ENOSYS speed selection not supported by drive.
+
+
+
+CDROM_SELECT_DISC
+ Select disc (for juke-boxes)
+
+
+ usage::
+
+ int disk;
+
+ ioctl(fd, CDROM_SELECT_DISC, disk);
+
+ inputs:
+ Disk to load into drive.
+
+
+ outputs:
+ none
+
+
+ error return:
+ - EINVAL Disk number beyond capacity of drive
+
+
+
+CDROM_MEDIA_CHANGED
+ Check is media changed
+
+
+ usage::
+
+ int slot;
+
+ ioctl(fd, CDROM_MEDIA_CHANGED, slot);
+
+ inputs:
+ Slot number to be tested, always zero except for jukeboxes.
+
+ May also be special values CDSL_NONE or CDSL_CURRENT
+
+ outputs:
+ Ioctl return value is 0 or 1 depending on whether the media
+
+ has been changed, or -1 on error.
+
+ error returns:
+ - ENOSYS Drive can't detect media change
+ - EINVAL Slot number beyond capacity of drive
+ - ENOMEM Out of memory
+
+
+
+CDROM_DRIVE_STATUS
+ Get tray position, etc.
+
+
+ usage::
+
+ int slot;
+
+ ioctl(fd, CDROM_DRIVE_STATUS, slot);
+
+ inputs:
+ Slot number to be tested, always zero except for jukeboxes.
+
+ May also be special values CDSL_NONE or CDSL_CURRENT
+
+ outputs:
+ Ioctl return value will be one of the following values
+
+ from <linux/cdrom.h>:
+
+ =================== ==========================
+ CDS_NO_INFO Information not available.
+ CDS_NO_DISC
+ CDS_TRAY_OPEN
+ CDS_DRIVE_NOT_READY
+ CDS_DISC_OK
+ -1 error
+ =================== ==========================
+
+ error returns:
+ - ENOSYS Drive can't detect drive status
+ - EINVAL Slot number beyond capacity of drive
+ - ENOMEM Out of memory
+
+
+
+
+CDROM_DISC_STATUS
+ Get disc type, etc.
+
+
+ usage::
+
+ ioctl(fd, CDROM_DISC_STATUS, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ Ioctl return value will be one of the following values
+
+ from <linux/cdrom.h>:
+
+ - CDS_NO_INFO
+ - CDS_AUDIO
+ - CDS_MIXED
+ - CDS_XA_2_2
+ - CDS_XA_2_1
+ - CDS_DATA_1
+
+ error returns:
+ none at present
+
+ notes:
+ - Source code comments state::
+
+
+ Ok, this is where problems start. The current interface for
+ the CDROM_DISC_STATUS ioctl is flawed. It makes the false
+ assumption that CDs are all CDS_DATA_1 or all CDS_AUDIO, etc.
+ Unfortunately, while this is often the case, it is also
+ very common for CDs to have some tracks with data, and some
+ tracks with audio. Just because I feel like it, I declare
+ the following to be the best way to cope. If the CD has
+ ANY data tracks on it, it will be returned as a data CD.
+ If it has any XA tracks, I will return it as that. Now I
+ could simplify this interface by combining these returns with
+ the above, but this more clearly demonstrates the problem
+ with the current interface. Too bad this wasn't designed
+ to use bitmasks... -Erik
+
+ Well, now we have the option CDS_MIXED: a mixed-type CD.
+ User level programmers might feel the ioctl is not very
+ useful.
+ ---david
+
+
+
+
+CDROM_CHANGER_NSLOTS
+ Get number of slots
+
+
+ usage::
+
+ ioctl(fd, CDROM_CHANGER_NSLOTS, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ The ioctl return value will be the number of slots in a
+ CD changer. Typically 1 for non-multi-disk devices.
+
+ error returns:
+ none
+
+
+
+CDROM_LOCKDOOR
+ lock or unlock door
+
+
+ usage::
+
+ int lock;
+
+ ioctl(fd, CDROM_LOCKDOOR, lock);
+
+ inputs:
+ Door lock flag, 1=lock, 0=unlock
+
+
+ outputs:
+ none
+
+
+ error returns:
+ - EDRIVE_CANT_DO_THIS
+
+ Door lock function not supported.
+ - EBUSY
+
+ Attempt to unlock when multiple users
+ have the drive open and not CAP_SYS_ADMIN
+
+ notes:
+ As of 2.6.8.1, the lock flag is a global lock, meaning that
+ all CD drives will be locked or unlocked together. This is
+ probably a bug.
+
+ The EDRIVE_CANT_DO_THIS value is defined in <linux/cdrom.h>
+ and is currently (2.6.8.1) the same as EOPNOTSUPP
+
+
+
+CDROM_DEBUG
+ Turn debug messages on/off
+
+
+ usage::
+
+ int debug;
+
+ ioctl(fd, CDROM_DEBUG, debug);
+
+ inputs:
+ Cdrom debug flag, 0=disable, 1=enable
+
+
+ outputs:
+ The ioctl return value will be the new debug flag.
+
+
+ error return:
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+
+
+
+CDROM_GET_CAPABILITY
+ get capabilities
+
+
+ usage::
+
+ ioctl(fd, CDROM_GET_CAPABILITY, 0);
+
+
+ inputs:
+ none
+
+
+ outputs:
+ The ioctl return value is the current device capability
+ flags. See CDC_CLOSE_TRAY, CDC_OPEN_TRAY, etc.
+
+
+
+CDROMAUDIOBUFSIZ
+ set the audio buffer size
+
+
+ usage::
+
+ int arg;
+
+ ioctl(fd, CDROMAUDIOBUFSIZ, val);
+
+ inputs:
+ New audio buffer size
+
+
+ outputs:
+ The ioctl return value is the new audio buffer size, or -1
+ on error.
+
+ error return:
+ - ENOSYS Not supported by this driver.
+
+ notes:
+ Not supported by all drivers.
+
+
+
+
+DVD_READ_STRUCT Read structure
+
+ usage::
+
+ dvd_struct s;
+
+ ioctl(fd, DVD_READ_STRUCT, &s);
+
+ inputs:
+ dvd_struct structure, containing:
+
+ =================== ==========================================
+ type specifies the information desired, one of
+ DVD_STRUCT_PHYSICAL, DVD_STRUCT_COPYRIGHT,
+ DVD_STRUCT_DISCKEY, DVD_STRUCT_BCA,
+ DVD_STRUCT_MANUFACT
+ physical.layer_num desired layer, indexed from 0
+ copyright.layer_num desired layer, indexed from 0
+ disckey.agid
+ =================== ==========================================
+
+ outputs:
+ dvd_struct structure, containing:
+
+ =================== ================================
+ physical for type == DVD_STRUCT_PHYSICAL
+ copyright for type == DVD_STRUCT_COPYRIGHT
+ disckey.value for type == DVD_STRUCT_DISCKEY
+ bca.{len,value} for type == DVD_STRUCT_BCA
+ manufact.{len,valu} for type == DVD_STRUCT_MANUFACT
+ =================== ================================
+
+ error returns:
+ - EINVAL physical.layer_num exceeds number of layers
+ - EIO Received invalid response from drive
+
+
+
+DVD_WRITE_STRUCT Write structure
+
+ Not implemented, as of 2.6.8.1
+
+
+
+DVD_AUTH Authentication
+
+ usage::
+
+ dvd_authinfo ai;
+
+ ioctl(fd, DVD_AUTH, &ai);
+
+ inputs:
+ dvd_authinfo structure. See <linux/cdrom.h>
+
+
+ outputs:
+ dvd_authinfo structure.
+
+
+ error return:
+ - ENOTTY ai.type not recognized.
+
+
+
+CDROM_SEND_PACKET
+ send a packet to the drive
+
+
+ usage::
+
+ struct cdrom_generic_command cgc;
+
+ ioctl(fd, CDROM_SEND_PACKET, &cgc);
+
+ inputs:
+ cdrom_generic_command structure containing the packet to send.
+
+
+ outputs:
+ none
+
+ cdrom_generic_command structure containing results.
+
+ error return:
+ - EIO
+
+ command failed.
+ - EPERM
+
+ Operation not permitted, either because a
+ write command was attempted on a drive which
+ is opened read-only, or because the command
+ requires CAP_SYS_RAWIO
+ - EINVAL
+
+ cgc.data_direction not set
+
+
+
+CDROM_NEXT_WRITABLE
+ get next writable block
+
+
+ usage::
+
+ long next;
+
+ ioctl(fd, CDROM_NEXT_WRITABLE, &next);
+
+ inputs:
+ none
+
+
+ outputs:
+ The next writable block.
+
+
+ notes:
+ If the device does not support this ioctl directly, the
+
+ ioctl will return CDROM_LAST_WRITTEN + 7.
+
+
+
+CDROM_LAST_WRITTEN
+ get last block written on disc
+
+
+ usage::
+
+ long last;
+
+ ioctl(fd, CDROM_LAST_WRITTEN, &last);
+
+ inputs:
+ none
+
+
+ outputs:
+ The last block written on disc
+
+
+ notes:
+ If the device does not support this ioctl directly, the
+ result is derived from the disc's table of contents. If the
+ table of contents can't be read, this ioctl returns an
+ error.
+++ /dev/null
- Summary of CDROM ioctl calls.
- ============================
-
- Edward A. Falk <efalk@google.com>
-
- November, 2004
-
-This document attempts to describe the ioctl(2) calls supported by
-the CDROM layer. These are by-and-large implemented (as of Linux 2.6)
-in drivers/cdrom/cdrom.c and drivers/block/scsi_ioctl.c
-
-ioctl values are listed in <linux/cdrom.h>. As of this writing, they
-are as follows:
-
- CDROMPAUSE Pause Audio Operation
- CDROMRESUME Resume paused Audio Operation
- CDROMPLAYMSF Play Audio MSF (struct cdrom_msf)
- CDROMPLAYTRKIND Play Audio Track/index (struct cdrom_ti)
- CDROMREADTOCHDR Read TOC header (struct cdrom_tochdr)
- CDROMREADTOCENTRY Read TOC entry (struct cdrom_tocentry)
- CDROMSTOP Stop the cdrom drive
- CDROMSTART Start the cdrom drive
- CDROMEJECT Ejects the cdrom media
- CDROMVOLCTRL Control output volume (struct cdrom_volctrl)
- CDROMSUBCHNL Read subchannel data (struct cdrom_subchnl)
- CDROMREADMODE2 Read CDROM mode 2 data (2336 Bytes)
- (struct cdrom_read)
- CDROMREADMODE1 Read CDROM mode 1 data (2048 Bytes)
- (struct cdrom_read)
- CDROMREADAUDIO (struct cdrom_read_audio)
- CDROMEJECT_SW enable(1)/disable(0) auto-ejecting
- CDROMMULTISESSION Obtain the start-of-last-session
- address of multi session disks
- (struct cdrom_multisession)
- CDROM_GET_MCN Obtain the "Universal Product Code"
- if available (struct cdrom_mcn)
- CDROM_GET_UPC Deprecated, use CDROM_GET_MCN instead.
- CDROMRESET hard-reset the drive
- CDROMVOLREAD Get the drive's volume setting
- (struct cdrom_volctrl)
- CDROMREADRAW read data in raw mode (2352 Bytes)
- (struct cdrom_read)
- CDROMREADCOOKED read data in cooked mode
- CDROMSEEK seek msf address
- CDROMPLAYBLK scsi-cd only, (struct cdrom_blk)
- CDROMREADALL read all 2646 bytes
- CDROMGETSPINDOWN return 4-bit spindown value
- CDROMSETSPINDOWN set 4-bit spindown value
- CDROMCLOSETRAY pendant of CDROMEJECT
- CDROM_SET_OPTIONS Set behavior options
- CDROM_CLEAR_OPTIONS Clear behavior options
- CDROM_SELECT_SPEED Set the CD-ROM speed
- CDROM_SELECT_DISC Select disc (for juke-boxes)
- CDROM_MEDIA_CHANGED Check is media changed
- CDROM_DRIVE_STATUS Get tray position, etc.
- CDROM_DISC_STATUS Get disc type, etc.
- CDROM_CHANGER_NSLOTS Get number of slots
- CDROM_LOCKDOOR lock or unlock door
- CDROM_DEBUG Turn debug messages on/off
- CDROM_GET_CAPABILITY get capabilities
- CDROMAUDIOBUFSIZ set the audio buffer size
- DVD_READ_STRUCT Read structure
- DVD_WRITE_STRUCT Write structure
- DVD_AUTH Authentication
- CDROM_SEND_PACKET send a packet to the drive
- CDROM_NEXT_WRITABLE get next writable block
- CDROM_LAST_WRITTEN get last block written on disc
-
-
-The information that follows was determined from reading kernel source
-code. It is likely that some corrections will be made over time.
-
-
-
-
-
-
-
-General:
-
- Unless otherwise specified, all ioctl calls return 0 on success
- and -1 with errno set to an appropriate value on error. (Some
- ioctls return non-negative data values.)
-
- Unless otherwise specified, all ioctl calls return -1 and set
- errno to EFAULT on a failed attempt to copy data to or from user
- address space.
-
- Individual drivers may return error codes not listed here.
-
- Unless otherwise specified, all data structures and constants
- are defined in <linux/cdrom.h>
-
-
-
-
-CDROMPAUSE Pause Audio Operation
-
- usage:
-
- ioctl(fd, CDROMPAUSE, 0);
-
- inputs: none
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
-
-CDROMRESUME Resume paused Audio Operation
-
- usage:
-
- ioctl(fd, CDROMRESUME, 0);
-
- inputs: none
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
-
-CDROMPLAYMSF Play Audio MSF (struct cdrom_msf)
-
- usage:
-
- struct cdrom_msf msf;
- ioctl(fd, CDROMPLAYMSF, &msf);
-
- inputs:
- cdrom_msf structure, describing a segment of music to play
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
- notes:
- MSF stands for minutes-seconds-frames
- LBA stands for logical block address
-
- Segment is described as start and end times, where each time
- is described as minutes:seconds:frames. A frame is 1/75 of
- a second.
-
-
-CDROMPLAYTRKIND Play Audio Track/index (struct cdrom_ti)
-
- usage:
-
- struct cdrom_ti ti;
- ioctl(fd, CDROMPLAYTRKIND, &ti);
-
- inputs:
- cdrom_ti structure, describing a segment of music to play
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
- notes:
- Segment is described as start and end times, where each time
- is described as a track and an index.
-
-
-
-CDROMREADTOCHDR Read TOC header (struct cdrom_tochdr)
-
- usage:
-
- cdrom_tochdr header;
- ioctl(fd, CDROMREADTOCHDR, &header);
-
- inputs:
- cdrom_tochdr structure
-
- outputs:
- cdrom_tochdr structure
-
- error return:
- ENOSYS cd drive not audio-capable.
-
-
-
-CDROMREADTOCENTRY Read TOC entry (struct cdrom_tocentry)
-
- usage:
-
- struct cdrom_tocentry entry;
- ioctl(fd, CDROMREADTOCENTRY, &entry);
-
- inputs:
- cdrom_tocentry structure
-
- outputs:
- cdrom_tocentry structure
-
- error return:
- ENOSYS cd drive not audio-capable.
- EINVAL entry.cdte_format not CDROM_MSF or CDROM_LBA
- EINVAL requested track out of bounds
- EIO I/O error reading TOC
-
- notes:
- TOC stands for Table Of Contents
- MSF stands for minutes-seconds-frames
- LBA stands for logical block address
-
-
-
-CDROMSTOP Stop the cdrom drive
-
- usage:
-
- ioctl(fd, CDROMSTOP, 0);
-
- inputs: none
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
- notes:
- Exact interpretation of this ioctl depends on the device,
- but most seem to spin the drive down.
-
-
-CDROMSTART Start the cdrom drive
-
- usage:
-
- ioctl(fd, CDROMSTART, 0);
-
- inputs: none
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
- notes:
- Exact interpretation of this ioctl depends on the device,
- but most seem to spin the drive up and/or close the tray.
- Other devices ignore the ioctl completely.
-
-
-CDROMEJECT Ejects the cdrom media
-
- usage:
-
- ioctl(fd, CDROMEJECT, 0);
-
- inputs: none
-
- outputs: none
-
- error returns:
- ENOSYS cd drive not capable of ejecting
- EBUSY other processes are accessing drive, or door is locked
-
- notes:
- See CDROM_LOCKDOOR, below.
-
-
-
-CDROMCLOSETRAY pendant of CDROMEJECT
-
- usage:
-
- ioctl(fd, CDROMCLOSETRAY, 0);
-
- inputs: none
-
- outputs: none
-
- error returns:
- ENOSYS cd drive not capable of closing the tray
- EBUSY other processes are accessing drive, or door is locked
-
- notes:
- See CDROM_LOCKDOOR, below.
-
-
-
-CDROMVOLCTRL Control output volume (struct cdrom_volctrl)
-
- usage:
-
- struct cdrom_volctrl volume;
- ioctl(fd, CDROMVOLCTRL, &volume);
-
- inputs:
- cdrom_volctrl structure containing volumes for up to 4
- channels.
-
- outputs: none
-
- error return:
- ENOSYS cd drive not audio-capable.
-
-
-
-CDROMVOLREAD Get the drive's volume setting
- (struct cdrom_volctrl)
-
- usage:
-
- struct cdrom_volctrl volume;
- ioctl(fd, CDROMVOLREAD, &volume);
-
- inputs: none
-
- outputs:
- The current volume settings.
-
- error return:
- ENOSYS cd drive not audio-capable.
-
-
-
-CDROMSUBCHNL Read subchannel data (struct cdrom_subchnl)
-
- usage:
-
- struct cdrom_subchnl q;
- ioctl(fd, CDROMSUBCHNL, &q);
-
- inputs:
- cdrom_subchnl structure
-
- outputs:
- cdrom_subchnl structure
-
- error return:
- ENOSYS cd drive not audio-capable.
- EINVAL format not CDROM_MSF or CDROM_LBA
-
- notes:
- Format is converted to CDROM_MSF or CDROM_LBA
- as per user request on return
-
-
-
-CDROMREADRAW read data in raw mode (2352 Bytes)
- (struct cdrom_read)
-
- usage:
-
- union {
- struct cdrom_msf msf; /* input */
- char buffer[CD_FRAMESIZE_RAW]; /* return */
- } arg;
- ioctl(fd, CDROMREADRAW, &arg);
-
- inputs:
- cdrom_msf structure indicating an address to read.
- Only the start values are significant.
-
- outputs:
- Data written to address provided by user.
-
- error return:
- EINVAL address less than 0, or msf less than 0:2:0
- ENOMEM out of memory
-
- notes:
- As of 2.6.8.1, comments in <linux/cdrom.h> indicate that this
- ioctl accepts a cdrom_read structure, but actual source code
- reads a cdrom_msf structure and writes a buffer of data to
- the same address.
-
- MSF values are converted to LBA values via this formula:
-
- lba = (((m * CD_SECS) + s) * CD_FRAMES + f) - CD_MSF_OFFSET;
-
-
-
-
-CDROMREADMODE1 Read CDROM mode 1 data (2048 Bytes)
- (struct cdrom_read)
-
- notes:
- Identical to CDROMREADRAW except that block size is
- CD_FRAMESIZE (2048) bytes
-
-
-
-CDROMREADMODE2 Read CDROM mode 2 data (2336 Bytes)
- (struct cdrom_read)
-
- notes:
- Identical to CDROMREADRAW except that block size is
- CD_FRAMESIZE_RAW0 (2336) bytes
-
-
-
-CDROMREADAUDIO (struct cdrom_read_audio)
-
- usage:
-
- struct cdrom_read_audio ra;
- ioctl(fd, CDROMREADAUDIO, &ra);
-
- inputs:
- cdrom_read_audio structure containing read start
- point and length
-
- outputs:
- audio data, returned to buffer indicated by ra
-
- error return:
- EINVAL format not CDROM_MSF or CDROM_LBA
- EINVAL nframes not in range [1 75]
- ENXIO drive has no queue (probably means invalid fd)
- ENOMEM out of memory
-
-
-CDROMEJECT_SW enable(1)/disable(0) auto-ejecting
-
- usage:
-
- int val;
- ioctl(fd, CDROMEJECT_SW, val);
-
- inputs:
- Flag specifying auto-eject flag.
-
- outputs: none
-
- error return:
- ENOSYS Drive is not capable of ejecting.
- EBUSY Door is locked
-
-
-
-
-CDROMMULTISESSION Obtain the start-of-last-session
- address of multi session disks
- (struct cdrom_multisession)
- usage:
-
- struct cdrom_multisession ms_info;
- ioctl(fd, CDROMMULTISESSION, &ms_info);
-
- inputs:
- cdrom_multisession structure containing desired
- format.
-
- outputs:
- cdrom_multisession structure is filled with last_session
- information.
-
- error return:
- EINVAL format not CDROM_MSF or CDROM_LBA
-
-
-CDROM_GET_MCN Obtain the "Universal Product Code"
- if available (struct cdrom_mcn)
-
- usage:
-
- struct cdrom_mcn mcn;
- ioctl(fd, CDROM_GET_MCN, &mcn);
-
- inputs: none
-
- outputs:
- Universal Product Code
-
- error return:
- ENOSYS Drive is not capable of reading MCN data.
-
- notes:
- Source code comments state:
-
- The following function is implemented, although very few
- audio discs give Universal Product Code information, which
- should just be the Medium Catalog Number on the box. Note,
- that the way the code is written on the CD is /not/ uniform
- across all discs!
-
-
-
-
-CDROM_GET_UPC CDROM_GET_MCN (deprecated)
-
- Not implemented, as of 2.6.8.1
-
-
-
-CDROMRESET hard-reset the drive
-
- usage:
-
- ioctl(fd, CDROMRESET, 0);
-
- inputs: none
-
- outputs: none
-
- error return:
- EACCES Access denied: requires CAP_SYS_ADMIN
- ENOSYS Drive is not capable of resetting.
-
-
-
-
-CDROMREADCOOKED read data in cooked mode
-
- usage:
-
- u8 buffer[CD_FRAMESIZE]
- ioctl(fd, CDROMREADCOOKED, buffer);
-
- inputs: none
-
- outputs:
- 2048 bytes of data, "cooked" mode.
-
- notes:
- Not implemented on all drives.
-
-
-
-
-CDROMREADALL read all 2646 bytes
-
- Same as CDROMREADCOOKED, but reads 2646 bytes.
-
-
-
-CDROMSEEK seek msf address
-
- usage:
-
- struct cdrom_msf msf;
- ioctl(fd, CDROMSEEK, &msf);
-
- inputs:
- MSF address to seek to.
-
- outputs: none
-
-
-
-CDROMPLAYBLK scsi-cd only, (struct cdrom_blk)
-
- usage:
-
- struct cdrom_blk blk;
- ioctl(fd, CDROMPLAYBLK, &blk);
-
- inputs:
- Region to play
-
- outputs: none
-
-
-
-CDROMGETSPINDOWN
-
- usage:
-
- char spindown;
- ioctl(fd, CDROMGETSPINDOWN, &spindown);
-
- inputs: none
-
- outputs:
- The value of the current 4-bit spindown value.
-
-
-
-
-CDROMSETSPINDOWN
-
- usage:
-
- char spindown
- ioctl(fd, CDROMSETSPINDOWN, &spindown);
-
- inputs:
- 4-bit value used to control spindown (TODO: more detail here)
-
- outputs: none
-
-
-
-
-
-CDROM_SET_OPTIONS Set behavior options
-
- usage:
-
- int options;
- ioctl(fd, CDROM_SET_OPTIONS, options);
-
- inputs:
- New values for drive options. The logical 'or' of:
- CDO_AUTO_CLOSE close tray on first open(2)
- CDO_AUTO_EJECT open tray on last release
- CDO_USE_FFLAGS use O_NONBLOCK information on open
- CDO_LOCK lock tray on open files
- CDO_CHECK_TYPE check type on open for data
-
- outputs:
- Returns the resulting options settings in the
- ioctl return value. Returns -1 on error.
-
- error return:
- ENOSYS selected option(s) not supported by drive.
-
-
-
-
-CDROM_CLEAR_OPTIONS Clear behavior options
-
- Same as CDROM_SET_OPTIONS, except that selected options are
- turned off.
-
-
-
-CDROM_SELECT_SPEED Set the CD-ROM speed
-
- usage:
-
- int speed;
- ioctl(fd, CDROM_SELECT_SPEED, speed);
-
- inputs:
- New drive speed.
-
- outputs: none
-
- error return:
- ENOSYS speed selection not supported by drive.
-
-
-
-CDROM_SELECT_DISC Select disc (for juke-boxes)
-
- usage:
-
- int disk;
- ioctl(fd, CDROM_SELECT_DISC, disk);
-
- inputs:
- Disk to load into drive.
-
- outputs: none
-
- error return:
- EINVAL Disk number beyond capacity of drive
-
-
-
-CDROM_MEDIA_CHANGED Check is media changed
-
- usage:
-
- int slot;
- ioctl(fd, CDROM_MEDIA_CHANGED, slot);
-
- inputs:
- Slot number to be tested, always zero except for jukeboxes.
- May also be special values CDSL_NONE or CDSL_CURRENT
-
- outputs:
- Ioctl return value is 0 or 1 depending on whether the media
- has been changed, or -1 on error.
-
- error returns:
- ENOSYS Drive can't detect media change
- EINVAL Slot number beyond capacity of drive
- ENOMEM Out of memory
-
-
-
-CDROM_DRIVE_STATUS Get tray position, etc.
-
- usage:
-
- int slot;
- ioctl(fd, CDROM_DRIVE_STATUS, slot);
-
- inputs:
- Slot number to be tested, always zero except for jukeboxes.
- May also be special values CDSL_NONE or CDSL_CURRENT
-
- outputs:
- Ioctl return value will be one of the following values
- from <linux/cdrom.h>:
-
- CDS_NO_INFO Information not available.
- CDS_NO_DISC
- CDS_TRAY_OPEN
- CDS_DRIVE_NOT_READY
- CDS_DISC_OK
- -1 error
-
- error returns:
- ENOSYS Drive can't detect drive status
- EINVAL Slot number beyond capacity of drive
- ENOMEM Out of memory
-
-
-
-
-CDROM_DISC_STATUS Get disc type, etc.
-
- usage:
-
- ioctl(fd, CDROM_DISC_STATUS, 0);
-
- inputs: none
-
- outputs:
- Ioctl return value will be one of the following values
- from <linux/cdrom.h>:
- CDS_NO_INFO
- CDS_AUDIO
- CDS_MIXED
- CDS_XA_2_2
- CDS_XA_2_1
- CDS_DATA_1
-
- error returns: none at present
-
- notes:
- Source code comments state:
-
- Ok, this is where problems start. The current interface for
- the CDROM_DISC_STATUS ioctl is flawed. It makes the false
- assumption that CDs are all CDS_DATA_1 or all CDS_AUDIO, etc.
- Unfortunately, while this is often the case, it is also
- very common for CDs to have some tracks with data, and some
- tracks with audio. Just because I feel like it, I declare
- the following to be the best way to cope. If the CD has
- ANY data tracks on it, it will be returned as a data CD.
- If it has any XA tracks, I will return it as that. Now I
- could simplify this interface by combining these returns with
- the above, but this more clearly demonstrates the problem
- with the current interface. Too bad this wasn't designed
- to use bitmasks... -Erik
-
- Well, now we have the option CDS_MIXED: a mixed-type CD.
- User level programmers might feel the ioctl is not very
- useful.
- ---david
-
-
-
-
-CDROM_CHANGER_NSLOTS Get number of slots
-
- usage:
-
- ioctl(fd, CDROM_CHANGER_NSLOTS, 0);
-
- inputs: none
-
- outputs:
- The ioctl return value will be the number of slots in a
- CD changer. Typically 1 for non-multi-disk devices.
-
- error returns: none
-
-
-
-CDROM_LOCKDOOR lock or unlock door
-
- usage:
-
- int lock;
- ioctl(fd, CDROM_LOCKDOOR, lock);
-
- inputs:
- Door lock flag, 1=lock, 0=unlock
-
- outputs: none
-
- error returns:
- EDRIVE_CANT_DO_THIS Door lock function not supported.
- EBUSY Attempt to unlock when multiple users
- have the drive open and not CAP_SYS_ADMIN
-
- notes:
- As of 2.6.8.1, the lock flag is a global lock, meaning that
- all CD drives will be locked or unlocked together. This is
- probably a bug.
-
- The EDRIVE_CANT_DO_THIS value is defined in <linux/cdrom.h>
- and is currently (2.6.8.1) the same as EOPNOTSUPP
-
-
-
-CDROM_DEBUG Turn debug messages on/off
-
- usage:
-
- int debug;
- ioctl(fd, CDROM_DEBUG, debug);
-
- inputs:
- Cdrom debug flag, 0=disable, 1=enable
-
- outputs:
- The ioctl return value will be the new debug flag.
-
- error return:
- EACCES Access denied: requires CAP_SYS_ADMIN
-
-
-
-CDROM_GET_CAPABILITY get capabilities
-
- usage:
-
- ioctl(fd, CDROM_GET_CAPABILITY, 0);
-
- inputs: none
-
- outputs:
- The ioctl return value is the current device capability
- flags. See CDC_CLOSE_TRAY, CDC_OPEN_TRAY, etc.
-
-
-
-CDROMAUDIOBUFSIZ set the audio buffer size
-
- usage:
-
- int arg;
- ioctl(fd, CDROMAUDIOBUFSIZ, val);
-
- inputs:
- New audio buffer size
-
- outputs:
- The ioctl return value is the new audio buffer size, or -1
- on error.
-
- error return:
- ENOSYS Not supported by this driver.
-
- notes:
- Not supported by all drivers.
-
-
-
-DVD_READ_STRUCT Read structure
-
- usage:
-
- dvd_struct s;
- ioctl(fd, DVD_READ_STRUCT, &s);
-
- inputs:
- dvd_struct structure, containing:
- type specifies the information desired, one of
- DVD_STRUCT_PHYSICAL, DVD_STRUCT_COPYRIGHT,
- DVD_STRUCT_DISCKEY, DVD_STRUCT_BCA,
- DVD_STRUCT_MANUFACT
- physical.layer_num desired layer, indexed from 0
- copyright.layer_num desired layer, indexed from 0
- disckey.agid
-
- outputs:
- dvd_struct structure, containing:
- physical for type == DVD_STRUCT_PHYSICAL
- copyright for type == DVD_STRUCT_COPYRIGHT
- disckey.value for type == DVD_STRUCT_DISCKEY
- bca.{len,value} for type == DVD_STRUCT_BCA
- manufact.{len,valu} for type == DVD_STRUCT_MANUFACT
-
- error returns:
- EINVAL physical.layer_num exceeds number of layers
- EIO Received invalid response from drive
-
-
-
-DVD_WRITE_STRUCT Write structure
-
- Not implemented, as of 2.6.8.1
-
-
-
-DVD_AUTH Authentication
-
- usage:
-
- dvd_authinfo ai;
- ioctl(fd, DVD_AUTH, &ai);
-
- inputs:
- dvd_authinfo structure. See <linux/cdrom.h>
-
- outputs:
- dvd_authinfo structure.
-
- error return:
- ENOTTY ai.type not recognized.
-
-
-
-CDROM_SEND_PACKET send a packet to the drive
-
- usage:
-
- struct cdrom_generic_command cgc;
- ioctl(fd, CDROM_SEND_PACKET, &cgc);
-
- inputs:
- cdrom_generic_command structure containing the packet to send.
-
- outputs: none
- cdrom_generic_command structure containing results.
-
- error return:
- EIO command failed.
- EPERM Operation not permitted, either because a
- write command was attempted on a drive which
- is opened read-only, or because the command
- requires CAP_SYS_RAWIO
- EINVAL cgc.data_direction not set
-
-
-
-CDROM_NEXT_WRITABLE get next writable block
-
- usage:
-
- long next;
- ioctl(fd, CDROM_NEXT_WRITABLE, &next);
-
- inputs: none
-
- outputs:
- The next writable block.
-
- notes:
- If the device does not support this ioctl directly, the
- ioctl will return CDROM_LAST_WRITTEN + 7.
-
-
-
-CDROM_LAST_WRITTEN get last block written on disc
-
- usage:
-
- long last;
- ioctl(fd, CDROM_LAST_WRITTEN, &last);
-
- inputs: none
-
- outputs:
- The last block written on disc
-
- notes:
- If the device does not support this ioctl directly, the
- result is derived from the disc's table of contents. If the
- table of contents can't be read, this ioctl returns an
- error.
- Summary of HDIO_ ioctl calls.
- ============================
+==============================
+Summary of `HDIO_` ioctl calls
+==============================
- Edward A. Falk <efalk@google.com>
+- Edward A. Falk <efalk@google.com>
- November, 2004
+November, 2004
This document attempts to describe the ioctl(2) calls supported by
the HD/IDE layer. These are by-and-large implemented (as of Linux 2.6)
ioctls that pass argument pointers to user space:
+ ======================= =======================================
HDIO_GETGEO get device geometry
HDIO_GET_UNMASKINTR get current unmask setting
HDIO_GET_MULTCOUNT get current IDE blockmode setting
HDIO_DRIVE_TASK execute task and special drive command
HDIO_DRIVE_CMD execute a special drive command
HDIO_DRIVE_CMD_AEB HDIO_DRIVE_TASK
+ ======================= =======================================
ioctls that pass non-pointer values:
+ ======================= =======================================
HDIO_SET_MULTCOUNT change IDE blockmode
HDIO_SET_UNMASKINTR permit other irqs during I/O
HDIO_SET_KEEPSETTINGS keep ioctl settings on reset
HDIO_SET_IDE_SCSI Set scsi emulation mode on/off
HDIO_SET_SCSI_IDE not implemented yet
+ ======================= =======================================
The information that follows was determined from reading kernel source
code. It is likely that some corrections will be made over time.
-
-
-
-
-
+------------------------------------------------------------------------------
General:
Unless otherwise specified, all data structures and constants
are defined in <linux/hdreg.h>
+------------------------------------------------------------------------------
+HDIO_GETGEO
+ get device geometry
-HDIO_GETGEO get device geometry
- usage:
+ usage::
struct hd_geometry geom;
+
ioctl(fd, HDIO_GETGEO, &geom);
- inputs: none
+ inputs:
+ none
+
+
outputs:
+ hd_geometry structure containing:
- hd_geometry structure containing:
+ ========= ==================================
heads number of heads
sectors number of sectors/track
cylinders number of cylinders, mod 65536
start starting sector of this partition.
+ ========= ==================================
error returns:
- EINVAL if the device is not a disk drive or floppy drive,
- or if the user passes a null pointer
+ - EINVAL
+
+ if the device is not a disk drive or floppy drive,
+ or if the user passes a null pointer
notes:
+ Not particularly useful with modern disk drives, whose geometry
+ is a polite fiction anyway. Modern drives are addressed
+ purely by sector number nowadays (lba addressing), and the
+ drive geometry is an abstraction which is actually subject
+ to change. Currently (as of Nov 2004), the geometry values
+ are the "bios" values -- presumably the values the drive had
+ when Linux first booted.
- Not particularly useful with modern disk drives, whose geometry
- is a polite fiction anyway. Modern drives are addressed
- purely by sector number nowadays (lba addressing), and the
- drive geometry is an abstraction which is actually subject
- to change. Currently (as of Nov 2004), the geometry values
- are the "bios" values -- presumably the values the drive had
- when Linux first booted.
+ In addition, the cylinders field of the hd_geometry is an
+ unsigned short, meaning that on most architectures, this
+ ioctl will not return a meaningful value on drives with more
+ than 65535 tracks.
- In addition, the cylinders field of the hd_geometry is an
- unsigned short, meaning that on most architectures, this
- ioctl will not return a meaningful value on drives with more
- than 65535 tracks.
+ The start field is unsigned long, meaning that it will not
+ contain a meaningful value for disks over 219 Gb in size.
- The start field is unsigned long, meaning that it will not
- contain a meaningful value for disks over 219 Gb in size.
+HDIO_GET_UNMASKINTR
+ get current unmask setting
-HDIO_GET_UNMASKINTR get current unmask setting
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_GET_UNMASKINTR, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the drive's current unmask setting
+ The value of the drive's current unmask setting
-HDIO_SET_UNMASKINTR permit other irqs during I/O
- usage:
+
+HDIO_SET_UNMASKINTR
+ permit other irqs during I/O
+
+
+ usage::
unsigned long val;
+
ioctl(fd, HDIO_SET_UNMASKINTR, val);
inputs:
- New value for unmask flag
+ New value for unmask flag
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_GET_MULTCOUNT
+ get current IDE blockmode setting
-HDIO_GET_MULTCOUNT get current IDE blockmode setting
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_GET_MULTCOUNT, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current IDE block mode setting. This
- controls how many sectors the drive will transfer per
- interrupt.
+ The value of the current IDE block mode setting. This
+ controls how many sectors the drive will transfer per
+ interrupt.
+
+HDIO_SET_MULTCOUNT
+ change IDE blockmode
-HDIO_SET_MULTCOUNT change IDE blockmode
- usage:
+ usage::
int val;
+
ioctl(fd, HDIO_SET_MULTCOUNT, val);
inputs:
- New value for IDE block mode setting. This controls how many
- sectors the drive will transfer per interrupt.
+ New value for IDE block mode setting. This controls how many
+ sectors the drive will transfer per interrupt.
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range supported by disk.
- EBUSY Controller busy or blockmode already set.
- EIO Drive did not accept new block mode.
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range supported by disk.
+ - EBUSY Controller busy or blockmode already set.
+ - EIO Drive did not accept new block mode.
notes:
-
- Source code comments read:
+ Source code comments read::
This is tightly woven into the driver->do_special cannot
touch. DON'T do it again until a total personality rewrite
is committed.
If blockmode has already been set, this ioctl will fail with
- EBUSY
+ -EBUSY
-HDIO_GET_QDMA get use-qdma flag
+HDIO_GET_QDMA
+ get use-qdma flag
+
Not implemented, as of 2.6.8.1
-HDIO_SET_XFER set transfer rate via proc
+HDIO_SET_XFER
+ set transfer rate via proc
+
Not implemented, as of 2.6.8.1
-HDIO_OBSOLETE_IDENTITY OBSOLETE, DO NOT USE
+HDIO_OBSOLETE_IDENTITY
+ OBSOLETE, DO NOT USE
+
Same as HDIO_GET_IDENTITY (see below), except that it only
returns the first 142 bytes of drive identity information.
-HDIO_GET_IDENTITY get IDE identification info
+HDIO_GET_IDENTITY
+ get IDE identification info
+
- usage:
+ usage::
unsigned char identity[512];
+
ioctl(fd, HDIO_GET_IDENTITY, identity);
- inputs: none
+ inputs:
+ none
- outputs:
- ATA drive identity information. For full description, see
- the IDENTIFY DEVICE and IDENTIFY PACKET DEVICE commands in
- the ATA specification.
+
+ outputs:
+ ATA drive identity information. For full description, see
+ the IDENTIFY DEVICE and IDENTIFY PACKET DEVICE commands in
+ the ATA specification.
error returns:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- ENOMSG IDENTIFY DEVICE information not available
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - ENOMSG IDENTIFY DEVICE information not available
notes:
+ Returns information that was obtained when the drive was
+ probed. Some of this information is subject to change, and
+ this ioctl does not re-probe the drive to update the
+ information.
- Returns information that was obtained when the drive was
- probed. Some of this information is subject to change, and
- this ioctl does not re-probe the drive to update the
- information.
+ This information is also available from /proc/ide/hdX/identify
- This information is also available from /proc/ide/hdX/identify
+HDIO_GET_KEEPSETTINGS
+ get keep-settings-on-reset flag
-HDIO_GET_KEEPSETTINGS get keep-settings-on-reset flag
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_GET_KEEPSETTINGS, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current "keep settings" flag
+ The value of the current "keep settings" flag
+
+
notes:
+ When set, indicates that kernel should restore settings
+ after a drive reset.
- When set, indicates that kernel should restore settings
- after a drive reset.
+HDIO_SET_KEEPSETTINGS
+ keep ioctl settings on reset
-HDIO_SET_KEEPSETTINGS keep ioctl settings on reset
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_SET_KEEPSETTINGS, val);
inputs:
- New value for keep_settings flag
+ New value for keep_settings flag
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_GET_32BIT
+ get current io_32bit setting
-HDIO_GET_32BIT get current io_32bit setting
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_GET_32BIT, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current io_32bit setting
+ The value of the current io_32bit setting
+
+
notes:
+ 0=16-bit, 1=32-bit, 2,3 = 32bit+sync
+
- 0=16-bit, 1=32-bit, 2,3 = 32bit+sync
-HDIO_GET_NOWERR get ignore-write-error flag
+HDIO_GET_NOWERR
+ get ignore-write-error flag
- usage:
+
+ usage::
long val;
+
ioctl(fd, HDIO_GET_NOWERR, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current ignore-write-error flag
+ The value of the current ignore-write-error flag
-HDIO_GET_DMA get use-dma flag
- usage:
+
+HDIO_GET_DMA
+ get use-dma flag
+
+
+ usage::
long val;
+
ioctl(fd, HDIO_GET_DMA, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current use-dma flag
+ The value of the current use-dma flag
-HDIO_GET_NICE get nice flags
- usage:
+
+HDIO_GET_NICE
+ get nice flags
+
+
+ usage::
long nice;
+
ioctl(fd, HDIO_GET_NICE, &nice);
- inputs: none
+ inputs:
+ none
+
+
outputs:
+ The drive's "nice" values.
+
- The drive's "nice" values.
notes:
+ Per-drive flags which determine when the system will give more
+ bandwidth to other devices sharing the same IDE bus.
- Per-drive flags which determine when the system will give more
- bandwidth to other devices sharing the same IDE bus.
- See <linux/hdreg.h>, near symbol IDE_NICE_DSC_OVERLAP.
+ See <linux/hdreg.h>, near symbol IDE_NICE_DSC_OVERLAP.
-HDIO_SET_NICE set nice flags
+HDIO_SET_NICE
+ set nice flags
- usage:
+
+ usage::
unsigned long nice;
+
...
ioctl(fd, HDIO_SET_NICE, nice);
inputs:
- bitmask of nice flags.
+ bitmask of nice flags.
+
+
+
+ outputs:
+ none
+
- outputs: none
error returns:
- EACCES Access denied: requires CAP_SYS_ADMIN
- EPERM Flags other than DSC_OVERLAP and NICE_1 set.
- EPERM DSC_OVERLAP specified but not supported by drive
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EPERM Flags other than DSC_OVERLAP and NICE_1 set.
+ - EPERM DSC_OVERLAP specified but not supported by drive
notes:
+ This ioctl sets the DSC_OVERLAP and NICE_1 flags from values
+ provided by the user.
- This ioctl sets the DSC_OVERLAP and NICE_1 flags from values
- provided by the user.
+ Nice flags are listed in <linux/hdreg.h>, starting with
+ IDE_NICE_DSC_OVERLAP. These values represent shifts.
- Nice flags are listed in <linux/hdreg.h>, starting with
- IDE_NICE_DSC_OVERLAP. These values represent shifts.
+HDIO_GET_WCACHE
+ get write cache mode on|off
-HDIO_GET_WCACHE get write cache mode on|off
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_GET_WCACHE, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current write cache mode
+ The value of the current write cache mode
-HDIO_GET_ACOUSTIC get acoustic value
- usage:
+
+HDIO_GET_ACOUSTIC
+ get acoustic value
+
+
+ usage::
long val;
+
ioctl(fd, HDIO_GET_ACOUSTIC, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current acoustic settings
+ The value of the current acoustic settings
+
+
notes:
+ See HDIO_SET_ACOUSTIC
+
- See HDIO_SET_ACOUSTIC
HDIO_GET_ADDRESS
+ usage::
- usage:
long val;
+
ioctl(fd, HDIO_GET_ADDRESS, &val);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- The value of the current addressing mode:
- 0 = 28-bit
- 1 = 48-bit
- 2 = 48-bit doing 28-bit
- 3 = 64-bit
+ The value of the current addressing mode:
+
+ = ===================
+ 0 28-bit
+ 1 48-bit
+ 2 48-bit doing 28-bit
+ 3 64-bit
+ = ===================
-HDIO_GET_BUSSTATE get the bus state of the hwif
+HDIO_GET_BUSSTATE
+ get the bus state of the hwif
- usage:
+
+ usage::
long state;
+
ioctl(fd, HDIO_SCAN_HWIF, &state);
- inputs: none
+ inputs:
+ none
+
+
outputs:
- Current power state of the IDE bus. One of BUSSTATE_OFF,
- BUSSTATE_ON, or BUSSTATE_TRISTATE
+ Current power state of the IDE bus. One of BUSSTATE_OFF,
+ BUSSTATE_ON, or BUSSTATE_TRISTATE
error returns:
- EACCES Access denied: requires CAP_SYS_ADMIN
+ - EACCES Access denied: requires CAP_SYS_ADMIN
-HDIO_SET_BUSSTATE set the bus state of the hwif
+HDIO_SET_BUSSTATE
+ set the bus state of the hwif
- usage:
+
+ usage::
int state;
+
...
ioctl(fd, HDIO_SCAN_HWIF, state);
inputs:
- Desired IDE power state. One of BUSSTATE_OFF, BUSSTATE_ON,
- or BUSSTATE_TRISTATE
+ Desired IDE power state. One of BUSSTATE_OFF, BUSSTATE_ON,
+ or BUSSTATE_TRISTATE
+
+ outputs:
+ none
+
- outputs: none
error returns:
- EACCES Access denied: requires CAP_SYS_RAWIO
- EOPNOTSUPP Hardware interface does not support bus power control
+ - EACCES Access denied: requires CAP_SYS_RAWIO
+ - EOPNOTSUPP Hardware interface does not support bus power control
+
+HDIO_TRISTATE_HWIF
+ execute a channel tristate
-HDIO_TRISTATE_HWIF execute a channel tristate
Not implemented, as of 2.6.8.1. See HDIO_SET_BUSSTATE
-HDIO_DRIVE_RESET execute a device reset
+HDIO_DRIVE_RESET
+ execute a device reset
+
- usage:
+ usage::
int args[3]
+
...
ioctl(fd, HDIO_DRIVE_RESET, args);
- inputs: none
+ inputs:
+ none
+
+
+
+ outputs:
+ none
+
- outputs: none
error returns:
- EACCES Access denied: requires CAP_SYS_ADMIN
- ENXIO No such device: phy dead or ctl_addr == 0
- EIO I/O error: reset timed out or hardware error
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - ENXIO No such device: phy dead or ctl_addr == 0
+ - EIO I/O error: reset timed out or hardware error
notes:
- Execute a reset on the device as soon as the current IO
- operation has completed.
+ - Execute a reset on the device as soon as the current IO
+ operation has completed.
- Executes an ATAPI soft reset if applicable, otherwise
- executes an ATA soft reset on the controller.
+ - Executes an ATAPI soft reset if applicable, otherwise
+ executes an ATA soft reset on the controller.
-HDIO_DRIVE_TASKFILE execute raw taskfile
+HDIO_DRIVE_TASKFILE
+ execute raw taskfile
- Note: If you don't have a copy of the ANSI ATA specification
- handy, you should probably ignore this ioctl.
- Execute an ATA disk command directly by writing the "taskfile"
- registers of the drive. Requires ADMIN and RAWIO access
- privileges.
+ Note:
+ If you don't have a copy of the ANSI ATA specification
+ handy, you should probably ignore this ioctl.
+
+ - Execute an ATA disk command directly by writing the "taskfile"
+ registers of the drive. Requires ADMIN and RAWIO access
+ privileges.
- usage:
+ usage::
struct {
+
ide_task_request_t req_task;
u8 outbuf[OUTPUT_SIZE];
u8 inbuf[INPUT_SIZE];
(See below for details on memory area passed to ioctl.)
+ ============ ===================================================
io_ports[8] values to be written to taskfile registers
hob_ports[8] high-order bytes, for extended commands.
out_flags flags indicating which registers are valid
out_size size of output buffer
outbuf buffer of data to be transmitted to disk
inbuf buffer of data to be received from disk (see [1])
+ ============ ===================================================
outputs:
+ =========== ====================================================
io_ports[] values returned in the taskfile registers
hob_ports[] high-order bytes, for extended commands.
out_flags flags indicating which registers are valid (see [2])
in_flags flags indicating which registers should be returned
outbuf buffer of data to be transmitted to disk (see [1])
inbuf buffer of data to be received from disk
+ =========== ====================================================
error returns:
- EACCES CAP_SYS_ADMIN or CAP_SYS_RAWIO privilege not set.
- ENOMSG Device is not a disk drive.
- ENOMEM Unable to allocate memory for task
- EFAULT req_cmd == TASKFILE_IN_OUT (not implemented as of 2.6.8)
- EPERM req_cmd == TASKFILE_MULTI_OUT and drive
- multi-count not yet set.
- EIO Drive failed the command.
+ - EACCES CAP_SYS_ADMIN or CAP_SYS_RAWIO privilege not set.
+ - ENOMSG Device is not a disk drive.
+ - ENOMEM Unable to allocate memory for task
+ - EFAULT req_cmd == TASKFILE_IN_OUT (not implemented as of 2.6.8)
+ - EPERM
+
+ req_cmd == TASKFILE_MULTI_OUT and drive
+ multi-count not yet set.
+ - EIO Drive failed the command.
notes:
Command is passed to the disk drive via the ide_task_request_t
structure, which contains these fields:
+ ============ ===============================================
io_ports[8] values for the taskfile registers
hob_ports[8] high-order bytes, for extended commands
out_flags flags indicating which entries in the
- io_ports[] and hob_ports[] arrays
+ io_ports[] and hob_ports[] arrays
contain valid values. Type ide_reg_valid_t.
in_flags flags indicating which entries in the
- io_ports[] and hob_ports[] arrays
+ io_ports[] and hob_ports[] arrays
are expected to contain valid values
on return.
data_phase See below
req_cmd Command type, see below
out_size output (user->drive) buffer size, bytes
in_size input (drive->user) buffer size, bytes
+ ============ ===============================================
When out_flags is zero, the following registers are loaded.
+ ============ ===============================================
HOB_FEATURE If the drive supports LBA48
HOB_NSECTOR If the drive supports LBA48
HOB_SECTOR If the drive supports LBA48
SELECT First, masked with 0xE0 if LBA48, 0xEF
otherwise; then, or'ed with the default
value of SELECT.
+ ============ ===============================================
If any bit in out_flags is set, the following registers are loaded.
+ ============ ===============================================
HOB_DATA If out_flags.b.data is set. HOB_DATA will
travel on DD8-DD15 on little endian machines
and on DD0-DD7 on big endian machines.
HCYL If out_flags.b.hcyl is set
SELECT Or'ed with the default value of SELECT and
loaded regardless of out_flags.b.select.
+ ============ ===============================================
Taskfile registers are read back from the drive into
{io|hob}_ports[] after the command completes iff one of the
2. One or more than one bits are set in out_flags.
3. The requested data_phase is TASKFILE_NO_DATA.
+ ============ ===============================================
HOB_DATA If in_flags.b.data is set. It will contain
DD8-DD15 on little endian machines and
DD0-DD7 on big endian machines.
SECTOR
LCYL
HCYL
+ ============ ===============================================
The data_phase field describes the data transfer to be
performed. Value is one of:
+ =================== ========================================
TASKFILE_IN
TASKFILE_MULTI_IN
TASKFILE_OUT
TASKFILE_P_OUT unimplemented
TASKFILE_P_OUT_DMA unimplemented
TASKFILE_P_OUT_DMAQ unimplemented
+ =================== ========================================
The req_cmd field classifies the command type. It may be
one of:
+ ======================== =======================================
IDE_DRIVE_TASK_NO_DATA
IDE_DRIVE_TASK_SET_XFER unimplemented
IDE_DRIVE_TASK_IN
IDE_DRIVE_TASK_OUT unimplemented
IDE_DRIVE_TASK_RAW_WRITE
+ ======================== =======================================
[6] Do not access {in|out}_flags->all except for resetting
all the bits. Always access individual bit fields. ->all
-HDIO_DRIVE_CMD execute a special drive command
+HDIO_DRIVE_CMD
+ execute a special drive command
+
Note: If you don't have a copy of the ANSI ATA specification
handy, you should probably ignore this ioctl.
- usage:
+ usage::
u8 args[4+XFER_SIZE];
+
...
ioctl(fd, HDIO_DRIVE_CMD, args);
inputs:
+ Commands other than WIN_SMART:
- Commands other than WIN_SMART
+ ======= =======
args[0] COMMAND
args[1] NSECTOR
args[2] FEATURE
args[3] NSECTOR
+ ======= =======
+
+ WIN_SMART:
- WIN_SMART
+ ======= =======
args[0] COMMAND
args[1] SECTOR
args[2] FEATURE
args[3] NSECTOR
+ ======= =======
outputs:
+ args[] buffer is filled with register values followed by any
+
- args[] buffer is filled with register values followed by any
data returned by the disk.
+
+ ======== ====================================================
args[0] status
args[1] error
args[2] NSECTOR
args[3] undefined
args[4+] NSECTOR * 512 bytes of data returned by the command.
+ ======== ====================================================
error returns:
- EACCES Access denied: requires CAP_SYS_RAWIO
- ENOMEM Unable to allocate memory for task
- EIO Drive reports error
+ - EACCES Access denied: requires CAP_SYS_RAWIO
+ - ENOMEM Unable to allocate memory for task
+ - EIO Drive reports error
notes:
-HDIO_DRIVE_TASK execute task and special drive command
+HDIO_DRIVE_TASK
+ execute task and special drive command
+
Note: If you don't have a copy of the ANSI ATA specification
handy, you should probably ignore this ioctl.
- usage:
+ usage::
u8 args[7];
+
...
ioctl(fd, HDIO_DRIVE_TASK, args);
inputs:
+ Taskfile register values:
- Taskfile register values:
+ ======= =======
args[0] COMMAND
args[1] FEATURE
args[2] NSECTOR
args[4] LCYL
args[5] HCYL
args[6] SELECT
+ ======= =======
outputs:
+ Taskfile register values:
+
- Taskfile register values:
+ ======= =======
args[0] status
args[1] error
args[2] NSECTOR
args[4] LCYL
args[5] HCYL
args[6] SELECT
+ ======= =======
error returns:
- EACCES Access denied: requires CAP_SYS_RAWIO
- ENOMEM Unable to allocate memory for task
- ENOMSG Device is not a disk drive.
- EIO Drive failed the command.
+ - EACCES Access denied: requires CAP_SYS_RAWIO
+ - ENOMEM Unable to allocate memory for task
+ - ENOMSG Device is not a disk drive.
+ - EIO Drive failed the command.
notes:
-HDIO_DRIVE_CMD_AEB HDIO_DRIVE_TASK
+HDIO_DRIVE_CMD_AEB
+ HDIO_DRIVE_TASK
+
Not implemented, as of 2.6.8.1
-HDIO_SET_32BIT change io_32bit flags
+HDIO_SET_32BIT
+ change io_32bit flags
+
- usage:
+ usage::
int val;
+
ioctl(fd, HDIO_SET_32BIT, val);
inputs:
- New value for io_32bit flag
+ New value for io_32bit flag
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 3]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 3]
+ - EBUSY Controller busy
-HDIO_SET_NOWERR change ignore-write-error flag
+HDIO_SET_NOWERR
+ change ignore-write-error flag
- usage:
+
+ usage::
int val;
+
ioctl(fd, HDIO_SET_NOWERR, val);
inputs:
- New value for ignore-write-error flag. Used for ignoring
+ New value for ignore-write-error flag. Used for ignoring
+
+
WRERR_STAT
- outputs: none
+ outputs:
+ none
+
+
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_SET_DMA
+ change use-dma flag
-HDIO_SET_DMA change use-dma flag
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_SET_DMA, val);
inputs:
- New value for use-dma flag
+ New value for use-dma flag
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_SET_PIO_MODE
+ reconfig interface to new speed
-HDIO_SET_PIO_MODE reconfig interface to new speed
- usage:
+ usage::
long val;
+
ioctl(fd, HDIO_SET_PIO_MODE, val);
inputs:
- New interface speed.
+ New interface speed.
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 255]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 255]
+ - EBUSY Controller busy
+
+HDIO_SCAN_HWIF
+ register and (re)scan interface
-HDIO_SCAN_HWIF register and (re)scan interface
- usage:
+ usage::
int args[3]
+
...
ioctl(fd, HDIO_SCAN_HWIF, args);
inputs:
+
+ ======= =========================
args[0] io address to probe
+
+
args[1] control address to probe
args[2] irq number
+ ======= =========================
+
+ outputs:
+ none
+
- outputs: none
error returns:
- EACCES Access denied: requires CAP_SYS_RAWIO
- EIO Probe failed.
+ - EACCES Access denied: requires CAP_SYS_RAWIO
+ - EIO Probe failed.
notes:
+ This ioctl initializes the addresses and irq for a disk
+ controller, probes for drives, and creates /proc/ide
+ interfaces as appropriate.
- This ioctl initializes the addresses and irq for a disk
- controller, probes for drives, and creates /proc/ide
- interfaces as appropriate.
+HDIO_UNREGISTER_HWIF
+ unregister interface
-HDIO_UNREGISTER_HWIF unregister interface
- usage:
+ usage::
int index;
+
ioctl(fd, HDIO_UNREGISTER_HWIF, index);
inputs:
- index index of hardware interface to unregister
+ index index of hardware interface to unregister
+
+
+
+ outputs:
+ none
+
- outputs: none
error returns:
- EACCES Access denied: requires CAP_SYS_RAWIO
+ - EACCES Access denied: requires CAP_SYS_RAWIO
notes:
+ This ioctl removes a hardware interface from the kernel.
- This ioctl removes a hardware interface from the kernel.
+ Currently (2.6.8) this ioctl silently fails if any drive on
+ the interface is busy.
- Currently (2.6.8) this ioctl silently fails if any drive on
- the interface is busy.
+HDIO_SET_WCACHE
+ change write cache enable-disable
-HDIO_SET_WCACHE change write cache enable-disable
- usage:
+ usage::
int val;
+
ioctl(fd, HDIO_SET_WCACHE, val);
inputs:
- New value for write cache enable
+ New value for write cache enable
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_SET_ACOUSTIC
+ change acoustic behavior
-HDIO_SET_ACOUSTIC change acoustic behavior
- usage:
+ usage::
int val;
+
ioctl(fd, HDIO_SET_ACOUSTIC, val);
inputs:
- New value for drive acoustic settings
+ New value for drive acoustic settings
+
+
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 254]
- EBUSY Controller busy
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 254]
+ - EBUSY Controller busy
-HDIO_SET_QDMA change use-qdma flag
+HDIO_SET_QDMA
+ change use-qdma flag
+
Not implemented, as of 2.6.8.1
-HDIO_SET_ADDRESS change lba addressing modes
+HDIO_SET_ADDRESS
+ change lba addressing modes
+
- usage:
+ usage::
int val;
+
ioctl(fd, HDIO_SET_ADDRESS, val);
inputs:
- New value for addressing mode
- 0 = 28-bit
- 1 = 48-bit
- 2 = 48-bit doing 28-bit
+ New value for addressing mode
+
+ = ===================
+ 0 28-bit
+ 1 48-bit
+ 2 48-bit doing 28-bit
+ = ===================
+
+ outputs:
+ none
+
- outputs: none
error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 2]
- EBUSY Controller busy
- EIO Drive does not support lba48 mode.
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 2]
+ - EBUSY Controller busy
+ - EIO Drive does not support lba48 mode.
HDIO_SET_IDE_SCSI
+ usage::
- usage:
long val;
+
ioctl(fd, HDIO_SET_IDE_SCSI, val);
inputs:
- New value for scsi emulation mode (?)
+ New value for scsi emulation mode (?)
- outputs: none
- error return:
- EINVAL (bdev != bdev->bd_contains) (not sure what this means)
- EACCES Access denied: requires CAP_SYS_ADMIN
- EINVAL value out of range [0 1]
- EBUSY Controller busy
+ outputs:
+ none
-HDIO_SET_SCSI_IDE
- Not implemented, as of 2.6.8.1
+ error return:
+ - EINVAL (bdev != bdev->bd_contains) (not sure what this means)
+ - EACCES Access denied: requires CAP_SYS_ADMIN
+ - EINVAL value out of range [0 1]
+ - EBUSY Controller busy
+
+HDIO_SET_SCSI_IDE
+ Not implemented, as of 2.6.8.1
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+======
+IOCTLs
+======
+
+.. toctree::
+ :maxdepth: 1
+
+ ioctl-number
+
+ botching-up-ioctls
+ ioctl-decoding
+
+ cdrom
+ hdio
+==============================
+Decoding an IOCTL Magic Number
+==============================
+
To decode a hex IOCTL code:
Most architectures use this generic format, but check
include/ARCH/ioctl.h for specifics, e.g. powerpc
uses 3 bits to encode read/write and 13 bits for size.
- bits meaning
+ ====== ==================================
+ bits meaning
+ ====== ==================================
31-30 00 - no parameters: uses _IO macro
10 - read: _IOR
01 - write: _IOW
unique to each driver
7-0 function #
+ ====== ==================================
So for example 0x82187201 is a read with arg length of 0x218,
-character 'r' function 1. Grepping the source reveals this is:
+character 'r' function 1. Grepping the source reveals this is::
-#define VFAT_IOCTL_READDIR_BOTH _IOR('r', 1, struct dirent [2])
+ #define VFAT_IOCTL_READDIR_BOTH _IOR('r', 1, struct dirent [2])
--- /dev/null
+=============
+Ioctl Numbers
+=============
+
+19 October 1999
+
+Michael Elizabeth Chastain
+<mec@shout.net>
+
+If you are adding new ioctl's to the kernel, you should use the _IO
+macros defined in <linux/ioctl.h>:
+
+ ====== == ============================================
+ _IO an ioctl with no parameters
+ _IOW an ioctl with write parameters (copy_from_user)
+ _IOR an ioctl with read parameters (copy_to_user)
+ _IOWR an ioctl with both write and read parameters.
+ ====== == ============================================
+
+'Write' and 'read' are from the user's point of view, just like the
+system calls 'write' and 'read'. For example, a SET_FOO ioctl would
+be _IOW, although the kernel would actually read data from user space;
+a GET_FOO ioctl would be _IOR, although the kernel would actually write
+data to user space.
+
+The first argument to _IO, _IOW, _IOR, or _IOWR is an identifying letter
+or number from the table below. Because of the large number of drivers,
+many drivers share a partial letter with other drivers.
+
+If you are writing a driver for a new device and need a letter, pick an
+unused block with enough room for expansion: 32 to 256 ioctl commands.
+You can register the block by patching this file and submitting the
+patch to Linus Torvalds. Or you can e-mail me at <mec@shout.net> and
+I'll register one for you.
+
+The second argument to _IO, _IOW, _IOR, or _IOWR is a sequence number
+to distinguish ioctls from each other. The third argument to _IOW,
+_IOR, or _IOWR is the type of the data going into the kernel or coming
+out of the kernel (e.g. 'int' or 'struct foo'). NOTE! Do NOT use
+sizeof(arg) as the third argument as this results in your ioctl thinking
+it passes an argument of type size_t.
+
+Some devices use their major number as the identifier; this is OK, as
+long as it is unique. Some devices are irregular and don't follow any
+convention at all.
+
+Following this convention is good because:
+
+(1) Keeping the ioctl's globally unique helps error checking:
+ if a program calls an ioctl on the wrong device, it will get an
+ error rather than some unexpected behaviour.
+
+(2) The 'strace' build procedure automatically finds ioctl numbers
+ defined with _IO, _IOW, _IOR, or _IOWR.
+
+(3) 'strace' can decode numbers back into useful names when the
+ numbers are unique.
+
+(4) People looking for ioctls can grep for them more easily when
+ this convention is used to define the ioctl numbers.
+
+(5) When following the convention, the driver code can use generic
+ code to copy the parameters between user and kernel space.
+
+This table lists ioctls visible from user land for Linux/x86. It contains
+most drivers up to 2.6.31, but I know I am missing some. There has been
+no attempt to list non-X86 architectures or ioctls from drivers/staging/.
+
+==== ===== ======================================================= ================================================================
+Code Seq# Include File Comments
+ (hex)
+==== ===== ======================================================= ================================================================
+0x00 00-1F linux/fs.h conflict!
+0x00 00-1F scsi/scsi_ioctl.h conflict!
+0x00 00-1F linux/fb.h conflict!
+0x00 00-1F linux/wavefront.h conflict!
+0x02 all linux/fd.h
+0x03 all linux/hdreg.h
+0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these.
+0x06 all linux/lp.h
+0x09 all linux/raid/md_u.h
+0x10 00-0F drivers/char/s390/vmcp.h
+0x10 10-1F arch/s390/include/uapi/sclp_ctl.h
+0x10 20-2F arch/s390/include/uapi/asm/hypfs.h
+0x12 all linux/fs.h
+ linux/blkpg.h
+0x1b all InfiniBand Subsystem
+ <http://infiniband.sourceforge.net/>
+0x20 all drivers/cdrom/cm206.h
+0x22 all scsi/sg.h
+'!' 00-1F uapi/linux/seccomp.h
+'#' 00-3F IEEE 1394 Subsystem
+ Block for the entire subsystem
+'$' 00-0F linux/perf_counter.h, linux/perf_event.h
+'%' 00-0F include/uapi/linux/stm.h System Trace Module subsystem
+ <mailto:alexander.shishkin@linux.intel.com>
+'&' 00-07 drivers/firewire/nosy-user.h
+'1' 00-1F linux/timepps.h PPS kit from Ulrich Windl
+ <ftp://ftp.de.kernel.org/pub/linux/daemons/ntp/PPS/>
+'2' 01-04 linux/i2o.h
+'3' 00-0F drivers/s390/char/raw3270.h conflict!
+'3' 00-1F linux/suspend_ioctls.h, conflict!
+ kernel/power/user.c
+'8' all SNP8023 advanced NIC card
+ <mailto:mcr@solidum.com>
+';' 64-7F linux/vfio.h
+'@' 00-0F linux/radeonfb.h conflict!
+'@' 00-0F drivers/video/aty/aty128fb.c conflict!
+'A' 00-1F linux/apm_bios.h conflict!
+'A' 00-0F linux/agpgart.h, conflict!
+ drivers/char/agp/compat_ioctl.h
+'A' 00-7F sound/asound.h conflict!
+'B' 00-1F linux/cciss_ioctl.h conflict!
+'B' 00-0F include/linux/pmu.h conflict!
+'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de>
+'C' all linux/soundcard.h conflict!
+'C' 01-2F linux/capi.h conflict!
+'C' F0-FF drivers/net/wan/cosa.h conflict!
+'D' all arch/s390/include/asm/dasd.h
+'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h
+'D' 05 drivers/scsi/pmcraid.h
+'E' all linux/input.h conflict!
+'E' 00-0F xen/evtchn.h conflict!
+'F' all linux/fb.h conflict!
+'F' 01-02 drivers/scsi/pmcraid.h conflict!
+'F' 20 drivers/video/fsl-diu-fb.h conflict!
+'F' 20 drivers/video/intelfb/intelfb.h conflict!
+'F' 20 linux/ivtvfb.h conflict!
+'F' 20 linux/matroxfb.h conflict!
+'F' 20 drivers/video/aty/atyfb_base.c conflict!
+'F' 00-0F video/da8xx-fb.h conflict!
+'F' 80-8F linux/arcfb.h conflict!
+'F' DD video/sstfb.h conflict!
+'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!
+'G' 00-0F linux/gigaset_dev.h conflict!
+'H' 00-7F linux/hiddev.h conflict!
+'H' 00-0F linux/hidraw.h conflict!
+'H' 01 linux/mei.h conflict!
+'H' 02 linux/mei.h conflict!
+'H' 03 linux/mei.h conflict!
+'H' 00-0F sound/asound.h conflict!
+'H' 20-40 sound/asound_fm.h conflict!
+'H' 80-8F sound/sfnt_info.h conflict!
+'H' 10-8F sound/emu10k1.h conflict!
+'H' 10-1F sound/sb16_csp.h conflict!
+'H' 10-1F sound/hda_hwdep.h conflict!
+'H' 40-4F sound/hdspm.h conflict!
+'H' 40-4F sound/hdsp.h conflict!
+'H' 90 sound/usb/usx2y/usb_stream.h
+'H' A0 uapi/linux/usb/cdc-wdm.h
+'H' C0-F0 net/bluetooth/hci.h conflict!
+'H' C0-DF net/bluetooth/hidp/hidp.h conflict!
+'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict!
+'H' C0-DF net/bluetooth/bnep/bnep.h conflict!
+'H' F1 linux/hid-roccat.h <mailto:erazor_de@users.sourceforge.net>
+'H' F8-FA sound/firewire.h
+'I' all linux/isdn.h conflict!
+'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict!
+'I' 40-4F linux/mISDNif.h conflict!
+'J' 00-1F drivers/scsi/gdth_ioctl.h
+'K' all linux/kd.h
+'L' 00-1F linux/loop.h conflict!
+'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict!
+'L' 20-2F linux/lightnvm.h
+'L' E0-FF linux/ppdd.h encrypted disk device driver
+ <http://linux01.gwdg.de/~alatham/ppdd.html>
+'M' all linux/soundcard.h conflict!
+'M' 01-16 mtd/mtd-abi.h conflict!
+ and drivers/mtd/mtdchar.c
+'M' 01-03 drivers/scsi/megaraid/megaraid_sas.h
+'M' 00-0F drivers/video/fsl-diu-fb.h conflict!
+'N' 00-1F drivers/usb/scanner.h
+'N' 40-7F drivers/block/nvme.c
+'O' 00-06 mtd/ubi-user.h UBI
+'P' all linux/soundcard.h conflict!
+'P' 60-6F sound/sscape_ioctl.h conflict!
+'P' 00-0F drivers/usb/class/usblp.c conflict!
+'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!
+'Q' all linux/soundcard.h
+'R' 00-1F linux/random.h conflict!
+'R' 01 linux/rfkill.h conflict!
+'R' C0-DF net/bluetooth/rfcomm.h
+'S' all linux/cdrom.h conflict!
+'S' 80-81 scsi/scsi_ioctl.h conflict!
+'S' 82-FF scsi/scsi.h conflict!
+'S' 00-7F sound/asequencer.h conflict!
+'T' all linux/soundcard.h conflict!
+'T' 00-AF sound/asound.h conflict!
+'T' all arch/x86/include/asm/ioctls.h conflict!
+'T' C0-DF linux/if_tun.h conflict!
+'U' all sound/asound.h conflict!
+'U' 00-CF linux/uinput.h conflict!
+'U' 00-EF linux/usbdevice_fs.h
+'U' C0-CF drivers/bluetooth/hci_uart.h
+'V' all linux/vt.h conflict!
+'V' all linux/videodev2.h conflict!
+'V' C0 linux/ivtvfb.h conflict!
+'V' C0 linux/ivtv.h conflict!
+'V' C0 media/davinci/vpfe_capture.h conflict!
+'V' C0 media/si4713.h conflict!
+'W' 00-1F linux/watchdog.h conflict!
+'W' 00-1F linux/wanrouter.h conflict! (pre 3.9)
+'W' 00-3F sound/asound.h conflict!
+'W' 40-5F drivers/pci/switch/switchtec.c
+'X' all fs/xfs/xfs_fs.h, conflict!
+ fs/xfs/linux-2.6/xfs_ioctl32.h,
+ include/linux/falloc.h,
+ linux/fs.h,
+'X' all fs/ocfs2/ocfs_fs.h conflict!
+'X' 01 linux/pktcdvd.h conflict!
+'Y' all linux/cyclades.h
+'Z' 14-15 drivers/message/fusion/mptctl.h
+'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices
+ <mailto:gregkh@linuxfoundation.org>
+'a' all linux/atm*.h, linux/sonet.h ATM on linux
+ <http://lrcwww.epfl.ch/>
+'a' 00-0F drivers/crypto/qat/qat_common/adf_cfg_common.h conflict! qat driver
+'b' 00-FF conflict! bit3 vme host bridge
+ <mailto:natalia@nikhefk.nikhef.nl>
+'c' all linux/cm4000_cs.h conflict!
+'c' 00-7F linux/comstats.h conflict!
+'c' 00-7F linux/coda.h conflict!
+'c' 00-1F linux/chio.h conflict!
+'c' 80-9F arch/s390/include/asm/chsc.h conflict!
+'c' A0-AF arch/x86/include/asm/msr.h conflict!
+'d' 00-FF linux/char/drm/drm.h conflict!
+'d' 02-40 pcmcia/ds.h conflict!
+'d' F0-FF linux/digi1.h
+'e' all linux/digi1.h conflict!
+'f' 00-1F linux/ext2_fs.h conflict!
+'f' 00-1F linux/ext3_fs.h conflict!
+'f' 00-0F fs/jfs/jfs_dinode.h conflict!
+'f' 00-0F fs/ext4/ext4.h conflict!
+'f' 00-0F linux/fs.h conflict!
+'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict!
+'g' 00-0F linux/usb/gadgetfs.h
+'g' 20-2F linux/usb/g_printer.h
+'h' 00-7F conflict! Charon filesystem
+ <mailto:zapman@interlan.net>
+'h' 00-1F linux/hpet.h conflict!
+'h' 80-8F fs/hfsplus/ioctl.c
+'i' 00-3F linux/i2o-dev.h conflict!
+'i' 0B-1F linux/ipmi.h conflict!
+'i' 80-8F linux/i8k.h
+'j' 00-3F linux/joystick.h
+'k' 00-0F linux/spi/spidev.h conflict!
+'k' 00-05 video/kyro.h conflict!
+'k' 10-17 linux/hsi/hsi_char.h HSI character device
+'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system
+ <http://web.archive.org/web/%2A/http://mikonos.dia.unisa.it/tcfs>
+'l' 40-7F linux/udf_fs_i.h in development:
+ <http://sourceforge.net/projects/linux-udf/>
+'m' 00-09 linux/mmtimer.h conflict!
+'m' all linux/mtio.h conflict!
+'m' all linux/soundcard.h conflict!
+'m' all linux/synclink.h conflict!
+'m' 00-19 drivers/message/fusion/mptctl.h conflict!
+'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict!
+'n' 00-7F linux/ncp_fs.h and fs/ncpfs/ioctl.c
+'n' 80-8F uapi/linux/nilfs2_api.h NILFS2
+'n' E0-FF linux/matroxfb.h matroxfb
+'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2
+'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps)
+'o' 40-41 mtd/ubi-user.h UBI
+'o' 01-A1 `linux/dvb/*.h` DVB
+'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this)
+'p' 00-1F linux/rtc.h conflict!
+'p' 00-3F linux/mc146818rtc.h conflict!
+'p' 40-7F linux/nvram.h
+'p' 80-9F linux/ppdev.h user-space parport
+ <mailto:tim@cyberelk.net>
+'p' A1-A5 linux/pps.h LinuxPPS
+ <mailto:giometti@linux.it>
+'q' 00-1F linux/serio.h
+'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK
+ linux/ixjuser.h <http://web.archive.org/web/%2A/http://www.quicknet.net>
+'r' 00-1F linux/msdos_fs.h and fs/fat/dir.c
+'s' all linux/cdk.h
+'t' 00-7F linux/ppp-ioctl.h
+'t' 80-8F linux/isdn_ppp.h
+'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM
+'u' 00-1F linux/smb_fs.h gone
+'u' 20-3F linux/uvcvideo.h USB video class host driver
+'u' 40-4f linux/udmabuf.h userspace dma-buf misc device
+'v' 00-1F linux/ext2_fs.h conflict!
+'v' 00-1F linux/fs.h conflict!
+'v' 00-0F linux/sonypi.h conflict!
+'v' 00-0F media/v4l2-subdev.h conflict!
+'v' C0-FF linux/meye.h conflict!
+'w' all CERN SCI driver
+'y' 00-1F packet based user level communications
+ <mailto:zapman@interlan.net>
+'z' 00-3F CAN bus card conflict!
+ <mailto:hdstich@connectu.ulm.circular.de>
+'z' 40-7F CAN bus card conflict!
+ <mailto:oe@port.de>
+'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict!
+'|' 00-7F linux/media.h
+0x80 00-1F linux/fb.h
+0x89 00-06 arch/x86/include/asm/sockios.h
+0x89 0B-DF linux/sockios.h
+0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range
+0x89 E0-EF linux/dn.h PROTOPRIVATE range
+0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range
+0x8B all linux/wireless.h
+0x8C 00-3F WiNRADiO driver
+ <http://www.winradio.com.au/>
+0x90 00 drivers/cdrom/sbpcd.h
+0x92 00-0F drivers/usb/mon/mon_bin.c
+0x93 60-7F linux/auto_fs.h
+0x94 all fs/btrfs/ioctl.h Btrfs filesystem
+ and linux/fs.h some lifted to vfs/generic
+0x97 00-7F fs/ceph/ioctl.h Ceph file system
+0x99 00-0F 537-Addinboard driver
+ <mailto:buk@buks.ipn.de>
+0xA0 all linux/sdp/sdp.h Industrial Device Project
+ <mailto:kenji@bitgate.com>
+0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver
+0xA3 80-8F Port ACL in development:
+ <mailto:tlewis@mindspring.com>
+0xA3 90-9F linux/dtlk.h
+0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem
+0xAA 00-3F linux/uapi/linux/userfaultfd.h
+0xAB 00-1F linux/nbd.h
+0xAC 00-1F linux/raw.h
+0xAD 00 Netfilter device in development:
+ <mailto:rusty@rustcorp.com.au>
+0xAE all linux/kvm.h Kernel-based Virtual Machine
+ <mailto:kvm@vger.kernel.org>
+0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor
+0xB0 all RATIO devices in development:
+ <mailto:vgo@ratio.de>
+0xB1 00-1F PPPoX
+ <mailto:mostrows@styx.uwaterloo.ca>
+0xB3 00 linux/mmc/ioctl.h
+0xB4 00-0F linux/gpio.h <mailto:linux-gpio@vger.kernel.org>
+0xB5 00-0F uapi/linux/rpmsg.h <mailto:linux-remoteproc@vger.kernel.org>
+0xB6 all linux/fpga-dfl.h
+0xC0 00-0F linux/usb/iowarrior.h
+0xCA 00-0F uapi/misc/cxl.h
+0xCA 10-2F uapi/misc/ocxl.h
+0xCA 80-BF uapi/scsi/cxlflash_ioctl.h
+0xCB 00-1F CBM serial IEC bus in development:
+ <mailto:michael.klein@puffin.lb.shuttle.de>
+0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
+0xCD 01 linux/reiserfs_fs.h
+0xCF 02 fs/cifs/ioctl.c
+0xDB 00-0F drivers/char/mwave/mwavepub.h
+0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
+ <mailto:aherrman@de.ibm.com>
+0xE5 00-3F linux/fuse.h
+0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver
+0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development)
+ <mailto:thomas@winischhofer.net>
+0xF4 00-1F video/mbxfb.h mbxfb
+ <mailto:raph@8d.com>
+0xF6 all LTTng Linux Trace Toolkit Next Generation
+ <mailto:mathieu.desnoyers@efficios.com>
+0xFD all linux/dm-ioctl.h
+0xFE all linux/isst_if.h
+==== ===== ======================================================= ================================================================
+++ /dev/null
-Ioctl Numbers
-19 October 1999
-Michael Elizabeth Chastain
-<mec@shout.net>
-
-If you are adding new ioctl's to the kernel, you should use the _IO
-macros defined in <linux/ioctl.h>:
-
- _IO an ioctl with no parameters
- _IOW an ioctl with write parameters (copy_from_user)
- _IOR an ioctl with read parameters (copy_to_user)
- _IOWR an ioctl with both write and read parameters.
-
-'Write' and 'read' are from the user's point of view, just like the
-system calls 'write' and 'read'. For example, a SET_FOO ioctl would
-be _IOW, although the kernel would actually read data from user space;
-a GET_FOO ioctl would be _IOR, although the kernel would actually write
-data to user space.
-
-The first argument to _IO, _IOW, _IOR, or _IOWR is an identifying letter
-or number from the table below. Because of the large number of drivers,
-many drivers share a partial letter with other drivers.
-
-If you are writing a driver for a new device and need a letter, pick an
-unused block with enough room for expansion: 32 to 256 ioctl commands.
-You can register the block by patching this file and submitting the
-patch to Linus Torvalds. Or you can e-mail me at <mec@shout.net> and
-I'll register one for you.
-
-The second argument to _IO, _IOW, _IOR, or _IOWR is a sequence number
-to distinguish ioctls from each other. The third argument to _IOW,
-_IOR, or _IOWR is the type of the data going into the kernel or coming
-out of the kernel (e.g. 'int' or 'struct foo'). NOTE! Do NOT use
-sizeof(arg) as the third argument as this results in your ioctl thinking
-it passes an argument of type size_t.
-
-Some devices use their major number as the identifier; this is OK, as
-long as it is unique. Some devices are irregular and don't follow any
-convention at all.
-
-Following this convention is good because:
-
-(1) Keeping the ioctl's globally unique helps error checking:
- if a program calls an ioctl on the wrong device, it will get an
- error rather than some unexpected behaviour.
-
-(2) The 'strace' build procedure automatically finds ioctl numbers
- defined with _IO, _IOW, _IOR, or _IOWR.
-
-(3) 'strace' can decode numbers back into useful names when the
- numbers are unique.
-
-(4) People looking for ioctls can grep for them more easily when
- this convention is used to define the ioctl numbers.
-
-(5) When following the convention, the driver code can use generic
- code to copy the parameters between user and kernel space.
-
-This table lists ioctls visible from user land for Linux/x86. It contains
-most drivers up to 2.6.31, but I know I am missing some. There has been
-no attempt to list non-X86 architectures or ioctls from drivers/staging/.
-
-Code Seq#(hex) Include File Comments
-========================================================
-0x00 00-1F linux/fs.h conflict!
-0x00 00-1F scsi/scsi_ioctl.h conflict!
-0x00 00-1F linux/fb.h conflict!
-0x00 00-1F linux/wavefront.h conflict!
-0x02 all linux/fd.h
-0x03 all linux/hdreg.h
-0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these.
-0x06 all linux/lp.h
-0x09 all linux/raid/md_u.h
-0x10 00-0F drivers/char/s390/vmcp.h
-0x10 10-1F arch/s390/include/uapi/sclp_ctl.h
-0x10 20-2F arch/s390/include/uapi/asm/hypfs.h
-0x12 all linux/fs.h
- linux/blkpg.h
-0x1b all InfiniBand Subsystem <http://infiniband.sourceforge.net/>
-0x20 all drivers/cdrom/cm206.h
-0x22 all scsi/sg.h
-'!' 00-1F uapi/linux/seccomp.h
-'#' 00-3F IEEE 1394 Subsystem Block for the entire subsystem
-'$' 00-0F linux/perf_counter.h, linux/perf_event.h
-'%' 00-0F include/uapi/linux/stm.h
- System Trace Module subsystem
- <mailto:alexander.shishkin@linux.intel.com>
-'&' 00-07 drivers/firewire/nosy-user.h
-'1' 00-1F <linux/timepps.h> PPS kit from Ulrich Windl
- <ftp://ftp.de.kernel.org/pub/linux/daemons/ntp/PPS/>
-'2' 01-04 linux/i2o.h
-'3' 00-0F drivers/s390/char/raw3270.h conflict!
-'3' 00-1F linux/suspend_ioctls.h conflict!
- and kernel/power/user.c
-'8' all SNP8023 advanced NIC card
- <mailto:mcr@solidum.com>
-';' 64-7F linux/vfio.h
-'@' 00-0F linux/radeonfb.h conflict!
-'@' 00-0F drivers/video/aty/aty128fb.c conflict!
-'A' 00-1F linux/apm_bios.h conflict!
-'A' 00-0F linux/agpgart.h conflict!
- and drivers/char/agp/compat_ioctl.h
-'A' 00-7F sound/asound.h conflict!
-'B' 00-1F linux/cciss_ioctl.h conflict!
-'B' 00-0F include/linux/pmu.h conflict!
-'B' C0-FF advanced bbus
- <mailto:maassen@uni-freiburg.de>
-'C' all linux/soundcard.h conflict!
-'C' 01-2F linux/capi.h conflict!
-'C' F0-FF drivers/net/wan/cosa.h conflict!
-'D' all arch/s390/include/asm/dasd.h
-'D' 40-5F drivers/scsi/dpt/dtpi_ioctl.h
-'D' 05 drivers/scsi/pmcraid.h
-'E' all linux/input.h conflict!
-'E' 00-0F xen/evtchn.h conflict!
-'F' all linux/fb.h conflict!
-'F' 01-02 drivers/scsi/pmcraid.h conflict!
-'F' 20 drivers/video/fsl-diu-fb.h conflict!
-'F' 20 drivers/video/intelfb/intelfb.h conflict!
-'F' 20 linux/ivtvfb.h conflict!
-'F' 20 linux/matroxfb.h conflict!
-'F' 20 drivers/video/aty/atyfb_base.c conflict!
-'F' 00-0F video/da8xx-fb.h conflict!
-'F' 80-8F linux/arcfb.h conflict!
-'F' DD video/sstfb.h conflict!
-'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict!
-'G' 00-0F linux/gigaset_dev.h conflict!
-'H' 00-7F linux/hiddev.h conflict!
-'H' 00-0F linux/hidraw.h conflict!
-'H' 01 linux/mei.h conflict!
-'H' 02 linux/mei.h conflict!
-'H' 03 linux/mei.h conflict!
-'H' 00-0F sound/asound.h conflict!
-'H' 20-40 sound/asound_fm.h conflict!
-'H' 80-8F sound/sfnt_info.h conflict!
-'H' 10-8F sound/emu10k1.h conflict!
-'H' 10-1F sound/sb16_csp.h conflict!
-'H' 10-1F sound/hda_hwdep.h conflict!
-'H' 40-4F sound/hdspm.h conflict!
-'H' 40-4F sound/hdsp.h conflict!
-'H' 90 sound/usb/usx2y/usb_stream.h
-'H' A0 uapi/linux/usb/cdc-wdm.h
-'H' C0-F0 net/bluetooth/hci.h conflict!
-'H' C0-DF net/bluetooth/hidp/hidp.h conflict!
-'H' C0-DF net/bluetooth/cmtp/cmtp.h conflict!
-'H' C0-DF net/bluetooth/bnep/bnep.h conflict!
-'H' F1 linux/hid-roccat.h <mailto:erazor_de@users.sourceforge.net>
-'H' F8-FA sound/firewire.h
-'I' all linux/isdn.h conflict!
-'I' 00-0F drivers/isdn/divert/isdn_divert.h conflict!
-'I' 40-4F linux/mISDNif.h conflict!
-'J' 00-1F drivers/scsi/gdth_ioctl.h
-'K' all linux/kd.h
-'L' 00-1F linux/loop.h conflict!
-'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict!
-'L' 20-2F linux/lightnvm.h
-'L' E0-FF linux/ppdd.h encrypted disk device driver
- <http://linux01.gwdg.de/~alatham/ppdd.html>
-'M' all linux/soundcard.h conflict!
-'M' 01-16 mtd/mtd-abi.h conflict!
- and drivers/mtd/mtdchar.c
-'M' 01-03 drivers/scsi/megaraid/megaraid_sas.h
-'M' 00-0F drivers/video/fsl-diu-fb.h conflict!
-'N' 00-1F drivers/usb/scanner.h
-'N' 40-7F drivers/block/nvme.c
-'O' 00-06 mtd/ubi-user.h UBI
-'P' all linux/soundcard.h conflict!
-'P' 60-6F sound/sscape_ioctl.h conflict!
-'P' 00-0F drivers/usb/class/usblp.c conflict!
-'P' 01-09 drivers/misc/pci_endpoint_test.c conflict!
-'Q' all linux/soundcard.h
-'R' 00-1F linux/random.h conflict!
-'R' 01 linux/rfkill.h conflict!
-'R' C0-DF net/bluetooth/rfcomm.h
-'S' all linux/cdrom.h conflict!
-'S' 80-81 scsi/scsi_ioctl.h conflict!
-'S' 82-FF scsi/scsi.h conflict!
-'S' 00-7F sound/asequencer.h conflict!
-'T' all linux/soundcard.h conflict!
-'T' 00-AF sound/asound.h conflict!
-'T' all arch/x86/include/asm/ioctls.h conflict!
-'T' C0-DF linux/if_tun.h conflict!
-'U' all sound/asound.h conflict!
-'U' 00-CF linux/uinput.h conflict!
-'U' 00-EF linux/usbdevice_fs.h
-'U' C0-CF drivers/bluetooth/hci_uart.h
-'V' all linux/vt.h conflict!
-'V' all linux/videodev2.h conflict!
-'V' C0 linux/ivtvfb.h conflict!
-'V' C0 linux/ivtv.h conflict!
-'V' C0 media/davinci/vpfe_capture.h conflict!
-'V' C0 media/si4713.h conflict!
-'W' 00-1F linux/watchdog.h conflict!
-'W' 00-1F linux/wanrouter.h conflict! (pre 3.9)
-'W' 00-3F sound/asound.h conflict!
-'W' 40-5F drivers/pci/switch/switchtec.c
-'X' all fs/xfs/xfs_fs.h conflict!
- and fs/xfs/linux-2.6/xfs_ioctl32.h
- and include/linux/falloc.h
- and linux/fs.h
-'X' all fs/ocfs2/ocfs_fs.h conflict!
-'X' 01 linux/pktcdvd.h conflict!
-'Y' all linux/cyclades.h
-'Z' 14-15 drivers/message/fusion/mptctl.h
-'[' 00-3F linux/usb/tmc.h USB Test and Measurement Devices
- <mailto:gregkh@linuxfoundation.org>
-'a' all linux/atm*.h, linux/sonet.h ATM on linux
- <http://lrcwww.epfl.ch/>
-'a' 00-0F drivers/crypto/qat/qat_common/adf_cfg_common.h conflict! qat driver
-'b' 00-FF conflict! bit3 vme host bridge
- <mailto:natalia@nikhefk.nikhef.nl>
-'c' all linux/cm4000_cs.h conflict!
-'c' 00-7F linux/comstats.h conflict!
-'c' 00-7F linux/coda.h conflict!
-'c' 00-1F linux/chio.h conflict!
-'c' 80-9F arch/s390/include/asm/chsc.h conflict!
-'c' A0-AF arch/x86/include/asm/msr.h conflict!
-'d' 00-FF linux/char/drm/drm.h conflict!
-'d' 02-40 pcmcia/ds.h conflict!
-'d' F0-FF linux/digi1.h
-'e' all linux/digi1.h conflict!
-'f' 00-1F linux/ext2_fs.h conflict!
-'f' 00-1F linux/ext3_fs.h conflict!
-'f' 00-0F fs/jfs/jfs_dinode.h conflict!
-'f' 00-0F fs/ext4/ext4.h conflict!
-'f' 00-0F linux/fs.h conflict!
-'f' 00-0F fs/ocfs2/ocfs2_fs.h conflict!
-'g' 00-0F linux/usb/gadgetfs.h
-'g' 20-2F linux/usb/g_printer.h
-'h' 00-7F conflict! Charon filesystem
- <mailto:zapman@interlan.net>
-'h' 00-1F linux/hpet.h conflict!
-'h' 80-8F fs/hfsplus/ioctl.c
-'i' 00-3F linux/i2o-dev.h conflict!
-'i' 0B-1F linux/ipmi.h conflict!
-'i' 80-8F linux/i8k.h
-'j' 00-3F linux/joystick.h
-'k' 00-0F linux/spi/spidev.h conflict!
-'k' 00-05 video/kyro.h conflict!
-'k' 10-17 linux/hsi/hsi_char.h HSI character device
-'l' 00-3F linux/tcfs_fs.h transparent cryptographic file system
- <http://web.archive.org/web/*/http://mikonos.dia.unisa.it/tcfs>
-'l' 40-7F linux/udf_fs_i.h in development:
- <http://sourceforge.net/projects/linux-udf/>
-'m' 00-09 linux/mmtimer.h conflict!
-'m' all linux/mtio.h conflict!
-'m' all linux/soundcard.h conflict!
-'m' all linux/synclink.h conflict!
-'m' 00-19 drivers/message/fusion/mptctl.h conflict!
-'m' 00 drivers/scsi/megaraid/megaraid_ioctl.h conflict!
-'n' 00-7F linux/ncp_fs.h and fs/ncpfs/ioctl.c
-'n' 80-8F uapi/linux/nilfs2_api.h NILFS2
-'n' E0-FF linux/matroxfb.h matroxfb
-'o' 00-1F fs/ocfs2/ocfs2_fs.h OCFS2
-'o' 00-03 mtd/ubi-user.h conflict! (OCFS2 and UBI overlaps)
-'o' 40-41 mtd/ubi-user.h UBI
-'o' 01-A1 linux/dvb/*.h DVB
-'p' 00-0F linux/phantom.h conflict! (OpenHaptics needs this)
-'p' 00-1F linux/rtc.h conflict!
-'p' 00-3F linux/mc146818rtc.h conflict!
-'p' 40-7F linux/nvram.h
-'p' 80-9F linux/ppdev.h user-space parport
- <mailto:tim@cyberelk.net>
-'p' A1-A5 linux/pps.h LinuxPPS
- <mailto:giometti@linux.it>
-'q' 00-1F linux/serio.h
-'q' 80-FF linux/telephony.h Internet PhoneJACK, Internet LineJACK
- linux/ixjuser.h <http://web.archive.org/web/*/http://www.quicknet.net>
-'r' 00-1F linux/msdos_fs.h and fs/fat/dir.c
-'s' all linux/cdk.h
-'t' 00-7F linux/ppp-ioctl.h
-'t' 80-8F linux/isdn_ppp.h
-'t' 90-91 linux/toshiba.h toshiba and toshiba_acpi SMM
-'u' 00-1F linux/smb_fs.h gone
-'u' 20-3F linux/uvcvideo.h USB video class host driver
-'u' 40-4f linux/udmabuf.h userspace dma-buf misc device
-'v' 00-1F linux/ext2_fs.h conflict!
-'v' 00-1F linux/fs.h conflict!
-'v' 00-0F linux/sonypi.h conflict!
-'v' 00-0F media/v4l2-subdev.h conflict!
-'v' C0-FF linux/meye.h conflict!
-'w' all CERN SCI driver
-'y' 00-1F packet based user level communications
- <mailto:zapman@interlan.net>
-'z' 00-3F CAN bus card conflict!
- <mailto:hdstich@connectu.ulm.circular.de>
-'z' 40-7F CAN bus card conflict!
- <mailto:oe@port.de>
-'z' 10-4F drivers/s390/crypto/zcrypt_api.h conflict!
-'|' 00-7F linux/media.h
-0x80 00-1F linux/fb.h
-0x89 00-06 arch/x86/include/asm/sockios.h
-0x89 0B-DF linux/sockios.h
-0x89 E0-EF linux/sockios.h SIOCPROTOPRIVATE range
-0x89 E0-EF linux/dn.h PROTOPRIVATE range
-0x89 F0-FF linux/sockios.h SIOCDEVPRIVATE range
-0x8B all linux/wireless.h
-0x8C 00-3F WiNRADiO driver
- <http://www.winradio.com.au/>
-0x90 00 drivers/cdrom/sbpcd.h
-0x92 00-0F drivers/usb/mon/mon_bin.c
-0x93 60-7F linux/auto_fs.h
-0x94 all fs/btrfs/ioctl.h Btrfs filesystem
- and linux/fs.h some lifted to vfs/generic
-0x97 00-7F fs/ceph/ioctl.h Ceph file system
-0x99 00-0F 537-Addinboard driver
- <mailto:buk@buks.ipn.de>
-0xA0 all linux/sdp/sdp.h Industrial Device Project
- <mailto:kenji@bitgate.com>
-0xA1 0 linux/vtpm_proxy.h TPM Emulator Proxy Driver
-0xA3 80-8F Port ACL in development:
- <mailto:tlewis@mindspring.com>
-0xA3 90-9F linux/dtlk.h
-0xA4 00-1F uapi/linux/tee.h Generic TEE subsystem
-0xAA 00-3F linux/uapi/linux/userfaultfd.h
-0xAB 00-1F linux/nbd.h
-0xAC 00-1F linux/raw.h
-0xAD 00 Netfilter device in development:
- <mailto:rusty@rustcorp.com.au>
-0xAE all linux/kvm.h Kernel-based Virtual Machine
- <mailto:kvm@vger.kernel.org>
-0xAF 00-1F linux/fsl_hypervisor.h Freescale hypervisor
-0xB0 all RATIO devices in development:
- <mailto:vgo@ratio.de>
-0xB1 00-1F PPPoX <mailto:mostrows@styx.uwaterloo.ca>
-0xB3 00 linux/mmc/ioctl.h
-0xB4 00-0F linux/gpio.h <mailto:linux-gpio@vger.kernel.org>
-0xB5 00-0F uapi/linux/rpmsg.h <mailto:linux-remoteproc@vger.kernel.org>
-0xB6 all linux/fpga-dfl.h
-0xC0 00-0F linux/usb/iowarrior.h
-0xCA 00-0F uapi/misc/cxl.h
-0xCA 10-2F uapi/misc/ocxl.h
-0xCA 80-BF uapi/scsi/cxlflash_ioctl.h
-0xCB 00-1F CBM serial IEC bus in development:
- <mailto:michael.klein@puffin.lb.shuttle.de>
-0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
-0xCD 01 linux/reiserfs_fs.h
-0xCF 02 fs/cifs/ioctl.c
-0xDB 00-0F drivers/char/mwave/mwavepub.h
-0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
- <mailto:aherrman@de.ibm.com>
-0xE5 00-3F linux/fuse.h
-0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver
-0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development)
- <mailto:thomas@winischhofer.net>
-0xF4 00-1F video/mbxfb.h mbxfb
- <mailto:raph@8d.com>
-0xF6 all LTTng Linux Trace Toolkit Next Generation
- <mailto:mathieu.desnoyers@efficios.com>
-0xFD all linux/dm-ioctl.h
-0xFE all linux/isst_if.h
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
===================
Kernel Build System
-Recursion issue #1
-------------------
+================
+Recursion issues
+================
- .. include:: Kconfig.recursion-issue-01
- :literal:
+issue #1
+--------
-Recursion issue #2
-------------------
+.. literalinclude:: Kconfig.recursion-issue-01
+ :language: kconfig
- .. include:: Kconfig.recursion-issue-02
- :literal:
+issue #2
+--------
+
+.. literalinclude:: Kconfig.recursion-issue-02
+ :language: kconfig
by modprobe to not fail when trying to load something builtin.
modules.builtin.modinfo
---------------------------------------------------
+-----------------------
This file contains modinfo from all modules that are built into the kernel.
Unlike modinfo of a separate module, all fields are prefixed with module name.
The default name is "installkernel".
The script will be called with the following arguments:
+
- $1 - kernel version
- $2 - kernel image file
- $3 - kernel map file
applicable everywhere (see syntax).
- type definition: "bool"/"tristate"/"string"/"hex"/"int"
+
Every config option must have a type. There are only two basic types:
tristate and string; the other types are based on these two. The type
definition optionally accepts an input prompt, so these two examples
prompt "Networking support"
- input prompt: "prompt" <prompt> ["if" <expr>]
+
Every menu entry can have at most one prompt, which is used to display
to the user. Optionally dependencies only for this prompt can be added
with "if".
- default value: "default" <expr> ["if" <expr>]
+
A config option can have any number of default values. If multiple
default values are visible, only the first defined one is active.
Default values are not limited to the menu entry where they are
Optionally dependencies for this default value can be added with "if".
- dependencies: "depends on" <expr>
+
This defines a dependency for this menu entry. If multiple
dependencies are defined, they are connected with '&&'. Dependencies
are applied to all other options within this menu entry (which also
default y
- reverse dependencies: "select" <symbol> ["if" <expr>]
+
While normal dependencies reduce the upper limit of a symbol (see
below), reverse dependencies can be used to force a lower limit of
another symbol. The value of the current menu symbol is used as the
the illegal configurations all over.
- weak reverse dependencies: "imply" <symbol> ["if" <expr>]
+
This is similar to "select" as it enforces a lower limit on another
symbol except that the "implied" symbol's value may still be set to n
from a direct dependency or with a visible prompt.
configure that subsystem out without also having to unset these drivers.
- limiting menu display: "visible if" <expr>
+
This attribute is only applicable to menu blocks, if the condition is
false, the menu block is not displayed to the user (the symbols
contained there can still be selected by other symbols, though). It is
entries. Default value of "visible" is true.
- numerical ranges: "range" <symbol> <symbol> ["if" <expr>]
+
This allows to limit the range of possible input values for int
and hex symbols. The user can only input a value which is larger than
or equal to the first symbol and smaller than or equal to the second
symbol.
- help text: "help" or "---help---"
+
This defines a help text. The end of the help text is determined by
the indentation level, this means it ends at the first line which has
a smaller indentation than the first line of the help text.
the file as an aid to developers.
- misc options: "option" <symbol>[=<value>]
+
Various less common options can be defined via this option syntax,
which can modify the behaviour of the menu entry and its config
symbol. These options are currently possible:
The first five also start the definition of a menu entry.
config::
+
"config" <symbol>
<config options>
attributes as options.
menuconfig::
+
"menuconfig" <symbol>
<config options>
This mode shows all sub-menus in one large tree.
Example::
+
make NCONFIG_MODE=single_menu nconfig
----------------------------------------------------------------------
names, so you have to know something close to what you are
looking for.
- Example:
+ Example::
+
Ctrl-F hotplug
- or
+
+ or::
+
Menu: File, Search, hotplug
lists all config symbol entries that contain "hotplug" in
-----------------------
Kbuild tracks dependencies on the following:
+
1) All prerequisite files (both `*.c` and `*.h`)
2) `CONFIG_` options used in all prerequisite files
3) Command-line used to compile target
Further reading
===============
-- ``Documentation/locking/spinlocks.txt``: Linus Torvalds' spinlocking
+- ``Documentation/locking/spinlocks.rst``: Linus Torvalds' spinlocking
tutorial in the kernel sources.
- Unix Systems for Modern Architectures: Symmetric Multiprocessing and
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
====
LEDs
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
===================
Kernel Livepatching
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=======
+locking
+=======
+
+.. toctree::
+ :maxdepth: 1
+
+ lockdep-design
+ lockstat
+ locktorture
+ mutex-design
+ rt-mutex-design
+ rt-mutex
+ spinlocks
+ ww-mutex-design
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
=====================================
started by Ingo Molnar <mingo@redhat.com>
+
additions by Arjan van de Ven <arjan@linux.intel.com>
Lock-class
When locking rules are violated, these usage bits are presented in the
locking error messages, inside curlies, with a total of 2 * n STATEs bits.
-A contrived example:
+A contrived example::
modprobe/2287 is trying to acquire lock:
(&sio_locks[i].lock){-.-.}, at: [<c02867fd>] mutex_lock+0x21/0x24
above respectively, and the character displayed at each bit position
indicates:
+ === ===================================================
'.' acquired while irqs disabled and not in irq context
'-' acquired in irq context
'+' acquired with irqs enabled
'?' acquired in irq context with irqs enabled.
+ === ===================================================
-The bits are illustrated with an example:
+The bits are illustrated with an example::
(&sio_locks[i].lock){-.-.}, at: [<c02867fd>] mutex_lock+0x21/0x24
||||
shown in the table below. The bit character is able to indicate which
exact case is for the lock as of the reporting time.
- -------------------------------------------
+ +--------------+-------------+--------------+
| | irq enabled | irq disabled |
- |-------------------------------------------|
+ +--------------+-------------+--------------+
| ever in irq | ? | - |
- |-------------------------------------------|
+ +--------------+-------------+--------------+
| never in irq | + | . |
- -------------------------------------------
+ +--------------+-------------+--------------+
The character '-' suggests irq is disabled because if otherwise the
charactor '?' would have been shown instead. Similar deduction can be
A softirq-unsafe lock-class is automatically hardirq-unsafe as well. The
following states must be exclusive: only one of them is allowed to be set
-for any lock-class based on its usage:
+for any lock-class based on its usage::
<hardirq-safe> or <hardirq-unsafe>
<softirq-safe> or <softirq-unsafe>
The same lock-class must not be acquired twice, because this could lead
to lock recursion deadlocks.
-Furthermore, two locks can not be taken in inverse order:
+Furthermore, two locks can not be taken in inverse order::
<L1> -> <L2>
<L2> -> <L1>
acquired in a circular fashion.
Furthermore, the following usage based lock dependencies are not allowed
-between any two lock-classes:
+between any two lock-classes::
<hardirq-safe> -> <hardirq-unsafe>
<softirq-safe> -> <softirq-unsafe>
In order to teach the validator about this correct usage model, new
versions of the various locking primitives were added that allow you to
specify a "nesting level". An example call, for the block device mutex,
-looks like this:
+looks like this::
-enum bdev_bd_mutex_lock_class
-{
+ enum bdev_bd_mutex_lock_class
+ {
BD_MUTEX_NORMAL,
BD_MUTEX_WHOLE,
BD_MUTEX_PARTITION
-};
+ };
- mutex_lock_nested(&bdev->bd_contains->bd_mutex, BD_MUTEX_PARTITION);
+mutex_lock_nested(&bdev->bd_contains->bd_mutex, BD_MUTEX_PARTITION);
In this case the locking is done on a bdev object that is known to be a
partition.
As the name suggests, lockdep_assert_held* family of macros assert that a
particular lock is held at a certain time (and generate a WARN() otherwise).
This annotation is largely used all over the kernel, e.g. kernel/sched/
-core.c
+core.c::
void update_rq_clock(struct rq *rq)
{
layer assumes a lock remains taken, but a lower layer thinks it can maybe drop
and reacquire the lock ("unwittingly" introducing races). lockdep_pin_lock()
returns a 'struct pin_cookie' that is then used by lockdep_unpin_lock() to check
-that nobody tampered with the lock, e.g. kernel/sched/sched.h
+that nobody tampered with the lock, e.g. kernel/sched/sched.h::
static inline void rq_pin_lock(struct rq *rq, struct rq_flags *rf)
{
locking sequence that occurred at least once during the lifetime of the
kernel, the validator proves it with a 100% certainty that no
combination and timing of these locking sequences can cause any class of
-lock related deadlock. [*]
+lock related deadlock. [1]_
I.e. complex multi-CPU and multi-task locking scenarios do not have to
occur in practice to prove a deadlock: only the simple 'component'
every possible hardirq and softirq nesting scenario (which is impossible
to do in practice).
-[*] assuming that the validator itself is 100% correct, and no other
+.. [1]
+
+ assuming that the validator itself is 100% correct, and no other
part of the system corrupts the state of the validator in any way.
We also assume that all NMI/SMM paths [which could interrupt
even hardirq-disabled codepaths] are correct and do not interfere
Performance:
------------
-The above rules require _massive_ amounts of runtime checking. If we did
+The above rules require **massive** amounts of runtime checking. If we did
that for every lock taken and for every irqs-enable event, it would
render the system practically unusably slow. The complexity of checking
is O(N^2), so even with just a few hundred lock-classes we'd have to do
Of course, if you do run out of lock classes, the next thing to do is
to find the offending lock classes. First, the following command gives
-you the number of lock classes currently in use along with the maximum:
+you the number of lock classes currently in use along with the maximum::
grep "lock-classes" /proc/lockdep_stats
-This command produces the following output on a modest system:
+This command produces the following output on a modest system::
- lock-classes: 748 [max: 8191]
+ lock-classes: 748 [max: 8191]
If the number allocated (748 above) increases continually over time,
then there is likely a leak. The following command can be used to
-identify the leaking lock classes:
+identify the leaking lock classes::
grep "BD" /proc/lockdep
--- /dev/null
+===============
+Lock Statistics
+===============
+
+What
+====
+
+As the name suggests, it provides statistics on locks.
+
+
+Why
+===
+
+Because things like lock contention can severely impact performance.
+
+How
+===
+
+Lockdep already has hooks in the lock functions and maps lock instances to
+lock classes. We build on that (see Documentation/locking/lockdep-design.rst).
+The graph below shows the relation between the lock functions and the various
+hooks therein::
+
+ __acquire
+ |
+ lock _____
+ | \
+ | __contended
+ | |
+ | <wait>
+ | _______/
+ |/
+ |
+ __acquired
+ |
+ .
+ <hold>
+ .
+ |
+ __release
+ |
+ unlock
+
+ lock, unlock - the regular lock functions
+ __* - the hooks
+ <> - states
+
+With these hooks we provide the following statistics:
+
+ con-bounces
+ - number of lock contention that involved x-cpu data
+ contentions
+ - number of lock acquisitions that had to wait
+ wait time
+ min
+ - shortest (non-0) time we ever had to wait for a lock
+ max
+ - longest time we ever had to wait for a lock
+ total
+ - total time we spend waiting on this lock
+ avg
+ - average time spent waiting on this lock
+ acq-bounces
+ - number of lock acquisitions that involved x-cpu data
+ acquisitions
+ - number of times we took the lock
+ hold time
+ min
+ - shortest (non-0) time we ever held the lock
+ max
+ - longest time we ever held the lock
+ total
+ - total time this lock was held
+ avg
+ - average time this lock was held
+
+These numbers are gathered per lock class, per read/write state (when
+applicable).
+
+It also tracks 4 contention points per class. A contention point is a call site
+that had to wait on lock acquisition.
+
+Configuration
+-------------
+
+Lock statistics are enabled via CONFIG_LOCK_STAT.
+
+Usage
+-----
+
+Enable collection of statistics::
+
+ # echo 1 >/proc/sys/kernel/lock_stat
+
+Disable collection of statistics::
+
+ # echo 0 >/proc/sys/kernel/lock_stat
+
+Look at the current lock statistics::
+
+ ( line numbers not part of actual output, done for clarity in the explanation
+ below )
+
+ # less /proc/lock_stat
+
+ 01 lock_stat version 0.4
+ 02-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ 03 class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
+ 04-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ 05
+ 06 &mm->mmap_sem-W: 46 84 0.26 939.10 16371.53 194.90 47291 2922365 0.16 2220301.69 17464026916.32 5975.99
+ 07 &mm->mmap_sem-R: 37 100 1.31 299502.61 325629.52 3256.30 212344 34316685 0.10 7744.91 95016910.20 2.77
+ 08 ---------------
+ 09 &mm->mmap_sem 1 [<ffffffff811502a7>] khugepaged_scan_mm_slot+0x57/0x280
+ 10 &mm->mmap_sem 96 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
+ 11 &mm->mmap_sem 34 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
+ 12 &mm->mmap_sem 17 [<ffffffff81127e71>] vm_munmap+0x41/0x80
+ 13 ---------------
+ 14 &mm->mmap_sem 1 [<ffffffff81046fda>] dup_mmap+0x2a/0x3f0
+ 15 &mm->mmap_sem 60 [<ffffffff81129e29>] SyS_mprotect+0xe9/0x250
+ 16 &mm->mmap_sem 41 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
+ 17 &mm->mmap_sem 68 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
+ 18
+ 19.............................................................................................................................................................................................................................
+ 20
+ 21 unix_table_lock: 110 112 0.21 49.24 163.91 1.46 21094 66312 0.12 624.42 31589.81 0.48
+ 22 ---------------
+ 23 unix_table_lock 45 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
+ 24 unix_table_lock 47 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
+ 25 unix_table_lock 15 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
+ 26 unix_table_lock 5 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
+ 27 ---------------
+ 28 unix_table_lock 39 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
+ 29 unix_table_lock 49 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
+ 30 unix_table_lock 20 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
+ 31 unix_table_lock 4 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
+
+
+This excerpt shows the first two lock class statistics. Line 01 shows the
+output version - each time the format changes this will be updated. Line 02-04
+show the header with column descriptions. Lines 05-18 and 20-31 show the actual
+statistics. These statistics come in two parts; the actual stats separated by a
+short separator (line 08, 13) from the contention points.
+
+Lines 09-12 show the first 4 recorded contention points (the code
+which tries to get the lock) and lines 14-17 show the first 4 recorded
+contended points (the lock holder). It is possible that the max
+con-bounces point is missing in the statistics.
+
+The first lock (05-18) is a read/write lock, and shows two lines above the
+short separator. The contention points don't match the column descriptors,
+they have two: contentions and [<IP>] symbol. The second set of contention
+points are the points we're contending with.
+
+The integer part of the time values is in us.
+
+Dealing with nested locks, subclasses may appear::
+
+ 32...........................................................................................................................................................................................................................
+ 33
+ 34 &rq->lock: 13128 13128 0.43 190.53 103881.26 7.91 97454 3453404 0.00 401.11 13224683.11 3.82
+ 35 ---------
+ 36 &rq->lock 645 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
+ 37 &rq->lock 297 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
+ 38 &rq->lock 360 [<ffffffff8103c4c5>] select_task_rq_fair+0x1f0/0x74a
+ 39 &rq->lock 428 [<ffffffff81045f98>] scheduler_tick+0x46/0x1fb
+ 40 ---------
+ 41 &rq->lock 77 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
+ 42 &rq->lock 174 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
+ 43 &rq->lock 4715 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
+ 44 &rq->lock 893 [<ffffffff81340524>] schedule+0x157/0x7b8
+ 45
+ 46...........................................................................................................................................................................................................................
+ 47
+ 48 &rq->lock/1: 1526 11488 0.33 388.73 136294.31 11.86 21461 38404 0.00 37.93 109388.53 2.84
+ 49 -----------
+ 50 &rq->lock/1 11526 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
+ 51 -----------
+ 52 &rq->lock/1 5645 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
+ 53 &rq->lock/1 1224 [<ffffffff81340524>] schedule+0x157/0x7b8
+ 54 &rq->lock/1 4336 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
+ 55 &rq->lock/1 181 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
+
+Line 48 shows statistics for the second subclass (/1) of &rq->lock class
+(subclass starts from 0), since in this case, as line 50 suggests,
+double_rq_lock actually acquires a nested lock of two spinlocks.
+
+View the top contending locks::
+
+ # grep : /proc/lock_stat | head
+ clockevents_lock: 2926159 2947636 0.15 46882.81 1784540466.34 605.41 3381345 3879161 0.00 2260.97 53178395.68 13.71
+ tick_broadcast_lock: 346460 346717 0.18 2257.43 39364622.71 113.54 3642919 4242696 0.00 2263.79 49173646.60 11.59
+ &mapping->i_mmap_mutex: 203896 203899 3.36 645530.05 31767507988.39 155800.21 3361776 8893984 0.17 2254.15 14110121.02 1.59
+ &rq->lock: 135014 136909 0.18 606.09 842160.68 6.15 1540728 10436146 0.00 728.72 17606683.41 1.69
+ &(&zone->lru_lock)->rlock: 93000 94934 0.16 59.18 188253.78 1.98 1199912 3809894 0.15 391.40 3559518.81 0.93
+ tasklist_lock-W: 40667 41130 0.23 1189.42 428980.51 10.43 270278 510106 0.16 653.51 3939674.91 7.72
+ tasklist_lock-R: 21298 21305 0.20 1310.05 215511.12 10.12 186204 241258 0.14 1162.33 1179779.23 4.89
+ rcu_node_1: 47656 49022 0.16 635.41 193616.41 3.95 844888 1865423 0.00 764.26 1656226.96 0.89
+ &(&dentry->d_lockref.lock)->rlock: 39791 40179 0.15 1302.08 88851.96 2.21 2790851 12527025 0.10 1910.75 3379714.27 0.27
+ rcu_node_0: 29203 30064 0.16 786.55 1555573.00 51.74 88963 244254 0.00 398.87 428872.51 1.76
+
+Clear the statistics::
+
+ # echo 0 > /proc/lock_stat
+++ /dev/null
-
-LOCK STATISTICS
-
-- WHAT
-
-As the name suggests, it provides statistics on locks.
-
-- WHY
-
-Because things like lock contention can severely impact performance.
-
-- HOW
-
-Lockdep already has hooks in the lock functions and maps lock instances to
-lock classes. We build on that (see Documentation/locking/lockdep-design.txt).
-The graph below shows the relation between the lock functions and the various
-hooks therein.
-
- __acquire
- |
- lock _____
- | \
- | __contended
- | |
- | <wait>
- | _______/
- |/
- |
- __acquired
- |
- .
- <hold>
- .
- |
- __release
- |
- unlock
-
-lock, unlock - the regular lock functions
-__* - the hooks
-<> - states
-
-With these hooks we provide the following statistics:
-
- con-bounces - number of lock contention that involved x-cpu data
- contentions - number of lock acquisitions that had to wait
- wait time min - shortest (non-0) time we ever had to wait for a lock
- max - longest time we ever had to wait for a lock
- total - total time we spend waiting on this lock
- avg - average time spent waiting on this lock
- acq-bounces - number of lock acquisitions that involved x-cpu data
- acquisitions - number of times we took the lock
- hold time min - shortest (non-0) time we ever held the lock
- max - longest time we ever held the lock
- total - total time this lock was held
- avg - average time this lock was held
-
-These numbers are gathered per lock class, per read/write state (when
-applicable).
-
-It also tracks 4 contention points per class. A contention point is a call site
-that had to wait on lock acquisition.
-
- - CONFIGURATION
-
-Lock statistics are enabled via CONFIG_LOCK_STAT.
-
- - USAGE
-
-Enable collection of statistics:
-
-# echo 1 >/proc/sys/kernel/lock_stat
-
-Disable collection of statistics:
-
-# echo 0 >/proc/sys/kernel/lock_stat
-
-Look at the current lock statistics:
-
-( line numbers not part of actual output, done for clarity in the explanation
- below )
-
-# less /proc/lock_stat
-
-01 lock_stat version 0.4
-02-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-03 class name con-bounces contentions waittime-min waittime-max waittime-total waittime-avg acq-bounces acquisitions holdtime-min holdtime-max holdtime-total holdtime-avg
-04-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-05
-06 &mm->mmap_sem-W: 46 84 0.26 939.10 16371.53 194.90 47291 2922365 0.16 2220301.69 17464026916.32 5975.99
-07 &mm->mmap_sem-R: 37 100 1.31 299502.61 325629.52 3256.30 212344 34316685 0.10 7744.91 95016910.20 2.77
-08 ---------------
-09 &mm->mmap_sem 1 [<ffffffff811502a7>] khugepaged_scan_mm_slot+0x57/0x280
-10 &mm->mmap_sem 96 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
-11 &mm->mmap_sem 34 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
-12 &mm->mmap_sem 17 [<ffffffff81127e71>] vm_munmap+0x41/0x80
-13 ---------------
-14 &mm->mmap_sem 1 [<ffffffff81046fda>] dup_mmap+0x2a/0x3f0
-15 &mm->mmap_sem 60 [<ffffffff81129e29>] SyS_mprotect+0xe9/0x250
-16 &mm->mmap_sem 41 [<ffffffff815351c4>] __do_page_fault+0x1d4/0x510
-17 &mm->mmap_sem 68 [<ffffffff81113d77>] vm_mmap_pgoff+0x87/0xd0
-18
-19.............................................................................................................................................................................................................................
-20
-21 unix_table_lock: 110 112 0.21 49.24 163.91 1.46 21094 66312 0.12 624.42 31589.81 0.48
-22 ---------------
-23 unix_table_lock 45 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
-24 unix_table_lock 47 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
-25 unix_table_lock 15 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
-26 unix_table_lock 5 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
-27 ---------------
-28 unix_table_lock 39 [<ffffffff8150b111>] unix_release_sock+0x31/0x250
-29 unix_table_lock 49 [<ffffffff8150ad8e>] unix_create1+0x16e/0x1b0
-30 unix_table_lock 20 [<ffffffff8150ca37>] unix_find_other+0x117/0x230
-31 unix_table_lock 4 [<ffffffff8150a09f>] unix_autobind+0x11f/0x1b0
-
-
-This excerpt shows the first two lock class statistics. Line 01 shows the
-output version - each time the format changes this will be updated. Line 02-04
-show the header with column descriptions. Lines 05-18 and 20-31 show the actual
-statistics. These statistics come in two parts; the actual stats separated by a
-short separator (line 08, 13) from the contention points.
-
-Lines 09-12 show the first 4 recorded contention points (the code
-which tries to get the lock) and lines 14-17 show the first 4 recorded
-contended points (the lock holder). It is possible that the max
-con-bounces point is missing in the statistics.
-
-The first lock (05-18) is a read/write lock, and shows two lines above the
-short separator. The contention points don't match the column descriptors,
-they have two: contentions and [<IP>] symbol. The second set of contention
-points are the points we're contending with.
-
-The integer part of the time values is in us.
-
-Dealing with nested locks, subclasses may appear:
-
-32...........................................................................................................................................................................................................................
-33
-34 &rq->lock: 13128 13128 0.43 190.53 103881.26 7.91 97454 3453404 0.00 401.11 13224683.11 3.82
-35 ---------
-36 &rq->lock 645 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
-37 &rq->lock 297 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
-38 &rq->lock 360 [<ffffffff8103c4c5>] select_task_rq_fair+0x1f0/0x74a
-39 &rq->lock 428 [<ffffffff81045f98>] scheduler_tick+0x46/0x1fb
-40 ---------
-41 &rq->lock 77 [<ffffffff8103bfc4>] task_rq_lock+0x43/0x75
-42 &rq->lock 174 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
-43 &rq->lock 4715 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
-44 &rq->lock 893 [<ffffffff81340524>] schedule+0x157/0x7b8
-45
-46...........................................................................................................................................................................................................................
-47
-48 &rq->lock/1: 1526 11488 0.33 388.73 136294.31 11.86 21461 38404 0.00 37.93 109388.53 2.84
-49 -----------
-50 &rq->lock/1 11526 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
-51 -----------
-52 &rq->lock/1 5645 [<ffffffff8103ed4b>] double_rq_lock+0x42/0x54
-53 &rq->lock/1 1224 [<ffffffff81340524>] schedule+0x157/0x7b8
-54 &rq->lock/1 4336 [<ffffffff8103ed58>] double_rq_lock+0x4f/0x54
-55 &rq->lock/1 181 [<ffffffff8104ba65>] try_to_wake_up+0x127/0x25a
-
-Line 48 shows statistics for the second subclass (/1) of &rq->lock class
-(subclass starts from 0), since in this case, as line 50 suggests,
-double_rq_lock actually acquires a nested lock of two spinlocks.
-
-View the top contending locks:
-
-# grep : /proc/lock_stat | head
- clockevents_lock: 2926159 2947636 0.15 46882.81 1784540466.34 605.41 3381345 3879161 0.00 2260.97 53178395.68 13.71
- tick_broadcast_lock: 346460 346717 0.18 2257.43 39364622.71 113.54 3642919 4242696 0.00 2263.79 49173646.60 11.59
- &mapping->i_mmap_mutex: 203896 203899 3.36 645530.05 31767507988.39 155800.21 3361776 8893984 0.17 2254.15 14110121.02 1.59
- &rq->lock: 135014 136909 0.18 606.09 842160.68 6.15 1540728 10436146 0.00 728.72 17606683.41 1.69
- &(&zone->lru_lock)->rlock: 93000 94934 0.16 59.18 188253.78 1.98 1199912 3809894 0.15 391.40 3559518.81 0.93
- tasklist_lock-W: 40667 41130 0.23 1189.42 428980.51 10.43 270278 510106 0.16 653.51 3939674.91 7.72
- tasklist_lock-R: 21298 21305 0.20 1310.05 215511.12 10.12 186204 241258 0.14 1162.33 1179779.23 4.89
- rcu_node_1: 47656 49022 0.16 635.41 193616.41 3.95 844888 1865423 0.00 764.26 1656226.96 0.89
- &(&dentry->d_lockref.lock)->rlock: 39791 40179 0.15 1302.08 88851.96 2.21 2790851 12527025 0.10 1910.75 3379714.27 0.27
- rcu_node_0: 29203 30064 0.16 786.55 1555573.00 51.74 88963 244254 0.00 398.87 428872.51 1.76
-
-Clear the statistics:
-
-# echo 0 > /proc/lock_stat
+==================================
Kernel Lock Torture Test Operation
+==================================
CONFIG_LOCK_TORTURE_TEST
+========================
The CONFIG LOCK_TORTURE_TEST config option provides a kernel module
that runs torture tests on core kernel locking primitives. The kernel
creating more kthreads.
-MODULE PARAMETERS
+Module Parameters
+=================
This module has the following parameters:
- ** Locktorture-specific **
+Locktorture-specific
+--------------------
-nwriters_stress Number of kernel threads that will stress exclusive lock
+nwriters_stress
+ Number of kernel threads that will stress exclusive lock
ownership (writers). The default value is twice the number
of online CPUs.
-nreaders_stress Number of kernel threads that will stress shared lock
+nreaders_stress
+ Number of kernel threads that will stress shared lock
ownership (readers). The default is the same amount of writer
locks. If the user did not specify nwriters_stress, then
both readers and writers be the amount of online CPUs.
-torture_type Type of lock to torture. By default, only spinlocks will
+torture_type
+ Type of lock to torture. By default, only spinlocks will
be tortured. This module can torture the following locks,
with string values as follows:
- o "lock_busted": Simulates a buggy lock implementation.
+ - "lock_busted":
+ Simulates a buggy lock implementation.
- o "spin_lock": spin_lock() and spin_unlock() pairs.
+ - "spin_lock":
+ spin_lock() and spin_unlock() pairs.
- o "spin_lock_irq": spin_lock_irq() and spin_unlock_irq()
- pairs.
+ - "spin_lock_irq":
+ spin_lock_irq() and spin_unlock_irq() pairs.
- o "rw_lock": read/write lock() and unlock() rwlock pairs.
+ - "rw_lock":
+ read/write lock() and unlock() rwlock pairs.
- o "rw_lock_irq": read/write lock_irq() and unlock_irq()
- rwlock pairs.
+ - "rw_lock_irq":
+ read/write lock_irq() and unlock_irq()
+ rwlock pairs.
- o "mutex_lock": mutex_lock() and mutex_unlock() pairs.
+ - "mutex_lock":
+ mutex_lock() and mutex_unlock() pairs.
- o "rtmutex_lock": rtmutex_lock() and rtmutex_unlock()
- pairs. Kernel must have CONFIG_RT_MUTEX=y.
+ - "rtmutex_lock":
+ rtmutex_lock() and rtmutex_unlock() pairs.
+ Kernel must have CONFIG_RT_MUTEX=y.
- o "rwsem_lock": read/write down() and up() semaphore pairs.
+ - "rwsem_lock":
+ read/write down() and up() semaphore pairs.
- ** Torture-framework (RCU + locking) **
+Torture-framework (RCU + locking)
+---------------------------------
-shutdown_secs The number of seconds to run the test before terminating
+shutdown_secs
+ The number of seconds to run the test before terminating
the test and powering off the system. The default is
zero, which disables test termination and system shutdown.
This capability is useful for automated testing.
-onoff_interval The number of seconds between each attempt to execute a
+onoff_interval
+ The number of seconds between each attempt to execute a
randomly selected CPU-hotplug operation. Defaults
to zero, which disables CPU hotplugging. In
CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently
refuse to do any CPU-hotplug operations regardless of
what value is specified for onoff_interval.
-onoff_holdoff The number of seconds to wait until starting CPU-hotplug
+onoff_holdoff
+ The number of seconds to wait until starting CPU-hotplug
operations. This would normally only be used when
locktorture was built into the kernel and started
automatically at boot time, in which case it is useful
coming and going. This parameter is only useful if
CONFIG_HOTPLUG_CPU is enabled.
-stat_interval Number of seconds between statistics-related printk()s.
+stat_interval
+ Number of seconds between statistics-related printk()s.
By default, locktorture will report stats every 60 seconds.
Setting the interval to zero causes the statistics to
be printed -only- when the module is unloaded, and this
is the default.
-stutter The length of time to run the test before pausing for this
+stutter
+ The length of time to run the test before pausing for this
same period of time. Defaults to "stutter=5", so as
to run and pause for (roughly) five-second intervals.
Specifying "stutter=0" causes the test to run continuously
without pausing, which is the old default behavior.
-shuffle_interval The number of seconds to keep the test threads affinitied
+shuffle_interval
+ The number of seconds to keep the test threads affinitied
to a particular subset of the CPUs, defaults to 3 seconds.
Used in conjunction with test_no_idle_hz.
-verbose Enable verbose debugging printing, via printk(). Enabled
+verbose
+ Enable verbose debugging printing, via printk(). Enabled
by default. This extra information is mostly related to
high-level errors and reports from the main 'torture'
framework.
-STATISTICS
+Statistics
+==========
-Statistics are printed in the following format:
+Statistics are printed in the following format::
-spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0
- (A) (B) (C) (D) (E)
+ spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0
+ (A) (B) (C) (D) (E)
-(A): Lock type that is being tortured -- torture_type parameter.
+ (A): Lock type that is being tortured -- torture_type parameter.
-(B): Number of writer lock acquisitions. If dealing with a read/write primitive
- a second "Reads" statistics line is printed.
+ (B): Number of writer lock acquisitions. If dealing with a read/write
+ primitive a second "Reads" statistics line is printed.
-(C): Number of times the lock was acquired.
+ (C): Number of times the lock was acquired.
-(D): Min and max number of times threads failed to acquire the lock.
+ (D): Min and max number of times threads failed to acquire the lock.
-(E): true/false values if there were errors acquiring the lock. This should
- -only- be positive if there is a bug in the locking primitive's
- implementation. Otherwise a lock should never fail (i.e., spin_lock()).
- Of course, the same applies for (C), above. A dummy example of this is
- the "lock_busted" type.
+ (E): true/false values if there were errors acquiring the lock. This should
+ -only- be positive if there is a bug in the locking primitive's
+ implementation. Otherwise a lock should never fail (i.e., spin_lock()).
+ Of course, the same applies for (C), above. A dummy example of this is
+ the "lock_busted" type.
-USAGE
+Usage
+=====
-The following script may be used to torture locks:
+The following script may be used to torture locks::
#!/bin/sh
+=======================
Generic Mutex Subsystem
+=======================
started by Ingo Molnar <mingo@redhat.com>
+
updated by Davidlohr Bueso <davidlohr@hp.com>
What are mutexes?
Mutexes are represented by 'struct mutex', defined in include/linux/mutex.h
and implemented in kernel/locking/mutex.c. These locks use an atomic variable
(->owner) to keep track of the lock state during its lifetime. Field owner
-actually contains 'struct task_struct *' to the current lock owner and it is
+actually contains `struct task_struct *` to the current lock owner and it is
therefore NULL if not currently owned. Since task_struct pointers are aligned
at at least L1_CACHE_BYTES, low bits (3) are used to store extra state (e.g.,
if waiter list is non-empty). In its most basic form it also includes a
Interfaces
----------
-Statically define the mutex:
+Statically define the mutex::
+
DEFINE_MUTEX(name);
-Dynamically initialize the mutex:
+Dynamically initialize the mutex::
+
mutex_init(mutex);
-Acquire the mutex, uninterruptible:
+Acquire the mutex, uninterruptible::
+
void mutex_lock(struct mutex *lock);
void mutex_lock_nested(struct mutex *lock, unsigned int subclass);
int mutex_trylock(struct mutex *lock);
-Acquire the mutex, interruptible:
+Acquire the mutex, interruptible::
+
int mutex_lock_interruptible_nested(struct mutex *lock,
unsigned int subclass);
int mutex_lock_interruptible(struct mutex *lock);
-Acquire the mutex, interruptible, if dec to 0:
+Acquire the mutex, interruptible, if dec to 0::
+
int atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock);
-Unlock the mutex:
+Unlock the mutex::
+
void mutex_unlock(struct mutex *lock);
-Test if the mutex is taken:
+Test if the mutex is taken::
+
int mutex_is_locked(struct mutex *lock);
Disadvantages
-#
-# Copyright (c) 2006 Steven Rostedt
-# Licensed under the GNU Free Documentation License, Version 1.2
-#
-
+==============================
RT-mutex implementation design
-------------------------------
+==============================
+
+Copyright (c) 2006 Steven Rostedt
+
+Licensed under the GNU Free Documentation License, Version 1.2
+
This document tries to describe the design of the rtmutex.c implementation.
It doesn't describe the reasons why rtmutex.c exists. For that please see
-Documentation/locking/rt-mutex.txt. Although this document does explain problems
+Documentation/locking/rt-mutex.rst. Although this document does explain problems
that happen without this code, but that is in the concept to understand
what the code actually is doing.
never give C a chance to release the lock. This is called unbounded priority
inversion.
-Here's a little ASCII art to show the problem.
+Here's a little ASCII art to show the problem::
- grab lock L1 (owned by C)
- |
-A ---+
- C preempted by B
- |
-C +----+
+ grab lock L1 (owned by C)
+ |
+ A ---+
+ C preempted by B
+ |
+ C +----+
-B +-------->
- B now keeps A from running.
+ B +-------->
+ B now keeps A from running.
Priority Inheritance (PI)
Here I explain some terminology that is used in this document to help describe
the design that is used to implement PI.
-PI chain - The PI chain is an ordered series of locks and processes that cause
+PI chain
+ - The PI chain is an ordered series of locks and processes that cause
processes to inherit priorities from a previous process that is
blocked on one of its locks. This is described in more detail
later in this document.
-mutex - In this document, to differentiate from locks that implement
+mutex
+ - In this document, to differentiate from locks that implement
PI and spin locks that are used in the PI code, from now on
the PI locks will be called a mutex.
-lock - In this document from now on, I will use the term lock when
+lock
+ - In this document from now on, I will use the term lock when
referring to spin locks that are used to protect parts of the PI
algorithm. These locks disable preemption for UP (when
CONFIG_PREEMPT is enabled) and on SMP prevents multiple CPUs from
entering critical sections simultaneously.
-spin lock - Same as lock above.
+spin lock
+ - Same as lock above.
-waiter - A waiter is a struct that is stored on the stack of a blocked
+waiter
+ - A waiter is a struct that is stored on the stack of a blocked
process. Since the scope of the waiter is within the code for
a process being blocked on the mutex, it is fine to allocate
the waiter on the process's stack (local variable). This
waiter is sometimes used in reference to the task that is waiting
on a mutex. This is the same as waiter->task.
-waiters - A list of processes that are blocked on a mutex.
+waiters
+ - A list of processes that are blocked on a mutex.
-top waiter - The highest priority process waiting on a specific mutex.
+top waiter
+ - The highest priority process waiting on a specific mutex.
-top pi waiter - The highest priority process waiting on one of the mutexes
+top pi waiter
+ - The highest priority process waiting on one of the mutexes
that a specific process owns.
-Note: task and process are used interchangeably in this document, mostly to
+Note:
+ task and process are used interchangeably in this document, mostly to
differentiate between two processes that are being described together.
would never diverge, since a process can't be blocked on more than one
mutex at a time.
-Example:
+Example::
Process: A, B, C, D, E
Mutexes: L1, L2, L3, L4
D owns L4
E blocked on L4
-The chain would be:
+The chain would be::
E->L4->D->L3->C->L2->B->L1->A
To show where two chains merge, we could add another process F and
another mutex L5 where B owns L5 and F is blocked on mutex L5.
-The chain for F would be:
+The chain for F would be::
F->L5->B->L1->A
Since a process may own more than one mutex, but never be blocked on more than
one, the chains merge.
-Here we show both chains:
+Here we show both chains::
E->L4->D->L3->C->L2-+
|
Also since a mutex may have more than one process blocked on it, we can
have multiple chains merge at mutexes. If we add another process G that is
-blocked on mutex L2:
+blocked on mutex L2::
G->L2->B->L1->A
And once again, to show how this can grow I will show the merging chains
-again.
+again::
E->L4->D->L3->C-+
+->L2-+
to that of G.
Mutex Waiters Tree
------------------
+------------------
Every mutex keeps track of all the waiters that are blocked on itself. The
mutex has a rbtree to store these waiters by priority. This tree is protected
the nesting of mutexes. Let's look at the example where we have 3 mutexes,
L1, L2, and L3, and four separate functions func1, func2, func3 and func4.
The following shows a locking order of L1->L2->L3, but may not actually
-be directly nested that way.
+be directly nested that way::
-void func1(void)
-{
+ void func1(void)
+ {
mutex_lock(L1);
/* do anything */
mutex_unlock(L1);
-}
+ }
-void func2(void)
-{
+ void func2(void)
+ {
mutex_lock(L1);
mutex_lock(L2);
mutex_unlock(L2);
mutex_unlock(L1);
-}
+ }
-void func3(void)
-{
+ void func3(void)
+ {
mutex_lock(L2);
mutex_lock(L3);
mutex_unlock(L3);
mutex_unlock(L2);
-}
+ }
-void func4(void)
-{
+ void func4(void)
+ {
mutex_lock(L3);
/* do something again */
mutex_unlock(L3);
-}
+ }
Now we add 4 processes that run each of these functions separately.
Processes A, B, C, and D which run functions func1, func2, func3 and func4
respectively, and such that D runs first and A last. With D being preempted
-in func4 in the "do something again" area, we have a locking that follows:
+in func4 in the "do something again" area, we have a locking that follows::
-D owns L3
- C blocked on L3
- C owns L2
- B blocked on L2
- B owns L1
- A blocked on L1
+ D owns L3
+ C blocked on L3
+ C owns L2
+ B blocked on L2
+ B owns L1
+ A blocked on L1
-And thus we have the chain A->L1->B->L2->C->L3->D.
+ And thus we have the chain A->L1->B->L2->C->L3->D.
This gives us a PI depth of 4 (four processes), but looking at any of the
functions individually, it seems as though they only have at most a locking
significant bit to be used as a flag. Bit 0 is used as the "Has Waiters"
flag. It's set whenever there are waiters on a mutex.
-See Documentation/locking/rt-mutex.txt for further details.
+See Documentation/locking/rt-mutex.rst for further details.
cmpxchg Tricks
--------------
is used (when applicable) to keep the fast path of grabbing and releasing
mutexes short.
-cmpxchg is basically the following function performed atomically:
+cmpxchg is basically the following function performed atomically::
-unsigned long _cmpxchg(unsigned long *A, unsigned long *B, unsigned long *C)
-{
+ unsigned long _cmpxchg(unsigned long *A, unsigned long *B, unsigned long *C)
+ {
unsigned long T = *A;
if (*A == *B) {
*A = *C;
}
return T;
-}
-#define cmpxchg(a,b,c) _cmpxchg(&a,&b,&c)
+ }
+ #define cmpxchg(a,b,c) _cmpxchg(&a,&b,&c)
This is really nice to have, since it allows you to only update a variable
if the variable is what you expect it to be. You know if it succeeded if
new priority. Note that rt_mutex_setprio is defined in kernel/sched/core.c
to implement the actual change in priority.
-(Note: For the "prio" field in task_struct, the lower the number, the
+Note:
+ For the "prio" field in task_struct, the lower the number, the
higher the priority. A "prio" of 5 is of higher priority than a
- "prio" of 10.)
+ "prio" of 10.
It is interesting to note that rt_mutex_adjust_prio can either increase
or decrease the priority of the task. In the case that a higher priority
forces the current owner to synchronize with this code.
The lock is taken if the following are true:
+
1) The lock has no owner
2) The current task is the highest priority against all other
waiters of the lock
-------
Author: Steven Rostedt <rostedt@goodmis.org>
+
Updated: Alex Shi <alex.shi@linaro.org> - 7/6/2017
-Original Reviewers: Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and
+Original Reviewers:
+ Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and
Randy Dunlap
+
Update (7/6/2017) Reviewers: Steven Rostedt and Sebastian Siewior
Updates
+==================================
RT-mutex subsystem with PI support
-----------------------------------
+==================================
RT-mutexes with priority inheritance are used to support PI-futexes,
which enable pthread_mutex_t priority inheritance attributes
structure:
lock->owner holds the task_struct pointer of the owner. Bit 0 is used to
-keep track of the "lock has waiters" state.
+keep track of the "lock has waiters" state:
- owner bit0
+ ============ ======= ================================================
+ owner bit0 Notes
+ ============ ======= ================================================
NULL 0 lock is free (fast acquire possible)
NULL 1 lock is free and has waiters and the top waiter
- is going to take the lock*
+ is going to take the lock [1]_
taskpointer 0 lock is held (fast release possible)
- taskpointer 1 lock is held and has waiters**
+ taskpointer 1 lock is held and has waiters [2]_
+ ============ ======= ================================================
The fast atomic compare exchange based acquire and release is only
possible when bit 0 of lock->owner is 0.
-(*) It also can be a transitional state when grabbing the lock
-with ->wait_lock is held. To prevent any fast path cmpxchg to the lock,
-we need to set the bit0 before looking at the lock, and the owner may be
-NULL in this small time, hence this can be a transitional state.
+.. [1] It also can be a transitional state when grabbing the lock
+ with ->wait_lock is held. To prevent any fast path cmpxchg to the lock,
+ we need to set the bit0 before looking at the lock, and the owner may
+ be NULL in this small time, hence this can be a transitional state.
-(**) There is a small time when bit 0 is set but there are no
-waiters. This can happen when grabbing the lock in the slow path.
-To prevent a cmpxchg of the owner releasing the lock, we need to
-set this bit before looking at the lock.
+.. [2] There is a small time when bit 0 is set but there are no
+ waiters. This can happen when grabbing the lock in the slow path.
+ To prevent a cmpxchg of the owner releasing the lock, we need to
+ set this bit before looking at the lock.
BTW, there is still technically a "Pending Owner", it's just not called
that anymore. The pending owner happens to be the top_waiter of a lock
+===============
+Locking lessons
+===============
+
Lesson 1: Spin locks
+====================
-The most basic primitive for locking is spinlock.
+The most basic primitive for locking is spinlock::
-static DEFINE_SPINLOCK(xxx_lock);
+ static DEFINE_SPINLOCK(xxx_lock);
unsigned long flags;
NOTE! Implications of spin_locks for memory are further described in:
Documentation/memory-barriers.txt
+
(5) LOCK operations.
+
(6) UNLOCK operations.
The above is usually pretty simple (you usually need and want only one
spinlock for most things - using more than one spinlock can make things a
lot more complex and even slower and is usually worth it only for
-sequences that you _know_ need to be split up: avoid it at all cost if you
+sequences that you **know** need to be split up: avoid it at all cost if you
aren't sure).
This is really the only really hard part about spinlocks: once you start
using spinlocks they tend to expand to areas you might not have noticed
before, because you have to make sure the spinlocks correctly protect the
-shared data structures _everywhere_ they are used. The spinlocks are most
+shared data structures **everywhere** they are used. The spinlocks are most
easily added to places that are completely independent of other code (for
example, internal driver data structures that nobody else ever touches).
- NOTE! The spin-lock is safe only when you _also_ use the lock itself
+ NOTE! The spin-lock is safe only when you **also** use the lock itself
to do locking across CPU's, which implies that EVERYTHING that
touches a shared variable has to agree about the spinlock they want
to use.
----
Lesson 2: reader-writer spinlocks.
+==================================
If your data accesses have a very natural pattern where you usually tend
to mostly read from the shared variables, the reader-writer locks
simple spinlocks. Unless the reader critical section is long, you
are better off just using spinlocks.
-The routines look the same as above:
+The routines look the same as above::
rwlock_t xxx_lock = __RW_LOCK_UNLOCKED(xxx_lock);
The above kind of lock may be useful for complex data structures like
linked lists, especially searching for entries without changing the list
itself. The read lock allows many concurrent readers. Anything that
-_changes_ the list will have to get the write lock.
+**changes** the list will have to get the write lock.
NOTE! RCU is better for list traversal, but requires careful
attention to design detail (see Documentation/RCU/listRCU.txt).
----
Lesson 3: spinlocks revisited.
+==============================
The single spin-lock primitives above are by no means the only ones. They
are the most safe ones, and the ones that work under all circumstances,
-but partly _because_ they are safe they are also fairly slow. They are slower
+but partly **because** they are safe they are also fairly slow. They are slower
than they'd need to be, because they do have to disable interrupts
(which is just a single instruction on a x86, but it's an expensive one -
and on other architectures it can be worse).
If you have a case where you have to protect a data structure across
several CPU's and you want to use spinlocks you can potentially use
cheaper versions of the spinlocks. IFF you know that the spinlocks are
-never used in interrupt handlers, you can use the non-irq versions:
+never used in interrupt handlers, you can use the non-irq versions::
spin_lock(&lock);
...
manipulated from a "process context", ie no interrupts involved.
The reasons you mustn't use these versions if you have interrupts that
-play with the spinlock is that you can get deadlocks:
+play with the spinlock is that you can get deadlocks::
spin_lock(&lock);
...
----
Reference information:
+======================
For dynamic initialization, use spin_lock_init() or rwlock_init() as
-appropriate:
+appropriate::
spinlock_t xxx_lock;
rwlock_t xxx_rw_lock;
+======================================
Wound/Wait Deadlock-Proof Mutex Design
======================================
no deadlock potential and hence the ww_mutex_lock call will block and not
prematurely return -EDEADLK. The advantage of the _slow functions is in
interface safety:
+
- ww_mutex_lock has a __must_check int return type, whereas ww_mutex_lock_slow
has a void return type. Note that since ww mutex code needs loops/retries
anyway the __must_check doesn't result in spurious warnings, even though the
and you want to reduce the number of rollbacks.
Three different ways to acquire locks within the same w/w class. Common
-definitions for methods #1 and #2:
+definitions for methods #1 and #2::
-static DEFINE_WW_CLASS(ww_class);
+ static DEFINE_WW_CLASS(ww_class);
-struct obj {
+ struct obj {
struct ww_mutex lock;
/* obj data */
-};
+ };
-struct obj_entry {
+ struct obj_entry {
struct list_head head;
struct obj *obj;
-};
+ };
Method 1, using a list in execbuf->buffers that's not allowed to be reordered.
This is useful if a list of required objects is already tracked somewhere.
Furthermore the lock helper can use propagate the -EALREADY return code back to
the caller as a signal that an object is twice on the list. This is useful if
the list is constructed from userspace input and the ABI requires userspace to
-not have duplicate entries (e.g. for a gpu commandbuffer submission ioctl).
+not have duplicate entries (e.g. for a gpu commandbuffer submission ioctl)::
-int lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
-{
+ int lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
+ {
struct obj *res_obj = NULL;
struct obj_entry *contended_entry = NULL;
struct obj_entry *entry;
ww_acquire_init(ctx, &ww_class);
-retry:
+ retry:
list_for_each_entry (entry, list, head) {
if (entry->obj == res_obj) {
res_obj = NULL;
ww_acquire_done(ctx);
return 0;
-err:
+ err:
list_for_each_entry_continue_reverse (entry, list, head)
ww_mutex_unlock(&entry->obj->lock);
ww_acquire_fini(ctx);
return ret;
-}
+ }
Method 2, using a list in execbuf->buffers that can be reordered. Same semantics
of duplicate entry detection using -EALREADY as method 1 above. But the
-list-reordering allows for a bit more idiomatic code.
+list-reordering allows for a bit more idiomatic code::
-int lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
-{
+ int lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
+ {
struct obj_entry *entry, *entry2;
ww_acquire_init(ctx, &ww_class);
ww_acquire_done(ctx);
return 0;
-}
+ }
-Unlocking works the same way for both methods #1 and #2:
+Unlocking works the same way for both methods #1 and #2::
-void unlock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
-{
+ void unlock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
+ {
struct obj_entry *entry;
list_for_each_entry (entry, list, head)
ww_mutex_unlock(&entry->obj->lock);
ww_acquire_fini(ctx);
-}
+ }
Method 3 is useful if the list of objects is constructed ad-hoc and not upfront,
e.g. when adjusting edges in a graph where each node has its own ww_mutex lock,
and edges can only be changed when holding the locks of all involved nodes. w/w
mutexes are a natural fit for such a case for two reasons:
+
- They can handle lock-acquisition in any order which allows us to start walking
a graph from a starting point and then iteratively discovering new edges and
locking down the nodes those edges connect to.
as a starting point).
Note that this approach differs in two important ways from the above methods:
+
- Since the list of objects is dynamically constructed (and might very well be
different when retrying due to hitting the -EDEADLK die condition) there's
no need to keep any object on a persistent list when it's not locked. We can
Also, method 3 can't fail the lock acquisition step since it doesn't return
-EALREADY. Of course this would be different when using the _interruptible
-variants, but that's outside of the scope of these examples here.
+variants, but that's outside of the scope of these examples here::
-struct obj {
+ struct obj {
struct ww_mutex ww_mutex;
struct list_head locked_list;
-};
+ };
-static DEFINE_WW_CLASS(ww_class);
+ static DEFINE_WW_CLASS(ww_class);
-void __unlock_objs(struct list_head *list)
-{
+ void __unlock_objs(struct list_head *list)
+ {
struct obj *entry, *temp;
list_for_each_entry_safe (entry, temp, list, locked_list) {
list_del(&entry->locked_list);
ww_mutex_unlock(entry->ww_mutex)
}
-}
+ }
-void lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
-{
+ void lock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
+ {
struct obj *obj;
ww_acquire_init(ctx, &ww_class);
-retry:
+ retry:
/* re-init loop start state */
loop {
/* magic code which walks over a graph and decides which objects
ww_acquire_done(ctx);
return 0;
-}
+ }
-void unlock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
-{
+ void unlock_objs(struct list_head *list, struct ww_acquire_ctx *ctx)
+ {
__unlock_objs(list);
ww_acquire_fini(ctx);
-}
+ }
Method 4: Only lock one single objects. In that case deadlock detection and
prevention is obviously overkill, since with grabbing just one lock you can't
----------------------
Design:
+^^^^^^^
+
ww_mutex currently encapsulates a struct mutex, this means no extra overhead for
normal mutex locks, which are far more common. As such there is only a small
increase in code size if wait/wound mutexes are not used.
We maintain the following invariants for the wait list:
+
(1) Waiters with an acquire context are sorted by stamp order; waiters
without an acquire context are interspersed in FIFO order.
(2) For Wait-Die, among waiters with contexts, only the first one can have
therefore be directed towards the uncontended cases.
Lockdep:
+^^^^^^^^
+
Special care has been taken to warn for as many cases of api abuse
as possible. Some common api abuses will be caught with
CONFIG_DEBUG_MUTEXES, but CONFIG_PROVE_LOCKING is recommended.
having called ww_acquire_fini on the first.
- 'normal' deadlocks that can occur.
-FIXME: Update this section once we have the TASK_DEADLOCK task state flag magic
-implemented.
+FIXME:
+ Update this section once we have the TASK_DEADLOCK task state flag magic
+ implemented.
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+m68k Architecture
+=================
+
+.. toctree::
+ :maxdepth: 2
+
+ kernel-options
+
+.. only:: subproject and html
+
+ Indices
+ =======
+
+ * :ref:`genindex`
-
-
- Command Line Options for Linux/m68k
- ===================================
+===================================
+Command Line Options for Linux/m68k
+===================================
Last Update: 2 May 1999
+
Linux/m68k version: 2.2.6
+
Author: Roman.Hodek@informatik.uni-erlangen.de (Roman Hodek)
+
Update: jds@kom.auc.dk (Jes Sorensen) and faq@linux-m68k.org (Chris Lawrence)
0) Introduction
===============
- Often I've been asked which command line options the Linux/m68k
+Often I've been asked which command line options the Linux/m68k
kernel understands, or how the exact syntax for the ... option is, or
... about the option ... . I hope, this document supplies all the
answers...
- Note that some options might be outdated, their descriptions being
+Note that some options might be outdated, their descriptions being
incomplete or missing. Please update the information and send in the
patches.
into init's environment. All other arguments are passed to init as
command line options.
- This document describes the valid kernel options for Linux/m68k in
+This document describes the valid kernel options for Linux/m68k in
the version mentioned at the start of this file. Later revisions may
add new such options, and some may be missing in older versions.
- In general, the value (the part after the '=') of an option is a
+In general, the value (the part after the '=') of an option is a
list of values separated by commas. The interpretation of these values
is up to the driver that "owns" the option. This association of
options with drivers is also the reason that some are further
2.1) root=
----------
-Syntax: root=/dev/<device>
- or: root=<hex_number>
+:Syntax: root=/dev/<device>
+:or: root=<hex_number>
This tells the kernel which device it should mount as the root
filesystem. The device must be a block device with a valid filesystem
on it.
- The first syntax gives the device by name. These names are converted
+The first syntax gives the device by name. These names are converted
into a major/minor number internally in the kernel in an unusual way.
Normally, this "conversion" is done by the device files in /dev, but
this isn't possible here, because the root filesystem (with /dev)
isn't mounted yet... So the kernel parses the name itself, with some
hardcoded name to number mappings. The name must always be a
combination of two or three letters, followed by a decimal number.
-Valid names are:
+Valid names are::
/dev/ram: -> 0x0100 (initial ramdisk)
/dev/hda: -> 0x0300 (first IDE disk)
/dev/sde: -> 0x0840 (fifth SCSI disk)
/dev/fd : -> 0x0200 (floppy disk)
- The name must be followed by a decimal number, that stands for the
+The name must be followed by a decimal number, that stands for the
partition number. Internally, the value of the number is just
added to the device number mentioned in the table above. The
exceptions are /dev/ram and /dev/fd, where /dev/ram refers to an
[Strange and maybe uninteresting stuff ON]
- This unusual translation of device names has some strange
+This unusual translation of device names has some strange
consequences: If, for example, you have a symbolic link from /dev/fd
to /dev/fd0D720 as an abbreviation for floppy driver #0 in DD format,
you cannot use this name for specifying the root device, because the
kernel cannot see this symlink before mounting the root FS and it
-isn't in the table above. If you use it, the root device will not be
+isn't in the table above. If you use it, the root device will not be
set at all, without an error message. Another example: You cannot use a
partition on e.g. the sixth SCSI disk as the root filesystem, if you
want to specify it by name. This is, because only the devices up to
[Strange and maybe uninteresting stuff OFF]
- If the device containing your root partition isn't in the table
+If the device containing your root partition isn't in the table
above, you can also specify it by major and minor numbers. These are
written in hex, with no prefix and no separator between. E.g., if you
have a CD with contents appropriate as a root filesystem in the first
if partition 5 of the device has the UUID of
00112233-4455-6677-8899-AABBCCDDEEFF then partition 3 may be found as
follows:
+
PARTUUID=00112233-4455-6677-8899-AABBCCDDEEFF/PARTNROFF=-2
Authoritative information can be found in
2.2) ro, rw
-----------
-Syntax: ro
- or: rw
+:Syntax: ro
+:or: rw
These two options tell the kernel whether it should mount the root
filesystem read-only or read-write. The default is read-only, except
2.3) debug
----------
-Syntax: debug
+:Syntax: debug
This raises the kernel log level to 10 (the default is 7). This is the
same level as set by the "dmesg" command, just that the maximum level
2.4) debug=
-----------
-Syntax: debug=<device>
+:Syntax: debug=<device>
This option causes certain kernel messages be printed to the selected
debugging device. This can aid debugging the kernel, since the
for the validity of the device name. If the device isn't implemented,
nothing happens.
- Messages logged this way are in general stack dumps after kernel
+Messages logged this way are in general stack dumps after kernel
memory faults or bad kernel traps, and kernel panics. To be exact: all
messages of level 0 (panic messages) and all messages printed while
the log level is 8 or more (their level doesn't matter). Before stack
Devices possible for Amiga:
- - "ser": built-in serial port; parameters: 9600bps, 8N1
- - "mem": Save the messages to a reserved area in chip mem. After
+ - "ser":
+ built-in serial port; parameters: 9600bps, 8N1
+ - "mem":
+ Save the messages to a reserved area in chip mem. After
rebooting, they can be read under AmigaOS with the tool
'dmesg'.
Devices possible for Atari:
- - "ser1": ST-MFP serial port ("Modem1"); parameters: 9600bps, 8N1
- - "ser2": SCC channel B serial port ("Modem2"); parameters: 9600bps, 8N1
- - "ser" : default serial port
+ - "ser1":
+ ST-MFP serial port ("Modem1"); parameters: 9600bps, 8N1
+ - "ser2":
+ SCC channel B serial port ("Modem2"); parameters: 9600bps, 8N1
+ - "ser" :
+ default serial port
This is "ser2" for a Falcon, and "ser1" for any other machine
- - "midi": The MIDI port; parameters: 31250bps, 8N1
- - "par" : parallel port
+ - "midi":
+ The MIDI port; parameters: 31250bps, 8N1
+ - "par" :
+ parallel port
+
The printing routine for this implements a timeout for the
case there's no printer connected (else the kernel would
lock up). The timeout is not exact, but usually a few
2.6) ramdisk_size=
--------------
+------------------
-Syntax: ramdisk_size=<size>
+:Syntax: ramdisk_size=<size>
- This option instructs the kernel to set up a ramdisk of the given
+This option instructs the kernel to set up a ramdisk of the given
size in KBytes. Do not use this option if the ramdisk contents are
passed by bootstrap! In this case, the size is selected automatically
and should not be overwritten.
- The only application is for root filesystems on floppy disks, that
+The only application is for root filesystems on floppy disks, that
should be loaded into memory. To do that, select the corresponding
size of the disk as ramdisk size, and set the root device to the disk
drive (with "root=").
2.7) swap=
+
+ I can't find any sign of this option in 2.2.6.
+
2.8) buff=
-----------
- I can't find any sign of these options in 2.2.6.
+ I can't find any sign of this option in 2.2.6.
3) General Device Options (Amiga and Atari)
3.1) ether=
-----------
-Syntax: ether=[<irq>[,<base_addr>[,<mem_start>[,<mem_end>]]]],<dev-name>
+:Syntax: ether=[<irq>[,<base_addr>[,<mem_start>[,<mem_end>]]]],<dev-name>
- <dev-name> is the name of a net driver, as specified in
+<dev-name> is the name of a net driver, as specified in
drivers/net/Space.c in the Linux source. Most prominent are eth0, ...
eth3, sl0, ... sl3, ppp0, ..., ppp3, dummy, and lo.
- The non-ethernet drivers (sl, ppp, dummy, lo) obviously ignore the
+The non-ethernet drivers (sl, ppp, dummy, lo) obviously ignore the
settings by this options. Also, the existing ethernet drivers for
Linux/m68k (ariadne, a2065, hydra) don't use them because Zorro boards
are really Plug-'n-Play, so the "ether=" option is useless altogether
3.2) hd=
--------
-Syntax: hd=<cylinders>,<heads>,<sectors>
+:Syntax: hd=<cylinders>,<heads>,<sectors>
- This option sets the disk geometry of an IDE disk. The first hd=
+This option sets the disk geometry of an IDE disk. The first hd=
option is for the first IDE disk, the second for the second one.
(I.e., you can give this option twice.) In most cases, you won't have
to use this option, since the kernel can obtain the geometry data
3.3) max_scsi_luns=
-------------------
-Syntax: max_scsi_luns=<n>
+:Syntax: max_scsi_luns=<n>
- Sets the maximum number of LUNs (logical units) of SCSI devices to
+Sets the maximum number of LUNs (logical units) of SCSI devices to
be scanned. Valid values for <n> are between 1 and 8. Default is 8 if
"Probe all LUNs on each SCSI device" was selected during the kernel
configuration, else 1.
3.4) st=
--------
-Syntax: st=<buffer_size>,[<write_thres>,[<max_buffers>]]
+:Syntax: st=<buffer_size>,[<write_thres>,[<max_buffers>]]
- Sets several parameters of the SCSI tape driver. <buffer_size> is
+Sets several parameters of the SCSI tape driver. <buffer_size> is
the number of 512-byte buffers reserved for tape operations for each
device. <write_thres> sets the number of blocks which must be filled
to start an actual write operation to the tape. Maximum value is the
3.5) dmasound=
--------------
-Syntax: dmasound=[<buffers>,<buffer-size>[,<catch-radius>]]
+:Syntax: dmasound=[<buffers>,<buffer-size>[,<catch-radius>]]
- This option controls some configurations of the Linux/m68k DMA sound
+This option controls some configurations of the Linux/m68k DMA sound
driver (Amiga and Atari): <buffers> is the number of buffers you want
to use (minimum 4, default 4), <buffer-size> is the size of each
buffer in kilobytes (minimum 4, default 32) and <catch-radius> says
4.1) video=
-----------
-Syntax: video=<fbname>:<sub-options...>
+:Syntax: video=<fbname>:<sub-options...>
The <fbname> parameter specifies the name of the frame buffer,
-eg. most atari users will want to specify `atafb' here. The
+eg. most atari users will want to specify `atafb` here. The
<sub-options> is a comma-separated list of the sub-options listed
below.
-NB: Please notice that this option was renamed from `atavideo' to
- `video' during the development of the 1.3.x kernels, thus you
+NB:
+ Please notice that this option was renamed from `atavideo` to
+ `video` during the development of the 1.3.x kernels, thus you
might need to update your boot-scripts if upgrading to 2.x from
an 1.2.x kernel.
-NBB: The behavior of video= was changed in 2.1.57 so the recommended
-option is to specify the name of the frame buffer.
+NBB:
+ The behavior of video= was changed in 2.1.57 so the recommended
+ option is to specify the name of the frame buffer.
4.1.1) Video Mode
-----------------
- falh2 : 896x608x1, Falcon only
- falh16 : 896x608x4, Falcon only
- If no video mode is given on the command line, the kernel tries the
+If no video mode is given on the command line, the kernel tries the
modes names "default<n>" in turn, until one is possible with the
hardware in use.
- A video mode setting doesn't make sense, if the external driver is
+A video mode setting doesn't make sense, if the external driver is
activated by a "external:" sub-option.
4.1.2) inverse
4.1.3) font
-----------
-Syntax: font:<fontname>
+:Syntax: font:<fontname>
Specify the font to use in text modes. Currently you can choose only
-between `VGA8x8', `VGA8x16' and `PEARL8x8'. `VGA8x8' is default, if the
+between `VGA8x8`, `VGA8x16` and `PEARL8x8`. `VGA8x8` is default, if the
vertical size of the display is less than 400 pixel rows. Otherwise, the
-`VGA8x16' font is the default.
+`VGA8x16` font is the default.
-4.1.4) hwscroll_
-----------------
+4.1.4) `hwscroll_`
+------------------
-Syntax: hwscroll_<n>
+:Syntax: `hwscroll_<n>`
The number of additional lines of video memory to reserve for
speeding up the scrolling ("hardware scrolling"). Hardware scrolling
base address must be on a 256 byte boundary there, the latter because
the kernel doesn't know how to set the base address at all.)
- By default, <n> is set to the number of visible text lines on the
+By default, <n> is set to the number of visible text lines on the
display. Thus, the amount of video memory is doubled, compared to no
hardware scrolling. You can turn off the hardware scrolling altogether
by setting <n> to 0.
4.1.5) internal:
----------------
-Syntax: internal:<xres>;<yres>[;<xres_max>;<yres_max>;<offset>]
+:Syntax: internal:<xres>;<yres>[;<xres_max>;<yres_max>;<offset>]
This option specifies the capabilities of some extended internal video
hardware, like e.g. OverScan. <xres> and <yres> give the (extended)
dimensions of the screen.
- If your OverScan needs a black border, you have to write the last
+If your OverScan needs a black border, you have to write the last
three arguments of the "internal:". <xres_max> is the maximum line
length the hardware allows, <yres_max> the maximum number of lines.
<offset> is the offset of the visible part of the screen memory to its
physical start, in bytes.
- Often, extended interval video hardware has to be activated somehow.
+Often, extended interval video hardware has to be activated somehow.
For this, see the "sw_*" options below.
4.1.6) external:
----------------
-Syntax:
- external:<xres>;<yres>;<depth>;<org>;<scrmem>[;<scrlen>[;<vgabase>\
- [;<colw>[;<coltype>[;<xres_virtual>]]]]]
+:Syntax:
+ external:<xres>;<yres>;<depth>;<org>;<scrmem>[;<scrlen>[;<vgabase>
+ [;<colw>[;<coltype>[;<xres_virtual>]]]]]
-[I had to break this line...]
+.. I had to break this line...
- This is probably the most complicated parameter... It specifies that
+This is probably the most complicated parameter... It specifies that
you have some external video hardware (a graphics board), and how to
use it under Linux/m68k. The kernel cannot know more about the hardware
than you tell it here! The kernel also is unable to set or change any
have to switch to that video mode before you start Linux, and cannot
switch to another mode once Linux has started.
- The first 3 parameters of this sub-option should be obvious: <xres>,
+The first 3 parameters of this sub-option should be obvious: <xres>,
<yres> and <depth> give the dimensions of the screen and the number of
planes (depth). The depth is the logarithm to base 2 of the number
of colors possible. (Or, the other way round: The number of colors is
2^depth).
- You have to tell the kernel furthermore how the video memory is
+You have to tell the kernel furthermore how the video memory is
organized. This is done by a letter as <org> parameter:
- 'n': "normal planes", i.e. one whole plane after another
- 'i': "interleaved planes", i.e. 16 bit of the first plane, than 16 bit
+ 'n':
+ "normal planes", i.e. one whole plane after another
+ 'i':
+ "interleaved planes", i.e. 16 bit of the first plane, than 16 bit
of the next, and so on... This mode is used only with the
- built-in Atari video modes, I think there is no card that
- supports this mode.
- 'p': "packed pixels", i.e. <depth> consecutive bits stand for all
- planes of one pixel; this is the most common mode for 8 planes
- (256 colors) on graphic cards
- 't': "true color" (more or less packed pixels, but without a color
- lookup table); usually depth is 24
+ built-in Atari video modes, I think there is no card that
+ supports this mode.
+ 'p':
+ "packed pixels", i.e. <depth> consecutive bits stand for all
+ planes of one pixel; this is the most common mode for 8 planes
+ (256 colors) on graphic cards
+ 't':
+ "true color" (more or less packed pixels, but without a color
+ lookup table); usually depth is 24
For monochrome modes (i.e., <depth> is 1), the <org> letter has a
different meaning:
- 'n': normal colors, i.e. 0=white, 1=black
- 'i': inverted colors, i.e. 0=black, 1=white
+ 'n':
+ normal colors, i.e. 0=white, 1=black
+ 'i':
+ inverted colors, i.e. 0=black, 1=white
- The next important information about the video hardware is the base
+The next important information about the video hardware is the base
address of the video memory. That is given in the <scrmem> parameter,
as a hexadecimal number with a "0x" prefix. You have to find out this
address in the documentation of your hardware.
- The next parameter, <scrlen>, tells the kernel about the size of the
+The next parameter, <scrlen>, tells the kernel about the size of the
video memory. If it's missing, the size is calculated from <xres>,
<yres>, and <depth>. For now, it is not useful to write a value here.
It would be used only for hardware scrolling (which isn't possible
writing two consecutive semicolons, if you want to give a <vgabase>
(it is allowed to leave this parameter empty).
- The <vgabase> parameter is optional. If it is not given, the kernel
+The <vgabase> parameter is optional. If it is not given, the kernel
cannot read or write any color registers of the video hardware, and
thus you have to set appropriate colors before you start Linux. But if
your card is somehow VGA compatible, you can tell the kernel the base
parameter is written in hexadecimal with a "0x" prefix, just as
<scrmem>.
- <colw> is meaningful only if <vgabase> is specified. It tells the
+<colw> is meaningful only if <vgabase> is specified. It tells the
kernel how wide each of the color register is, i.e. the number of bits
per single color (red/green/blue). Default is 6, another quite usual
value is 8.
- Also <coltype> is used together with <vgabase>. It tells the kernel
+Also <coltype> is used together with <vgabase>. It tells the kernel
about the color register model of your gfx board. Currently, the types
"vga" (which is also the default) and "mv300" (SANG MV300) are
implemented.
- Parameter <xres_virtual> is required for ProMST or ET4000 cards where
-the physical linelength differs from the visible length. With ProMST,
+Parameter <xres_virtual> is required for ProMST or ET4000 cards where
+the physical linelength differs from the visible length. With ProMST,
xres_virtual must be set to 2048. For ET4000, xres_virtual depends on the
initialisation of the video-card.
If you're missing a corresponding yres_virtual: the external part is legacy,
4.1.8) monitorcap:
-------------------
-Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
+:Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
This describes the capabilities of a multisync monitor. Don't use it
with a fixed-frequency monitor! For now, only the Falcon frame buffer
uses the settings of "monitorcap:".
- <vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
+<vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
your monitor can work with, in Hz. <hmin> and <hmax> are the same for
the horizontal frequency, in kHz.
mode calculations and settings on its own. The only Atari fb device
that does this currently is the Falcon.
- What you reach with this: Settings for unknown video extensions
+What you reach with this: Settings for unknown video extensions
aren't overridden by the driver, so you can still use the mode found
when booting, when the driver doesn't know to set this mode itself.
But this also means, that you can't switch video modes anymore...
- An example where you may want to use "keep" is the ScreenBlaster for
+An example where you may want to use "keep" is the ScreenBlaster for
the Falcon.
4.2) atamouse=
--------------
-Syntax: atamouse=<x-threshold>,[<y-threshold>]
+:Syntax: atamouse=<x-threshold>,[<y-threshold>]
- With this option, you can set the mouse movement reporting threshold.
+With this option, you can set the mouse movement reporting threshold.
This is the number of pixels of mouse movement that have to accumulate
before the IKBD sends a new mouse packet to the kernel. Higher values
reduce the mouse interrupt load and thus reduce the chance of keyboard
overruns. Lower values give a slightly faster mouse responses and
slightly better mouse tracking.
- You can set the threshold in x and y separately, but usually this is
+You can set the threshold in x and y separately, but usually this is
of little practical use. If there's just one number in the option, it
is used for both dimensions. The default value is 2 for both
thresholds.
4.3) ataflop=
-------------
-Syntax: ataflop=<drive type>[,<trackbuffering>[,<steprateA>[,<steprateB>]]]
+:Syntax: ataflop=<drive type>[,<trackbuffering>[,<steprateA>[,<steprateB>]]]
The drive type may be 0, 1, or 2, for DD, HD, and ED, resp. This
setting affects how many buffers are reserved and which formats are
no for the Medusa and yes for all others.
With the two following parameters, you can change the default
- steprate used for drive A and B, resp.
+ steprate used for drive A and B, resp.
4.4) atascsi=
-------------
-Syntax: atascsi=<can_queue>[,<cmd_per_lun>[,<scat-gat>[,<host-id>[,<tagged>]]]]
+:Syntax: atascsi=<can_queue>[,<cmd_per_lun>[,<scat-gat>[,<host-id>[,<tagged>]]]]
- This option sets some parameters for the Atari native SCSI driver.
+This option sets some parameters for the Atari native SCSI driver.
Generally, any number of arguments can be omitted from the end. And
for each of the numbers, a negative value means "use default". The
defaults depend on whether TT-style or Falcon-style SCSI is used.
32). Default: 8/1. (Note: Values > 1 seem to cause problems on a
Falcon, cause not yet known.)
- The <cmd_per_lun> value at a great part determines the amount of
+ The <cmd_per_lun> value at a great part determines the amount of
memory SCSI reserves for itself. The formula is rather
complicated, but I can give you some hints:
- no scatter-gather : cmd_per_lun * 232 bytes
- full scatter-gather: cmd_per_lun * approx. 17 Kbytes
+
+ no scatter-gather:
+ cmd_per_lun * 232 bytes
+ full scatter-gather:
+ cmd_per_lun * approx. 17 Kbytes
<scat-gat>:
Size of the scatter-gather table, i.e. the number of requests
4.5 switches=
-------------
-Syntax: switches=<list of switches>
+:Syntax: switches=<list of switches>
- With this option you can switch some hardware lines that are often
+With this option you can switch some hardware lines that are often
used to enable/disable certain hardware extensions. Examples are
OverScan, overclocking, ...
- The <list of switches> is a comma-separated list of the following
+The <list of switches> is a comma-separated list of the following
items:
- ikbd: set RTS of the keyboard ACIA high
- midi: set RTS of the MIDI ACIA high
- snd6: set bit 6 of the PSG port A
- snd7: set bit 6 of the PSG port A
+ ikbd:
+ set RTS of the keyboard ACIA high
+ midi:
+ set RTS of the MIDI ACIA high
+ snd6:
+ set bit 6 of the PSG port A
+ snd7:
+ set bit 6 of the PSG port A
It doesn't make sense to mention a switch more than once (no
difference to only once), but you can give as many switches as you
as possible during kernel initialization (even before determining the
present hardware.)
- All of the items can also be prefixed with "ov_", i.e. "ov_ikbd",
-"ov_midi", ... These options are meant for switching on an OverScan
+All of the items can also be prefixed with `ov_`, i.e. `ov_ikbd`,
+`ov_midi`, ... These options are meant for switching on an OverScan
video extension. The difference to the bare option is that the
switch-on is done after video initialization, and somehow synchronized
to the HBLANK. A speciality is that ov_ikbd and ov_midi are switched
off before rebooting, so that OverScan is disabled and TOS boots
correctly.
- If you give an option both, with and without the "ov_" prefix, the
-earlier initialization ("ov_"-less) takes precedence. But the
+If you give an option both, with and without the `ov_` prefix, the
+earlier initialization (`ov_`-less) takes precedence. But the
switching-off on reset still happens in this case.
5) Options for Amiga Only:
5.1) video=
-----------
-Syntax: video=<fbname>:<sub-options...>
+:Syntax: video=<fbname>:<sub-options...>
The <fbname> parameter specifies the name of the frame buffer, valid
-options are `amifb', `cyber', 'virge', `retz3' and `clgen', provided
+options are `amifb`, `cyber`, 'virge', `retz3` and `clgen`, provided
that the respective frame buffer devices have been compiled into the
kernel (or compiled as loadable modules). The behavior of the <fbname>
option was changed in 2.1.57 so it is now recommended to specify this
NTSC modes:
- ntsc : 640x200, 15 kHz, 60 Hz
- ntsc-lace : 640x400, 15 kHz, 60 Hz interlaced
+
PAL modes:
- pal : 640x256, 15 kHz, 50 Hz
- pal-lace : 640x512, 15 kHz, 50 Hz interlaced
+
ECS modes:
- multiscan : 640x480, 29 kHz, 57 Hz
- multiscan-lace : 640x960, 29 kHz, 57 Hz interlaced
- dblpal-lace : 640x1024, 27 kHz, 47 Hz interlaced
- dblntsc : 640x200, 27 kHz, 57 Hz doublescan
- dblpal : 640x256, 27 kHz, 47 Hz doublescan
+
VGA modes:
- vga : 640x480, 31 kHz, 60 Hz
- vga70 : 640x400, 31 kHz, 70 Hz
5.1.2) depth
------------
-Syntax: depth:<nr. of bit-planes>
+:Syntax: depth:<nr. of bit-planes>
Specify the number of bit-planes for the selected video-mode.
5.1.4) font
-----------
-Syntax: font:<fontname>
+:Syntax: font:<fontname>
Specify the font to use in text modes. Functionally the same as the
-"font" sub-option for the Atari, except that `PEARL8x8' is used instead
-of `VGA8x8' if the vertical size of the display is less than 400 pixel
+"font" sub-option for the Atari, except that `PEARL8x8` is used instead
+of `VGA8x8` if the vertical size of the display is less than 400 pixel
rows.
5.1.5) monitorcap:
-------------------
-Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
+:Syntax: monitorcap:<vmin>;<vmax>;<hmin>;<hmax>
This describes the capabilities of a multisync monitor. For now, only
the color frame buffer uses the settings of "monitorcap:".
- <vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
+<vmin> and <vmax> are the minimum and maximum, resp., vertical frequencies
your monitor can work with, in Hz. <hmin> and <hmax> are the same for
the horizontal frequency, in kHz.
- The defaults are 50;90;15;38 (Generic Amiga multisync monitor).
+The defaults are 50;90;15;38 (Generic Amiga multisync monitor).
5.2) fd_def_df0=
----------------
-Syntax: fd_def_df0=<value>
+:Syntax: fd_def_df0=<value>
Sets the df0 value for "silent" floppy drives. The value should be in
hexadecimal with "0x" prefix.
5.3) wd33c93=
-------------
-Syntax: wd33c93=<sub-options...>
+:Syntax: wd33c93=<sub-options...>
These options affect the A590/A2091, A3000 and GVP Series II SCSI
controllers.
5.3.1) nosync
-------------
-Syntax: nosync:bitmask
+:Syntax: nosync:bitmask
- bitmask is a byte where the 1st 7 bits correspond with the 7
+bitmask is a byte where the 1st 7 bits correspond with the 7
possible SCSI devices. Set a bit to prevent sync negotiation on that
device. To maintain backwards compatibility, a command-line such as
"wd33c93=255" will be automatically translated to
5.3.2) period
-------------
-Syntax: period:ns
+:Syntax: period:ns
- `ns' is the minimum # of nanoseconds in a SCSI data transfer
+`ns` is the minimum # of nanoseconds in a SCSI data transfer
period. Default is 500; acceptable values are 250 - 1000.
5.3.3) disconnect
-----------------
-Syntax: disconnect:x
+:Syntax: disconnect:x
- Specify x = 0 to never allow disconnects, 2 to always allow them.
+Specify x = 0 to never allow disconnects, 2 to always allow them.
x = 1 does 'adaptive' disconnects, which is the default and generally
the best choice.
5.3.4) debug
------------
-Syntax: debug:x
+:Syntax: debug:x
- If `DEBUGGING_ON' is defined, x is a bit mask that causes various
+If `DEBUGGING_ON` is defined, x is a bit mask that causes various
types of debug output to printed - see the DB_xxx defines in
wd33c93.h.
5.3.5) clock
------------
-Syntax: clock:x
+:Syntax: clock:x
- x = clock input in MHz for WD33c93 chip. Normal values would be from
+x = clock input in MHz for WD33c93 chip. Normal values would be from
8 through 20. The default value depends on your hostadapter(s),
default for the A3000 internal controller is 14, for the A2091 it's 8
and for the GVP hostadapters it's either 8 or 14, depending on the
5.3.6) next
-----------
- No argument. Used to separate blocks of keywords when there's more
+No argument. Used to separate blocks of keywords when there's more
than one wd33c93-based host adapter in the system.
5.3.7) nodma
------------
-Syntax: nodma:x
+:Syntax: nodma:x
- If x is 1 (or if the option is just written as "nodma"), the WD33c93
+If x is 1 (or if the option is just written as "nodma"), the WD33c93
controller will not use DMA (= direct memory access) to access the
Amiga's memory. This is useful for some systems (like A3000's and
A4000's with the A3640 accelerator, revision 3.0) that have problems
5.4) gvp11=
-----------
-Syntax: gvp11=<addr-mask>
+
:Syntax: gvp11=<addr-mask>
- The earlier versions of the GVP driver did not handle DMA
+The earlier versions of the GVP driver did not handle DMA
address-mask settings correctly which made it necessary for some
people to use this option, in order to get their GVP controller
running under Linux. These problems have hopefully been solved and the
use of this option is now highly unrecommended!
- Incorrect use can lead to unpredictable behavior, so please only use
+Incorrect use can lead to unpredictable behavior, so please only use
this option if you *know* what you are doing and have a reason to do
so. In any case if you experience problems and need to use this
option, please inform us about it by mailing to the Linux/68k kernel
mailing list.
- The address mask set by this option specifies which addresses are
+The address mask set by this option specifies which addresses are
valid for DMA with the GVP Series II SCSI controller. An address is
valid, if no bits are set except the bits that are set in the mask,
too.
- Some versions of the GVP can only DMA into a 24 bit address range,
+Some versions of the GVP can only DMA into a 24 bit address range,
some can address a 25 bit address range while others can use the whole
32 bit address range for DMA. The correct setting depends on your
controller and should be autodetected by the driver. An example is the
24 bit region which is specified by a mask of 0x00fffffe.
-
-
-/* Local Variables: */
-/* mode: text */
-/* End: */
-:orphan:
-
=============================================
Intel Many Integrated Core (MIC) architecture
=============================================
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
========
NetLabel
unsolicited IPv6 Neighbor Advertisements) to be issued after a
failover event. As soon as the link is up on the new slave
(possibly immediately) a peer notification is sent on the
- bonding device and each VLAN sub-device. This is repeated at
- each link monitor interval (arp_interval or miimon, whichever
- is active) if the number is greater than 1.
+ bonding device and each VLAN sub-device. This is repeated at
+ the rate specified by peer_notif_delay if the number is
+ greater than 1.
The valid range is 0 - 255; the default value is 1. These options
affect only the active-backup mode. These options were added for
The valid range is 0 - 65535; the default value is 1. This option
has effect only in balance-rr mode.
+peer_notif_delay
+
+ Specify the delay, in milliseconds, between each peer
+ notification (gratuitous ARP and unsolicited IPv6 Neighbor
+ Advertisement) when they are issued after a failover event.
+ This delay should be a multiple of the link monitor interval
+ (arp_interval or miimon, whichever is active). The default
+ value is 0 which means to match the value of the link monitor
+ interval.
+
primary
A string (eth0, eth2, etc) specifying which slave is the
/proc/sys/net/core/*
- Please see: Documentation/sysctl/net.txt for descriptions of these entries.
+ Please see: Documentation/admin-guide/sysctl/net.rst for descriptions of these entries.
/proc/sys/net/unix/*
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
======
pcmcia
robust-futex, PI-futex, robust+PI-futex.
More details about priority inheritance can be found in
-Documentation/locking/rt-mutex.txt.
+Documentation/locking/rt-mutex.rst.
and call the notification trees if the target was changed as a result of
removing the request.
-s32 dev_pm_qos_read_value(device):
+s32 dev_pm_qos_read_value(device, type):
Returns the aggregated value for a given device's constraints list.
enum pm_qos_flags_status dev_pm_qos_flags(device, mask)
The per-device PM QoS framework has a per-device notification tree.
-int dev_pm_qos_add_notifier(device, notifier):
- Adds a notification callback function for the device.
+int dev_pm_qos_add_notifier(device, notifier, type):
+ Adds a notification callback function for the device for a particular request
+ type.
+
The callback is called when the aggregated value of the device constraints list
- is changed (for resume latency device PM QoS only).
+ is changed.
-int dev_pm_qos_remove_notifier(device, notifier):
+int dev_pm_qos_remove_notifier(device, notifier, type):
Removes the notification callback function for the device.
the default calculated size. Use this option if default
boot memory size is not sufficient for second kernel to
boot successfully. For syntax of crashkernel= parameter,
- refer to Documentation/kdump/kdump.rst. If any offset is
+ refer to Documentation/admin-guide/kdump/kdump.rst. If any offset is
provided in crashkernel= parameter, it will be ignored
as fadump uses a predefined offset to reserve memory
for boot memory dump preservation in case of a crash.
and why.
26) If any ioctl's are added by the patch, then also update
- ``Documentation/ioctl/ioctl-number.txt``.
+ ``Documentation/ioctl/ioctl-number.rst``.
27) If your modified source code depends on or uses any of the kernel
APIs or features that are related to the following ``Kconfig`` symbols,
+++ /dev/null
-The Intel MID PTI project is HW implemented in Intel Atom
-system-on-a-chip designs based on the Parallel Trace
-Interface for MIPI P1149.7 cJTAG standard. The kernel solution
-for this platform involves the following files:
-
-./include/linux/pti.h
-./drivers/.../n_tracesink.h
-./drivers/.../n_tracerouter.c
-./drivers/.../n_tracesink.c
-./drivers/.../pti.c
-
-pti.c is the driver that enables various debugging features
-popular on platforms from certain mobile manufacturers.
-n_tracerouter.c and n_tracesink.c allow extra system information to
-be collected and routed to the pti driver, such as trace
-debugging data from a modem. Although n_tracerouter
-and n_tracesink are a part of the complete PTI solution,
-these two line disciplines can work separately from
-pti.c and route any data stream from one /dev/tty node
-to another /dev/tty node via kernel-space. This provides
-a stable, reliable connection that will not break unless
-the user-space application shuts down (plus avoids
-kernel->user->kernel context switch overheads of routing
-data).
-
-An example debugging usage for this driver system:
- *Hook /dev/ttyPTI0 to syslogd. Opening this port will also start
- a console device to further capture debugging messages to PTI.
- *Hook /dev/ttyPTI1 to modem debugging data to write to PTI HW.
- This is where n_tracerouter and n_tracesink are used.
- *Hook /dev/pti to a user-level debugging application for writing
- to PTI HW.
- *Use mipi_* Kernel Driver API in other device drivers for
- debugging to PTI by first requesting a PTI write address via
- mipi_request_masterchannel(1).
-
-Below is example pseudo-code on how a 'privileged' application
-can hook up n_tracerouter and n_tracesink to any tty on
-a system. 'Privileged' means the application has enough
-privileges to successfully manipulate the ldisc drivers
-but is not just blindly executing as 'root'. Keep in mind
-the use of ioctl(,TIOCSETD,) is not specific to the n_tracerouter
-and n_tracesink line discpline drivers but is a generic
-operation for a program to use a line discpline driver
-on a tty port other than the default n_tty.
-
-/////////// To hook up n_tracerouter and n_tracesink /////////
-
-// Note that n_tracerouter depends on n_tracesink.
-#include <errno.h>
-#define ONE_TTY "/dev/ttyOne"
-#define TWO_TTY "/dev/ttyTwo"
-
-// needed global to hand onto ldisc connection
-static int g_fd_source = -1;
-static int g_fd_sink = -1;
-
-// these two vars used to grab LDISC values from loaded ldisc drivers
-// in OS. Look at /proc/tty/ldiscs to get the right numbers from
-// the ldiscs loaded in the system.
-int source_ldisc_num, sink_ldisc_num = -1;
-int retval;
-
-g_fd_source = open(ONE_TTY, O_RDWR); // must be R/W
-g_fd_sink = open(TWO_TTY, O_RDWR); // must be R/W
-
-if (g_fd_source <= 0) || (g_fd_sink <= 0) {
- // doubt you'll want to use these exact error lines of code
- printf("Error on open(). errno: %d\n",errno);
- return errno;
-}
-
-retval = ioctl(g_fd_sink, TIOCSETD, &sink_ldisc_num);
-if (retval < 0) {
- printf("Error on ioctl(). errno: %d\n", errno);
- return errno;
-}
-
-retval = ioctl(g_fd_source, TIOCSETD, &source_ldisc_num);
-if (retval < 0) {
- printf("Error on ioctl(). errno: %d\n", errno);
- return errno;
-}
-
-/////////// To disconnect n_tracerouter and n_tracesink ////////
-
-// First make sure data through the ldiscs has stopped.
-
-// Second, disconnect ldiscs. This provides a
-// little cleaner shutdown on tty stack.
-sink_ldisc_num = 0;
-source_ldisc_num = 0;
-ioctl(g_fd_uart, TIOCSETD, &sink_ldisc_num);
-ioctl(g_fd_gadget, TIOCSETD, &source_ldisc_num);
-
-// Three, program closes connection, and cleanup:
-close(g_fd_uart);
-close(g_fd_gadget);
-g_fd_uart = g_fd_gadget = NULL;
overhead for maintanence; albeit larger memory footprint.
Similar to the rb_root structure, cached rbtrees are initialized to be
-empty via:
+empty via::
struct rb_root_cached mytree = RB_ROOT_CACHED;
Cached rbtree is simply a regular rb_root with an extra pointer to cache the
leftmost node. This allows rb_root_cached to exist wherever rb_root does,
which permits augmented trees to be supported as well as only a few extra
-interfaces:
+interfaces::
struct rb_node *rb_first_cached(struct rb_root_cached *tree);
void rb_insert_color_cached(struct rb_node *, struct rb_root_cached *, bool);
void rb_erase_cached(struct rb_node *node, struct rb_root_cached *);
Both insert and erase calls have their respective counterpart of augmented
-trees:
+trees::
void rb_insert_augmented_cached(struct rb_node *node, struct rb_root_cached *,
bool, struct rb_augment_callbacks *);
* @RSC_VDEV: declare support for a virtio device, and serve as its
* virtio header.
* @RSC_LAST: just keep this one at the end
+ * @RSC_VENDOR_START: start of the vendor specific resource types range
+ * @RSC_VENDOR_END: end of the vendor specific resource types range
*
* Please note that these values are used as indices to the rproc_handle_rsc
* lookup table, so please keep them sane. Moreover, @RSC_LAST is used to
* please update it as needed.
*/
enum fw_resource_type {
- RSC_CARVEOUT = 0,
- RSC_DEVMEM = 1,
- RSC_TRACE = 2,
- RSC_VDEV = 3,
- RSC_LAST = 4,
+ RSC_CARVEOUT = 0,
+ RSC_DEVMEM = 1,
+ RSC_TRACE = 2,
+ RSC_VDEV = 3,
+ RSC_LAST = 4,
+ RSC_VENDOR_START = 128,
+ RSC_VENDOR_END = 512,
};
For more details regarding a specific resource type, please see its
--- /dev/null
+ Boot image header in RISC-V Linux
+ =============================================
+
+Author: Atish Patra <atish.patra@wdc.com>
+Date : 20 May 2019
+
+This document only describes the boot image header details for RISC-V Linux.
+The complete booting guide will be available at Documentation/riscv/booting.txt.
+
+The following 64-byte header is present in decompressed Linux kernel image.
+
+ u32 code0; /* Executable code */
+ u32 code1; /* Executable code */
+ u64 text_offset; /* Image load offset, little endian */
+ u64 image_size; /* Effective Image size, little endian */
+ u64 flags; /* kernel flags, little endian */
+ u32 version; /* Version of this header */
+ u32 res1 = 0; /* Reserved */
+ u64 res2 = 0; /* Reserved */
+ u64 magic = 0x5643534952; /* Magic number, little endian, "RISCV" */
+ u32 res3; /* Reserved for additional RISC-V specific header */
+ u32 res4; /* Reserved for PE COFF offset */
+
+This header format is compliant with PE/COFF header and largely inspired from
+ARM64 header. Thus, both ARM64 & RISC-V header can be combined into one common
+header in future.
+
+Notes:
+- This header can also be reused to support EFI stub for RISC-V in future. EFI
+ specification needs PE/COFF image header in the beginning of the kernel image
+ in order to load it as an EFI application. In order to support EFI stub,
+ code0 should be replaced with "MZ" magic string and res5(at offset 0x3c) should
+ point to the rest of the PE/COFF header.
+
+- version field indicate header version number.
+ Bits 0:15 - Minor version
+ Bits 16:31 - Major version
+
+ This preserves compatibility across newer and older version of the header.
+ The current version is defined as 0.1.
+
+- res3 is reserved for offset to any other additional fields. This makes the
+ header extendible in future. One example would be to accommodate ISA
+ extension for RISC-V in future. For current version, it is set to be zero.
+
+- In current header, the flag field has only one field.
+ Bit 0: Kernel endianness. 1 if BE, 0 if LE.
+
+- Image size is mandatory for boot loader to load kernel image. Booting will
+ fail otherwise.
-:orphan:
-
===================
RISC-V architecture
===================
| +----------------+-------------------------------------------------+
| | 32 | Basic Addressing Mode |
| | | |
-| | | Used to set addressing mode |
+| | | Used to set addressing mode:: |
| | | |
| | | +---------+----------+----------+ |
| | | | PSW 31 | PSW 32 | | |
-:orphan:
-
=================
s390 Architecture
=================
qemu/hw/s390x/css.c
For vfio mediated device framework:
-- Documentation/vfio-mediated-device.txt
+- Documentation/driver-api/vfio-mediated-device.rst
Motivation of vfio-ccw
----------------------
2. ESA/390 Common I/O Device Commands manual (IBM Form. No. SA22-7204)
3. https://en.wikipedia.org/wiki/Channel_I/O
4. Documentation/s390/cds.rst
-5. Documentation/vfio.txt
-6. Documentation/vfio-mediated-device.txt
+5. Documentation/driver-api/vfio.rst
+6. Documentation/driver-api/vfio-mediated-device.rst
-:orphan:
-
===============
Linux Scheduler
===============
-deadline tasks cannot have an affinity mask smaller that the entire
root_domain they are created on. However, affinities can be specified
- through the cpuset facility (Documentation/cgroup-v1/cpusets.rst).
+ through the cpuset facility (Documentation/admin-guide/cgroup-v1/cpusets.rst).
5.1 SCHED_DEADLINE and cpusets HOWTO
------------------------------------
These options need CONFIG_CGROUPS to be defined, and let the administrator
create arbitrary groups of tasks, using the "cgroup" pseudo filesystem. See
- Documentation/cgroup-v1/cgroups.rst for more information about this filesystem.
+ Documentation/admin-guide/cgroup-v1/cgroups.rst for more information about this filesystem.
When CONFIG_FAIR_GROUP_SCHED is defined, a "cpu.shares" file is created for each
group created using the pseudo filesystem. See example steps below to create
to control the CPU time reserved for each control group.
For more information on working with control groups, you should read
-Documentation/cgroup-v1/cgroups.rst as well.
+Documentation/admin-guide/cgroup-v1/cgroups.rst as well.
Group settings are checked against the following limits in order to keep the
configuration schedulable:
credentials
IMA-templates
keys/index
- LSM
+ lsm
+ lsm-development
+ sak
SCTP
self-protection
+ siphash
tpm/index
.. toctree::
tpm_vtpm_proxy
+ xen-tpmfront
+=============================
Virtual TPM interface for Xen
+=============================
Authors: Matthew Fioravante (JHUAPL), Daniel De Graaf (NSA)
Xen. The reader is assumed to have familiarity with building and installing Xen,
Linux, and a basic understanding of the TPM and vTPM concepts.
-INTRODUCTION
+Introduction
+------------
The goal of this work is to provide a TPM functionality to a virtual guest
operating system (in Xen terms, a DomU). This allows programs to interact with
IBM and Intel corporation.
-DESIGN OVERVIEW
+Design Overview
---------------
-The architecture of vTPM is described below:
-
-+------------------+
-| Linux DomU | ...
-| | ^ |
-| v | |
-| xen-tpmfront |
-+------------------+
- | ^
- v |
-+------------------+
-| mini-os/tpmback |
-| | ^ |
-| v | |
-| vtpm-stubdom | ...
-| | ^ |
-| v | |
-| mini-os/tpmfront |
-+------------------+
- | ^
- v |
-+------------------+
-| mini-os/tpmback |
-| | ^ |
-| v | |
-| vtpmmgr-stubdom |
-| | ^ |
-| v | |
-| mini-os/tpm_tis |
-+------------------+
- | ^
- v |
-+------------------+
-| Hardware TPM |
-+------------------+
-
- * Linux DomU: The Linux based guest that wants to use a vTPM. There may be
+The architecture of vTPM is described below::
+
+ +------------------+
+ | Linux DomU | ...
+ | | ^ |
+ | v | |
+ | xen-tpmfront |
+ +------------------+
+ | ^
+ v |
+ +------------------+
+ | mini-os/tpmback |
+ | | ^ |
+ | v | |
+ | vtpm-stubdom | ...
+ | | ^ |
+ | v | |
+ | mini-os/tpmfront |
+ +------------------+
+ | ^
+ v |
+ +------------------+
+ | mini-os/tpmback |
+ | | ^ |
+ | v | |
+ | vtpmmgr-stubdom |
+ | | ^ |
+ | v | |
+ | mini-os/tpm_tis |
+ +------------------+
+ | ^
+ v |
+ +------------------+
+ | Hardware TPM |
+ +------------------+
+
+* Linux DomU:
+ The Linux based guest that wants to use a vTPM. There may be
more than one of these.
- * xen-tpmfront.ko: Linux kernel virtual TPM frontend driver. This driver
+* xen-tpmfront.ko:
+ Linux kernel virtual TPM frontend driver. This driver
provides vTPM access to a Linux-based DomU.
- * mini-os/tpmback: Mini-os TPM backend driver. The Linux frontend driver
+* mini-os/tpmback:
+ Mini-os TPM backend driver. The Linux frontend driver
connects to this backend driver to facilitate communications
between the Linux DomU and its vTPM. This driver is also
used by vtpmmgr-stubdom to communicate with vtpm-stubdom.
- * vtpm-stubdom: A mini-os stub domain that implements a vTPM. There is a
+* vtpm-stubdom:
+ A mini-os stub domain that implements a vTPM. There is a
one to one mapping between running vtpm-stubdom instances and
logical vtpms on the system. The vTPM Platform Configuration
Registers (PCRs) are normally all initialized to zero.
- * mini-os/tpmfront: Mini-os TPM frontend driver. The vTPM mini-os domain
+* mini-os/tpmfront:
+ Mini-os TPM frontend driver. The vTPM mini-os domain
vtpm-stubdom uses this driver to communicate with
vtpmmgr-stubdom. This driver is also used in mini-os
domains such as pv-grub that talk to the vTPM domain.
- * vtpmmgr-stubdom: A mini-os domain that implements the vTPM manager. There is
+* vtpmmgr-stubdom:
+ A mini-os domain that implements the vTPM manager. There is
only one vTPM manager and it should be running during the
entire lifetime of the machine. This domain regulates
access to the physical TPM on the system and secures the
persistent state of each vTPM.
- * mini-os/tpm_tis: Mini-os TPM version 1.2 TPM Interface Specification (TIS)
+* mini-os/tpm_tis:
+ Mini-os TPM version 1.2 TPM Interface Specification (TIS)
driver. This driver used by vtpmmgr-stubdom to talk directly to
the hardware TPM. Communication is facilitated by mapping
hardware memory pages into vtpmmgr-stubdom.
- * Hardware TPM: The physical TPM that is soldered onto the motherboard.
+* Hardware TPM:
+ The physical TPM that is soldered onto the motherboard.
-INTEGRATION WITH XEN
+Integration With Xen
--------------------
Support for the vTPM driver was added in Xen using the libxl toolstack in Xen
-:orphan:
-
==================
Sparc Architecture
==================
+++ /dev/null
-Documentation for /proc/sys/abi/* kernel version 2.6.0.test2
- (c) 2003, Fabian Frederick <ffrederick@users.sourceforge.net>
-
-For general info : README.
-
-==============================================================
-
-This path is binary emulation relevant aka personality types aka abi.
-When a process is executed, it's linked to an exec_domain whose
-personality is defined using values available from /proc/sys/abi.
-You can find further details about abi in include/linux/personality.h.
-
-Here are the files featuring in 2.6 kernel :
-
-- defhandler_coff
-- defhandler_elf
-- defhandler_lcall7
-- defhandler_libcso
-- fake_utsname
-- trace
-
-===========================================================
-defhandler_coff:
-defined value :
-PER_SCOSVR3
-0x0003 | STICKY_TIMEOUTS | WHOLE_SECONDS | SHORT_INODE
-
-===========================================================
-defhandler_elf:
-defined value :
-PER_LINUX
-0
-
-===========================================================
-defhandler_lcall7:
-defined value :
-PER_SVR4
-0x0001 | STICKY_TIMEOUTS | MMAP_PAGE_ZERO,
-
-===========================================================
-defhandler_libsco:
-defined value:
-PER_SVR4
-0x0001 | STICKY_TIMEOUTS | MMAP_PAGE_ZERO,
-
-===========================================================
-fake_utsname:
-Unused
-
-===========================================================
-trace:
-Unused
-
-===========================================================
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
==================
TCM Virtual Device
+=======================
CPU cooling APIs How To
-===================================
+=======================
Written by Amit Daniel Kachhap <amit.kachhap@linaro.org>
Copyright (c) 2012 Samsung Electronics Co., Ltd(http://www.samsung.com)
0. Introduction
+===============
The generic cpu cooling(freq clipping) provides registration/unregistration APIs
to the caller. The binding of the cooling devices to the trip point is left for
the user. The registration APIs returns the cooling device pointer.
1. cpu cooling APIs
+===================
1.1 cpufreq registration/unregistration APIs
-1.1.1 struct thermal_cooling_device *cpufreq_cooling_register(
- struct cpumask *clip_cpus)
+--------------------------------------------
+
+ ::
+
+ struct thermal_cooling_device
+ *cpufreq_cooling_register(struct cpumask *clip_cpus)
This interface function registers the cpufreq cooling device with the name
"thermal-cpufreq-%x". This api can support multiple instances of cpufreq
cooling devices.
- clip_cpus: cpumask of cpus where the frequency constraints will happen.
+ clip_cpus:
+ cpumask of cpus where the frequency constraints will happen.
+
+ ::
-1.1.2 struct thermal_cooling_device *of_cpufreq_cooling_register(
- struct cpufreq_policy *policy)
+ struct thermal_cooling_device
+ *of_cpufreq_cooling_register(struct cpufreq_policy *policy)
This interface function registers the cpufreq cooling device with
the name "thermal-cpufreq-%x" linking it with a device tree node, in
order to bind it via the thermal DT code. This api can support multiple
instances of cpufreq cooling devices.
- policy: CPUFreq policy.
+ policy:
+ CPUFreq policy.
+
+
+ ::
-1.1.3 void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
+ void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
This interface function unregisters the "thermal-cpufreq-%x" cooling device.
cdev: Cooling device pointer which has to be unregistered.
2. Power models
+===============
The power API registration functions provide a simple power model for
CPUs. The current power is calculated as dynamic power (static power isn't
variation. In pathological cases this variation can be significant,
but typically it is of a much lesser impact than the factors above.
-A high level dynamic power consumption model may then be represented as:
+A high level dynamic power consumption model may then be represented as::
-Pdyn = f(run) * Voltage^2 * Frequency * Utilisation
+ Pdyn = f(run) * Voltage^2 * Frequency * Utilisation
f(run) here represents the described execution behaviour and its
result has a units of Watts/Hz/Volt^2 (this often expressed in
represented as a constant coefficient. This is a simplification
consistent with the relative contribution to overall power variation.
-In this simplified representation our model becomes:
+In this simplified representation our model becomes::
-Pdyn = Capacitance * Voltage^2 * Frequency * Utilisation
+ Pdyn = Capacitance * Voltage^2 * Frequency * Utilisation
Where `capacitance` is a constant that represents an indicative
running time dynamic power coefficient in fundamental units of
+========================
Kernel driver exynos_tmu
-=================
+========================
Supported chips:
+
* ARM SAMSUNG EXYNOS4, EXYNOS5 series of SoC
+
Datasheet: Not publicly available
Authors: Donggeun Kim <dg77.kim@samsung.com>
There are three equations converting from temperature to temperature code.
The three equations are:
- 1. Two point trimming
+ 1. Two point trimming::
+
Tc = (T - 25) * (TI2 - TI1) / (85 - 25) + TI1
- 2. One point trimming
+ 2. One point trimming::
+
Tc = T + TI1 - 25
- 3. No trimming
+ 3. No trimming::
+
Tc = T + 50
- Tc: Temperature code, T: Temperature,
- TI1: Trimming info for 25 degree Celsius (stored at TRIMINFO register)
+ Tc:
+ Temperature code, T: Temperature,
+ TI1:
+ Trimming info for 25 degree Celsius (stored at TRIMINFO register)
Temperature code measured at 25 degree Celsius which is unchanged
- TI2: Trimming info for 85 degree Celsius (stored at TRIMINFO register)
+ TI2:
+ Trimming info for 85 degree Celsius (stored at TRIMINFO register)
Temperature code measured at 85 degree Celsius which is unchanged
TMU(Thermal Management Unit) in EXYNOS4/5 generates interrupt
when temperature exceeds pre-defined levels.
The maximum number of configurable threshold is five.
-The threshold levels are defined as follows:
+The threshold levels are defined as follows::
+
Level_0: current temperature > trigger_level_0 + threshold
Level_1: current temperature > trigger_level_1 + threshold
Level_2: current temperature > trigger_level_2 + threshold
Level_3: current temperature > trigger_level_3 + threshold
- The threshold and each trigger_level are set
- through the corresponding registers.
+The threshold and each trigger_level are set
+through the corresponding registers.
When an interrupt occurs, this driver notify kernel thermal framework
with the function exynos_report_trigger.
TMU driver description:
-----------------------
-The exynos thermal driver is structured as,
+The exynos thermal driver is structured as::
Kernel Core thermal framework
(thermal_core.c, step_wise.c, cpu_cooling.c)
^
|
|
-TMU configuration data -------> TMU Driver <------> Exynos Core thermal wrapper
-(exynos_tmu_data.c) (exynos_tmu.c) (exynos_thermal_common.c)
-(exynos_tmu_data.h) (exynos_tmu.h) (exynos_thermal_common.h)
+ TMU configuration data -----> TMU Driver <----> Exynos Core thermal wrapper
+ (exynos_tmu_data.c) (exynos_tmu.c) (exynos_thermal_common.c)
+ (exynos_tmu_data.h) (exynos_tmu.h) (exynos_thermal_common.h)
-a) TMU configuration data: This consist of TMU register offsets/bitfields
+a) TMU configuration data:
+ This consist of TMU register offsets/bitfields
described through structure exynos_tmu_registers. Also several
other platform data (struct exynos_tmu_platform_data) members
are used to configure the TMU.
-b) TMU driver: This component initialises the TMU controller and sets different
+b) TMU driver:
+ This component initialises the TMU controller and sets different
thresholds. It invokes core thermal implementation with the call
exynos_report_trigger.
-c) Exynos Core thermal wrapper: This provides 3 wrapper function to use the
+c) Exynos Core thermal wrapper:
+ This provides 3 wrapper function to use the
Kernel core thermal framework. They are exynos_unregister_thermal,
exynos_register_thermal and exynos_report_trigger.
+++ /dev/null
-EXYNOS EMULATION MODE
-========================
-
-Copyright (C) 2012 Samsung Electronics
-
-Written by Jonghwa Lee <jonghwa3.lee@samsung.com>
-
-Description
------------
-
-Exynos 4x12 (4212, 4412) and 5 series provide emulation mode for thermal management unit.
-Thermal emulation mode supports software debug for TMU's operation. User can set temperature
-manually with software code and TMU will read current temperature from user value not from
-sensor's value.
-
-Enabling CONFIG_THERMAL_EMULATION option will make this support available.
-When it's enabled, sysfs node will be created as
-/sys/devices/virtual/thermal/thermal_zone'zone id'/emul_temp.
-
-The sysfs node, 'emul_node', will contain value 0 for the initial state. When you input any
-temperature you want to update to sysfs node, it automatically enable emulation mode and
-current temperature will be changed into it.
-(Exynos also supports user changeable delay time which would be used to delay of
- changing temperature. However, this node only uses same delay of real sensing time, 938us.)
-
-Exynos emulation mode requires synchronous of value changing and enabling. It means when you
-want to update the any value of delay or next temperature, then you have to enable emulation
-mode at the same time. (Or you have to keep the mode enabling.) If you don't, it fails to
-change the value to updated one and just use last succeessful value repeatedly. That's why
-this node gives users the right to change termerpature only. Just one interface makes it more
-simply to use.
-
-Disabling emulation mode only requires writing value 0 to sysfs node.
-
-
-TEMP 120 |
- |
- 100 |
- |
- 80 |
- | +-----------
- 60 | | |
- | +-------------| |
- 40 | | | |
- | | | |
- 20 | | | +----------
- | | | | |
- 0 |______________|_____________|__________|__________|_________
- A A A A TIME
- |<----->| |<----->| |<----->| |
- | 938us | | | | | |
-emulation : 0 50 | 70 | 20 | 0
-current temp : sensor 50 70 20 sensor
--- /dev/null
+=====================
+Exynos Emulation Mode
+=====================
+
+Copyright (C) 2012 Samsung Electronics
+
+Written by Jonghwa Lee <jonghwa3.lee@samsung.com>
+
+Description
+-----------
+
+Exynos 4x12 (4212, 4412) and 5 series provide emulation mode for thermal
+management unit. Thermal emulation mode supports software debug for
+TMU's operation. User can set temperature manually with software code
+and TMU will read current temperature from user value not from sensor's
+value.
+
+Enabling CONFIG_THERMAL_EMULATION option will make this support
+available. When it's enabled, sysfs node will be created as
+/sys/devices/virtual/thermal/thermal_zone'zone id'/emul_temp.
+
+The sysfs node, 'emul_node', will contain value 0 for the initial state.
+When you input any temperature you want to update to sysfs node, it
+automatically enable emulation mode and current temperature will be
+changed into it.
+
+(Exynos also supports user changeable delay time which would be used to
+delay of changing temperature. However, this node only uses same delay
+of real sensing time, 938us.)
+
+Exynos emulation mode requires synchronous of value changing and
+enabling. It means when you want to update the any value of delay or
+next temperature, then you have to enable emulation mode at the same
+time. (Or you have to keep the mode enabling.) If you don't, it fails to
+change the value to updated one and just use last succeessful value
+repeatedly. That's why this node gives users the right to change
+termerpature only. Just one interface makes it more simply to use.
+
+Disabling emulation mode only requires writing value 0 to sysfs node.
+
+::
+
+
+ TEMP 120 |
+ |
+ 100 |
+ |
+ 80 |
+ | +-----------
+ 60 | | |
+ | +-------------| |
+ 40 | | | |
+ | | | |
+ 20 | | | +----------
+ | | | | |
+ 0 |______________|_____________|__________|__________|_________
+ A A A A TIME
+ |<----->| |<----->| |<----->| |
+ | 938us | | | | | |
+ emulation : 0 50 | 70 | 20 | 0
+ current temp: sensor 50 70 20 sensor
--- /dev/null
+:orphan:
+
+=======
+Thermal
+=======
+
+.. toctree::
+ :maxdepth: 1
+
+ cpu-cooling-api
+ sysfs-api
+ power_allocator
+
+ exynos_thermal
+ exynos_thermal_emulation
+ intel_powerclamp
+ nouveau_thermal
+ x86_pkg_temperature_thermal
- =======================
- INTEL POWERCLAMP DRIVER
- =======================
-By: Arjan van de Ven <arjan@linux.intel.com>
- Jacob Pan <jacob.jun.pan@linux.intel.com>
+=======================
+Intel Powerclamp Driver
+=======================
+
+By:
+ - Arjan van de Ven <arjan@linux.intel.com>
+ - Jacob Pan <jacob.jun.pan@linux.intel.com>
+
+.. Contents:
-Contents:
(*) Introduction
- Goals and Objectives
- Generic Thermal Layer (sysfs)
- Kernel APIs (TBD)
-============
INTRODUCTION
============
shown over taking the CPU offline or modulating the CPU clock.
-===================
THEORY OF OPERATION
===================
On modern Intel processors (Nehalem or later), package level C-state
residency is available in MSRs, thus also available to the kernel.
-These MSRs are:
- #define MSR_PKG_C2_RESIDENCY 0x60D
- #define MSR_PKG_C3_RESIDENCY 0x3F8
- #define MSR_PKG_C6_RESIDENCY 0x3F9
- #define MSR_PKG_C7_RESIDENCY 0x3FA
+These MSRs are::
+
+ #define MSR_PKG_C2_RESIDENCY 0x60D
+ #define MSR_PKG_C3_RESIDENCY 0x3F8
+ #define MSR_PKG_C6_RESIDENCY 0x3F9
+ #define MSR_PKG_C7_RESIDENCY 0x3FA
If the kernel can also inject idle time to the system, then a
closed-loop control system can be established that manages package
have a dramatic impact on the effectiveness of the powerclamp driver
on large scale systems (Westmere system with 80 processors).
-CPU0
- ____________ ____________
-kidle_inject/0 | sleep | mwait | sleep |
- _________| |________| |_______
- duration
-CPU1
- ____________ ____________
-kidle_inject/1 | sleep | mwait | sleep |
- _________| |________| |_______
- ^
- |
- |
- roundup(jiffies, interval)
+::
+
+ CPU0
+ ____________ ____________
+ kidle_inject/0 | sleep | mwait | sleep |
+ _________| |________| |_______
+ duration
+ CPU1
+ ____________ ____________
+ kidle_inject/1 | sleep | mwait | sleep |
+ _________| |________| |_______
+ ^
+ |
+ |
+ roundup(jiffies, interval)
Only one CPU is allowed to collect statistics and update global
control parameters. This CPU is referred to as the controlling CPU in
Compensation to each target ratio consists of two parts:
- a) steady state error compensation
+ a) steady state error compensation
This is to offset the error occurring when the system can
enter idle without extra wakeups (such as external interrupts).
slowing down CPU activities.
A debugfs file is provided for the user to examine compensation
-progress and results, such as on a Westmere system.
-[jacob@nex01 ~]$ cat
-/sys/kernel/debug/intel_powerclamp/powerclamp_calib
-controlling cpu: 0
-pct confidence steady dynamic (compensation)
-0 0 0 0
-1 1 0 0
-2 1 1 0
-3 3 1 0
-4 3 1 0
-5 3 1 0
-6 3 1 0
-7 3 1 0
-8 3 1 0
-...
-30 3 2 0
-31 3 2 0
-32 3 1 0
-33 3 2 0
-34 3 1 0
-35 3 2 0
-36 3 1 0
-37 3 2 0
-38 3 1 0
-39 3 2 0
-40 3 3 0
-41 3 1 0
-42 3 2 0
-43 3 1 0
-44 3 1 0
-45 3 2 0
-46 3 3 0
-47 3 0 0
-48 3 2 0
-49 3 3 0
+progress and results, such as on a Westmere system::
+
+ [jacob@nex01 ~]$ cat
+ /sys/kernel/debug/intel_powerclamp/powerclamp_calib
+ controlling cpu: 0
+ pct confidence steady dynamic (compensation)
+ 0 0 0 0
+ 1 1 0 0
+ 2 1 1 0
+ 3 3 1 0
+ 4 3 1 0
+ 5 3 1 0
+ 6 3 1 0
+ 7 3 1 0
+ 8 3 1 0
+ ...
+ 30 3 2 0
+ 31 3 2 0
+ 32 3 1 0
+ 33 3 2 0
+ 34 3 1 0
+ 35 3 2 0
+ 36 3 1 0
+ 37 3 2 0
+ 38 3 1 0
+ 39 3 2 0
+ 40 3 3 0
+ 41 3 1 0
+ 42 3 2 0
+ 43 3 1 0
+ 44 3 1 0
+ 45 3 2 0
+ 46 3 3 0
+ 47 3 0 0
+ 48 3 2 0
+ 49 3 3 0
Calibration occurs during runtime. No offline method is available.
Steady state compensation is used only when confidence levels of all
to other CPUs, after a CPU offline event.
-=====================
Performance Analysis
-=====================
+====================
This section describes the general performance data collected on
multiple systems, including Westmere (80P) and Ivy Bridge (4P, 8P).
counter summed over per CPU counting threads spawned for all running
CPUs).
-====================
Usage and Interfaces
====================
The powerclamp driver is registered to the generic thermal layer as a
-cooling device. Currently, it’s not bound to any thermal zones.
+cooling device. Currently, it’s not bound to any thermal zones::
-jacob@chromoly:/sys/class/thermal/cooling_device14$ grep . *
-cur_state:0
-max_state:50
-type:intel_powerclamp
+ jacob@chromoly:/sys/class/thermal/cooling_device14$ grep . *
+ cur_state:0
+ max_state:50
+ type:intel_powerclamp
cur_state allows user to set the desired idle percentage. Writing 0 to
cur_state will stop idle injection. Writing a value between 1 and
100% busy state with the disabled state.
Example usage:
-- To inject 25% idle time
-$ sudo sh -c "echo 25 > /sys/class/thermal/cooling_device80/cur_state
-"
+- To inject 25% idle time::
+
+ $ sudo sh -c "echo 25 > /sys/class/thermal/cooling_device80/cur_state
If the system is not busy and has more than 25% idle time already,
then the powerclamp driver will not start idle injection. Using Top
taken as the idle task.
In this example, 24.1% idle is shown. This helps the system admin or
-user determine the cause of slowdown, when a powerclamp driver is in action.
-
-
-Tasks: 197 total, 1 running, 196 sleeping, 0 stopped, 0 zombie
-Cpu(s): 71.2%us, 4.7%sy, 0.0%ni, 24.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
-Mem: 3943228k total, 1689632k used, 2253596k free, 74960k buffers
-Swap: 4087804k total, 0k used, 4087804k free, 945336k cached
-
- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
- 3352 jacob 20 0 262m 644 428 S 286 0.0 0:17.16 spin
- 3341 root -51 0 0 0 0 D 25 0.0 0:01.62 kidle_inject/0
- 3344 root -51 0 0 0 0 D 25 0.0 0:01.60 kidle_inject/3
- 3342 root -51 0 0 0 0 D 25 0.0 0:01.61 kidle_inject/1
- 3343 root -51 0 0 0 0 D 25 0.0 0:01.60 kidle_inject/2
- 2935 jacob 20 0 696m 125m 35m S 5 3.3 0:31.11 firefox
- 1546 root 20 0 158m 20m 6640 S 3 0.5 0:26.97 Xorg
- 2100 jacob 20 0 1223m 88m 30m S 3 2.3 0:23.68 compiz
+user determine the cause of slowdown, when a powerclamp driver is in action::
+
+
+ Tasks: 197 total, 1 running, 196 sleeping, 0 stopped, 0 zombie
+ Cpu(s): 71.2%us, 4.7%sy, 0.0%ni, 24.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
+ Mem: 3943228k total, 1689632k used, 2253596k free, 74960k buffers
+ Swap: 4087804k total, 0k used, 4087804k free, 945336k cached
+
+ PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
+ 3352 jacob 20 0 262m 644 428 S 286 0.0 0:17.16 spin
+ 3341 root -51 0 0 0 0 D 25 0.0 0:01.62 kidle_inject/0
+ 3344 root -51 0 0 0 0 D 25 0.0 0:01.60 kidle_inject/3
+ 3342 root -51 0 0 0 0 D 25 0.0 0:01.61 kidle_inject/1
+ 3343 root -51 0 0 0 0 D 25 0.0 0:01.60 kidle_inject/2
+ 2935 jacob 20 0 696m 125m 35m S 5 3.3 0:31.11 firefox
+ 1546 root 20 0 158m 20m 6640 S 3 0.5 0:26.97 Xorg
+ 2100 jacob 20 0 1223m 88m 30m S 3 2.3 0:23.68 compiz
Tests have shown that by using the powerclamp driver as a cooling
device, a PID based userspace thermal controller can manage to
+=====================
Kernel driver nouveau
-===================
+=====================
Supported chips:
+
* NV43+
Authors: Martin Peres (mupuf) <martin.peres@free.fr>
Description
----------
+-----------
This driver allows to read the GPU core temperature, drive the GPU fan and
set temperature alarms.
entirely.
Temperature management
---------------------
+----------------------
Temperature is exposed under as a read-only HWMON attribute temp1_input.
In order to protect the GPU from overheating, Nouveau supports 4 configurable
temperature thresholds:
- * Fan_boost: Fan speed is set to 100% when reaching this temperature;
- * Downclock: The GPU will be downclocked to reduce its power dissipation;
- * Critical: The GPU is put on hold to further lower power dissipation;
- * Shutdown: Shut the computer down to protect your GPU.
+ * Fan_boost:
+ Fan speed is set to 100% when reaching this temperature;
+ * Downclock:
+ The GPU will be downclocked to reduce its power dissipation;
+ * Critical:
+ The GPU is put on hold to further lower power dissipation;
+ * Shutdown:
+ Shut the computer down to protect your GPU.
-WARNING: Some of these thresholds may not be used by Nouveau depending
-on your chipset.
+WARNING:
+ Some of these thresholds may not be used by Nouveau depending
+ on your chipset.
The default value for these thresholds comes from the GPU's vbios. These
thresholds can be configured thanks to the following HWMON attributes:
to multiply!
Fan management
-------------
+--------------
Not all cards have a drivable fan. If you do, then the following HWMON
attributes should be available:
- * pwm1_enable: Current fan management mode (NONE, MANUAL or AUTO);
- * pwm1: Current PWM value (power percentage);
- * pwm1_min: The minimum PWM speed allowed;
- * pwm1_max: The maximum PWM speed allowed (bypassed when hitting Fan_boost);
+ * pwm1_enable:
+ Current fan management mode (NONE, MANUAL or AUTO);
+ * pwm1:
+ Current PWM value (power percentage);
+ * pwm1_min:
+ The minimum PWM speed allowed;
+ * pwm1_max:
+ The maximum PWM speed allowed (bypassed when hitting Fan_boost);
You may also have the following attribute:
- * fan1_input: Speed in RPM of your fan.
+ * fan1_input:
+ Speed in RPM of your fan.
Your fan can be driven in different modes:
* 1: The fan can be driven in manual (use pwm1 to change the speed);
* 2; The fan is driven automatically depending on the temperature.
-NOTE: Be sure to use the manual mode if you want to drive the fan speed manually
+NOTE:
+ Be sure to use the manual mode if you want to drive the fan speed manually
-NOTE2: When operating in manual mode outside the vbios-defined
-[PWM_min, PWM_max] range, the reported fan speed (RPM) may not be accurate
-depending on your hardware.
+NOTE2:
+ When operating in manual mode outside the vbios-defined
+ [PWM_min, PWM_max] range, the reported fan speed (RPM) may not be accurate
+ depending on your hardware.
Bug reports
----------
+-----------
Thermal management on Nouveau is new and may not work on all cards. If you have
inquiries, please ping mupuf on IRC (#nouveau, freenode).
+=================================
Power allocator governor tunables
=================================
P_max = k_p * e + k_i * err_integral + k_d * diff_err + sustainable_power
where
- e = desired_temperature - current_temperature
- err_integral is the sum of previous errors
- diff_err = e - previous_error
-
-It is similar to the one depicted below:
-
- k_d
- |
-current_temp |
- | v
- | +----------+ +---+
- | +----->| diff_err |-->| X |------+
- | | +----------+ +---+ |
- | | | tdp actor
- | | k_i | | get_requested_power()
- | | | | | | |
- | | | | | | | ...
- v | v v v v v
- +---+ | +-------+ +---+ +---+ +---+ +----------+
- | S |-------+----->| sum e |----->| X |--->| S |-->| S |-->|power |
- +---+ | +-------+ +---+ +---+ +---+ |allocation|
- ^ | ^ +----------+
- | | | | |
- | | +---+ | | |
- | +------->| X |-------------------+ v v
- | +---+ granted performance
-desired_temperature ^
- |
- |
- k_po/k_pu
+ - e = desired_temperature - current_temperature
+ - err_integral is the sum of previous errors
+ - diff_err = e - previous_error
+
+It is similar to the one depicted below::
+
+ k_d
+ |
+ current_temp |
+ | v
+ | +----------+ +---+
+ | +----->| diff_err |-->| X |------+
+ | | +----------+ +---+ |
+ | | | tdp actor
+ | | k_i | | get_requested_power()
+ | | | | | | |
+ | | | | | | | ...
+ v | v v v v v
+ +---+ | +-------+ +---+ +---+ +---+ +----------+
+ | S |-----+----->| sum e |----->| X |--->| S |-->| S |-->|power |
+ +---+ | +-------+ +---+ +---+ +---+ |allocation|
+ ^ | ^ +----------+
+ | | | | |
+ | | +---+ | | |
+ | +------->| X |-------------------+ v v
+ | +---+ granted performance
+ desired_temperature ^
+ |
+ |
+ k_po/k_pu
Sustainable power
-----------------
depending on screen size).
If you are using device tree, do add it as a property of the
-thermal-zone. For example:
+thermal-zone. For example::
thermal-zones {
soc_thermal {
Instead, if the thermal zone is registered from the platform code, pass a
`thermal_zone_params` that has a `sustainable_power`. If no
`thermal_zone_params` were being passed, then something like below
-will suffice:
+will suffice::
static const struct thermal_zone_params tz_params = {
.sustainable_power = 3500,
value of `k_pu` will result in the governor granting very high power
while temperature is low, and may lead to temperature overshooting.
-The default value for `k_pu` is:
+The default value for `k_pu` is::
2 * sustainable_power / (desired_temperature - switch_on_temp)
This means that at `switch_on_temp` the output of the controller's
proportional term will be 2 * `sustainable_power`. The default value
-for `k_po` is:
+for `k_po` is::
sustainable_power / (desired_temperature - switch_on_temp)
Focusing on the proportional and feed forward values of the PID
-controller equation we have:
+controller equation we have::
P_max = k_p * e + sustainable_power
thermal equilibrium under constant load. `sustainable_power` is only
an estimate, which is the reason for closed-loop control such as this.
-Expanding `k_pu` we get:
+Expanding `k_pu` we get::
+
P_max = 2 * sustainable_power * (T_set - T) / (T_set - T_on) +
- sustainable_power
+ sustainable_power
-where
- T_set is the desired temperature
- T is the current temperature
- T_on is the switch on temperature
+where:
+
+ - T_set is the desired temperature
+ - T is the current temperature
+ - T_on is the switch on temperature
When the current temperature is the switch_on temperature, the above
-formula becomes:
+formula becomes::
P_max = 2 * sustainable_power * (T_set - T_on) / (T_set - T_on) +
- sustainable_power = 2 * sustainable_power + sustainable_power =
- 3 * sustainable_power
+ sustainable_power = 2 * sustainable_power + sustainable_power =
+ 3 * sustainable_power
Therefore, the proportional term alone linearly decreases power from
3 * `sustainable_power` to `sustainable_power` as the temperature
Cooling devices controlled by this governor must supply the additional
"power" API in their `cooling_device_ops`. It consists on three ops:
-1. int get_requested_power(struct thermal_cooling_device *cdev,
- struct thermal_zone_device *tz, u32 *power);
-@cdev: The `struct thermal_cooling_device` pointer
-@tz: thermal zone in which we are currently operating
-@power: pointer in which to store the calculated power
+1. ::
+
+ int get_requested_power(struct thermal_cooling_device *cdev,
+ struct thermal_zone_device *tz, u32 *power);
+
+
+@cdev:
+ The `struct thermal_cooling_device` pointer
+@tz:
+ thermal zone in which we are currently operating
+@power:
+ pointer in which to store the calculated power
`get_requested_power()` calculates the power requested by the device
in milliwatts and stores it in @power . It should return 0 on
allocator governor to calculate how much power to give to each cooling
device.
-2. int state2power(struct thermal_cooling_device *cdev, struct
- thermal_zone_device *tz, unsigned long state, u32 *power);
-@cdev: The `struct thermal_cooling_device` pointer
-@tz: thermal zone in which we are currently operating
-@state: A cooling device state
-@power: pointer in which to store the equivalent power
+2. ::
+
+ int state2power(struct thermal_cooling_device *cdev, struct
+ thermal_zone_device *tz, unsigned long state,
+ u32 *power);
+
+@cdev:
+ The `struct thermal_cooling_device` pointer
+@tz:
+ thermal zone in which we are currently operating
+@state:
+ A cooling device state
+@power:
+ pointer in which to store the equivalent power
Convert cooling device state @state into power consumption in
milliwatts and store it in @power. It should return 0 on success, -E*
on failure. This is currently used by thermal core to calculate the
maximum power that an actor can consume.
-3. int power2state(struct thermal_cooling_device *cdev, u32 power,
- unsigned long *state);
-@cdev: The `struct thermal_cooling_device` pointer
-@power: power in milliwatts
-@state: pointer in which to store the resulting state
+3. ::
+
+ int power2state(struct thermal_cooling_device *cdev, u32 power,
+ unsigned long *state);
+
+@cdev:
+ The `struct thermal_cooling_device` pointer
+@power:
+ power in milliwatts
+@state:
+ pointer in which to store the resulting state
Calculate a cooling device state that would make the device consume at
most @power mW and store it in @state. It should return 0 on success,
+===================================
Generic Thermal Sysfs driver How To
===================================
0. Introduction
+===============
The generic thermal sysfs provides a set of interfaces for thermal zone
devices (sensors) and thermal cooling devices (fan, processor...) to register
inputs from thermal zone attributes (the current temperature and trip point
temperature) and throttle appropriate devices.
-[0-*] denotes any positive number starting from 0
-[1-*] denotes any positive number starting from 1
+- `[0-*]` denotes any positive number starting from 0
+- `[1-*]` denotes any positive number starting from 1
1. thermal sysfs driver interface functions
+===========================================
1.1 thermal zone device interface
-1.1.1 struct thermal_zone_device *thermal_zone_device_register(char *type,
- int trips, int mask, void *devdata,
- struct thermal_zone_device_ops *ops,
- const struct thermal_zone_params *tzp,
- int passive_delay, int polling_delay))
+---------------------------------
+
+ ::
+
+ struct thermal_zone_device
+ *thermal_zone_device_register(char *type,
+ int trips, int mask, void *devdata,
+ struct thermal_zone_device_ops *ops,
+ const struct thermal_zone_params *tzp,
+ int passive_delay, int polling_delay))
This interface function adds a new thermal zone device (sensor) to
- /sys/class/thermal folder as thermal_zone[0-*]. It tries to bind all the
+ /sys/class/thermal folder as `thermal_zone[0-*]`. It tries to bind all the
thermal cooling devices registered at the same time.
- type: the thermal zone type.
- trips: the total number of trip points this thermal zone supports.
- mask: Bit string: If 'n'th bit is set, then trip point 'n' is writeable.
- devdata: device private data
- ops: thermal zone device call-backs.
- .bind: bind the thermal zone device with a thermal cooling device.
- .unbind: unbind the thermal zone device with a thermal cooling device.
- .get_temp: get the current temperature of the thermal zone.
- .set_trips: set the trip points window. Whenever the current temperature
+ type:
+ the thermal zone type.
+ trips:
+ the total number of trip points this thermal zone supports.
+ mask:
+ Bit string: If 'n'th bit is set, then trip point 'n' is writeable.
+ devdata:
+ device private data
+ ops:
+ thermal zone device call-backs.
+
+ .bind:
+ bind the thermal zone device with a thermal cooling device.
+ .unbind:
+ unbind the thermal zone device with a thermal cooling device.
+ .get_temp:
+ get the current temperature of the thermal zone.
+ .set_trips:
+ set the trip points window. Whenever the current temperature
is updated, the trip points immediately below and above the
current temperature are found.
- .get_mode: get the current mode (enabled/disabled) of the thermal zone.
- - "enabled" means the kernel thermal management is enabled.
- - "disabled" will prevent kernel thermal driver action upon trip points
- so that user applications can take charge of thermal management.
- .set_mode: set the mode (enabled/disabled) of the thermal zone.
- .get_trip_type: get the type of certain trip point.
- .get_trip_temp: get the temperature above which the certain trip point
+ .get_mode:
+ get the current mode (enabled/disabled) of the thermal zone.
+
+ - "enabled" means the kernel thermal management is
+ enabled.
+ - "disabled" will prevent kernel thermal driver action
+ upon trip points so that user applications can take
+ charge of thermal management.
+ .set_mode:
+ set the mode (enabled/disabled) of the thermal zone.
+ .get_trip_type:
+ get the type of certain trip point.
+ .get_trip_temp:
+ get the temperature above which the certain trip point
will be fired.
- .set_emul_temp: set the emulation temperature which helps in debugging
+ .set_emul_temp:
+ set the emulation temperature which helps in debugging
different threshold temperature points.
- tzp: thermal zone platform parameters.
- passive_delay: number of milliseconds to wait between polls when
+ tzp:
+ thermal zone platform parameters.
+ passive_delay:
+ number of milliseconds to wait between polls when
performing passive cooling.
- polling_delay: number of milliseconds to wait between polls when checking
+ polling_delay:
+ number of milliseconds to wait between polls when checking
whether trip points have been crossed (0 for interrupt driven systems).
+ ::
-1.1.2 void thermal_zone_device_unregister(struct thermal_zone_device *tz)
+ void thermal_zone_device_unregister(struct thermal_zone_device *tz)
This interface function removes the thermal zone device.
It deletes the corresponding entry from /sys/class/thermal folder and
unbinds all the thermal cooling devices it uses.
-1.1.3 struct thermal_zone_device *thermal_zone_of_sensor_register(
- struct device *dev, int sensor_id, void *data,
- const struct thermal_zone_of_device_ops *ops)
+ ::
+
+ struct thermal_zone_device
+ *thermal_zone_of_sensor_register(struct device *dev, int sensor_id,
+ void *data,
+ const struct thermal_zone_of_device_ops *ops)
This interface adds a new sensor to a DT thermal zone.
This function will search the list of thermal zones described in
thermal zone device.
The parameters for this interface are:
- dev: Device node of sensor containing valid node pointer in
+
+ dev:
+ Device node of sensor containing valid node pointer in
dev->of_node.
- sensor_id: a sensor identifier, in case the sensor IP has more
+ sensor_id:
+ a sensor identifier, in case the sensor IP has more
than one sensors
- data: a private pointer (owned by the caller) that will be
+ data:
+ a private pointer (owned by the caller) that will be
passed back, when a temperature reading is needed.
- ops: struct thermal_zone_of_device_ops *.
+ ops:
+ `struct thermal_zone_of_device_ops *`.
- get_temp: a pointer to a function that reads the
+ ============== =======================================
+ get_temp a pointer to a function that reads the
sensor temperature. This is mandatory
callback provided by sensor driver.
- set_trips: a pointer to a function that sets a
+ set_trips a pointer to a function that sets a
temperature window. When this window is
left the driver must inform the thermal
core via thermal_zone_device_update.
- get_trend: a pointer to a function that reads the
+ get_trend a pointer to a function that reads the
sensor temperature trend.
- set_emul_temp: a pointer to a function that sets
+ set_emul_temp a pointer to a function that sets
sensor emulated temperature.
+ ============== =======================================
+
The thermal zone temperature is provided by the get_temp() function
pointer of thermal_zone_of_device_ops. When called, it will
have the private pointer @data back.
handle. Caller should check the return handle with IS_ERR() for finding
whether success or not.
-1.1.4 void thermal_zone_of_sensor_unregister(struct device *dev,
- struct thermal_zone_device *tzd)
+ ::
+
+ void thermal_zone_of_sensor_unregister(struct device *dev,
+ struct thermal_zone_device *tzd)
This interface unregisters a sensor from a DT thermal zone which was
successfully added by interface thermal_zone_of_sensor_register().
interface. It will also silent the zone by remove the .get_temp() and
get_trend() thermal zone device callbacks.
-1.1.5 struct thermal_zone_device *devm_thermal_zone_of_sensor_register(
- struct device *dev, int sensor_id,
- void *data, const struct thermal_zone_of_device_ops *ops)
+ ::
+
+ struct thermal_zone_device
+ *devm_thermal_zone_of_sensor_register(struct device *dev,
+ int sensor_id,
+ void *data,
+ const struct thermal_zone_of_device_ops *ops)
This interface is resource managed version of
thermal_zone_of_sensor_register().
+
All details of thermal_zone_of_sensor_register() described in
section 1.1.3 is applicable here.
+
The benefit of using this interface to register sensor is that it
is not require to explicitly call thermal_zone_of_sensor_unregister()
in error path or during driver unbinding as this is done by driver
resource manager.
-1.1.6 void devm_thermal_zone_of_sensor_unregister(struct device *dev,
- struct thermal_zone_device *tzd)
+ ::
+
+ void devm_thermal_zone_of_sensor_unregister(struct device *dev,
+ struct thermal_zone_device *tzd)
This interface is resource managed version of
thermal_zone_of_sensor_unregister().
Normally this function will not need to be called and the resource
management code will ensure that the resource is freed.
-1.1.7 int thermal_zone_get_slope(struct thermal_zone_device *tz)
+ ::
+
+ int thermal_zone_get_slope(struct thermal_zone_device *tz)
This interface is used to read the slope attribute value
for the thermal zone device, which might be useful for platform
drivers for temperature calculations.
-1.1.8 int thermal_zone_get_offset(struct thermal_zone_device *tz)
+ ::
+
+ int thermal_zone_get_offset(struct thermal_zone_device *tz)
This interface is used to read the offset attribute value
for the thermal zone device, which might be useful for platform
drivers for temperature calculations.
1.2 thermal cooling device interface
-1.2.1 struct thermal_cooling_device *thermal_cooling_device_register(char *name,
- void *devdata, struct thermal_cooling_device_ops *)
+------------------------------------
+
+
+ ::
+
+ struct thermal_cooling_device
+ *thermal_cooling_device_register(char *name,
+ void *devdata, struct thermal_cooling_device_ops *)
This interface function adds a new thermal cooling device (fan/processor/...)
- to /sys/class/thermal/ folder as cooling_device[0-*]. It tries to bind itself
+ to /sys/class/thermal/ folder as `cooling_device[0-*]`. It tries to bind itself
to all the thermal zone devices registered at the same time.
- name: the cooling device name.
- devdata: device private data.
- ops: thermal cooling devices call-backs.
- .get_max_state: get the Maximum throttle state of the cooling device.
- .get_cur_state: get the Currently requested throttle state of the cooling device.
- .set_cur_state: set the Current throttle state of the cooling device.
-1.2.2 void thermal_cooling_device_unregister(struct thermal_cooling_device *cdev)
+ name:
+ the cooling device name.
+ devdata:
+ device private data.
+ ops:
+ thermal cooling devices call-backs.
+
+ .get_max_state:
+ get the Maximum throttle state of the cooling device.
+ .get_cur_state:
+ get the Currently requested throttle state of the
+ cooling device.
+ .set_cur_state:
+ set the Current throttle state of the cooling device.
+
+ ::
+
+ void thermal_cooling_device_unregister(struct thermal_cooling_device *cdev)
This interface function removes the thermal cooling device.
It deletes the corresponding entry from /sys/class/thermal folder and
unbinds itself from all the thermal zone devices using it.
1.3 interface for binding a thermal zone device with a thermal cooling device
-1.3.1 int thermal_zone_bind_cooling_device(struct thermal_zone_device *tz,
- int trip, struct thermal_cooling_device *cdev,
- unsigned long upper, unsigned long lower, unsigned int weight);
+-----------------------------------------------------------------------------
+
+ ::
+
+ int thermal_zone_bind_cooling_device(struct thermal_zone_device *tz,
+ int trip, struct thermal_cooling_device *cdev,
+ unsigned long upper, unsigned long lower, unsigned int weight);
This interface function binds a thermal cooling device to a particular trip
point of a thermal zone device.
+
This function is usually called in the thermal zone device .bind callback.
- tz: the thermal zone device
- cdev: thermal cooling device
- trip: indicates which trip point in this thermal zone the cooling device
- is associated with.
- upper:the Maximum cooling state for this trip point.
- THERMAL_NO_LIMIT means no upper limit,
+
+ tz:
+ the thermal zone device
+ cdev:
+ thermal cooling device
+ trip:
+ indicates which trip point in this thermal zone the cooling device
+ is associated with.
+ upper:
+ the Maximum cooling state for this trip point.
+ THERMAL_NO_LIMIT means no upper limit,
and the cooling device can be in max_state.
- lower:the Minimum cooling state can be used for this trip point.
- THERMAL_NO_LIMIT means no lower limit,
+ lower:
+ the Minimum cooling state can be used for this trip point.
+ THERMAL_NO_LIMIT means no lower limit,
and the cooling device can be in cooling state 0.
- weight: the influence of this cooling device in this thermal
- zone. See 1.4.1 below for more information.
+ weight:
+ the influence of this cooling device in this thermal
+ zone. See 1.4.1 below for more information.
-1.3.2 int thermal_zone_unbind_cooling_device(struct thermal_zone_device *tz,
- int trip, struct thermal_cooling_device *cdev);
+ ::
+
+ int thermal_zone_unbind_cooling_device(struct thermal_zone_device *tz,
+ int trip, struct thermal_cooling_device *cdev);
This interface function unbinds a thermal cooling device from a particular
trip point of a thermal zone device. This function is usually called in
the thermal zone device .unbind callback.
- tz: the thermal zone device
- cdev: thermal cooling device
- trip: indicates which trip point in this thermal zone the cooling device
- is associated with.
+
+ tz:
+ the thermal zone device
+ cdev:
+ thermal cooling device
+ trip:
+ indicates which trip point in this thermal zone the cooling device
+ is associated with.
1.4 Thermal Zone Parameters
-1.4.1 struct thermal_bind_params
+---------------------------
+
+ ::
+
+ struct thermal_bind_params
+
This structure defines the following parameters that are used to bind
a zone with a cooling device for a particular trip point.
- .cdev: The cooling device pointer
- .weight: The 'influence' of a particular cooling device on this
- zone. This is relative to the rest of the cooling
- devices. For example, if all cooling devices have a
- weight of 1, then they all contribute the same. You can
- use percentages if you want, but it's not mandatory. A
- weight of 0 means that this cooling device doesn't
- contribute to the cooling of this zone unless all cooling
- devices have a weight of 0. If all weights are 0, then
- they all contribute the same.
- .trip_mask:This is a bit mask that gives the binding relation between
- this thermal zone and cdev, for a particular trip point.
- If nth bit is set, then the cdev and thermal zone are bound
- for trip point n.
- .binding_limits: This is an array of cooling state limits. Must have
- exactly 2 * thermal_zone.number_of_trip_points. It is an
- array consisting of tuples <lower-state upper-state> of
- state limits. Each trip will be associated with one state
- limit tuple when binding. A NULL pointer means
- <THERMAL_NO_LIMITS THERMAL_NO_LIMITS> on all trips.
- These limits are used when binding a cdev to a trip point.
- .match: This call back returns success(0) if the 'tz and cdev' need to
+
+ .cdev:
+ The cooling device pointer
+ .weight:
+ The 'influence' of a particular cooling device on this
+ zone. This is relative to the rest of the cooling
+ devices. For example, if all cooling devices have a
+ weight of 1, then they all contribute the same. You can
+ use percentages if you want, but it's not mandatory. A
+ weight of 0 means that this cooling device doesn't
+ contribute to the cooling of this zone unless all cooling
+ devices have a weight of 0. If all weights are 0, then
+ they all contribute the same.
+ .trip_mask:
+ This is a bit mask that gives the binding relation between
+ this thermal zone and cdev, for a particular trip point.
+ If nth bit is set, then the cdev and thermal zone are bound
+ for trip point n.
+ .binding_limits:
+ This is an array of cooling state limits. Must have
+ exactly 2 * thermal_zone.number_of_trip_points. It is an
+ array consisting of tuples <lower-state upper-state> of
+ state limits. Each trip will be associated with one state
+ limit tuple when binding. A NULL pointer means
+ <THERMAL_NO_LIMITS THERMAL_NO_LIMITS> on all trips.
+ These limits are used when binding a cdev to a trip point.
+ .match:
+ This call back returns success(0) if the 'tz and cdev' need to
be bound, as per platform data.
-1.4.2 struct thermal_zone_params
+
+ ::
+
+ struct thermal_zone_params
+
This structure defines the platform level parameters for a thermal zone.
This data, for each thermal zone should come from the platform layer.
This is an optional feature where some platforms can choose not to
provide this data.
- .governor_name: Name of the thermal governor used for this zone
- .no_hwmon: a boolean to indicate if the thermal to hwmon sysfs interface
- is required. when no_hwmon == false, a hwmon sysfs interface
- will be created. when no_hwmon == true, nothing will be done.
- In case the thermal_zone_params is NULL, the hwmon interface
- will be created (for backward compatibility).
- .num_tbps: Number of thermal_bind_params entries for this zone
- .tbp: thermal_bind_params entries
+
+ .governor_name:
+ Name of the thermal governor used for this zone
+ .no_hwmon:
+ a boolean to indicate if the thermal to hwmon sysfs interface
+ is required. when no_hwmon == false, a hwmon sysfs interface
+ will be created. when no_hwmon == true, nothing will be done.
+ In case the thermal_zone_params is NULL, the hwmon interface
+ will be created (for backward compatibility).
+ .num_tbps:
+ Number of thermal_bind_params entries for this zone
+ .tbp:
+ thermal_bind_params entries
2. sysfs attributes structure
+=============================
+== ================
RO read only value
WO write only value
RW read/write value
+== ================
Thermal sysfs attributes will be represented under /sys/class/thermal.
Hwmon sysfs I/F extension is also available under /sys/class/hwmon
if hwmon is compiled in or built as a module.
-Thermal zone device sys I/F, created once it's registered:
-/sys/class/thermal/thermal_zone[0-*]:
+Thermal zone device sys I/F, created once it's registered::
+
+ /sys/class/thermal/thermal_zone[0-*]:
|---type: Type of the thermal zone
|---temp: Current temperature
|---mode: Working mode of the thermal zone
|---slope: Slope constant applied as linear extrapolation
|---offset: Offset constant applied as linear extrapolation
-Thermal cooling device sys I/F, created once it's registered:
-/sys/class/thermal/cooling_device[0-*]:
+Thermal cooling device sys I/F, created once it's registered::
+
+ /sys/class/thermal/cooling_device[0-*]:
|---type: Type of the cooling device(processor/fan/...)
|---max_state: Maximum cooling state of the cooling device
|---cur_state: Current cooling state of the cooling device
They are created/removed for each successful execution of
thermal_zone_bind_cooling_device/thermal_zone_unbind_cooling_device.
-/sys/class/thermal/thermal_zone[0-*]:
+::
+
+ /sys/class/thermal/thermal_zone[0-*]:
|---cdev[0-*]: [0-*]th cooling device in current thermal zone
|---cdev[0-*]_trip_point: Trip point that cdev[0-*] is associated with
|---cdev[0-*]_weight: Influence of the cooling device in
- this thermal zone
+ this thermal zone
Besides the thermal zone device sysfs I/F and cooling device sysfs I/F,
the generic thermal driver also creates a hwmon sysfs I/F for each _type_
class device and build the associated hwmon sysfs I/F for all the registered
ACPI thermal zones.
-/sys/class/hwmon/hwmon[0-*]:
+::
+
+ /sys/class/hwmon/hwmon[0-*]:
|---name: The type of the thermal zone devices
|---temp[1-*]_input: The current temperature of thermal zone [1-*]
|---temp[1-*]_critical: The critical trip point of thermal zone [1-*]
Please read Documentation/hwmon/sysfs-interface.rst for additional information.
-***************************
-* Thermal zone attributes *
-***************************
+Thermal zone attributes
+-----------------------
type
Strings which represent the thermal zone type.
This file gives information about the algorithm that is currently
managing the thermal zone. It can be either default kernel based
algorithm or user space application.
- enabled = enable Kernel Thermal management.
- disabled = Preventing kernel thermal zone driver actions upon
+
+ enabled
+ enable Kernel Thermal management.
+ disabled
+ Preventing kernel thermal zone driver actions upon
trip points so that user application can take full
charge of the thermal management.
+
RW, Optional
policy
One of the various thermal governors used for a particular zone.
+
RW, Required
available_policies
Available thermal governors which can be used for a particular zone.
+
RO, Required
-trip_point_[0-*]_temp
+`trip_point_[0-*]_temp`
The temperature above which trip point will be fired.
+
Unit: millidegree Celsius
+
RO, Optional
-trip_point_[0-*]_type
+`trip_point_[0-*]_type`
Strings which indicate the type of the trip point.
- E.g. it can be one of critical, hot, passive, active[0-*] for ACPI
+
+ E.g. it can be one of critical, hot, passive, `active[0-*]` for ACPI
thermal zone.
+
RO, Optional
-trip_point_[0-*]_hyst
+`trip_point_[0-*]_hyst`
The hysteresis value for a trip point, represented as an integer
Unit: Celsius
RW, Optional
-cdev[0-*]
+`cdev[0-*]`
Sysfs link to the thermal cooling device node where the sys I/F
for cooling device throttling control represents.
+
RO, Optional
-cdev[0-*]_trip_point
- The trip point in this thermal zone which cdev[0-*] is associated
+`cdev[0-*]_trip_point`
+ The trip point in this thermal zone which `cdev[0-*]` is associated
with; -1 means the cooling device is not associated with any trip
point.
+
RO, Optional
-cdev[0-*]_weight
- The influence of cdev[0-*] in this thermal zone. This value
- is relative to the rest of cooling devices in the thermal
- zone. For example, if a cooling device has a weight double
- than that of other, it's twice as effective in cooling the
- thermal zone.
- RW, Optional
+`cdev[0-*]_weight`
+ The influence of `cdev[0-*]` in this thermal zone. This value
+ is relative to the rest of cooling devices in the thermal
+ zone. For example, if a cooling device has a weight double
+ than that of other, it's twice as effective in cooling the
+ thermal zone.
+
+ RW, Optional
passive
Attribute is only present for zones in which the passive cooling
and can be set to a temperature (in millidegrees) to enable a
passive trip point for the zone. Activation is done by polling with
an interval of 1 second.
+
Unit: millidegrees Celsius
+
Valid values: 0 (disabled) or greater than 1000
+
RW, Optional
emul_temp
threshold and its associated cooling action. This is write only node
and writing 0 on this node should disable emulation.
Unit: millidegree Celsius
+
WO, Optional
- WARNING: Be careful while enabling this option on production systems,
- because userland can easily disable the thermal policy by simply
- flooding this sysfs node with low temperature values.
+ WARNING:
+ Be careful while enabling this option on production systems,
+ because userland can easily disable the thermal policy by simply
+ flooding this sysfs node with low temperature values.
sustainable_power
An estimate of the sustained power that can be dissipated by
the thermal zone. Used by the power allocator governor. For
- more information see Documentation/thermal/power_allocator.txt
+ more information see Documentation/thermal/power_allocator.rst
+
Unit: milliwatts
+
RW, Optional
k_po
controller during temperature overshoot. Temperature overshoot
is when the current temperature is above the "desired
temperature" trip point. For more information see
- Documentation/thermal/power_allocator.txt
+ Documentation/thermal/power_allocator.rst
+
RW, Optional
k_pu
controller during temperature undershoot. Temperature undershoot
is when the current temperature is below the "desired
temperature" trip point. For more information see
- Documentation/thermal/power_allocator.txt
+ Documentation/thermal/power_allocator.rst
+
RW, Optional
k_i
The integral term of the power allocator governor's PID
controller. This term allows the PID controller to compensate
for long term drift. For more information see
- Documentation/thermal/power_allocator.txt
+ Documentation/thermal/power_allocator.rst
+
RW, Optional
k_d
The derivative term of the power allocator governor's PID
controller. For more information see
- Documentation/thermal/power_allocator.txt
+ Documentation/thermal/power_allocator.rst
+
RW, Optional
integral_cutoff
example, if integral_cutoff is 0, then the integral term only
accumulates error when temperature is above the desired
temperature trip point. For more information see
- Documentation/thermal/power_allocator.txt
+ Documentation/thermal/power_allocator.rst
+
Unit: millidegree Celsius
+
RW, Optional
slope
to determine a hotspot temperature based off the sensor's
raw readings. It is up to the device driver to determine
the usage of these values.
+
RW, Optional
offset
to determine a hotspot temperature based off the sensor's
raw readings. It is up to the device driver to determine
the usage of these values.
+
RW, Optional
-*****************************
-* Cooling device attributes *
-*****************************
+Cooling device attributes
+-------------------------
type
String which represents the type of device, e.g:
+
- for generic ACPI: should be "Fan", "Processor" or "LCD"
- for memory controller device on intel_menlow platform:
should be "Memory controller".
+
RO, Required
max_state
The maximum permissible cooling state of this cooling device.
+
RO, Required
cur_state
The current cooling state of this cooling device.
The value can any integer numbers between 0 and max_state:
+
- cur_state == 0 means no cooling
- cur_state == max_state means the maximum cooling.
+
RW, Required
stats/reset
units here is 10mS (similar to other time exported in /proc).
RO, Required
+
stats/total_trans:
A single positive value showing the total number of times the state of a
cooling device is changed.
+
RO, Required
stats/trans_table:
RO, Required
3. A simple implementation
+==========================
ACPI thermal zone may support multiple trip points like critical, hot,
passive, active. If an ACPI thermal zone supports critical, passive,
effectiveness in cooling the thermal zone.
If the processor is listed in _PSL method, and the fan is listed in _AL0
-method, the sys I/F structure will be built like this:
+method, the sys I/F structure will be built like this::
-/sys/class/thermal:
-
-|thermal_zone1:
+ /sys/class/thermal:
+ |thermal_zone1:
|---type: acpitz
|---temp: 37000
|---mode: enabled
|---cdev1_trip_point: 2 /* cdev1 can be used for active[0]*/
|---cdev1_weight: 1024
-|cooling_device0:
+ |cooling_device0:
|---type: Processor
|---max_state: 8
|---cur_state: 0
-|cooling_device3:
+ |cooling_device3:
|---type: Fan
|---max_state: 2
|---cur_state: 0
-/sys/class/hwmon:
-
-|hwmon0:
+ /sys/class/hwmon:
+ |hwmon0:
|---name: acpitz
|---temp1_input: 37000
|---temp1_crit: 100000
4. Event Notification
+=====================
The framework includes a simple notification mechanism, in the form of a
netlink event. Netlink socket initialization is done during the _init_
THERMAL_DEV_FAULT}. Notification can be sent when the current temperature
crosses any of the configured thresholds.
-5. Export Symbol APIs:
+5. Export Symbol APIs
+=====================
+
+5.1. get_tz_trend
+-----------------
-5.1: get_tz_trend:
This function returns the trend of a thermal zone, i.e the rate of change
of temperature of the thermal zone. Ideally, the thermal sensor drivers
are supposed to implement the callback. If they don't, the thermal
framework calculated the trend by comparing the previous and the current
temperature values.
-5.2:get_thermal_instance:
+5.2. get_thermal_instance
+-------------------------
+
This function returns the thermal_instance corresponding to a given
{thermal_zone, cooling_device, trip_point} combination. Returns NULL
if such an instance does not exist.
-5.3:thermal_notify_framework:
+5.3. thermal_notify_framework
+-----------------------------
+
This function handles the trip events from sensor drivers. It starts
throttling the cooling devices according to the policy configured.
For CRITICAL and HOT trip points, this notifies the respective drivers,
The throttling policy is based on the configured platform data; if no
platform data is provided, this uses the step_wise throttling policy.
-5.4:thermal_cdev_update:
+5.4. thermal_cdev_update
+------------------------
+
This function serves as an arbitrator to set the state of a cooling
device. It sets the cooling device to the deepest cooling state if
possible.
-6. thermal_emergency_poweroff:
+6. thermal_emergency_poweroff
+=============================
On an event of critical trip temperature crossing. Thermal framework
allows the system to shutdown gracefully by calling orderly_poweroff().
+===================================
Kernel driver: x86_pkg_temp_thermal
-===================
+===================================
Supported chips:
+
* x86: with package level thermal management
+
(Verify using: CPUID.06H:EAX[bit 6] =1)
Authors: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>
Reference
----
+---------
+
Intel® 64 and IA-32 Architectures Software Developer’s Manual (Jan, 2013):
Chapter 14.6: PACKAGE LEVEL THERMAL MANAGEMENT
Description
----------
+-----------
This driver register CPU digital temperature package level sensor as a thermal
zone with maximum two user mode configurable trip points. Number of trip points
Threshold management
--------------------
Each package will register as a thermal zone under /sys/class/thermal.
-Example:
-/sys/class/thermal/thermal_zone1
+
+Example::
+
+ /sys/class/thermal/thermal_zone1
This contains two trip points:
+
- trip_point_0_temp
- trip_point_1_temp
User can set any temperature between 0 to TJ-Max temperature. Temperature units
-are in milli-degree Celsius. Refer to "Documentation/thermal/sysfs-api.txt" for
+are in milli-degree Celsius. Refer to "Documentation/thermal/sysfs-api.rst" for
thermal sys-fs details.
Any value other than 0 in these trip points, can trigger thermal notifications.
Setting 0, stops sending thermal notifications.
-Thermal notifications: To get kobject-uevent notifications, set the thermal zone
-policy to "user_space". For example: echo -n "user_space" > policy
-
-
+Thermal notifications:
+To get kobject-uevent notifications, set the thermal zone
+policy to "user_space".
+For example::
+ echo -n "user_space" > policy
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
======
timers
$argN : Fetch the Nth function argument. (N >= 1) (\*1)
$retval : Fetch return value.(\*2)
$comm : Fetch current task comm.
- +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(\*3)
+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4)
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
(u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
- (x8/x16/x32/x64), "string" and bitfield are supported.
+ (x8/x16/x32/x64), "string", "ustring" and bitfield
+ are supported.
(\*1) only for the probe on function entry (offs == 0).
(\*2) only for return probe.
(\*3) this is useful for fetching a field of data structures.
+ (\*4) "u" means user-space dereference. See :ref:`user_mem_access`.
Types
-----
wrong, but '+8($stack):x8[8]' is OK.)
String type is a special type, which fetches a "null-terminated" string from
kernel space. This means it will fail and store NULL if the string container
-has been paged out.
+has been paged out. "ustring" type is an alternative of string for user-space.
+See :ref:`user_mem_access` for more info..
The string array type is a bit different from other types. For other base
types, <base-type>[1] is equal to <base-type> (e.g. +0(%di):x32[1] is same
as +0(%di):x32.) But string[1] is not equal to string. The string type itself
which shows given pointer in "symbol+offset" style.
For $comm, the default type is "string"; any other type is invalid.
+.. _user_mem_access:
+User Memory Access
+------------------
+Kprobe events supports user-space memory access. For that purpose, you can use
+either user-space dereference syntax or 'ustring' type.
+
+The user-space dereference syntax allows you to access a field of a data
+structure in user-space. This is done by adding the "u" prefix to the
+dereference syntax. For example, +u4(%si) means it will read memory from the
+address in the register %si offset by 4, and the memory is expected to be in
+user-space. You can use this for strings too, e.g. +u0(%si):string will read
+a string from the address in the register %si that is expected to be in user-
+space. 'ustring' is a shortcut way of performing the same task. That is,
++0(%si):ustring is equivalent to +u0(%si):string.
+
+Note that kprobe-event provides the user-memory access syntax but it doesn't
+use it transparently. This means if you use normal dereference or string type
+for user memory, it might fail, and may always fail on some archs. The user
+has to carefully check if the target data is in kernel or user space.
Per-Probe Event Filtering
-------------------------
The first column is event name, the second is the number of probe hits,
the third is the number of probe miss-hits.
+Kernel Boot Parameter
+---------------------
+You can add and enable new kprobe events when booting up the kernel by
+"kprobe_event=" parameter. The parameter accepts a semicolon-delimited
+kprobe events, which format is similar to the kprobe_events.
+The difference is that the probe definition parameters are comma-delimited
+instead of space. For example, adding myprobe event on do_sys_open like below
+
+ p:myprobe do_sys_open dfd=%ax filename=%dx flags=%cx mode=+4($stack)
+
+should be below for kernel boot parameter (just replace spaces with comma)
+
+ p:myprobe,do_sys_open,dfd=%ax,filename=%dx,flags=%cx,mode=+4($stack)
+
Usage examples
--------------
@+OFFSET : Fetch memory at OFFSET (OFFSET from same file as PATH)
$stackN : Fetch Nth entry of stack (N >= 0)
$stack : Fetch stack address.
- $retval : Fetch return value.(*)
+ $retval : Fetch return value.(\*1)
$comm : Fetch current task comm.
- +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(**)
+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*2)(\*3)
NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
(u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
(x8/x16/x32/x64), "string" and bitfield are supported.
- (*) only for return probe.
- (**) this is useful for fetching a field of data structures.
+ (\*1) only for return probe.
+ (\*2) this is useful for fetching a field of data structures.
+ (\*3) Unlike kprobe event, "u" prefix will just be ignored, becuse uprobe
+ events can access only user-space memory.
Types
-----
Approfondimenti
===============
-- ``Documentation/locking/spinlocks.txt``: la guida di Linus Torvalds agli
+- ``Documentation/locking/spinlocks.rst``: la guida di Linus Torvalds agli
spinlock del kernel.
- Unix Systems for Modern Architectures: Symmetric Multiprocessing and
sorgenti che ne spieghi la logica: cosa fanno e perché.
25) Se la patch aggiunge nuove chiamate ioctl, allora aggiornate
- ``Documentation/ioctl/ioctl-number.txt``.
+ ``Documentation/ioctl/ioctl-number.rst``.
26) Se il codice che avete modificato dipende o usa una qualsiasi interfaccia o
funzionalità del kernel che è associata a uno dei seguenti simboli
-Chinese translated version of Documentation/arm/Booting
+Chinese translated version of Documentation/arm/booting.rst
If you have any comment or update to the content, please contact the
original document maintainer directly. However, if you have a problem
Maintainer: Russell King <linux@arm.linux.org.uk>
Chinese maintainer: Fu Wei <tekkamanninja@gmail.com>
---------------------------------------------------------------------
-Documentation/arm/Booting 的中文翻译
+Documentation/arm/booting.rst 的中文翻译
如果想评论或更新本文的内容，请直接联系原文档的维护者。如果你使用英文
交流有困难的话，也可以向中文版维护者求助。如果本翻译更新不及时或者翻
-Chinese translated version of Documentation/arm/kernel_user_helpers.txt
+Chinese translated version of Documentation/arm/kernel_user_helpers.rst
If you have any comment or update to the content, please contact the
original document maintainer directly. However, if you have a problem
Dave Martin <dave.martin@linaro.org>
Chinese maintainer: Fu Wei <tekkamanninja@gmail.com>
---------------------------------------------------------------------
-Documentation/arm/kernel_user_helpers.txt 的中文翻译
+Documentation/arm/kernel_user_helpers.rst 的中文翻译
如果想评论或更新本文的内容，请直接联系原文档的维护者。如果你使用英文
交流有困难的话，也可以向中文版维护者求助。如果本翻译更新不及时或者翻
中相应的设备。/sys/dev 提供一个通过一个 stat(2) 操作结果，查找
设备 sysfs 接口快捷的方法。
-更多有关 driver-model 的特性信息可以在 Documentation/driver-model/
+更多有关 driver-model 的特性信息可以在 Documentation/driver-api/driver-model/
中找到。
-Chinese translated version of Documentation/gpio
+Chinese translated version of Documentation/admin-guide/gpio
If you have any comment or update to the content, please contact the
original document maintainer directly. However, if you have a problem
Linus Walleij <linus.walleij@linaro.org>
Chinese maintainer: Fu Wei <tekkamanninja@gmail.com>
---------------------------------------------------------------------
-Documentation/gpio 的中文翻译
+Documentation/admin-guide/gpio 的中文翻译
如果想评论或更新本文的内容，请直接联系原文档的维护者。如果你使用英文
交流有困难的话，也可以向中文版维护者求助。如果本翻译更新不及时或者翻
（2）用串口终端启动（请参看Documentation/admin-guide/serial-console.rst），运行一个null
modem到另一台机器并用你喜欢的通讯工具获取输出。Minicom工作地很好。
-（3）使用Kdump（请参看Documentation/kdump/kdump.rst），
-使用在Documentation/kdump/gdbmacros.txt中定义的dmesg gdb宏，从旧的内存中提取内核
+（3）使用Kdump（请参看Documentation/admin-guide/kdump/kdump.rst），
+使用在Documentation/admin-guide/kdump/gdbmacros.txt中定义的dmesg gdb宏，从旧的内存中提取内核
环形缓冲区。
完整信息
24) 所有内存屏障例如 ``barrier()``, ``rmb()``, ``wmb()`` 都需要源代码中的注
释来解释它们正在执行的操作及其原因的逻辑。
-25) 如果补丁添加了任何ioctl，那么也要更新 ``Documentation/ioctl/ioctl-number.txt``
+25) 如果补丁添加了任何ioctl，那么也要更新 ``Documentation/ioctl/ioctl-number.rst``
26) 如果修改后的源代码依赖或使用与以下 ``Kconfig`` 符号相关的任何内核API或
功能，则在禁用相关 ``Kconfig`` 符号和/或 ``=m`` （如果该选项可用）的情况
-:orphan:
-
========================================================
OpenCAPI (Open Coherent Accelerator Processor Interface)
========================================================
seccomp_filter
unshare
spec_ctrl
+ accelerators/ocxl
.. only:: subproject and html
of function calls. The vmemmap_populate() implementation may use the
`vmem_altmap` along with :c:func:`altmap_alloc_block_buf` helper to
allocate memory map on the persistent memory device.
+
+ZONE_DEVICE
+===========
+The `ZONE_DEVICE` facility builds upon `SPARSEMEM_VMEMMAP` to offer
+`struct page` `mem_map` services for device driver identified physical
+address ranges. The "device" aspect of `ZONE_DEVICE` relates to the fact
+that the page objects for these address ranges are never marked online,
+and that a reference must be taken against the device, not just the page
+to keep the memory pinned for active use. `ZONE_DEVICE`, via
+:c:func:`devm_memremap_pages`, performs just enough memory hotplug to
+turn on :c:func:`pfn_to_page`, :c:func:`page_to_pfn`, and
+:c:func:`get_user_pages` service for the given range of pfns. Since the
+page reference count never drops below 1 the page is never tracked as
+free memory and the page's `struct list_head lru` space is repurposed
+for back referencing to the host device / driver that mapped the memory.
+
+While `SPARSEMEM` presents memory as a collection of sections,
+optionally collected into memory blocks, `ZONE_DEVICE` users have a need
+for smaller granularity of populating the `mem_map`. Given that
+`ZONE_DEVICE` memory is never marked online it is subsequently never
+subject to its memory ranges being exposed through the sysfs memory
+hotplug api on memory block boundaries. The implementation relies on
+this lack of user-api constraint to allow sub-section sized memory
+ranges to be specified to :c:func:`arch_add_memory`, the top-half of
+memory hotplug. Sub-section support allows for 2MB as the cross-arch
+common alignment granularity for :c:func:`devm_memremap_pages`.
+
+The users of `ZONE_DEVICE` are:
+
+* pmem: Map platform persistent memory to be used as a direct-I/O target
+ via DAX mappings.
+
+* hmm: Extend `ZONE_DEVICE` with `->page_fault()` and `->page_free()`
+ event callbacks to allow a device-driver to coordinate memory management
+ events related to device-memory, typically GPU memory. See
+ Documentation/vm/hmm.rst.
+
+* p2pdma: Create `struct page` objects to allow peer devices in a
+ PCI/-E topology to coordinate direct-DMA operations between themselves,
+ i.e. bypass host memory.
physical memory. NUMA emluation is useful for testing NUMA kernel and
application features on non-NUMA platforms, and as a sort of memory resource
management mechanism when used together with cpusets.
-[see Documentation/cgroup-v1/cpusets.rst]
+[see Documentation/admin-guide/cgroup-v1/cpusets.rst]
For each node with memory, Linux constructs an independent memory management
subsystem, complete with its own free page lists, in-use page lists, usage
System administrators can restrict the CPUs and nodes' memories that a non-
privileged user can specify in the scheduling or NUMA commands and functions
-using control groups and CPUsets. [see Documentation/cgroup-v1/cpusets.rst]
+using control groups and CPUsets. [see Documentation/admin-guide/cgroup-v1/cpusets.rst]
On architectures that do not hide memoryless nodes, Linux will include only
zones [nodes] with memory in the zonelists. This means that for a memoryless
Larger installations usually partition the system using cpusets into
sections of nodes. Paul Jackson has equipped cpusets with the ability to
move pages when a task is moved to another cpuset (See
-Documentation/cgroup-v1/cpusets.rst).
+Documentation/admin-guide/cgroup-v1/cpusets.rst).
Cpusets allows the automation of process locality. If a task is moved to
a new cpuset then also all its pages are moved with it so that the
performance of the process does not sink dramatically. Also the pages
--------------------------------
The unevictable LRU facility interacts with the memory control group [aka
-memory controller; see Documentation/cgroup-v1/memory.rst] by extending the
+memory controller; see Documentation/admin-guide/cgroup-v1/memory.rst] by extending the
lru_list enum.
The memory controller data structure automatically gets a per-zone unevictable
The unevictable LRU can be scanned for compactable regions and the default
behavior is to do so. /proc/sys/vm/compact_unevictable_allowed controls
-this behavior (see Documentation/sysctl/vm.txt). Once scanning of the
+this behavior (see Documentation/admin-guide/sysctl/vm.rst). Once scanning of the
unevictable LRU is enabled, the work of compaction is mostly handled by
the page migration code and the same work flow as described in MIGRATING
MLOCKED PAGES will apply.
Additional documantion, source code examples.
============================================
-1. Documentation/connector
+1. Documentation/driver-api/connector.rst
2. http://www.ioremap.net/archive/w1
This archive includes userspace application w1d.c which uses
read/write/search commands for all master/slave devices found on the bus.
Default value is set when compiling the kernel. If it is set
to "Y", then there is no way of disabling the watchdog once
it has been started.
+ kdumptimeout Minimum timeout in seconds to apply upon receipt of an NMI
+ before calling panic. (-1) disables the watchdog. When value
+ is > 0, the timer is reprogrammed with the greater of
+ value or current timeout value.
============ ================================================================
NOTE:
-:orphan:
+.. SPDX-License-Identifier: GPL-2.0
======================
Linux Watchdog Support
-------------------------------------------------
+watchdog core:
+ open_timeout:
+ Maximum time, in seconds, for which the watchdog framework will take
+ care of pinging a running hardware watchdog until userspace opens the
+ corresponding /dev/watchdogN device. A value of 0 means an infinite
+ timeout. Setting this to a non-zero value can be useful to ensure that
+ either userspace comes up properly, or the board gets reset and allows
+ fallback logic in the bootloader to try something else.
+
+-------------------------------------------------
+
acquirewdt:
wdt_stop:
Acquire WDT 'stop' io port (default 0x43)
mtrr
pat
intel_mpx
+ intel-iommu
+ intel_txt
amd-memory-encryption
pti
mds
respective code.
The architecture-agnostic topology definitions are in
-Documentation/cputopology.txt. This file holds x86-specific
+Documentation/admin-guide/cputopology.rst. This file holds x86-specific
differences/specialities which must not necessarily apply to the generic
definitions. Thus, the way to read up on Linux topology on x86 is to start
with the generic one and look at this one in parallel for the x86 specifics.
amount of system memory that are available to a certain class of tasks.
For more information on the features of cpusets, see
-Documentation/cgroup-v1/cpusets.rst.
+Documentation/admin-guide/cgroup-v1/cpusets.rst.
There are a number of different configurations you can use for your needs. For
more information on the numa=fake command line option and its various ways of
configuring fake nodes, see Documentation/x86/x86_64/boot-options.rst.
On node 3 totalpages: 131072
Now following the instructions for mounting the cpusets filesystem from
-Documentation/cgroup-v1/cpusets.rst, you can assign fake nodes (i.e. contiguous memory
+Documentation/admin-guide/cgroup-v1/cpusets.rst, you can assign fake nodes (i.e. contiguous memory
address spaces) to individual cpusets::
[root@xroads /]# mkdir exampleset
+===========================================
+Atomic Operation Control (ATOMCTL) Register
+===========================================
+
We Have Atomic Operation Control (ATOMCTL) Register.
This register determines the effect of using a S32C1I instruction
with various combinations of:
2. With and without An Intelligent Memory Controller which
can do Atomic Transactions itself.
-The Core comes up with a default value of for the three types of cache ops:
+The Core comes up with a default value of for the three types of cache ops::
0x28: (WB: Internal, WT: Internal, BY:Exception)
Developers might find using RCW in Bypass mode convenient when testing
with the cache being bypassed; for example studying cache alias problems.
-See Section 4.3.12.4 of ISA; Bits:
+See Section 4.3.12.4 of ISA; Bits::
WB WT BY
5 4 | 3 2 | 1 0
+
+========= ================== ================== ===============
2 Bit
Field
Values WB - Write Back WT - Write Thru BY - Bypass
---------- --------------- ----------------- ----------------
+========= ================== ================== ===============
0 Exception Exception Exception
1 RCW Transaction RCW Transaction RCW Transaction
2 Internal Operation Internal Operation Reserved
3 Reserved Reserved Reserved
+========= ================== ================== ===============
-Passing boot parameters to the kernel.
+=====================================
+Passing boot parameters to the kernel
+=====================================
Boot parameters are represented as a TLV list in the memory. Please see
arch/xtensa/include/asm/bootparam.h for definition of the bp_tag structure and
tag value constants. First entry in the list must have type BP_TAG_FIRST, last
entry must have type BP_TAG_LAST. The address of the first list entry is
passed to the kernel in the register a2. The address type depends on MMU type:
+
- For configurations without MMU, with region protection or with MPU the
address must be the physical address.
- For configurations with region translarion MMU or with MMUv3 and CONFIG_MMU=n
--- /dev/null
+.. SPDX-License-Identifier: GPL-2.0
+
+===================
+Xtensa Architecture
+===================
+
+.. toctree::
+ :maxdepth: 1
+
+ atomctl
+ booting
+ mmu
--- /dev/null
+=============================
+MMUv3 initialization sequence
+=============================
+
+The code in the initialize_mmu macro sets up MMUv3 memory mapping
+identically to MMUv2 fixed memory mapping. Depending on
+CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX symbol this code is
+located in addresses it was linked for (symbol undefined), or not
+(symbol defined), so it needs to be position-independent.
+
+The code has the following assumptions:
+
+ - This code fragment is run only on an MMU v3.
+ - TLBs are in their reset state.
+ - ITLBCFG and DTLBCFG are zero (reset state).
+ - RASID is 0x04030201 (reset state).
+ - PS.RING is zero (reset state).
+ - LITBASE is zero (reset state, PC-relative literals); required to be PIC.
+
+TLB setup proceeds along the following steps.
+
+ Legend:
+
+ - VA = virtual address (two upper nibbles of it);
+ - PA = physical address (two upper nibbles of it);
+ - pc = physical range that contains this code;
+
+After step 2, we jump to virtual address in the range 0x40000000..0x5fffffff
+or 0x00000000..0x1fffffff, depending on whether the kernel was loaded below
+0x40000000 or above. That address corresponds to next instruction to execute
+in this code. After step 4, we jump to intended (linked) address of this code.
+The scheme below assumes that the kernel is loaded below 0x40000000.
+
+ ====== ===== ===== ===== ===== ====== ===== =====
+ - Step0 Step1 Step2 Step3 Step4 Step5
+
+ VA PA PA PA PA VA PA PA
+ ====== ===== ===== ===== ===== ====== ===== =====
+ E0..FF -> E0 -> E0 -> E0 F0..FF -> F0 -> F0
+ C0..DF -> C0 -> C0 -> C0 E0..EF -> F0 -> F0
+ A0..BF -> A0 -> A0 -> A0 D8..DF -> 00 -> 00
+ 80..9F -> 80 -> 80 -> 80 D0..D7 -> 00 -> 00
+ 60..7F -> 60 -> 60 -> 60
+ 40..5F -> 40 -> pc -> pc 40..5F -> pc
+ 20..3F -> 20 -> 20 -> 20
+ 00..1F -> 00 -> 00 -> 00
+ ====== ===== ===== ===== ===== ====== ===== =====
+
+The default location of IO peripherals is above 0xf0000000. This may be changed
+using a "ranges" property in a device tree simple-bus node. See the Devicetree
+Specification, section 4.5 for details on the syntax and semantics of
+simple-bus nodes. The following limitations apply:
+
+1. Only top level simple-bus nodes are considered
+
+2. Only one (first) simple-bus node is considered
+
+3. Empty "ranges" properties are not supported
+
+4. Only the first triplet in the "ranges" property is considered
+
+5. The parent-bus-address value is rounded down to the nearest 256MB boundary
+
+6. The IO area covers the entire 256MB segment of parent-bus-address; the
+ "ranges" triplet length field is ignored
+
+
+MMUv3 address space layouts.
+============================
+
+Default MMUv2-compatible layout::
+
+ Symbol VADDR Size
+ +------------------+
+ | Userspace | 0x00000000 TASK_SIZE
+ +------------------+ 0x40000000
+ +------------------+
+ | Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
+ +------------------+
+ | KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
+ +------------------+ 0x8e400000
+ +------------------+
+ | VMALLOC area | VMALLOC_START 0xc0000000 128MB - 64KB
+ +------------------+ VMALLOC_END
+ | Cache aliasing | TLBTEMP_BASE_1 0xc7ff0000 DCACHE_WAY_SIZE
+ | remap area 1 |
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
+ | remap area 2 |
+ +------------------+
+ +------------------+
+ | KMAP area | PKMAP_BASE PTRS_PER_PTE *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ | | (4MB * DCACHE_N_COLORS)
+ +------------------+
+ | Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
+ | | NR_CPUS *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ +------------------+ FIXADDR_TOP 0xcffff000
+ +------------------+
+ | Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xd0000000 128MB
+ +------------------+
+ | Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xd8000000 128MB
+ +------------------+
+ | Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
+ +------------------+
+ | Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
+ +------------------+
+
+
+256MB cached + 256MB uncached layout::
+
+ Symbol VADDR Size
+ +------------------+
+ | Userspace | 0x00000000 TASK_SIZE
+ +------------------+ 0x40000000
+ +------------------+
+ | Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
+ +------------------+
+ | KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
+ +------------------+ 0x8e400000
+ +------------------+
+ | VMALLOC area | VMALLOC_START 0xa0000000 128MB - 64KB
+ +------------------+ VMALLOC_END
+ | Cache aliasing | TLBTEMP_BASE_1 0xa7ff0000 DCACHE_WAY_SIZE
+ | remap area 1 |
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
+ | remap area 2 |
+ +------------------+
+ +------------------+
+ | KMAP area | PKMAP_BASE PTRS_PER_PTE *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ | | (4MB * DCACHE_N_COLORS)
+ +------------------+
+ | Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
+ | | NR_CPUS *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ +------------------+ FIXADDR_TOP 0xaffff000
+ +------------------+
+ | Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xb0000000 256MB
+ +------------------+
+ | Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xc0000000 256MB
+ +------------------+
+ +------------------+
+ | Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
+ +------------------+
+ | Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
+ +------------------+
+
+
+512MB cached + 512MB uncached layout::
+
+ Symbol VADDR Size
+ +------------------+
+ | Userspace | 0x00000000 TASK_SIZE
+ +------------------+ 0x40000000
+ +------------------+
+ | Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
+ +------------------+
+ | KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
+ +------------------+ 0x8e400000
+ +------------------+
+ | VMALLOC area | VMALLOC_START 0x90000000 128MB - 64KB
+ +------------------+ VMALLOC_END
+ | Cache aliasing | TLBTEMP_BASE_1 0x97ff0000 DCACHE_WAY_SIZE
+ | remap area 1 |
+ +------------------+
+ | Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
+ | remap area 2 |
+ +------------------+
+ +------------------+
+ | KMAP area | PKMAP_BASE PTRS_PER_PTE *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ | | (4MB * DCACHE_N_COLORS)
+ +------------------+
+ | Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
+ | | NR_CPUS *
+ | | DCACHE_N_COLORS *
+ | | PAGE_SIZE
+ +------------------+ FIXADDR_TOP 0x9ffff000
+ +------------------+
+ | Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xa0000000 512MB
+ +------------------+
+ | Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xc0000000 512MB
+ +------------------+
+ | Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
+ +------------------+
+ | Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
+ +------------------+
+++ /dev/null
-MMUv3 initialization sequence.
-
-The code in the initialize_mmu macro sets up MMUv3 memory mapping
-identically to MMUv2 fixed memory mapping. Depending on
-CONFIG_INITIALIZE_XTENSA_MMU_INSIDE_VMLINUX symbol this code is
-located in addresses it was linked for (symbol undefined), or not
-(symbol defined), so it needs to be position-independent.
-
-The code has the following assumptions:
- This code fragment is run only on an MMU v3.
- TLBs are in their reset state.
- ITLBCFG and DTLBCFG are zero (reset state).
- RASID is 0x04030201 (reset state).
- PS.RING is zero (reset state).
- LITBASE is zero (reset state, PC-relative literals); required to be PIC.
-
-TLB setup proceeds along the following steps.
-
- Legend:
- VA = virtual address (two upper nibbles of it);
- PA = physical address (two upper nibbles of it);
- pc = physical range that contains this code;
-
-After step 2, we jump to virtual address in the range 0x40000000..0x5fffffff
-or 0x00000000..0x1fffffff, depending on whether the kernel was loaded below
-0x40000000 or above. That address corresponds to next instruction to execute
-in this code. After step 4, we jump to intended (linked) address of this code.
-The scheme below assumes that the kernel is loaded below 0x40000000.
-
- Step0 Step1 Step2 Step3 Step4 Step5
- ===== ===== ===== ===== ===== =====
- VA PA PA PA PA VA PA PA
- ------ -- -- -- -- ------ -- --
- E0..FF -> E0 -> E0 -> E0 F0..FF -> F0 -> F0
- C0..DF -> C0 -> C0 -> C0 E0..EF -> F0 -> F0
- A0..BF -> A0 -> A0 -> A0 D8..DF -> 00 -> 00
- 80..9F -> 80 -> 80 -> 80 D0..D7 -> 00 -> 00
- 60..7F -> 60 -> 60 -> 60
- 40..5F -> 40 -> pc -> pc 40..5F -> pc
- 20..3F -> 20 -> 20 -> 20
- 00..1F -> 00 -> 00 -> 00
-
-The default location of IO peripherals is above 0xf0000000. This may be changed
-using a "ranges" property in a device tree simple-bus node. See the Devicetree
-Specification, section 4.5 for details on the syntax and semantics of
-simple-bus nodes. The following limitations apply:
-
-1. Only top level simple-bus nodes are considered
-
-2. Only one (first) simple-bus node is considered
-
-3. Empty "ranges" properties are not supported
-
-4. Only the first triplet in the "ranges" property is considered
-
-5. The parent-bus-address value is rounded down to the nearest 256MB boundary
-
-6. The IO area covers the entire 256MB segment of parent-bus-address; the
- "ranges" triplet length field is ignored
-
-
-MMUv3 address space layouts.
-============================
-
-Default MMUv2-compatible layout.
-
- Symbol VADDR Size
-+------------------+
-| Userspace | 0x00000000 TASK_SIZE
-+------------------+ 0x40000000
-+------------------+
-| Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
-+------------------+
-| KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
-+------------------+ 0x8e400000
-+------------------+
-| VMALLOC area | VMALLOC_START 0xc0000000 128MB - 64KB
-+------------------+ VMALLOC_END
-| Cache aliasing | TLBTEMP_BASE_1 0xc7ff0000 DCACHE_WAY_SIZE
-| remap area 1 |
-+------------------+
-| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
-| remap area 2 |
-+------------------+
-+------------------+
-| KMAP area | PKMAP_BASE PTRS_PER_PTE *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-| | (4MB * DCACHE_N_COLORS)
-+------------------+
-| Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
-| | NR_CPUS *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-+------------------+ FIXADDR_TOP 0xcffff000
-+------------------+
-| Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xd0000000 128MB
-+------------------+
-| Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xd8000000 128MB
-+------------------+
-| Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
-+------------------+
-| Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
-+------------------+
-
-
-256MB cached + 256MB uncached layout.
-
- Symbol VADDR Size
-+------------------+
-| Userspace | 0x00000000 TASK_SIZE
-+------------------+ 0x40000000
-+------------------+
-| Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
-+------------------+
-| KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
-+------------------+ 0x8e400000
-+------------------+
-| VMALLOC area | VMALLOC_START 0xa0000000 128MB - 64KB
-+------------------+ VMALLOC_END
-| Cache aliasing | TLBTEMP_BASE_1 0xa7ff0000 DCACHE_WAY_SIZE
-| remap area 1 |
-+------------------+
-| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
-| remap area 2 |
-+------------------+
-+------------------+
-| KMAP area | PKMAP_BASE PTRS_PER_PTE *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-| | (4MB * DCACHE_N_COLORS)
-+------------------+
-| Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
-| | NR_CPUS *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-+------------------+ FIXADDR_TOP 0xaffff000
-+------------------+
-| Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xb0000000 256MB
-+------------------+
-| Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xc0000000 256MB
-+------------------+
-+------------------+
-| Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
-+------------------+
-| Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
-+------------------+
-
-
-512MB cached + 512MB uncached layout.
-
- Symbol VADDR Size
-+------------------+
-| Userspace | 0x00000000 TASK_SIZE
-+------------------+ 0x40000000
-+------------------+
-| Page table | XCHAL_PAGE_TABLE_VADDR 0x80000000 XCHAL_PAGE_TABLE_SIZE
-+------------------+
-| KASAN shadow map | KASAN_SHADOW_START 0x80400000 KASAN_SHADOW_SIZE
-+------------------+ 0x8e400000
-+------------------+
-| VMALLOC area | VMALLOC_START 0x90000000 128MB - 64KB
-+------------------+ VMALLOC_END
-| Cache aliasing | TLBTEMP_BASE_1 0x97ff0000 DCACHE_WAY_SIZE
-| remap area 1 |
-+------------------+
-| Cache aliasing | TLBTEMP_BASE_2 DCACHE_WAY_SIZE
-| remap area 2 |
-+------------------+
-+------------------+
-| KMAP area | PKMAP_BASE PTRS_PER_PTE *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-| | (4MB * DCACHE_N_COLORS)
-+------------------+
-| Atomic KMAP area | FIXADDR_START KM_TYPE_NR *
-| | NR_CPUS *
-| | DCACHE_N_COLORS *
-| | PAGE_SIZE
-+------------------+ FIXADDR_TOP 0x9ffff000
-+------------------+
-| Cached KSEG | XCHAL_KSEG_CACHED_VADDR 0xa0000000 512MB
-+------------------+
-| Uncached KSEG | XCHAL_KSEG_BYPASS_VADDR 0xc0000000 512MB
-+------------------+
-| Cached KIO | XCHAL_KIO_CACHED_VADDR 0xe0000000 256MB
-+------------------+
-| Uncached KIO | XCHAL_KIO_BYPASS_VADDR 0xf0000000 256MB
-+------------------+
M: Khuong Dinh <khuong@os.amperecomputing.com>
S: Supported
F: drivers/perf/xgene_pmu.c
-F: Documentation/perf/xgene-pmu.txt
+F: Documentation/admin-guide/perf/xgene-pmu.rst
F: Documentation/devicetree/bindings/perf/apm-xgene-pmu.txt
APTINA CAMERA SENSOR PLL
F: drivers/*/*s5pv210*
F: drivers/memory/samsung/*
F: drivers/soc/samsung/*
-F: Documentation/arm/Samsung/
+F: Documentation/arm/samsung/
F: Documentation/devicetree/bindings/arm/samsung/
F: Documentation/devicetree/bindings/sram/samsung-sram.txt
F: Documentation/devicetree/bindings/power/pd-samsung.txt
M: "Justin Sanders" <justin@coraid.com>
W: http://www.openaoe.org/
S: Supported
-F: Documentation/aoe/
+F: Documentation/admin-guide/aoe/
F: drivers/block/aoe/
ATHEROS 71XX/9XXX GPIO DRIVER
L: linux-block@vger.kernel.org
S: Maintained
F: block/bfq-*
-F: Documentation/block/bfq-iosched.txt
+F: Documentation/block/bfq-iosched.rst
BFS FILE SYSTEM
M: "Tigran A. Aivazian" <aivazian.tigran@gmail.com>
F: arch/riscv/net/
BPF JIT for S390
+M: Ilya Leoshkevich <iii@linux.ibm.com>
M: Heiko Carstens <heiko.carstens@de.ibm.com>
M: Vasily Gorbik <gor@linux.ibm.com>
-M: Christian Borntraeger <borntraeger@de.ibm.com>
L: netdev@vger.kernel.org
L: bpf@vger.kernel.org
S: Maintained
CEPH COMMON CODE (LIBCEPH)
M: Ilya Dryomov <idryomov@gmail.com>
-M: "Yan, Zheng" <zyan@redhat.com>
+M: Jeff Layton <jlayton@kernel.org>
M: Sage Weil <sage@redhat.com>
L: ceph-devel@vger.kernel.org
W: http://ceph.com/
F: include/linux/crush/
CEPH DISTRIBUTED FILE SYSTEM CLIENT (CEPH)
-M: "Yan, Zheng" <zyan@redhat.com>
+M: Jeff Layton <jlayton@kernel.org>
M: Sage Weil <sage@redhat.com>
M: Ilya Dryomov <idryomov@gmail.com>
L: ceph-devel@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git
S: Maintained
F: Documentation/admin-guide/cgroup-v2.rst
-F: Documentation/cgroup-v1/
+F: Documentation/admin-guide/cgroup-v1/
F: include/linux/cgroup*
F: kernel/cgroup/
W: http://oss.sgi.com/projects/cpusets/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git
S: Maintained
-F: Documentation/cgroup-v1/cpusets.rst
+F: Documentation/admin-guide/cgroup-v1/cpusets.rst
F: include/linux/cpuset.h
F: kernel/cgroup/cpuset.c
M: Stuart Hayes <stuart.w.hayes@gmail.com>
L: platform-driver-x86@vger.kernel.org
S: Maintained
-F: Documentation/dcdbas.txt
+F: Documentation/driver-api/dcdbas.rst
F: drivers/platform/x86/dcdbas.*
DELL WMI NOTIFICATIONS DRIVER
S: Supported
F: drivers/mtd/nand/raw/denali*
+DESIGNWARE EDMA CORE IP DRIVER
+M: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+L: dmaengine@vger.kernel.org
+S: Maintained
+F: drivers/dma/dw-edma/
+F: include/linux/dma/edma.h
+
DESIGNWARE USB2 DRD IP DRIVER
M: Minas Harutyunyan <hminas@synopsys.com>
L: linux-usb@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git
T: quilt http://people.redhat.com/agk/patches/linux/editing/
S: Maintained
-F: Documentation/device-mapper/
+F: Documentation/admin-guide/device-mapper/
F: drivers/md/Makefile
F: drivers/md/Kconfig
F: drivers/md/dm*
S: Supported
F: drivers/block/drbd/
F: lib/lru_cache.c
-F: Documentation/blockdev/drbd/
+F: Documentation/admin-guide/blockdev/
DRIVER CORE, KOBJECTS, DEBUGFS AND SYSFS
M: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
L: linux-efi@vger.kernel.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/efi/efi.git
S: Maintained
-F: Documentation/efi-stub.txt
+F: Documentation/admin-guide/efi-stub.rst
F: arch/*/kernel/efi.c
F: arch/x86/boot/compressed/eboot.[ch]
F: arch/*/include/asm/efi.h
F: drivers/counter/ftm-quaddec.c
FLOPPY DRIVER
-M: Jiri Kosina <jikos@kernel.org>
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/jikos/floppy.git
-S: Odd fixes
+S: Orphan
+L: linux-block@vger.kernel.org
F: drivers/block/floppy.c
FMC SUBSYSTEM
F: scripts/gcc-plugins/
F: scripts/gcc-plugin.sh
F: scripts/Makefile.gcc-plugins
-F: Documentation/gcc-plugins.txt
+F: Documentation/core-api/gcc-plugins.rst
GASKET DRIVER FRAMEWORK
M: Rob Springer <rspringer@google.com>
S: Maintained
F: Documentation/devicetree/bindings/gpio/
F: Documentation/driver-api/gpio/
-F: Documentation/gpio/
+F: Documentation/admin-guide/gpio/
F: Documentation/ABI/testing/gpio-cdev
F: Documentation/ABI/obsolete/sysfs-gpio
F: drivers/gpio/
L: linux-crypto@vger.kernel.org
S: Odd fixes
F: Documentation/devicetree/bindings/rng/
-F: Documentation/hw_random.txt
+F: Documentation/admin-guide/hw_random.rst
F: drivers/char/hw_random/
F: include/linux/hw_random.h
W: http://www.hisilicon.com
S: Supported
F: drivers/perf/hisilicon
-F: Documentation/perf/hisi-pmu.txt
+F: Documentation/admin-guide/perf/hisi-pmu.rst
HISILICON ROCE DRIVER
M: Lijun Ou <oulijun@huawei.com>
S: Maintained
F: drivers/mtd/nand/raw/ingenic/
+INGENIC JZ47xx SoCs
+M: Paul Cercueil <paul@crapouillou.net>
+S: Maintained
+F: arch/mips/boot/dts/ingenic/
+F: arch/mips/include/asm/mach-jz4740/
+F: arch/mips/jz4740/
+F: drivers/clk/ingenic/
+F: drivers/dma/dma-jz4780.c
+F: drivers/gpu/drm/ingenic/
+F: drivers/i2c/busses/i2c-jz4780.c
+F: drivers/iio/adc/ingenic-adc.c
+F: drivers/irqchip/irq-ingenic.c
+F: drivers/memory/jz4780-nemc.c
+F: drivers/mmc/host/jz4740_mmc.c
+F: drivers/mtd/nand/raw/ingenic/
+F: drivers/pinctrl/pinctrl-ingenic.c
+F: drivers/power/supply/ingenic-battery.c
+F: drivers/pwm/pwm-jz4740.c
+F: drivers/rtc/rtc-jz4740.c
+F: drivers/tty/serial/8250/8250_ingenic.c
+F: drivers/usb/musb/jz4740.c
+F: drivers/watchdog/jz4740_wdt.c
+F: include/dt-bindings/iio/adc/ingenic,adc.h
+F: include/linux/mfd/ingenic-tcu.h
+F: sound/soc/jz4740/
+F: sound/soc/codecs/jz47*
+
INOTIFY
M: Jan Kara <jack@suse.cz>
R: Amir Goldstein <amir73il@gmail.com>
W: http://tboot.sourceforge.net
T: hg http://tboot.hg.sourceforge.net:8000/hgroot/tboot/tboot
S: Supported
-F: Documentation/intel_txt.txt
+F: Documentation/x86/intel_txt.rst
F: include/linux/tboot.h
F: arch/x86/kernel/tboot.c
M: Georgi Djakov <georgi.djakov@linaro.org>
L: linux-pm@vger.kernel.org
S: Maintained
-F: Documentation/interconnect/
+F: Documentation/driver-api/interconnect.rst
F: Documentation/devicetree/bindings/interconnect/
F: drivers/interconnect/
F: include/dt-bindings/interconnect/
T: git git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git
S: Supported
F: fs/iomap.c
+F: fs/iomap/
F: include/linux/iomap.h
IOMMU DRIVERS
ISA
M: William Breathitt Gray <vilhelm.gray@gmail.com>
S: Maintained
-F: Documentation/isa.txt
+F: Documentation/driver-api/isa.rst
F: drivers/base/isa.c
F: include/linux/isa.h
ISAPNP
M: Jaroslav Kysela <perex@perex.cz>
S: Maintained
-F: Documentation/isapnp.txt
+F: Documentation/driver-api/isapnp.rst
F: drivers/pnp/isapnp/
F: include/linux/isapnp.h
L: kexec@lists.infradead.org
W: http://lse.sourceforge.net/kdump/
S: Maintained
-F: Documentation/kdump/
+F: Documentation/admin-guide/kdump/
KEENE FM RADIO TRANSMITTER DRIVER
M: Hans Verkuil <hverkuil@xs4all.nl>
L: platform-driver-x86@vger.kernel.org
S: Maintained
F: Documentation/ABI/testing/sysfs-platform-lg-laptop
-F: Documentation/laptops/lg-laptop.rst
+F: Documentation/admin-guide/laptops/lg-laptop.rst
F: drivers/platform/x86/lg-laptop.c
LG2160 MEDIA DRIVER
L: linux-ntfs-dev@lists.sourceforge.net
W: http://www.linux-ntfs.org/content/view/19/37/
S: Maintained
-F: Documentation/ldm.txt
+F: Documentation/admin-guide/ldm.rst
F: block/partitions/ldm.*
LSILOGIC MPT FUSION DRIVERS (FC/SAS/SPI)
S: Maintained
F: drivers/mcb/
F: include/linux/mcb.h
-F: Documentation/men-chameleon-bus.txt
+F: Documentation/driver-api/men-chameleon-bus.rst
MEN F21BMC (Board Management Controller)
M: Andreas Werner <andreas.werner@men.de>
MOXA SMARTIO/INDUSTIO/INTELLIO SERIAL CARD
M: Jiri Slaby <jirislaby@gmail.com>
S: Maintained
-F: Documentation/serial/moxa-smartio.rst
+F: Documentation/driver-api/serial/moxa-smartio.rst
F: drivers/tty/mxser.*
MR800 AVERMEDIA USB FM RADIO DRIVER
S: Maintained
L: linux-block@vger.kernel.org
L: nbd@other.debian.org
-F: Documentation/blockdev/nbd.txt
+F: Documentation/admin-guide/blockdev/nbd.rst
F: drivers/block/nbd.c
F: include/trace/events/nbd.h
F: include/uapi/linux/nbd.h
F: drivers/misc/ocxl/
F: include/misc/ocxl*
F: include/uapi/misc/ocxl.h
-F: Documentation/accelerators/ocxl.rst
+F: Documentation/userspace-api/accelerators/ocxl.rst
OMAP AUDIO SUPPORT
M: Peter Ujfalusi <peter.ujfalusi@ti.com>
L: linux-fbdev@vger.kernel.org
S: Orphan
F: drivers/video/fbdev/omap2/
-F: Documentation/arm/OMAP/DSS
+F: Documentation/arm/omap/dss.rst
OMAP FRAMEBUFFER SUPPORT
L: linux-fbdev@vger.kernel.org
M: Willy Tarreau <willy@haproxy.com>
M: Ksenija Stanojevic <ksenija.stanojevic@gmail.com>
S: Odd Fixes
-F: Documentation/auxdisplay/lcd-panel-cgram.txt
+F: Documentation/admin-guide/lcd-panel-cgram.rst
F: drivers/auxdisplay/panel.c
PARALLEL PORT SUBSYSTEM
F: include/linux/parport*.h
F: drivers/char/ppdev.c
F: include/uapi/linux/ppdev.h
-F: Documentation/parport*.txt
+F: Documentation/driver-api/parport*.rst
PARAVIRT_OPS INTERFACE
M: Juergen Gross <jgross@suse.com>
M: Tim Waugh <tim@cyberelk.net>
L: linux-parport@lists.infradead.org (subscribers-only)
S: Maintained
-F: Documentation/blockdev/paride.txt
+F: Documentation/admin-guide/blockdev/paride.rst
F: drivers/block/paride/
PARISC ARCHITECTURE
M: Logan Gunthorpe <logang@deltatee.com>
L: linux-pci@vger.kernel.org
S: Maintained
-F: Documentation/switchtec.txt
+F: Documentation/driver-api/switchtec.rst
F: Documentation/ABI/testing/sysfs-class-switchtec
F: drivers/pci/switch/switchtec*
F: include/uapi/linux/switchtec_ioctl.h
F: arch/arm/mach-picoxcell/
F: drivers/crypto/picoxcell*
+PIDFD API
+M: Christian Brauner <christian@brauner.io>
+L: linux-kernel@vger.kernel.org
+S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git
+F: samples/pidfd/
+F: tools/testing/selftests/pidfd/
+K: (?i)pidfd
+K: (?i)clone3
+K: \b(clone_args|kernel_clone_args)\b
+
PIN CONTROL SUBSYSTEM
M: Linus Walleij <linus.walleij@linaro.org>
L: linux-gpio@vger.kernel.org
F: include/linux/pm.h
F: include/linux/pm_*
F: include/linux/powercap.h
+F: include/linux/intel_rapl.h
F: drivers/powercap/
F: kernel/configs/nopm.config
L: linux-pwm@vger.kernel.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/thierry.reding/linux-pwm.git
-F: Documentation/pwm.txt
+F: Documentation/driver-api/pwm.rst
F: Documentation/devicetree/bindings/pwm/
F: include/linux/pwm.h
F: drivers/pwm/
RAMDISK RAM BLOCK DEVICE DRIVER
M: Jens Axboe <axboe@kernel.dk>
S: Maintained
-F: Documentation/blockdev/ramdisk.txt
+F: Documentation/admin-guide/blockdev/ramdisk.rst
F: drivers/block/brd.c
RANCHU VIRTUAL BOARD FOR MIPS
T: git git://git.kernel.org/pub/scm/linux/kernel/git/abelloni/linux.git
S: Maintained
F: Documentation/devicetree/bindings/rtc/
-F: Documentation/rtc.txt
+F: Documentation/admin-guide/rtc.rst
F: drivers/rtc/
F: include/linux/rtc.h
F: include/uapi/linux/rtc.h
T: git git://git.kernel.org/pub/scm/linux/kernel/git/ohad/remoteproc.git
S: Maintained
F: Documentation/devicetree/bindings/remoteproc/
+F: Documentation/ABI/testing/sysfs-class-remoteproc
F: Documentation/remoteproc.txt
F: drivers/remoteproc/
F: include/linux/remoteproc.h
+F: include/linux/remoteproc/
REMOTE PROCESSOR MESSAGING (RPMSG) SUBSYSTEM
M: Ohad Ben-Cohen <ohad@wizery.com>
S: Maintained
F: drivers/rpmsg/
F: Documentation/rpmsg.txt
+F: Documentation/ABI/testing/sysfs-bus-rpmsg
F: include/linux/rpmsg.h
F: include/linux/rpmsg/
+F: include/uapi/linux/rpmsg.h
+F: samples/rpmsg/
RENESAS CLOCK DRIVERS
M: Geert Uytterhoeven <geert+renesas@glider.be>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/jberg/mac80211-next.git
S: Maintained
-F: Documentation/rfkill.txt
+F: Documentation/driver-api/rfkill.rst
F: Documentation/ABI/stable/sysfs-class-rfkill
F: net/rfkill/
F: include/linux/rfkill.h
M: Palmer Dabbelt <palmer@sifive.com>
M: Albert Ou <aou@eecs.berkeley.edu>
L: linux-riscv@lists.infradead.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/palmer/riscv-linux.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git
S: Supported
F: arch/riscv/
K: riscv
P: Comtrol Corp.
W: http://www.comtrol.com
S: Maintained
-F: Documentation/serial/rocket.rst
+F: Documentation/driver-api/serial/rocket.rst
F: drivers/tty/rocket*
ROCKETPORT EXPRESS/INFINITY DRIVER
L: linux-kernel@vger.kernel.org
S: Supported
F: Documentation/devicetree/bindings/phy/samsung-phy.txt
-F: Documentation/phy/samsung-usb2.txt
+F: Documentation/driver-api/phy/samsung-usb2.rst
F: drivers/phy/samsung/phy-exynos4210-usb2.c
F: drivers/phy/samsung/phy-exynos4x12-usb2.c
F: drivers/phy/samsung/phy-exynos5250-usb2.c
M: Pat Gefre <pfg@sgi.com>
L: linux-ia64@vger.kernel.org
S: Supported
-F: Documentation/ia64/serial.txt
+F: Documentation/ia64/serial.rst
F: drivers/tty/serial/ioc?_serial.c
F: include/linux/ioc?.h
L: linux-riscv@lists.infradead.org
T: git git://github.com/sifive/riscv-linux.git
S: Supported
-K: sifive
+K: [^@]sifive
N: sifive
SIFIVE FU540 SYSTEM-ON-CHIP
SOCIONEXT (SNI) NETSEC NETWORK DRIVER
M: Jassi Brar <jaswinder.singh@linaro.org>
+M: Ilias Apalodimas <ilias.apalodimas@linaro.org>
L: netdev@vger.kernel.org
S: Maintained
F: drivers/net/ethernet/socionext/netsec.c
L: platform-driver-x86@vger.kernel.org
W: http://www.linux.it/~malattia/wiki/index.php/Sony_drivers
S: Maintained
-F: Documentation/laptops/sony-laptop.txt
+F: Documentation/admin-guide/laptops/sony-laptop.rst
F: drivers/char/sonypi.c
F: drivers/platform/x86/sony-laptop.c
F: include/linux/sony-laptop.h
M: Martin Mares <mj@ucw.cz>
L: linux-video@atrey.karlin.mff.cuni.cz
S: Maintained
-F: Documentation/svga.txt
+F: Documentation/admin-guide/svga.rst
F: arch/x86/boot/video*
SWIOTLB SUBSYSTEM
F: drivers/dma-buf/sw_sync.c
F: include/linux/sync_file.h
F: include/uapi/linux/sync_file.h
-F: Documentation/sync_file.txt
+F: Documentation/driver-api/sync_file.rst
T: git git://anongit.freedesktop.org/drm/drm-misc
SYNOPSYS ARC ARCHITECTURE
M: Javi Merino <javi.merino@kernel.org>
L: linux-pm@vger.kernel.org
S: Supported
-F: Documentation/thermal/cpu-cooling-api.txt
+F: Documentation/thermal/cpu-cooling-api.rst
F: drivers/thermal/cpu_cooling.c
F: include/linux/cpu_cooling.h
TI LP855x BACKLIGHT DRIVER
M: Milo Kim <milo.kim@ti.com>
S: Maintained
-F: Documentation/backlight/lp855x-driver.txt
+F: Documentation/driver-api/backlight/lp855x-driver.rst
F: drivers/video/backlight/lp855x_bl.c
F: include/linux/platform_data/lp855x.h
M: Jiri Slaby <jslaby@suse.com>
S: Supported
T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty.git
-F: Documentation/serial/
+F: Documentation/driver-api/serial/
F: drivers/tty/
F: drivers/tty/serial/serial_core.c
F: include/linux/serial_core.h
L: kvm@vger.kernel.org
T: git git://github.com/awilliam/linux-vfio.git
S: Maintained
-F: Documentation/vfio.txt
+F: Documentation/driver-api/vfio.rst
F: drivers/vfio/
F: include/linux/vfio.h
F: include/uapi/linux/vfio.h
M: Kirti Wankhede <kwankhede@nvidia.com>
L: kvm@vger.kernel.org
S: Maintained
-F: Documentation/vfio-mediated-device.txt
+F: Documentation/driver-api/vfio-mediated-device.rst
F: drivers/vfio/mdev/
F: include/linux/mdev.h
F: samples/vfio-mdev/
F: drivers/virtio/virtio_input.c
F: include/uapi/linux/virtio_input.h
+VIRTIO IOMMU DRIVER
+M: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
+L: virtualization@lists.linux-foundation.org
+S: Maintained
+F: drivers/iommu/virtio-iommu.c
+F: include/uapi/linux/virtio_iommu.h
+
VIRTUAL BOX GUEST DEVICE DRIVER
M: Hans de Goede <hdegoede@redhat.com>
M: Arnd Bergmann <arnd@arndb.de>
W: http://xfs.org/
T: git git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git
S: Supported
-F: Documentation/filesystems/xfs.txt
+F: Documentation/admin-guide/xfs.rst
F: Documentation/ABI/testing/sysfs-fs-xfs
-F: Documentation/filesystems/xfs.txt
F: Documentation/filesystems/xfs-delayed-logging-design.txt
F: Documentation/filesystems/xfs-self-describing-metadata.txt
F: fs/xfs/
L: linux-kernel@vger.kernel.org
S: Maintained
F: drivers/block/zram/
-F: Documentation/blockdev/zram.txt
+F: Documentation/admin-guide/blockdev/zram.rst
ZS DECSTATION Z85C30 SERIAL DRIVER
M: "Maciej W. Rozycki" <macro@linux-mips.org>
managed by the kernel and kept transparent to the probed
application. )
-config HAVE_64BIT_ALIGNED_ACCESS
- def_bool 64BIT && !HAVE_EFFICIENT_UNALIGNED_ACCESS
- help
- Some architectures require 64 bit accesses to be 64 bit
- aligned, which also requires structs containing 64 bit values
- to be 64 bit aligned too. This includes some 32 bit
- architectures which can do 64 bit accesses, as well as 64 bit
- architectures without unaligned access.
-
- This symbol should be selected by an architecture if 64 bit
- accesses are required to be 64 bit aligned in this way even
- though it is not a 64 bit architecture.
-
- See Documentation/unaligned-memory-access.txt for more
- information on the topic of unaligned memory accesses.
-
config HAVE_EFFICIENT_UNALIGNED_ACCESS
bool
help
config HAVE_ARCH_HUGE_VMAP
bool
+config ARCH_WANT_HUGE_PMD_SHARE
+ bool
+
config HAVE_ARCH_SOFT_DIRTY
bool
#define page_to_phys(page) page_to_pa(page)
-static inline dma_addr_t __deprecated isa_page_to_bus(struct page *page)
-{
- return page_to_phys(page);
-}
-
/* Maximum PIO space address supported? */
#define IO_SPACE_LIMIT 0xffff
542 common fsmount sys_fsmount
543 common fspick sys_fspick
544 common pidfd_open sys_pidfd_open
+# 545 reserved for clone3
#interrupt-cells = <1>;
interrupts = <20>;
};
+
+ virtio0: virtio@f0100000 {
+ compatible = "virtio,mmio";
+ reg = <0xf0100000 0x2000>;
+ interrupts = <31>;
+ };
+
+ virtio1: virtio@f0102000 {
+ compatible = "virtio,mmio";
+ reg = <0xf0102000 0x2000>;
+ interrupts = <32>;
+ };
+
+ virtio2: virtio@f0104000 {
+ compatible = "virtio,mmio";
+ reg = <0xf0104000 0x2000>;
+ interrupts = <33>;
+ };
+
+ virtio3: virtio@f0106000 {
+ compatible = "virtio,mmio";
+ reg = <0xf0106000 0x2000>;
+ interrupts = <34>;
+ };
+
+ virtio4: virtio@f0108000 {
+ compatible = "virtio,mmio";
+ reg = <0xf0108000 0x2000>;
+ interrupts = <35>;
+ };
};
};
*/
/dts-v1/;
+#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/reset/snps,hsdk-reset.h>
/ {
dma-coherent;
};
+ spi0: spi@20000 {
+ compatible = "snps,dw-apb-ssi";
+ reg = <0x20000 0x100>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ interrupts = <16>;
+ num-cs = <2>;
+ reg-io-width = <4>;
+ clocks = <&input_clk>;
+ cs-gpios = <&creg_gpio 0 GPIO_ACTIVE_LOW>,
+ <&creg_gpio 1 GPIO_ACTIVE_LOW>;
+ };
+
creg_gpio: gpio@14b0 {
compatible = "snps,creg-gpio-hsdk";
reg = <0x14b0 0x4>;
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
-# CONFIG_BLK_DEV is not set
+CONFIG_VIRTIO_BLK=y
CONFIG_NETDEVICES=y
+CONFIG_VIRTIO_NET=y
# CONFIG_NET_VENDOR_ARC is not set
# CONFIG_NET_VENDOR_BROADCOM is not set
# CONFIG_NET_VENDOR_INTEL is not set
CONFIG_LOGO=y
# CONFIG_HID is not set
# CONFIG_USB_SUPPORT is not set
+CONFIG_VIRTIO_MMIO=y
# CONFIG_IOMMU_SUPPORT is not set
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_SERIAL_8250_DW=y
CONFIG_SERIAL_OF_PLATFORM=y
# CONFIG_HW_RANDOM is not set
+CONFIG_SPI=y
+CONFIG_SPI_DESIGNWARE=y
+CONFIG_SPI_DW_MMIO=y
CONFIG_GPIOLIB=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_DWAPB=y
CONFIG_MMC_SDHCI=y
CONFIG_MMC_SDHCI_PLTFM=y
CONFIG_MMC_DW=y
+CONFIG_DMADEVICES=y
+CONFIG_DW_AXI_DMAC=y
CONFIG_EXT3_FS=y
CONFIG_VFAT_FS=y
CONFIG_TMPFS=y
#include <asm/irqflags-arcv2.h>
#include <asm/thread_info.h> /* For THREAD_SIZE */
+/*
+ * Interrupt/Exception stack layout (pt_regs) for ARCv2
+ * (End of struct aligned to end of page [unless nested])
+ *
+ * INTERRUPT EXCEPTION
+ *
+ * manual --------------------- manual
+ * | orig_r0 |
+ * | event/ECR |
+ * | bta |
+ * | user_r25 |
+ * | gp |
+ * | fp |
+ * | sp |
+ * | r12 |
+ * | r30 |
+ * | r58 |
+ * | r59 |
+ * hw autosave ---------------------
+ * optional | r0 |
+ * | r1 |
+ * ~ ~
+ * | r9 |
+ * | r10 |
+ * | r11 |
+ * | blink |
+ * | lpe |
+ * | lps |
+ * | lpc |
+ * | ei base |
+ * | ldi base |
+ * | jli base |
+ * ---------------------
+ * hw autosave | pc / eret |
+ * mandatory | stat32 / erstatus |
+ * ---------------------
+ */
+
/*------------------------------------------------------------------------*/
-.macro INTERRUPT_PROLOGUE called_from
+.macro INTERRUPT_PROLOGUE
- ; Before jumping to Interrupt Vector, hardware micro-ops did following:
+ ; (A) Before jumping to Interrupt Vector, hardware micro-ops did following:
; 1. SP auto-switched to kernel mode stack
- ; 2. STATUS32.Z flag set to U mode at time of interrupt (U:1, K:0)
- ; 3. Auto saved: r0-r11, blink, LPE,LPS,LPC, JLI,LDI,EI, PC, STAT32
+ ; 2. STATUS32.Z flag set if in U mode at time of interrupt (U:1,K:0)
+ ; 3. Auto save: (mandatory) Push PC and STAT32 on stack
+ ; hardware does even if CONFIG_ARC_IRQ_NO_AUTOSAVE
+ ; 4. Auto save: (optional) r0-r11, blink, LPE,LPS,LPC, JLI,LDI,EI
;
- ; Now manually save: r12, sp, fp, gp, r25
+ ; (B) Manually saved some regs: r12,r25,r30, sp,fp,gp, ACCL pair
#ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE
-.ifnc \called_from, exception
- st.as r9, [sp, -10] ; save r9 in it's final stack slot
- sub sp, sp, 12 ; skip JLI, LDI, EI
-
- PUSH lp_count
- PUSHAX lp_start
- PUSHAX lp_end
- PUSH blink
-
- PUSH r11
- PUSH r10
-
- sub sp, sp, 4 ; skip r9
-
- PUSH r8
- PUSH r7
- PUSH r6
- PUSH r5
- PUSH r4
- PUSH r3
- PUSH r2
- PUSH r1
- PUSH r0
-.endif
-#endif
+ ; carve pt_regs on stack (case #3), PC/STAT32 already on stack
+ sub sp, sp, SZ_PT_REGS - 8
-#ifdef CONFIG_ARC_HAS_ACCL_REGS
- PUSH r59
- PUSH r58
+ __SAVE_REGFILE_HARD
+#else
+ ; carve pt_regs on stack (case #4), which grew partially already
+ sub sp, sp, PT_r0
#endif
- PUSH r30
- PUSH r12
+ __SAVE_REGFILE_SOFT
+.endm
+
+/*------------------------------------------------------------------------*/
+.macro EXCEPTION_PROLOGUE
+
+ ; (A) Before jumping to Exception Vector, hardware micro-ops did following:
+ ; 1. SP auto-switched to kernel mode stack
+ ; 2. STATUS32.Z flag set if in U mode at time of exception (U:1,K:0)
+ ;
+ ; (B) Manually save the complete reg file below
+
+ sub sp, sp, SZ_PT_REGS ; carve pt_regs
+
+ ; _HARD saves r10 clobbered by _SOFT as scratch hence comes first
+
+ __SAVE_REGFILE_HARD
+ __SAVE_REGFILE_SOFT
+
+ st r0, [sp] ; orig_r0
+
+ lr r10, [eret]
+ lr r11, [erstatus]
+ ST2 r10, r11, PT_ret
+
+ lr r10, [ecr]
+ lr r11, [erbta]
+ ST2 r10, r11, PT_event
+
+ ; OUTPUT: r10 has ECR expected by EV_Trap
+.endm
+
+/*------------------------------------------------------------------------
+ * This macro saves the registers manually which would normally be autosaved
+ * by hardware on taken interrupts. It is used by
+ * - exception handlers (which don't have autosave)
+ * - interrupt autosave disabled due to CONFIG_ARC_IRQ_NO_AUTOSAVE
+ */
+.macro __SAVE_REGFILE_HARD
+
+ ST2 r0, r1, PT_r0
+ ST2 r2, r3, PT_r2
+ ST2 r4, r5, PT_r4
+ ST2 r6, r7, PT_r6
+ ST2 r8, r9, PT_r8
+ ST2 r10, r11, PT_r10
+
+ st blink, [sp, PT_blink]
+
+ lr r10, [lp_end]
+ lr r11, [lp_start]
+ ST2 r10, r11, PT_lpe
+
+ st lp_count, [sp, PT_lpc]
+
+ ; skip JLI, LDI, EI for now
+.endm
+
+/*------------------------------------------------------------------------
+ * This macros saves a bunch of other registers which can't be autosaved for
+ * various reasons:
+ * - r12: the last caller saved scratch reg since hardware saves in pairs so r0-r11
+ * - r30: free reg, used by gcc as scratch
+ * - ACCL/ACCH pair when they exist
+ */
+.macro __SAVE_REGFILE_SOFT
+
+ ST2 gp, fp, PT_r26 ; gp (r26), fp (r27)
+
+ st r12, [sp, PT_sp + 4]
+ st r30, [sp, PT_sp + 8]
; Saving pt_regs->sp correctly requires some extra work due to the way
; Auto stack switch works
; - U mode: retrieve it from AUX_USER_SP
; - K mode: add the offset from current SP where H/w starts auto push
;
- ; Utilize the fact that Z bit is set if Intr taken in U mode
- mov.nz r9, sp
- add.nz r9, r9, SZ_PT_REGS - PT_sp - 4
- bnz 1f
+ ; 1. Utilize the fact that Z bit is set if Intr taken in U mode
+ ; 2. Upon entry SP is always saved (for any inspection, unwinding etc),
+ ; but on return, restored only if U mode
- lr r9, [AUX_USER_SP]
-1:
- PUSH r9 ; SP
+ lr r10, [AUX_USER_SP] ; U mode SP
+
+ ; ISA requires ADD.nz to have same dest and src reg operands
+ mov.nz r10, sp
+ add.nz r10, r10, SZ_PT_REGS ; K mode SP
- PUSH fp
- PUSH gp
+ st r10, [sp, PT_sp] ; SP (pt_regs->sp)
#ifdef CONFIG_ARC_CURR_IN_REG
- PUSH r25 ; user_r25
+ st r25, [sp, PT_user_r25]
GET_CURR_TASK_ON_CPU r25
-#else
- sub sp, sp, 4
#endif
-.ifnc \called_from, exception
- sub sp, sp, 12 ; BTA/ECR/orig_r0 placeholder per pt_regs
-.endif
+#ifdef CONFIG_ARC_HAS_ACCL_REGS
+ ST2 r58, r59, PT_sp + 12
+#endif
.endm
/*------------------------------------------------------------------------*/
-.macro INTERRUPT_EPILOGUE called_from
+.macro __RESTORE_REGFILE_SOFT
-.ifnc \called_from, exception
- add sp, sp, 12 ; skip BTA/ECR/orig_r0 placeholderss
-.endif
+ LD2 gp, fp, PT_r26 ; gp (r26), fp (r27)
-#ifdef CONFIG_ARC_CURR_IN_REG
- POP r25
-#else
- add sp, sp, 4
-#endif
+ ld r12, [sp, PT_sp + 4]
+ ld r30, [sp, PT_sp + 8]
- POP gp
- POP fp
-
- ; Don't touch AUX_USER_SP if returning to K mode (Z bit set)
- ; (Z bit set on K mode is inverse of INTERRUPT_PROLOGUE)
- add.z sp, sp, 4
+ ; Restore SP (into AUX_USER_SP) only if returning to U mode
+ ; - for K mode, it will be implicitly restored as stack is unwound
+ ; - Z flag set on K is inverse of what hardware does on interrupt entry
+ ; but that doesn't really matter
bz 1f
- POPAX AUX_USER_SP
+ ld r10, [sp, PT_sp] ; SP (pt_regs->sp)
+ sr r10, [AUX_USER_SP]
1:
- POP r12
- POP r30
-#ifdef CONFIG_ARC_HAS_ACCL_REGS
- POP r58
- POP r59
+#ifdef CONFIG_ARC_CURR_IN_REG
+ ld r25, [sp, PT_user_r25]
#endif
-#ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE
-.ifnc \called_from, exception
- POP r0
- POP r1
- POP r2
- POP r3
- POP r4
- POP r5
- POP r6
- POP r7
- POP r8
- POP r9
- POP r10
- POP r11
-
- POP blink
- POPAX lp_end
- POPAX lp_start
-
- POP r9
- mov lp_count, r9
-
- add sp, sp, 12 ; skip JLI, LDI, EI
- ld.as r9, [sp, -10] ; reload r9 which got clobbered
-.endif
+#ifdef CONFIG_ARC_HAS_ACCL_REGS
+ LD2 r58, r59, PT_sp + 12
#endif
+.endm
+/*------------------------------------------------------------------------*/
+.macro __RESTORE_REGFILE_HARD
+
+ ld blink, [sp, PT_blink]
+
+ LD2 r10, r11, PT_lpe
+ sr r10, [lp_end]
+ sr r11, [lp_start]
+
+ ld r10, [sp, PT_lpc] ; lp_count can't be target of LD
+ mov lp_count, r10
+
+ LD2 r0, r1, PT_r0
+ LD2 r2, r3, PT_r2
+ LD2 r4, r5, PT_r4
+ LD2 r6, r7, PT_r6
+ LD2 r8, r9, PT_r8
+ LD2 r10, r11, PT_r10
.endm
+
/*------------------------------------------------------------------------*/
-.macro EXCEPTION_PROLOGUE
+.macro INTERRUPT_EPILOGUE
- ; Before jumping to Exception Vector, hardware micro-ops did following:
- ; 1. SP auto-switched to kernel mode stack
- ; 2. STATUS32.Z flag set to U mode at time of interrupt (U:1,K:0)
- ;
- ; Now manually save the complete reg file
-
- PUSH r9 ; freeup a register: slot of erstatus
-
- PUSHAX eret
- sub sp, sp, 12 ; skip JLI, LDI, EI
- PUSH lp_count
- PUSHAX lp_start
- PUSHAX lp_end
- PUSH blink
-
- PUSH r11
- PUSH r10
-
- ld.as r9, [sp, 10] ; load stashed r9 (status32 stack slot)
- lr r10, [erstatus]
- st.as r10, [sp, 10] ; save status32 at it's right stack slot
-
- PUSH r9
- PUSH r8
- PUSH r7
- PUSH r6
- PUSH r5
- PUSH r4
- PUSH r3
- PUSH r2
- PUSH r1
- PUSH r0
-
- ; -- for interrupts, regs above are auto-saved by h/w in that order --
- ; Now do what ISR prologue does (manually save r12, sp, fp, gp, r25)
- ;
- ; Set Z flag if this was from U mode (expected by INTERRUPT_PROLOGUE)
- ; Although H/w exception micro-ops do set Z flag for U mode (just like
- ; for interrupts), it could get clobbered in case we soft land here from
- ; a TLB Miss exception handler (tlbex.S)
+ ; INPUT: r0 has STAT32 of calling context
+ ; INPUT: Z flag set if returning to K mode
- and r10, r10, STATUS_U_MASK
- xor.f 0, r10, STATUS_U_MASK
+ ; _SOFT clobbers r10 restored by _HARD hence the order
- INTERRUPT_PROLOGUE exception
+ __RESTORE_REGFILE_SOFT
- PUSHAX erbta
- PUSHAX ecr ; r9 contains ECR, expected by EV_Trap
+#ifdef CONFIG_ARC_IRQ_NO_AUTOSAVE
+ __RESTORE_REGFILE_HARD
+ add sp, sp, SZ_PT_REGS - 8
+#else
+ add sp, sp, PT_r0
+#endif
- PUSH r0 ; orig_r0
.endm
/*------------------------------------------------------------------------*/
.macro EXCEPTION_EPILOGUE
- ; Assumes r0 has PT_status32
- btst r0, STATUS_U_BIT ; Z flag set if K, used in INTERRUPT_EPILOGUE
-
- add sp, sp, 8 ; orig_r0/ECR don't need restoring
- POPAX erbta
-
- INTERRUPT_EPILOGUE exception
+ ; INPUT: r0 has STAT32 of calling context
- POP r0
- POP r1
- POP r2
- POP r3
- POP r4
- POP r5
- POP r6
- POP r7
- POP r8
- POP r9
- POP r10
- POP r11
+ btst r0, STATUS_U_BIT ; Z flag set if K, used in restoring SP
- POP blink
- POPAX lp_end
- POPAX lp_start
+ ld r10, [sp, PT_event + 4]
+ sr r10, [erbta]
- POP r9
- mov lp_count, r9
+ LD2 r10, r11, PT_ret
+ sr r10, [eret]
+ sr r11, [erstatus]
- add sp, sp, 12 ; skip JLI, LDI, EI
- POPAX eret
- POPAX erstatus
+ __RESTORE_REGFILE_SOFT
+ __RESTORE_REGFILE_HARD
- ld.as r9, [sp, -12] ; reload r9 which got clobbered
+ add sp, sp, SZ_PT_REGS
.endm
.macro FAKE_RET_FROM_EXCPN
PUSHAX CTOP_AUX_EFLAGS
#endif
- lr r9, [ecr]
- st r9, [sp, PT_event] /* EV_Trap expects r9 to have ECR */
+ lr r10, [ecr]
+ st r10, [sp, PT_event] /* EV_Trap expects r10 to have ECR */
.endm
/*--------------------------------------------------------------
#ifdef __ASSEMBLY__
+.macro ST2 e, o, off
+#ifdef CONFIG_ARC_HAS_LL64
+ std \e, [sp, \off]
+#else
+ st \e, [sp, \off]
+ st \o, [sp, \off+4]
+#endif
+.endm
+
+.macro LD2 e, o, off
+#ifdef CONFIG_ARC_HAS_LL64
+ ldd \e, [sp, \off]
+#else
+ ld \e, [sp, \off]
+ ld \o, [sp, \off+4]
+#endif
+.endm
+
#define ASM_NL ` /* use '`' to mark new line in macro */
/* annotation for data we want in DCCM - if enabled in .config */
#ifndef _ASM_ARC_PGTABLE_H
#define _ASM_ARC_PGTABLE_H
-#include <linux/const.h>
+#include <linux/bits.h>
#define __ARCH_USE_5LEVEL_HACK
#include <asm-generic/pgtable-nopmd.h>
#include <asm/page.h>
#define BITS_FOR_PTE (PGDIR_SHIFT - PAGE_SHIFT)
#define BITS_FOR_PGD (32 - PGDIR_SHIFT)
-#define PGDIR_SIZE _BITUL(PGDIR_SHIFT) /* vaddr span, not PDG sz */
+#define PGDIR_SIZE BIT(PGDIR_SHIFT) /* vaddr span, not PDG sz */
#define PGDIR_MASK (~(PGDIR_SIZE-1))
-#define PTRS_PER_PTE _BITUL(BITS_FOR_PTE)
-#define PTRS_PER_PGD _BITUL(BITS_FOR_PGD)
+#define PTRS_PER_PTE BIT(BITS_FOR_PTE)
+#define PTRS_PER_PGD BIT(BITS_FOR_PGD)
/*
* Number of entries a user land program use.
DEFINE(PT_r5, offsetof(struct pt_regs, r5));
DEFINE(PT_r6, offsetof(struct pt_regs, r6));
DEFINE(PT_r7, offsetof(struct pt_regs, r7));
+ DEFINE(PT_r8, offsetof(struct pt_regs, r8));
+ DEFINE(PT_r10, offsetof(struct pt_regs, r10));
+ DEFINE(PT_r26, offsetof(struct pt_regs, r26));
DEFINE(PT_ret, offsetof(struct pt_regs, ret));
+ DEFINE(PT_blink, offsetof(struct pt_regs, blink));
+ DEFINE(PT_lpe, offsetof(struct pt_regs, lp_end));
+ DEFINE(PT_lpc, offsetof(struct pt_regs, lp_count));
+ DEFINE(PT_user_r25, offsetof(struct pt_regs, user_r25));
DEFINE(SZ_CALLEE_REGS, sizeof(struct callee_regs));
DEFINE(SZ_PT_REGS, sizeof(struct pt_regs));
ENTRY(handle_interrupt)
- INTERRUPT_PROLOGUE irq
+ INTERRUPT_PROLOGUE
# irq control APIs local_irq_save/restore/disable/enable fiddle with
# global interrupt enable bits in STATUS32 (.IE for 1 prio, .E[] for 2 prio)
#
# Note this disable is only for consistent book-keeping as further interrupts
# will be disabled anyways even w/o this. Hardware tracks active interrupts
- # seperately in AUX_IRQ_ACTIVE.active and will not take new interrupts
+ # seperately in AUX_IRQ_ACT.active and will not take new interrupts
# unless this one returns (or higher prio becomes pending in 2-prio scheme)
IRQ_DISABLE
ld r0, [sp, PT_status32] ; U/K mode at time of entry
lr r10, [AUX_IRQ_ACT]
- bmsk r11, r10, 15 ; AUX_IRQ_ACT.ACTIVE
+ bmsk r11, r10, 15 ; extract AUX_IRQ_ACT.active
breq r11, 0, .Lexcept_ret ; No intr active, ret from Exception
;####### Return from Intr #######
+.Lisr_ret:
+
debug_marker_l1:
; bbit1.nt r0, STATUS_DE_BIT, .Lintr_ret_to_delay_slot
btst r0, STATUS_DE_BIT ; Z flag set if bit clear
bnz .Lintr_ret_to_delay_slot ; branch if STATUS_DE_BIT set
-.Lisr_ret_fast_path:
; Handle special case #1: (Entry via Exception, Return via IRQ)
;
; Exception in U mode, preempted in kernel, Intr taken (K mode), orig
bset.nz r11, r11, AUX_IRQ_ACT_BIT_U ; NZ means U
sr r11, [AUX_IRQ_ACT]
- INTERRUPT_EPILOGUE irq
+ INTERRUPT_EPILOGUE
rtie
;####### Return from Exception / pure kernel mode #######
;
; IRQ RTIE won't reliably restore DE bit and/or BTA, needs workaround
;
-; Solution is return from Intr w/o any delay slot quirks into a kernel trampoline
-; and from pure kernel mode return to delay slot which handles DS bit/BTA correctly
+; Solution is to drop out of interrupt context into pure kernel mode
+; and return from pure kernel mode which does right things for delay slot
.Lintr_ret_to_delay_slot:
debug_marker_ds:
add r2, r2, 1
st r2, [@intr_to_DE_cnt]
- ld r2, [sp, PT_ret]
- ld r3, [sp, PT_status32]
-
- ; STAT32 for Int return created from scratch
- ; (No delay dlot, disable Further intr in trampoline)
-
- bic r0, r3, STATUS_U_MASK|STATUS_DE_MASK|STATUS_IE_MASK|STATUS_L_MASK
- st r0, [sp, PT_status32]
-
- mov r1, .Lintr_ret_to_delay_slot_2
- st r1, [sp, PT_ret]
-
- ; Orig exception PC/STAT32 safekept @orig_r0 and @event stack slots
- st r2, [sp, 0]
- st r3, [sp, 4]
-
- b .Lisr_ret_fast_path
-
-.Lintr_ret_to_delay_slot_2:
- ; Trampoline to restore orig exception PC/STAT32/BTA/AUX_USER_SP
- sub sp, sp, SZ_PT_REGS
- st r9, [sp, -4]
-
- ld r9, [sp, 0]
- sr r9, [eret]
-
- ld r9, [sp, 4]
- sr r9, [erstatus]
-
- ; restore AUX_USER_SP if returning to U mode
- bbit0 r9, STATUS_U_BIT, 1f
- ld r9, [sp, PT_sp]
- sr r9, [AUX_USER_SP]
-
-1:
- ld r9, [sp, 8]
- sr r9, [erbta]
-
- ld r9, [sp, -4]
- add sp, sp, SZ_PT_REGS
-
- ; return from pure kernel mode to delay slot
- rtie
+ ; drop out of interrupt context (clear AUX_IRQ_ACT.active)
+ bmskn r11, r10, 15
+ sr r11, [AUX_IRQ_ACT]
+ b .Lexcept_ret
END(ret_from_exception)
EXCEPTION_PROLOGUE
- mov r2, r9 ; ECR set into r9 already
+ mov r2, r10 ; ECR set into r10 already
lr r0, [efa] ; Faulting Data address (not part of pt_regs saved above)
; Exception auto-disables further Intr/exceptions.
EXCEPTION_PROLOGUE
;============ TRAP 1 :breakpoints
- ; Check ECR for trap with arg (PROLOGUE ensures r9 has ECR)
- bmsk.f 0, r9, 7
+ ; Check ECR for trap with arg (PROLOGUE ensures r10 has ECR)
+ bmsk.f 0, r10, 7
bnz trap_with_param
;============ TRAP (no param): syscall top level
return memblock_alloc_from(sz, sizeof(unsigned int), MAX_DMA_ADDRESS);
}
-static void *unw_hdr_alloc(unsigned long sz)
-{
- return kmalloc(sz, GFP_KERNEL);
-}
-
static void init_unwind_table(struct unwind_table *table, const char *name,
const void *core_start, unsigned long core_size,
const void *init_start, unsigned long init_size,
}
#ifdef CONFIG_MODULES
+static void *unw_hdr_alloc(unsigned long sz)
+{
+ return kmalloc(sz, GFP_KERNEL);
+}
static struct unwind_table *last_table;
struct vm_area_struct *vma = NULL;
struct task_struct *tsk = current;
struct mm_struct *mm = tsk->mm;
- int si_code = SEGV_MAPERR;
- int ret;
- vm_fault_t fault;
- int write = regs->ecr_cause & ECR_C_PROTV_STORE; /* ST/EX */
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ int sig, si_code = SEGV_MAPERR;
+ unsigned int write = 0, exec = 0, mask;
+ vm_fault_t fault = VM_FAULT_SIGSEGV; /* handle_mm_fault() output */
+ unsigned int flags; /* handle_mm_fault() input */
/*
- * We fault-in kernel-space virtual memory on-demand. The
- * 'reference' page table is init_mm.pgd.
- *
* NOTE! We MUST NOT take any locks for this case. We may
* be in an interrupt or a critical region, and should
* only copy the information from the master page table,
* nothing more.
*/
if (address >= VMALLOC_START && !user_mode(regs)) {
- ret = handle_kernel_vaddr_fault(address);
- if (unlikely(ret))
+ if (unlikely(handle_kernel_vaddr_fault(address)))
goto no_context;
else
return;
if (faulthandler_disabled() || !mm)
goto no_context;
+ if (regs->ecr_cause & ECR_C_PROTV_STORE) /* ST/EX */
+ write = 1;
+ else if ((regs->ecr_vec == ECR_V_PROTV) &&
+ (regs->ecr_cause == ECR_C_PROTV_INST_FETCH))
+ exec = 1;
+
+ flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
if (user_mode(regs))
flags |= FAULT_FLAG_USER;
+ if (write)
+ flags |= FAULT_FLAG_WRITE;
+
retry:
down_read(&mm->mmap_sem);
+
vma = find_vma(mm, address);
if (!vma)
goto bad_area;
- if (vma->vm_start <= address)
- goto good_area;
- if (!(vma->vm_flags & VM_GROWSDOWN))
- goto bad_area;
- if (expand_stack(vma, address))
- goto bad_area;
+ if (unlikely(address < vma->vm_start)) {
+ if (!(vma->vm_flags & VM_GROWSDOWN) || expand_stack(vma, address))
+ goto bad_area;
+ }
/*
- * Ok, we have a good vm_area for this memory access, so
- * we can handle it..
+ * vm_area is good, now check permissions for this memory access
*/
-good_area:
- si_code = SEGV_ACCERR;
-
- /* Handle protection violation, execute on heap or stack */
-
- if ((regs->ecr_vec == ECR_V_PROTV) &&
- (regs->ecr_cause == ECR_C_PROTV_INST_FETCH))
+ mask = VM_READ;
+ if (write)
+ mask = VM_WRITE;
+ if (exec)
+ mask = VM_EXEC;
+
+ if (!(vma->vm_flags & mask)) {
+ si_code = SEGV_ACCERR;
goto bad_area;
-
- if (write) {
- if (!(vma->vm_flags & VM_WRITE))
- goto bad_area;
- flags |= FAULT_FLAG_WRITE;
- } else {
- if (!(vma->vm_flags & (VM_READ | VM_EXEC)))
- goto bad_area;
}
- /*
- * If for any reason at all we couldn't handle the fault,
- * make sure we exit gracefully rather than endlessly redo
- * the fault.
- */
fault = handle_mm_fault(vma, address, flags);
- if (fatal_signal_pending(current)) {
+ /*
+ * Fault retry nuances
+ */
+ if (unlikely(fault & VM_FAULT_RETRY)) {
/*
- * if fault retry, mmap_sem already relinquished by core mm
- * so OK to return to user mode (with signal handled first)
+ * If fault needs to be retried, handle any pending signals
+ * first (by returning to user mode).
+ * mmap_sem already relinquished by core mm for RETRY case
*/
- if (fault & VM_FAULT_RETRY) {
+ if (fatal_signal_pending(current)) {
if (!user_mode(regs))
goto no_context;
return;
}
- }
-
- perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
-
- if (likely(!(fault & VM_FAULT_ERROR))) {
+ /*
+ * retry state machine
+ */
if (flags & FAULT_FLAG_ALLOW_RETRY) {
- /* To avoid updating stats twice for retry case */
- if (fault & VM_FAULT_MAJOR) {
- tsk->maj_flt++;
- perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
- regs, address);
- } else {
- tsk->min_flt++;
- perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
- regs, address);
- }
-
- if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
- flags |= FAULT_FLAG_TRIED;
- goto retry;
- }
+ flags &= ~FAULT_FLAG_ALLOW_RETRY;
+ flags |= FAULT_FLAG_TRIED;
+ goto retry;
}
-
- /* Fault Handled Gracefully */
- up_read(&mm->mmap_sem);
- return;
}
- if (fault & VM_FAULT_OOM)
- goto out_of_memory;
- else if (fault & VM_FAULT_SIGSEGV)
- goto bad_area;
- else if (fault & VM_FAULT_SIGBUS)
- goto do_sigbus;
-
- /* no man's land */
- BUG();
+bad_area:
+ up_read(&mm->mmap_sem);
/*
- * Something tried to access memory that isn't in our memory map..
- * Fix it, but check if it's kernel or user first..
+ * Major/minor page fault accounting
+ * (in case of retry we only land here once)
*/
-bad_area:
- up_read(&mm->mmap_sem);
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
- /* User mode accesses just cause a SIGSEGV */
- if (user_mode(regs)) {
- tsk->thread.fault_address = address;
- force_sig_fault(SIGSEGV, si_code, (void __user *)address);
- return;
- }
+ if (likely(!(fault & VM_FAULT_ERROR))) {
+ if (fault & VM_FAULT_MAJOR) {
+ tsk->maj_flt++;
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+ regs, address);
+ } else {
+ tsk->min_flt++;
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+ regs, address);
+ }
-no_context:
- /* Are we prepared to handle this kernel fault?
- *
- * (The kernel has valid exception-points in the source
- * when it accesses user-memory. When it fails in one
- * of those points, we find it in a table and do a jump
- * to some fixup code that loads an appropriate error
- * code)
- */
- if (fixup_exception(regs))
+ /* Normal return path: fault Handled Gracefully */
return;
+ }
- die("Oops", regs, address);
-
-out_of_memory:
- up_read(&mm->mmap_sem);
+ if (!user_mode(regs))
+ goto no_context;
- if (user_mode(regs)) {
+ if (fault & VM_FAULT_OOM) {
pagefault_out_of_memory();
return;
}
- goto no_context;
+ if (fault & VM_FAULT_SIGBUS) {
+ sig = SIGBUS;
+ si_code = BUS_ADRERR;
+ }
+ else {
+ sig = SIGSEGV;
+ }
-do_sigbus:
- up_read(&mm->mmap_sem);
+ tsk->thread.fault_address = address;
+ force_sig_fault(sig, si_code, (void __user *)address);
+ return;
- if (!user_mode(regs))
- goto no_context;
+no_context:
+ if (fixup_exception(regs))
+ return;
- tsk->thread.fault_address = address;
- force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *)address);
+ die("Oops", regs, address);
}
;-------- Common routine to call Linux Page Fault Handler -----------
do_slow_path_pf:
+#ifdef CONFIG_ISA_ARCV2
+ ; Set Z flag if exception in U mode. Hardware micro-ops do this on any
+ ; taken interrupt/exception, and thus is already the case at the entry
+ ; above, but ensuing code would have already clobbered.
+ ; EXCEPTION_PROLOGUE called in slow path, relies on correct Z flag set
+
+ lr r2, [erstatus]
+ and r2, r2, STATUS_U_MASK
+ bxor.f 0, r2, STATUS_U_BIT
+#endif
+
; Restore the 4-scratch regs saved by fast path miss handler
TLBMISS_RESTORE_REGS
#error "Incorrect ctop.h include"
#endif
+#include <linux/bits.h>
#include <linux/types.h>
#include <soc/nps/common.h>
#define CTOP_INST_AXOR_DI_R2_R2_R3 0x4A664C06
/* Do not use D$ for address in 2G-3G */
-#define HW_COMPLY_KRN_NOT_D_CACHED _BITUL(28)
+#define HW_COMPLY_KRN_NOT_D_CACHED BIT(28)
#define NPS_MSU_EN_CFG 0x80
#define NPS_CRG_BLKID 0x480
-#define NPS_CRG_SYNC_BIT _BITUL(0)
+#define NPS_CRG_SYNC_BIT BIT(0)
#define NPS_GIM_BLKID 0x5C0
/* GIM registers and fields*/
-#define NPS_GIM_UART_LINE _BITUL(7)
-#define NPS_GIM_DBG_LAN_EAST_TX_DONE_LINE _BITUL(10)
-#define NPS_GIM_DBG_LAN_EAST_RX_RDY_LINE _BITUL(11)
-#define NPS_GIM_DBG_LAN_WEST_TX_DONE_LINE _BITUL(25)
-#define NPS_GIM_DBG_LAN_WEST_RX_RDY_LINE _BITUL(26)
+#define NPS_GIM_UART_LINE BIT(7)
+#define NPS_GIM_DBG_LAN_EAST_TX_DONE_LINE BIT(10)
+#define NPS_GIM_DBG_LAN_EAST_RX_RDY_LINE BIT(11)
+#define NPS_GIM_DBG_LAN_WEST_TX_DONE_LINE BIT(25)
+#define NPS_GIM_DBG_LAN_WEST_RX_RDY_LINE BIT(26)
#ifndef __ASSEMBLY__
/* Functional registers definition */
will run faster if you say N here.
See also <file:Documentation/x86/i386/IO-APIC.rst>,
- <file:Documentation/lockup-watchdogs.txt> and the SMP-HOWTO available at
+ <file:Documentation/admin-guide/lockup-watchdogs.rst> and the SMP-HOWTO available at
<http://tldp.org/HOWTO/SMP-HOWTO.html>.
If you don't know what to do here, say N.
kdump/kexec. The crash dump kernel must be compiled to a
memory address not used by the main kernel
- For more details see Documentation/kdump/kdump.rst
+ For more details see Documentation/admin-guide/kdump/kdump.rst
config AUTO_ZRELADDR
bool "Auto calculation of the decompressed kernel image address"
Say Y to include VFP support code in the kernel. This is needed
if your hardware includes a VFP unit.
- Please see <file:Documentation/arm/VFP/release-notes.txt> for
+ Please see <file:Documentation/arm/vfp/release-notes.rst> for
release notes and additional status information.
Say N if your target does not have VFP hardware.
/*
* The public API for this code is documented in arch/arm/include/asm/mcpm.h.
* For a comprehensive description of the main algorithm used here, please
- * see Documentation/arm/cluster-pm-race-avoidance.txt.
+ * see Documentation/arm/cluster-pm-race-avoidance.rst.
*/
struct sync_struct mcpm_sync;
* Created by: Nicolas Pitre, March 2012
* Copyright: (C) 2012-2013 Linaro Limited
*
- * Refer to Documentation/arm/cluster-pm-race-avoidance.txt
+ * Refer to Documentation/arm/cluster-pm-race-avoidance.rst
* for details of the synchronisation algorithms used here.
*/
* Copyright: (C) 2012-2013 Linaro Limited
*
* This algorithm is described in more detail in
- * Documentation/arm/vlocks.txt.
+ * Documentation/arm/vlocks.rst.
*/
#include <linux/linkage.h>
* ISA I/O bus memory addresses are 1:1 with the physical address.
*/
#define isa_virt_to_bus virt_to_phys
-#define isa_page_to_bus page_to_phys
#define isa_bus_to_virt phys_to_virt
/*
* Copyright (C) 1997-1999 Russell King
*
* Structure passed to kernel to tell it about the
- * hardware it's running on. See Documentation/arm/Setup
+ * hardware it's running on. See Documentation/arm/setup.rst
* for more info.
*/
#ifndef __ASMARM_SETUP_H
* published by the Free Software Foundation.
*
* Structure passed to kernel to tell it about the
- * hardware it's running on. See Documentation/arm/Setup
+ * hardware it's running on. See Documentation/arm/setup.rst
* for more info.
*/
#ifndef _UAPI__ASMARM_SETUP_H
* existing ones. This mechanism should be used only for things that are
* really small and justified, and not be abused freely.
*
- * See Documentation/arm/kernel_user_helpers.txt for formal definitions.
+ * See Documentation/arm/kernel_user_helpers.rst for formal definitions.
*/
THUMB( .arm )
}
#endif
+bool module_exit_section(const char *name)
+{
+ return strstarts(name, ".exit") ||
+ strstarts(name, ".ARM.extab.exit") ||
+ strstarts(name, ".ARM.exidx.exit");
+}
+
int
apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, unsigned int symindex,
unsigned int relindex, struct module *module)
#define C2_STATE (1 << 3)
/*
* Magic values for bootloader indicating chosen low power mode.
- * See also Documentation/arm/Samsung/Bootloader-interface.txt
+ * See also Documentation/arm/samsung/bootloader-interface.rst
*/
#define EXYNOS_SLEEP_MAGIC 0x00000bad
#define EXYNOS_AFTR_MAGIC 0xfcba0d10
help
Say 'Y' here if you want your kernel to support the Gateworks
Avila Network Platform. For more information on this platform,
- see <file:Documentation/arm/IXP4xx>.
+ see <file:Documentation/arm/ixp4xx.rst>.
config MACH_LOFT
bool "Loft"
help
Say 'Y' here if you want your kernel to support the ADI
Engineering Coyote Gateway Reference Platform. For more
- information on this platform, see <file:Documentation/arm/IXP4xx>.
+ information on this platform, see <file:Documentation/arm/ixp4xx.rst>.
config MACH_GATEWAY7001
bool "Gateway 7001"
help
Say 'Y' here if you want your kernel to support Intel's
IXDP425 Development Platform (Also known as Richfield).
- For more information on this platform, see <file:Documentation/arm/IXP4xx>.
+ For more information on this platform, see <file:Documentation/arm/ixp4xx.rst>.
config MACH_IXDPG425
bool "IXDPG425"
help
Say 'Y' here if you want your kernel to support Intel's
IXDPG425 Development Platform (Also known as Montajade).
- For more information on this platform, see <file:Documentation/arm/IXP4xx>.
+ For more information on this platform, see <file:Documentation/arm/ixp4xx.rst>.
config MACH_IXDP465
bool "IXDP465"
help
Say 'Y' here if you want your kernel to support Intel's
IXDP465 Development Platform (Also known as BMP).
- For more information on this platform, see <file:Documentation/arm/IXP4xx>.
+ For more information on this platform, see <file:Documentation/arm/ixp4xx.rst>.
config MACH_GORAMO_MLR
bool "GORAMO Multi Link Router"
help
Say 'Y' here if you want your kernel to support Intel's
KIXRP435 Reference Platform.
- For more information on this platform, see <file:Documentation/arm/IXP4xx>.
+ For more information on this platform, see <file:Documentation/arm/ixp4xx.rst>.
#
# IXCDP1100 is the exact same HW as IXDP425, but with a different machine
help
Say 'Y' here if you want your kernel to support the Motorola
PrPCM1100 Processor Mezanine Module. For more information on
- this platform, see <file:Documentation/arm/IXP4xx>.
+ this platform, see <file:Documentation/arm/ixp4xx.rst>.
config MACH_NAS100D
bool
//
// S3C24XX Power Manager (Suspend-To-RAM) support
//
-// See Documentation/arm/Samsung-S3C24XX/Suspend.txt for more information
+// See Documentation/arm/samsung-s3c24xx/suspend.rst for more information
//
// Parts based on arch/arm/mach-pxa/pm.c
//
assistance.
A compliant bootloader is required in order to make maximum
- use of this feature. Refer to Documentation/arm/Booting for
+ use of this feature. Refer to Documentation/arm/booting.rst for
details.
config SWP_EMULATE
the CPU type fitted to the system. This permits binaries to be
run on ARMv4 through to ARMv7 without modification.
- See Documentation/arm/kernel_user_helpers.txt for details.
+ See Documentation/arm/kernel_user_helpers.rst for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
#ifdef CONFIG_MMU
-#ifdef CONFIG_KPROBES
-static inline int notify_page_fault(struct pt_regs *regs, unsigned int fsr)
-{
- int ret = 0;
-
- if (!user_mode(regs)) {
- /* kprobe_running() needs smp_processor_id() */
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, fsr))
- ret = 1;
- preempt_enable();
- }
-
- return ret;
-}
-#else
-static inline int notify_page_fault(struct pt_regs *regs, unsigned int fsr)
-{
- return 0;
-}
-#endif
-
/*
* This is useful to dump out the page tables associated with
* 'addr' in mm 'mm'.
vm_fault_t fault;
unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
- if (notify_page_fault(regs, fsr))
+ if (kprobe_page_fault(regs, fsr))
return 0;
tsk = current;
depends on DEBUG_EXYNOS_UART || DEBUG_S3C24XX_UART || DEBUG_S3C2410_UART
help
Say Y here if you want verbose debugging from the PM Suspend and
- Resume code. See <file:Documentation/arm/Samsung-S3C24XX/Suspend.txt>
+ Resume code. See <file:Documentation/arm/samsung-s3c24xx/suspend.rst>
for more information.
config S3C_PM_DEBUG_LED_SMDK
Note, this can take several seconds depending on memory size
and CPU speed.
- See <file:Documentation/arm/Samsung-S3C24XX/Suspend.txt>
+ See <file:Documentation/arm/samsung-s3c24xx/suspend.rst>
config SAMSUNG_PM_CHECK_CHUNKSIZE
int "S3C2410 PM Suspend CRC Chunksize (KiB)"
the CRC data block will take more memory, but will identify any
faults with better precision.
- See <file:Documentation/arm/Samsung-S3C24XX/Suspend.txt>
+ See <file:Documentation/arm/samsung-s3c24xx/suspend.rst>
config SAMSUNG_WAKEMASK
bool
# http://www.arm.linux.org.uk/developer/machines/download.php
#
# Please do not send patches to this file; it is automatically generated!
-# To add an entry into this database, please see Documentation/arm/README,
+# To add an entry into this database, please see Documentation/arm/arm.rst,
# or visit:
#
# http://www.arm.linux.org.uk/developer/machines/?action=new
select ARCH_HAS_KCOV
select ARCH_HAS_KEEPINITRD
select ARCH_HAS_MEMBARRIER_SYNC_CORE
+ select ARCH_HAS_PTE_DEVMAP
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_SETUP_DMA_OPS
select ARCH_HAS_SET_DIRECT_MAP
select ARCH_SUPPORTS_NUMA_BALANCING
select ARCH_WANT_COMPAT_IPC_PARSE_VERSION if COMPAT
select ARCH_WANT_FRAME_POINTERS
+ select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
def_bool y
config ARCH_WANT_HUGE_PMD_SHARE
- def_bool y if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36)
config ARCH_HAS_CACHE_LINE_SIZE
def_bool y
reserved region and then later executed after a crash by
kdump/kexec.
- For more details see Documentation/kdump/kdump.rst
+ For more details see Documentation/admin-guide/kdump/kdump.rst
config XEN_DOM0
def_bool y
the system. This permits binaries to be run on ARMv4 through
to ARMv8 without modification.
- See Documentation/arm/kernel_user_helpers.txt for details.
+ See Documentation/arm/kernel_user_helpers.rst for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
#define PTE_WRITE (PTE_DBM) /* same as DBM (51) */
#define PTE_DIRTY (_AT(pteval_t, 1) << 55)
#define PTE_SPECIAL (_AT(pteval_t, 1) << 56)
+#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)
#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58) /* only when !PTE_VALID */
#ifndef __ASSEMBLY__
#define pte_write(pte) (!!(pte_val(pte) & PTE_WRITE))
#define pte_user_exec(pte) (!(pte_val(pte) & PTE_UXN))
#define pte_cont(pte) (!!(pte_val(pte) & PTE_CONT))
+#define pte_devmap(pte) (!!(pte_val(pte) & PTE_DEVMAP))
#define pte_cont_addr_end(addr, end) \
({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \
return __pmd(pmd_val(pmd) | PMD_SECT_CONT);
}
+static inline pte_t pte_mkdevmap(pte_t pte)
+{
+ return set_pte_bit(pte, __pgprot(PTE_DEVMAP));
+}
+
static inline void set_pte(pte_t *ptep, pte_t pte)
{
WRITE_ONCE(*ptep, pte);
#define pmd_mkhuge(pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT))
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#define pmd_devmap(pmd) pte_devmap(pmd_pte(pmd))
+#endif
+#define pmd_mkdevmap(pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd)))
+
#define __pmd_to_phys(pmd) __pte_to_phys(pmd_pte(pmd))
#define __phys_to_pmd_val(phys) __phys_to_pte_val(phys)
#define pmd_pfn(pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)
{
return ptep_set_access_flags(vma, address, (pte_t *)pmdp, pmd_pte(entry), dirty);
}
+
+static inline int pud_devmap(pud_t pud)
+{
+ return 0;
+}
+
+static inline int pgd_devmap(pgd_t pgd)
+{
+ return 0;
+}
#endif
/*
#ifndef __ASM_SYSREG_H
#define __ASM_SYSREG_H
-#include <linux/const.h>
+#include <linux/bits.h>
#include <linux/stringify.h>
/*
#define SYS_CNTV_CVAL_EL02 sys_reg(3, 5, 14, 3, 2)
/* Common SCTLR_ELx flags. */
-#define SCTLR_ELx_DSSBS (_BITUL(44))
-#define SCTLR_ELx_ENIA (_BITUL(31))
-#define SCTLR_ELx_ENIB (_BITUL(30))
-#define SCTLR_ELx_ENDA (_BITUL(27))
-#define SCTLR_ELx_EE (_BITUL(25))
-#define SCTLR_ELx_IESB (_BITUL(21))
-#define SCTLR_ELx_WXN (_BITUL(19))
-#define SCTLR_ELx_ENDB (_BITUL(13))
-#define SCTLR_ELx_I (_BITUL(12))
-#define SCTLR_ELx_SA (_BITUL(3))
-#define SCTLR_ELx_C (_BITUL(2))
-#define SCTLR_ELx_A (_BITUL(1))
-#define SCTLR_ELx_M (_BITUL(0))
+#define SCTLR_ELx_DSSBS (BIT(44))
+#define SCTLR_ELx_ENIA (BIT(31))
+#define SCTLR_ELx_ENIB (BIT(30))
+#define SCTLR_ELx_ENDA (BIT(27))
+#define SCTLR_ELx_EE (BIT(25))
+#define SCTLR_ELx_IESB (BIT(21))
+#define SCTLR_ELx_WXN (BIT(19))
+#define SCTLR_ELx_ENDB (BIT(13))
+#define SCTLR_ELx_I (BIT(12))
+#define SCTLR_ELx_SA (BIT(3))
+#define SCTLR_ELx_C (BIT(2))
+#define SCTLR_ELx_A (BIT(1))
+#define SCTLR_ELx_M (BIT(0))
#define SCTLR_ELx_FLAGS (SCTLR_ELx_M | SCTLR_ELx_A | SCTLR_ELx_C | \
SCTLR_ELx_SA | SCTLR_ELx_I | SCTLR_ELx_IESB)
/* SCTLR_EL2 specific flags. */
-#define SCTLR_EL2_RES1 ((_BITUL(4)) | (_BITUL(5)) | (_BITUL(11)) | (_BITUL(16)) | \
- (_BITUL(18)) | (_BITUL(22)) | (_BITUL(23)) | (_BITUL(28)) | \
- (_BITUL(29)))
-#define SCTLR_EL2_RES0 ((_BITUL(6)) | (_BITUL(7)) | (_BITUL(8)) | (_BITUL(9)) | \
- (_BITUL(10)) | (_BITUL(13)) | (_BITUL(14)) | (_BITUL(15)) | \
- (_BITUL(17)) | (_BITUL(20)) | (_BITUL(24)) | (_BITUL(26)) | \
- (_BITUL(27)) | (_BITUL(30)) | (_BITUL(31)) | \
+#define SCTLR_EL2_RES1 ((BIT(4)) | (BIT(5)) | (BIT(11)) | (BIT(16)) | \
+ (BIT(18)) | (BIT(22)) | (BIT(23)) | (BIT(28)) | \
+ (BIT(29)))
+#define SCTLR_EL2_RES0 ((BIT(6)) | (BIT(7)) | (BIT(8)) | (BIT(9)) | \
+ (BIT(10)) | (BIT(13)) | (BIT(14)) | (BIT(15)) | \
+ (BIT(17)) | (BIT(20)) | (BIT(24)) | (BIT(26)) | \
+ (BIT(27)) | (BIT(30)) | (BIT(31)) | \
(0xffffefffUL << 32))
#ifdef CONFIG_CPU_BIG_ENDIAN
#endif
/* SCTLR_EL1 specific flags. */
-#define SCTLR_EL1_UCI (_BITUL(26))
-#define SCTLR_EL1_E0E (_BITUL(24))
-#define SCTLR_EL1_SPAN (_BITUL(23))
-#define SCTLR_EL1_NTWE (_BITUL(18))
-#define SCTLR_EL1_NTWI (_BITUL(16))
-#define SCTLR_EL1_UCT (_BITUL(15))
-#define SCTLR_EL1_DZE (_BITUL(14))
-#define SCTLR_EL1_UMA (_BITUL(9))
-#define SCTLR_EL1_SED (_BITUL(8))
-#define SCTLR_EL1_ITD (_BITUL(7))
-#define SCTLR_EL1_CP15BEN (_BITUL(5))
-#define SCTLR_EL1_SA0 (_BITUL(4))
-
-#define SCTLR_EL1_RES1 ((_BITUL(11)) | (_BITUL(20)) | (_BITUL(22)) | (_BITUL(28)) | \
- (_BITUL(29)))
-#define SCTLR_EL1_RES0 ((_BITUL(6)) | (_BITUL(10)) | (_BITUL(13)) | (_BITUL(17)) | \
- (_BITUL(27)) | (_BITUL(30)) | (_BITUL(31)) | \
+#define SCTLR_EL1_UCI (BIT(26))
+#define SCTLR_EL1_E0E (BIT(24))
+#define SCTLR_EL1_SPAN (BIT(23))
+#define SCTLR_EL1_NTWE (BIT(18))
+#define SCTLR_EL1_NTWI (BIT(16))
+#define SCTLR_EL1_UCT (BIT(15))
+#define SCTLR_EL1_DZE (BIT(14))
+#define SCTLR_EL1_UMA (BIT(9))
+#define SCTLR_EL1_SED (BIT(8))
+#define SCTLR_EL1_ITD (BIT(7))
+#define SCTLR_EL1_CP15BEN (BIT(5))
+#define SCTLR_EL1_SA0 (BIT(4))
+
+#define SCTLR_EL1_RES1 ((BIT(11)) | (BIT(20)) | (BIT(22)) | (BIT(28)) | \
+ (BIT(29)))
+#define SCTLR_EL1_RES0 ((BIT(6)) | (BIT(10)) | (BIT(13)) | (BIT(17)) | \
+ (BIT(27)) | (BIT(30)) | (BIT(31)) | \
(0xffffefffUL << 32))
#ifdef CONFIG_CPU_BIG_ENDIAN
#define ZCR_ELx_LEN_SIZE 9
#define ZCR_ELx_LEN_MASK 0x1ff
-#define CPACR_EL1_ZEN_EL1EN (_BITUL(16)) /* enable EL1 access */
-#define CPACR_EL1_ZEN_EL0EN (_BITUL(17)) /* enable EL0 access, if EL1EN set */
+#define CPACR_EL1_ZEN_EL1EN (BIT(16)) /* enable EL1 access */
+#define CPACR_EL1_ZEN_EL0EN (BIT(17)) /* enable EL0 access, if EL1EN set */
#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
/* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
-#define SYS_MPIDR_SAFE_VAL (_BITUL(31))
+#define SYS_MPIDR_SAFE_VAL (BIT(31))
#ifdef __ASSEMBLY__
* aarch32_setup_additional_pages() and are provided for compatibility
* reasons with 32 bit (aarch32) applications that need them.
*
- * See Documentation/arm/kernel_user_helpers.txt for formal definitions.
+ * See Documentation/arm/kernel_user_helpers.rst for formal definitions.
*/
#include <asm/unistd.h>
return debug_fault_info + DBG_ESR_EVT(esr);
}
-#ifdef CONFIG_KPROBES
-static inline int notify_page_fault(struct pt_regs *regs, unsigned int esr)
-{
- int ret = 0;
-
- /* kprobe_running() needs smp_processor_id() */
- if (!user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, esr))
- ret = 1;
- preempt_enable();
- }
-
- return ret;
-}
-#else
-static inline int notify_page_fault(struct pt_regs *regs, unsigned int esr)
-{
- return 0;
-}
-#endif
-
static void data_abort_decode(unsigned int esr)
{
pr_alert("Data abort info:\n");
unsigned long vm_flags = VM_READ | VM_WRITE;
unsigned int mm_flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
- if (notify_page_fault(regs, esr))
+ if (kprobe_page_fault(regs, esr))
return 0;
/*
return dt_virt;
}
+int __init arch_ioremap_p4d_supported(void)
+{
+ return 0;
+}
+
int __init arch_ioremap_pud_supported(void)
{
/*
return __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT,
restrictions);
}
+void arch_remove_memory(int nid, u64 start, u64 size,
+ struct vmem_altmap *altmap)
+{
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ struct zone *zone;
+
+ /*
+ * FIXME: Cleanup page tables (also in arch_add_memory() in case
+ * adding fails). Until then, this function should only be used
+ * during memory hotplug (adding memory), not for memory
+ * unplug. ARCH_ENABLE_MEMORY_HOTREMOVE must not be
+ * unlocked yet.
+ */
+ zone = page_zone(pfn_to_page(start_pfn));
+ __remove_pages(zone, start_pfn, nr_pages, altmap);
+}
#endif
} \
}
-/*
- * clear_bit() doesn't provide any barrier for the compiler.
- */
-#define smp_mb__before_clear_bit() barrier()
-#define smp_mb__after_clear_bit() barrier()
-
H8300_GEN_BITOP(set_bit, "bset")
H8300_GEN_BITOP(clear_bit, "bclr")
H8300_GEN_BITOP(change_bit, "bnot")
#define _ASM_HEXAGON_SYSCALL_H
#include <uapi/linux/audit.h>
+#include <linux/err.h>
+#include <asm/ptrace.h>
typedef long (*syscall_fn)(unsigned long, unsigned long,
unsigned long, unsigned long,
memcpy(args, &(®s->r00)[0], 6 * sizeof(args[0]));
}
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return IS_ERR_VALUE(regs->r00) ? regs->r00 : 0;
+}
+
+static inline long syscall_get_return_value(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->r00;
+}
+
static inline int syscall_get_arch(struct task_struct *task)
{
return AUDIT_ARCH_HEXAGON;
* /dev/mem reads and writes use copy_to_user(), which implicitly
* uses a granule-sized kernel identity mapping. It's really
* only safe to do this for regions in kern_memmap. For more
- * details, see Documentation/ia64/aliasing.txt.
+ * details, see Documentation/ia64/aliasing.rst.
*/
attr = kern_mem_attribute(phys_addr, size);
if (attr & EFI_MEMORY_WB || attr & EFI_MEMORY_UC)
#include <asm/native/inst.h>
/*
- * See Documentation/ia64/fsys.txt for details on fsyscalls.
+ * See Documentation/ia64/fsys.rst for details on fsyscalls.
*
* On entry to an fsyscall handler:
* r10 = 0 (i.e., defaults to "successful syscall return")
#include <linux/smp.h>
#include <linux/pagemap.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/bitops.h>
#include <linux/capability.h>
#include <linux/rcupdate.h>
/* forward declaration */
static const struct dentry_operations pfmfs_dentry_operations;
-static struct dentry *
-pfmfs_mount(struct file_system_type *fs_type, int flags, const char *dev_name, void *data)
+static int pfmfs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "pfm:", NULL, &pfmfs_dentry_operations,
- PFMFS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, PFMFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->dops = &pfmfs_dentry_operations;
+ return 0;
}
static struct file_system_type pfm_fs_type = {
- .name = "pfmfs",
- .mount = pfmfs_mount,
- .kill_sb = kill_anon_super,
+ .name = "pfmfs",
+ .init_fs_context = pfmfs_init_fs_context,
+ .kill_sb = kill_anon_super,
};
MODULE_ALIAS_FS("pfmfs");
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+# 435 reserved for clone3
extern int die(char *, struct pt_regs *, long);
-#ifdef CONFIG_KPROBES
-static inline int notify_page_fault(struct pt_regs *regs, int trap)
-{
- int ret = 0;
-
- if (!user_mode(regs)) {
- /* kprobe_running() needs smp_processor_id() */
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, trap))
- ret = 1;
- preempt_enable();
- }
-
- return ret;
-}
-#else
-static inline int notify_page_fault(struct pt_regs *regs, int trap)
-{
- return 0;
-}
-#endif
-
/*
* Return TRUE if ADDRESS points at a page in the kernel's mapped segment
* (inside region 5, on ia64) and that page is present.
/*
* This is to handle the kprobes on user space access instructions
*/
- if (notify_page_fault(regs, TRAP_BRKPT))
+ if (kprobe_page_fault(regs, TRAP_BRKPT))
return;
if (user_mode(regs))
return ret;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
__remove_pages(zone, start_pfn, nr_pages, altmap);
}
#endif
-#endif
/*
* For things in kern_memmap, we must use the same attribute
* as the rest of the kernel. For more details, see
- * Documentation/ia64/aliasing.txt.
+ * Documentation/ia64/aliasing.rst.
*/
attr = kern_mem_attribute(phys_addr, size);
if (attr & EFI_MEMORY_WB)
return -ENOSYS;
/*
- * Avoid attribute aliasing. See Documentation/ia64/aliasing.txt
+ * Avoid attribute aliasing. See Documentation/ia64/aliasing.rst
* for more details.
*/
if (!valid_mmap_phys_addr_range(vma->vm_pgoff, size))
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+# 435 reserved for clone3
case TITAN_CHIP_1060:
return "TI AR7 (TNETV1060)";
}
+ /* fall through */
default:
return "TI AR7 (unknown)";
}
case REV_ID_MAJOR_QCA9533_V2:
ver = 2;
ath79_soc_rev = 2;
- /* drop through */
+ /* fall through */
case REV_ID_MAJOR_QCA9533:
ath79_soc = ATH79_SOC_QCA9533;
case STRAPBUS_6368_BOOT_SEL_PARALLEL:
return BCM63XX_FLASH_TYPE_PARALLEL;
}
+ /* fall through */
default:
return -EINVAL;
}
ioport_resource.start = 0;
ioport_resource.end = ~0;
- /* intended to somewhat resemble ARM; see Documentation/arm/Booting */
+ /* intended to somewhat resemble ARM; see Documentation/arm/booting.rst */
if (fw_arg0 == 0 && fw_arg1 == 0xffffffff)
dtb = phys_to_virt(fw_arg2);
else if (fw_passed_dtb) /* UHI interface or appended dtb */
+// SPDX-License-Identifier: GPL-2.0
+
/ {
#address-cells = <1>;
#size-cells = <1>;
sysc: system-controller@0 {
compatible = "ralink,mt7620a-sysc", "syscon";
- reg = <0x0 0x100>;
+ reg = <0x0 0x60>;
+ };
+
+ pinmux: pinmux@60 {
+ compatible = "pinctrl-single";
+ reg = <0x60 0x8>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ #pinctrl-cells = <2>;
+ pinctrl-single,bit-per-mux;
+ pinctrl-single,register-width = <32>;
+ pinctrl-single,function-mask = <0x1>;
+
+ pinmux_gpio_gpio: pinmux_gpio_gpio {
+ pinctrl-single,bits = <0x0 0x0 0x3>;
+ };
+
+ pinmux_spi_cs1_cs: pinmux_spi_cs1_cs {
+ pinctrl-single,bits = <0x0 0x0 0x30>;
+ };
+
+ pinmux_i2s_gpio: pinmux_i2s_gpio {
+ pinctrl-single,bits = <0x0 0x40 0xc0>;
+ };
+
+ pinmux_uart0_uart: pinmux_uart0_uart0 {
+ pinctrl-single,bits = <0x0 0x0 0x300>;
+ };
+
+ pinmux_sdmode_sdxc: pinmux_sdmode_sdxc {
+ pinctrl-single,bits = <0x0 0x0 0xc00>;
+ };
+
+ pinmux_sdmode_gpio: pinmux_sdmode_gpio {
+ pinctrl-single,bits = <0x0 0x400 0xc00>;
+ };
+
+ pinmux_spi_spi: pinmux_spi_spi {
+ pinctrl-single,bits = <0x0 0x0 0x1000>;
+ };
+
+ pinmux_refclk_gpio: pinmux_refclk_gpio {
+ pinctrl-single,bits = <0x0 0x40000 0x40000>;
+ };
+
+ pinmux_i2c_i2c: pinmux_i2c_i2c {
+ pinctrl-single,bits = <0x0 0x0 0x300000>;
+ };
+
+ pinmux_uart1_uart: pinmux_uart1_uart1 {
+ pinctrl-single,bits = <0x0 0x0 0x3000000>;
+ };
+
+ pinmux_uart2_uart: pinmux_uart2_uart {
+ pinctrl-single,bits = <0x0 0x0 0xc000000>;
+ };
+
+ pinmux_pwm0_pwm: pinmux_pwm0_pwm {
+ pinctrl-single,bits = <0x0 0x0 0x30000000>;
+ };
+
+ pinmux_pwm0_gpio: pinmux_pwm0_gpio {
+ pinctrl-single,bits = <0x0 0x10000000
+ 0x30000000>;
+ };
+
+ pinmux_pwm1_pwm: pinmux_pwm1_pwm {
+ pinctrl-single,bits = <0x0 0x0 0xc0000000>;
+ };
+
+ pinmux_pwm1_gpio: pinmux_pwm1_gpio {
+ pinctrl-single,bits = <0x0 0x40000000
+ 0xc0000000>;
+ };
+
+ pinmux_p0led_an_gpio: pinmux_p0led_an_gpio {
+ pinctrl-single,bits = <0x4 0x4 0xc>;
+ };
+
+ pinmux_p1led_an_gpio: pinmux_p1led_an_gpio {
+ pinctrl-single,bits = <0x4 0x10 0x30>;
+ };
+
+ pinmux_p2led_an_gpio: pinmux_p2led_an_gpio {
+ pinctrl-single,bits = <0x4 0x40 0xc0>;
+ };
+
+ pinmux_p3led_an_gpio: pinmux_p3led_an_gpio {
+ pinctrl-single,bits = <0x4 0x100 0x300>;
+ };
+
+ pinmux_p4led_an_gpio: pinmux_p4led_an_gpio {
+ pinctrl-single,bits = <0x4 0x400 0xc00>;
+ };
+ };
+
+ watchdog: watchdog@100 {
+ compatible = "mediatek,mt7621-wdt";
+ reg = <0x100 0x30>;
+
+ resets = <&resetc 8>;
+ reset-names = "wdt";
+
+ interrupt-parent = <&intc>;
+ interrupts = <24>;
+
+ status = "disabled";
};
intc: interrupt-controller@200 {
reg = <0x300 0x100>;
};
+ gpio: gpio@600 {
+ compatible = "mediatek,mt7621-gpio";
+ reg = <0x600 0x100>;
+
+ gpio-controller;
+ interrupt-controller;
+ #gpio-cells = <2>;
+ #interrupt-cells = <2>;
+
+ interrupt-parent = <&intc>;
+ interrupts = <6>;
+ };
+
+ spi: spi@b00 {
+ compatible = "ralink,mt7621-spi";
+ reg = <0xb00 0x100>;
+
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinmux_spi_spi>;
+
+ resets = <&resetc 18>;
+ reset-names = "spi";
+
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ status = "disabled";
+ };
+
uart0: uartlite@c00 {
compatible = "ns16550a";
reg = <0xc00 0x100>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinmux_uart0_uart>;
+
resets = <&resetc 12>;
reset-names = "uart0";
compatible = "ns16550a";
reg = <0xd00 0x100>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinmux_uart1_uart>;
+
resets = <&resetc 19>;
reset-names = "uart1";
compatible = "ns16550a";
reg = <0xe00 0x100>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinmux_uart2_uart>;
+
resets = <&resetc 20>;
reset-names = "uart2";
config.s.qos_mask = 0xff;
break;
case CVMX_PKO_QUEUE_STATIC_PRIORITY:
- /* Pass 1 will fall through to the error case */
if (!cvmx_octeon_is_pass1()) {
config.s.qos_mask = 0xff;
break;
}
+ /* fall through - to the error case, when Pass 1 */
default:
cvmx_dprintf("ERROR: cvmx_pko_config_port: Invalid "
"priority %llu\n",
CONFIG_HAMRADIO=y
CONFIG_CFG80211=m
CONFIG_MAC80211=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_CFG80211=m
CONFIG_MAC80211=m
CONFIG_MAC80211_DEBUGFS=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_REDBOOT_PARTS=y
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-2
CONFIG_CFG80211=m
CONFIG_MAC80211=m
CONFIG_MAC80211_DEBUGFS=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_REDBOOT_PARTS=y
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-2
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
CONFIG_MAC80211=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_MTD=y
CONFIG_BAYCOM_SER_FDX=m
CONFIG_BAYCOM_SER_HDX=m
CONFIG_YAM=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
CONFIG_MAC80211=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
CONFIG_CFG80211=y
CONFIG_NL80211_TESTMODE=y
CONFIG_MAC80211=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
CONFIG_SYN_COOKIES=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FW_LOADER is not set
# CONFIG_ALLOW_DEV_COREDUMP is not set
CONFIG_NET_KEY_MIGRATE=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_JEDECPROBE=y
CONFIG_MTD_CFI_AMDSTD=y
CONFIG_IP_NF_ARP_MANGLE=m
CONFIG_PHONET=m
CONFIG_NET_9P=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_MTD=m
CONFIG_MTD_BLOCK=m
CONFIG_SSB=m
CONFIG_SSB_DRIVER_PCICORE=y
# CONFIG_VGA_ARB is not set
-CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_GENERIC is not set
CONFIG_CFG80211=m
CONFIG_MAC80211=m
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_CDROM_PKTCDVD=m
CONFIG_INET6_IPCOMP=m
CONFIG_IPV6_TUNNEL=m
CONFIG_NETWORK_SECMARK=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_CONNECTOR=y
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_MAC80211_LEDS=y
CONFIG_RFKILL=m
CONFIG_RFKILL_INPUT=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_RAM=y
CONFIG_FB_SIS=y
CONFIG_FB_SIS_300=y
CONFIG_FB_SIS_315=y
-CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_GENERIC=m
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# CONFIG_STANDALONE is not set
CONFIG_MAC80211=m
CONFIG_RFKILL=m
CONFIG_RFKILL_INPUT=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_MTD=m
CONFIG_MAC80211=m
CONFIG_MAC80211_MESH=y
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_CONNECTOR=m
CONFIG_MTD=y
CONFIG_MAC80211=m
CONFIG_MAC80211_MESH=y
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_CONNECTOR=m
CONFIG_MTD=y
CONFIG_MAC80211=m
CONFIG_MAC80211_MESH=y
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_CONNECTOR=m
CONFIG_MTD=y
CONFIG_MAC80211=m
CONFIG_MAC80211_MESH=y
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_CONNECTOR=m
CONFIG_IP_PIMSM_V2=y
CONFIG_SYN_COOKIES=y
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_VIRTIO_BLK=y
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FW_LOADER is not set
# CONFIG_ALLOW_DEV_COREDUMP is not set
CONFIG_MEDIA_SUPPORT=y
CONFIG_FB=y
CONFIG_FB_MODE_HELPERS=y
-CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_SOUND=y
CONFIG_IP_PNP_DHCP=y
CONFIG_INET_AH=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_TCP_CONG_WESTWOOD=y
# CONFIG_TCP_CONG_HTCP is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_RAW_NAND=y
CONFIG_REGULATOR_FIXED_VOLTAGE=y
CONFIG_FB=y
CONFIG_FB_JZ4740=y
-CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_LCD_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_CLASS_DEVICE is not set
# CONFIG_VGA_CONSOLE is not set
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_PEDIT=m
CONFIG_HAMRADIO=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_BLOCK2MTD=y
CONFIG_VLAN_8021Q=y
CONFIG_NET_SCHED=y
CONFIG_HAMRADIO=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_CFG80211=m
CONFIG_MAC80211=m
CONFIG_RFKILL=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_CONNECTOR=m
CONFIG_BLK_DEV_RAM=y
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
CONFIG_NETWORK_SECMARK=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
CONFIG_NETWORK_SECMARK=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
CONFIG_TCP_CONG_CUBIC=m
# CONFIG_IPV6 is not set
CONFIG_NETWORK_SECMARK=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RAM=y
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
# CONFIG_FW_LOADER is not set
# CONFIG_ALLOW_DEV_COREDUMP is not set
CONFIG_VLAN_8021Q=y
CONFIG_NET_SCHED=y
CONFIG_HAMRADIO=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
#ifndef _ASM_CPU_H
#define _ASM_CPU_H
+#include <linux/bits.h>
+
/*
As of the MIPS32 and MIPS64 specs from MTI, the PRId register (CP0
register 15, select 0) is defined in this (backwards compatible) way:
MIPS_CPU_ISA_M64R6)
/*
- * Private version of BIT_ULL() to escape include file recursion hell.
- * We soon will have to switch to another mechanism that will work with
- * more than 64 bits anyway.
- */
-#define MBIT_ULL(bit) (1ULL << (bit))
-
-/*
* CPU Option encodings
*/
-#define MIPS_CPU_TLB MBIT_ULL( 0) /* CPU has TLB */
-#define MIPS_CPU_4KEX MBIT_ULL( 1) /* "R4K" exception model */
-#define MIPS_CPU_3K_CACHE MBIT_ULL( 2) /* R3000-style caches */
-#define MIPS_CPU_4K_CACHE MBIT_ULL( 3) /* R4000-style caches */
-#define MIPS_CPU_TX39_CACHE MBIT_ULL( 4) /* TX3900-style caches */
-#define MIPS_CPU_FPU MBIT_ULL( 5) /* CPU has FPU */
-#define MIPS_CPU_32FPR MBIT_ULL( 6) /* 32 dbl. prec. FP registers */
-#define MIPS_CPU_COUNTER MBIT_ULL( 7) /* Cycle count/compare */
-#define MIPS_CPU_WATCH MBIT_ULL( 8) /* watchpoint registers */
-#define MIPS_CPU_DIVEC MBIT_ULL( 9) /* dedicated interrupt vector */
-#define MIPS_CPU_VCE MBIT_ULL(10) /* virt. coherence conflict possible */
-#define MIPS_CPU_CACHE_CDEX_P MBIT_ULL(11) /* Create_Dirty_Exclusive CACHE op */
-#define MIPS_CPU_CACHE_CDEX_S MBIT_ULL(12) /* ... same for seconary cache ... */
-#define MIPS_CPU_MCHECK MBIT_ULL(13) /* Machine check exception */
-#define MIPS_CPU_EJTAG MBIT_ULL(14) /* EJTAG exception */
-#define MIPS_CPU_NOFPUEX MBIT_ULL(15) /* no FPU exception */
-#define MIPS_CPU_LLSC MBIT_ULL(16) /* CPU has ll/sc instructions */
-#define MIPS_CPU_INCLUSIVE_CACHES MBIT_ULL(17) /* P-cache subset enforced */
-#define MIPS_CPU_PREFETCH MBIT_ULL(18) /* CPU has usable prefetch */
-#define MIPS_CPU_VINT MBIT_ULL(19) /* CPU supports MIPSR2 vectored interrupts */
-#define MIPS_CPU_VEIC MBIT_ULL(20) /* CPU supports MIPSR2 external interrupt controller mode */
-#define MIPS_CPU_ULRI MBIT_ULL(21) /* CPU has ULRI feature */
-#define MIPS_CPU_PCI MBIT_ULL(22) /* CPU has Perf Ctr Int indicator */
-#define MIPS_CPU_RIXI MBIT_ULL(23) /* CPU has TLB Read/eXec Inhibit */
-#define MIPS_CPU_MICROMIPS MBIT_ULL(24) /* CPU has microMIPS capability */
-#define MIPS_CPU_TLBINV MBIT_ULL(25) /* CPU supports TLBINV/F */
-#define MIPS_CPU_SEGMENTS MBIT_ULL(26) /* CPU supports Segmentation Control registers */
-#define MIPS_CPU_EVA MBIT_ULL(27) /* CPU supports Enhanced Virtual Addressing */
-#define MIPS_CPU_HTW MBIT_ULL(28) /* CPU support Hardware Page Table Walker */
-#define MIPS_CPU_RIXIEX MBIT_ULL(29) /* CPU has unique exception codes for {Read, Execute}-Inhibit exceptions */
-#define MIPS_CPU_MAAR MBIT_ULL(30) /* MAAR(I) registers are present */
-#define MIPS_CPU_FRE MBIT_ULL(31) /* FRE & UFE bits implemented */
-#define MIPS_CPU_RW_LLB MBIT_ULL(32) /* LLADDR/LLB writes are allowed */
-#define MIPS_CPU_LPA MBIT_ULL(33) /* CPU supports Large Physical Addressing */
-#define MIPS_CPU_CDMM MBIT_ULL(34) /* CPU has Common Device Memory Map */
-#define MIPS_CPU_BP_GHIST MBIT_ULL(35) /* R12K+ Branch Prediction Global History */
-#define MIPS_CPU_SP MBIT_ULL(36) /* Small (1KB) page support */
-#define MIPS_CPU_FTLB MBIT_ULL(37) /* CPU has Fixed-page-size TLB */
-#define MIPS_CPU_NAN_LEGACY MBIT_ULL(38) /* Legacy NaN implemented */
-#define MIPS_CPU_NAN_2008 MBIT_ULL(39) /* 2008 NaN implemented */
-#define MIPS_CPU_VP MBIT_ULL(40) /* MIPSr6 Virtual Processors (multi-threading) */
-#define MIPS_CPU_LDPTE MBIT_ULL(41) /* CPU has ldpte/lddir instructions */
-#define MIPS_CPU_MVH MBIT_ULL(42) /* CPU supports MFHC0/MTHC0 */
-#define MIPS_CPU_EBASE_WG MBIT_ULL(43) /* CPU has EBase.WG */
-#define MIPS_CPU_BADINSTR MBIT_ULL(44) /* CPU has BadInstr register */
-#define MIPS_CPU_BADINSTRP MBIT_ULL(45) /* CPU has BadInstrP register */
-#define MIPS_CPU_CTXTC MBIT_ULL(46) /* CPU has [X]ConfigContext registers */
-#define MIPS_CPU_PERF MBIT_ULL(47) /* CPU has MIPS performance counters */
-#define MIPS_CPU_GUESTCTL0EXT MBIT_ULL(48) /* CPU has VZ GuestCtl0Ext register */
-#define MIPS_CPU_GUESTCTL1 MBIT_ULL(49) /* CPU has VZ GuestCtl1 register */
-#define MIPS_CPU_GUESTCTL2 MBIT_ULL(50) /* CPU has VZ GuestCtl2 register */
-#define MIPS_CPU_GUESTID MBIT_ULL(51) /* CPU uses VZ ASE GuestID feature */
-#define MIPS_CPU_DRG MBIT_ULL(52) /* CPU has VZ Direct Root to Guest (DRG) */
-#define MIPS_CPU_UFR MBIT_ULL(53) /* CPU supports User mode FR switching */
+#define MIPS_CPU_TLB BIT_ULL( 0) /* CPU has TLB */
+#define MIPS_CPU_4KEX BIT_ULL( 1) /* "R4K" exception model */
+#define MIPS_CPU_3K_CACHE BIT_ULL( 2) /* R3000-style caches */
+#define MIPS_CPU_4K_CACHE BIT_ULL( 3) /* R4000-style caches */
+#define MIPS_CPU_TX39_CACHE BIT_ULL( 4) /* TX3900-style caches */
+#define MIPS_CPU_FPU BIT_ULL( 5) /* CPU has FPU */
+#define MIPS_CPU_32FPR BIT_ULL( 6) /* 32 dbl. prec. FP registers */
+#define MIPS_CPU_COUNTER BIT_ULL( 7) /* Cycle count/compare */
+#define MIPS_CPU_WATCH BIT_ULL( 8) /* watchpoint registers */
+#define MIPS_CPU_DIVEC BIT_ULL( 9) /* dedicated interrupt vector */
+#define MIPS_CPU_VCE BIT_ULL(10) /* virt. coherence conflict possible */
+#define MIPS_CPU_CACHE_CDEX_P BIT_ULL(11) /* Create_Dirty_Exclusive CACHE op */
+#define MIPS_CPU_CACHE_CDEX_S BIT_ULL(12) /* ... same for seconary cache ... */
+#define MIPS_CPU_MCHECK BIT_ULL(13) /* Machine check exception */
+#define MIPS_CPU_EJTAG BIT_ULL(14) /* EJTAG exception */
+#define MIPS_CPU_NOFPUEX BIT_ULL(15) /* no FPU exception */
+#define MIPS_CPU_LLSC BIT_ULL(16) /* CPU has ll/sc instructions */
+#define MIPS_CPU_INCLUSIVE_CACHES BIT_ULL(17) /* P-cache subset enforced */
+#define MIPS_CPU_PREFETCH BIT_ULL(18) /* CPU has usable prefetch */
+#define MIPS_CPU_VINT BIT_ULL(19) /* CPU supports MIPSR2 vectored interrupts */
+#define MIPS_CPU_VEIC BIT_ULL(20) /* CPU supports MIPSR2 external interrupt controller mode */
+#define MIPS_CPU_ULRI BIT_ULL(21) /* CPU has ULRI feature */
+#define MIPS_CPU_PCI BIT_ULL(22) /* CPU has Perf Ctr Int indicator */
+#define MIPS_CPU_RIXI BIT_ULL(23) /* CPU has TLB Read/eXec Inhibit */
+#define MIPS_CPU_MICROMIPS BIT_ULL(24) /* CPU has microMIPS capability */
+#define MIPS_CPU_TLBINV BIT_ULL(25) /* CPU supports TLBINV/F */
+#define MIPS_CPU_SEGMENTS BIT_ULL(26) /* CPU supports Segmentation Control registers */
+#define MIPS_CPU_EVA BIT_ULL(27) /* CPU supports Enhanced Virtual Addressing */
+#define MIPS_CPU_HTW BIT_ULL(28) /* CPU support Hardware Page Table Walker */
+#define MIPS_CPU_RIXIEX BIT_ULL(29) /* CPU has unique exception codes for {Read, Execute}-Inhibit exceptions */
+#define MIPS_CPU_MAAR BIT_ULL(30) /* MAAR(I) registers are present */
+#define MIPS_CPU_FRE BIT_ULL(31) /* FRE & UFE bits implemented */
+#define MIPS_CPU_RW_LLB BIT_ULL(32) /* LLADDR/LLB writes are allowed */
+#define MIPS_CPU_LPA BIT_ULL(33) /* CPU supports Large Physical Addressing */
+#define MIPS_CPU_CDMM BIT_ULL(34) /* CPU has Common Device Memory Map */
+#define MIPS_CPU_BP_GHIST BIT_ULL(35) /* R12K+ Branch Prediction Global History */
+#define MIPS_CPU_SP BIT_ULL(36) /* Small (1KB) page support */
+#define MIPS_CPU_FTLB BIT_ULL(37) /* CPU has Fixed-page-size TLB */
+#define MIPS_CPU_NAN_LEGACY BIT_ULL(38) /* Legacy NaN implemented */
+#define MIPS_CPU_NAN_2008 BIT_ULL(39) /* 2008 NaN implemented */
+#define MIPS_CPU_VP BIT_ULL(40) /* MIPSr6 Virtual Processors (multi-threading) */
+#define MIPS_CPU_LDPTE BIT_ULL(41) /* CPU has ldpte/lddir instructions */
+#define MIPS_CPU_MVH BIT_ULL(42) /* CPU supports MFHC0/MTHC0 */
+#define MIPS_CPU_EBASE_WG BIT_ULL(43) /* CPU has EBase.WG */
+#define MIPS_CPU_BADINSTR BIT_ULL(44) /* CPU has BadInstr register */
+#define MIPS_CPU_BADINSTRP BIT_ULL(45) /* CPU has BadInstrP register */
+#define MIPS_CPU_CTXTC BIT_ULL(46) /* CPU has [X]ConfigContext registers */
+#define MIPS_CPU_PERF BIT_ULL(47) /* CPU has MIPS performance counters */
+#define MIPS_CPU_GUESTCTL0EXT BIT_ULL(48) /* CPU has VZ GuestCtl0Ext register */
+#define MIPS_CPU_GUESTCTL1 BIT_ULL(49) /* CPU has VZ GuestCtl1 register */
+#define MIPS_CPU_GUESTCTL2 BIT_ULL(50) /* CPU has VZ GuestCtl2 register */
+#define MIPS_CPU_GUESTID BIT_ULL(51) /* CPU uses VZ ASE GuestID feature */
+#define MIPS_CPU_DRG BIT_ULL(52) /* CPU has VZ Direct Root to Guest (DRG) */
+#define MIPS_CPU_UFR BIT_ULL(53) /* CPU supports User mode FR switching */
#define MIPS_CPU_SHARED_FTLB_RAM \
- MBIT_ULL(54) /* CPU shares FTLB RAM with another */
+ BIT_ULL(54) /* CPU shares FTLB RAM with another */
#define MIPS_CPU_SHARED_FTLB_ENTRIES \
- MBIT_ULL(55) /* CPU shares FTLB entries with another */
+ BIT_ULL(55) /* CPU shares FTLB entries with another */
#define MIPS_CPU_MT_PER_TC_PERF_COUNTERS \
- MBIT_ULL(56) /* CPU has perf counters implemented per TC (MIPSMT ASE) */
-#define MIPS_CPU_MMID MBIT_ULL(57) /* CPU supports MemoryMapIDs */
+ BIT_ULL(56) /* CPU has perf counters implemented per TC (MIPSMT ASE) */
+#define MIPS_CPU_MMID BIT_ULL(57) /* CPU supports MemoryMapIDs */
/*
* CPU ASE encodings
return phys_to_virt(address);
}
-#define isa_page_to_bus page_to_phys
-
/*
* However PCI ones are not necessarily 1:1 and therefore these interfaces
* are forbidden in portable PCI drivers.
BUILDIO_MEM(b, u8)
BUILDIO_MEM(w, u16)
BUILDIO_MEM(l, u32)
+#ifdef CONFIG_64BIT
BUILDIO_MEM(q, u64)
+#else
+__BUILD_MEMORY_PFX(__raw_, q, u64, 0)
+__BUILD_MEMORY_PFX(__mem_, q, u64, 0)
+#endif
#define __BUILD_IOPORT_PFX(bus, bwlq, type) \
__BUILD_IOPORT_SINGLE(bus, bwlq, type, 1, 0,) \
#define readb_relaxed __relaxed_readb
#define readw_relaxed __relaxed_readw
#define readl_relaxed __relaxed_readl
+#ifdef CONFIG_64BIT
#define readq_relaxed __relaxed_readq
+#endif
#define writeb_relaxed __relaxed_writeb
#define writew_relaxed __relaxed_writew
#define writel_relaxed __relaxed_writel
+#ifdef CONFIG_64BIT
#define writeq_relaxed __relaxed_writeq
+#endif
#define readb_be(addr) \
__raw_readb((__force unsigned *)(addr))
/*
* Some code tests for these symbols
*/
+#ifdef CONFIG_64BIT
#define readq readq
#define writeq writeq
+#endif
#define __BUILD_MEMORY_STRING(bwlq, type) \
\
#define kretprobe_blacklist_size 0
void arch_remove_kprobe(struct kprobe *p);
+int kprobe_fault_handler(struct pt_regs *regs, int trapnr);
/* Architecture specific copy of original instruction*/
struct arch_specific_insn {
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/*
- * Copyright (C) 2010, Lars-Peter Clausen <lars@metafoo.de>
- */
-
-#ifndef __ASM_JZ4740_CLOCK_H__
-#define __ASM_JZ4740_CLOCK_H__
-
-enum jz4740_wait_mode {
- JZ4740_WAIT_MODE_IDLE,
- JZ4740_WAIT_MODE_SLEEP,
-};
-
-void jz4740_clock_set_wait_mode(enum jz4740_wait_mode mode);
-
-void jz4740_clock_suspend(void);
-void jz4740_clock_resume(void);
-
-void jz4740_clock_udc_enable_auto_suspend(void);
-void jz4740_clock_udc_disable_auto_suspend(void);
-
-#endif
/* SPDX-License-Identifier: GPL-2.0-only */
/*
- *
* Copyright (C) 2012 John Crispin <john@phrozen.org>
*/
unreachable();
}
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ return regs->regs[7] ? -regs->regs[2] : 0;
+}
+
static inline long syscall_get_return_value(struct task_struct *task,
struct pt_regs *regs)
{
#include <asm/mach-jz4740/platform.h>
-#include "clock.h"
-
/* GPIOs */
#define QI_LB60_GPIO_KEYOUT(x) (JZ_GPIO_PORTC(10) + (x))
#define QI_LB60_GPIO_KEYIN(x) (JZ_GPIO_PORTD(18) + (x))
static struct pinctrl_map pin_map[] __initdata = {
/* NAND pin configuration */
PIN_MAP_MUX_GROUP_DEFAULT("jz4740-nand",
- "10010000.jz4740-pinctrl", "nand", "nand-cs1"),
+ "10010000.pin-controller", "nand-cs1", "nand"),
/* fbdev pin configuration */
PIN_MAP_MUX_GROUP("jz4740-fb", PINCTRL_STATE_DEFAULT,
- "10010000.jz4740-pinctrl", "lcd", "lcd-8bit"),
+ "10010000.pin-controller", "lcd-8bit", "lcd"),
PIN_MAP_MUX_GROUP("jz4740-fb", PINCTRL_STATE_SLEEP,
- "10010000.jz4740-pinctrl", "lcd", "lcd-no-pins"),
+ "10010000.pin-controller", "lcd-no-pins", "lcd"),
/* MMC pin configuration */
PIN_MAP_MUX_GROUP_DEFAULT("jz4740-mmc.0",
- "10010000.jz4740-pinctrl", "mmc", "mmc-1bit"),
+ "10010000.pin-controller", "mmc-1bit", "mmc"),
PIN_MAP_MUX_GROUP_DEFAULT("jz4740-mmc.0",
- "10010000.jz4740-pinctrl", "mmc", "mmc-4bit"),
+ "10010000.pin-controller", "mmc-4bit", "mmc"),
PIN_MAP_CONFIGS_PIN_DEFAULT("jz4740-mmc.0",
- "10010000.jz4740-pinctrl", "PD0", pin_cfg_bias_disable),
+ "10010000.pin-controller", "PD0", pin_cfg_bias_disable),
PIN_MAP_CONFIGS_PIN_DEFAULT("jz4740-mmc.0",
- "10010000.jz4740-pinctrl", "PD2", pin_cfg_bias_disable),
+ "10010000.pin-controller", "PD2", pin_cfg_bias_disable),
/* PWM pin configuration */
PIN_MAP_MUX_GROUP_DEFAULT("jz4740-pwm",
- "10010000.jz4740-pinctrl", "pwm4", "pwm4"),
+ "10010000.pin-controller", "pwm4", "pwm4"),
};
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
-#include "clock.h"
-
/* USB Device Controller */
struct platform_device jz4740_udc_xceiv_device = {
.name = "usb_phy_generic",
#include <linux/delay.h>
#include <linux/suspend.h>
-#include <asm/mach-jz4740/clock.h>
-
static int jz4740_pm_enter(suspend_state_t state)
{
- jz4740_clock_suspend();
-
- jz4740_clock_set_wait_mode(JZ4740_WAIT_MODE_SLEEP);
-
__asm__(".set\tmips3\n\t"
"wait\n\t"
".set\tmips0");
- jz4740_clock_set_wait_mode(JZ4740_WAIT_MODE_IDLE);
- jz4740_clock_resume();
return 0;
}
#include <linux/clockchips.h>
#include <linux/sched_clock.h>
-#include <asm/mach-jz4740/clock.h>
#include <asm/mach-jz4740/irq.h>
#include <asm/mach-jz4740/timer.h>
#include <asm/time.h>
-#include "clock.h"
-
#define TIMER_CLOCKEVENT 0
#define TIMER_CLOCKSOURCE 1
return;
/*
- * "parent_ra_addr" is the stack address saved the return address of
- * the caller of _mcount.
+ * "parent_ra_addr" is the stack address where the return address of
+ * the caller of _mcount is saved.
*
- * if the gcc < 4.5, a leaf function does not save the return address
- * in the stack address, so, we "emulate" one in _mcount's stack space,
- * and hijack it directly, but for a non-leaf function, it save the
- * return address to the its own stack space, we can not hijack it
- * directly, but need to find the real stack address,
- * ftrace_get_parent_addr() does it!
+ * If gcc < 4.5, a leaf function does not save the return address
+ * in the stack address, so we "emulate" one in _mcount's stack space,
+ * and hijack it directly.
+ * For a non-leaf function, it does save the return address to its own
+ * stack space, so we can not hijack it directly, but need to find the
+ * real stack address, which is done by ftrace_get_parent_addr().
*
- * if gcc>= 4.5, with the new -mmcount-ra-address option, for a
+ * If gcc >= 4.5, with the new -mmcount-ra-address option, for a
* non-leaf function, the location of the return address will be saved
- * to $12 for us, and for a leaf function, only put a zero into $12. we
- * do it in ftrace_graph_caller of mcount.S.
+ * to $12 for us.
+ * For a leaf function, it just puts a zero into $12, so we handle
+ * it in ftrace_graph_caller() of mcount.S.
*/
/* old_parent_ra = *parent_ra_addr; */
return 1;
}
-static inline int kprobe_fault_handler(struct pt_regs *regs, int trapnr)
+int kprobe_fault_handler(struct pt_regs *regs, int trapnr)
{
struct kprobe *cur = kprobe_running();
struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
case 4:
mipsxx_pmu_write_control(3, 0);
mipspmu.write_counter(3, 0);
+ /* fall through */
case 3:
mipsxx_pmu_write_control(2, 0);
mipspmu.write_counter(2, 0);
+ /* fall through */
case 2:
mipsxx_pmu_write_control(1, 0);
mipspmu.write_counter(1, 0);
+ /* fall through */
case 1:
mipsxx_pmu_write_control(0, 0);
mipspmu.write_counter(0, 0);
+ /* fall through */
}
}
struct perf_sample_data data;
unsigned int counters = mipspmu.num_counters;
u64 counter;
- int handled = IRQ_NONE;
+ int n, handled = IRQ_NONE;
struct pt_regs *regs;
if (cpu_has_perf_cntr_intr_bit && !(read_c0_cause() & CAUSEF_PCI))
perf_sample_data_init(&data, 0, 0);
- switch (counters) {
-#define HANDLE_COUNTER(n) \
- case n + 1: \
- if (test_bit(n, cpuc->used_mask)) { \
- counter = mipspmu.read_counter(n); \
- if (counter & mipspmu.overflow) { \
- handle_associated_event(cpuc, n, &data, regs); \
- handled = IRQ_HANDLED; \
- } \
- }
- HANDLE_COUNTER(3)
- HANDLE_COUNTER(2)
- HANDLE_COUNTER(1)
- HANDLE_COUNTER(0)
+ for (n = counters - 1; n >= 0; n--) {
+ if (!test_bit(n, cpuc->used_mask))
+ continue;
+
+ counter = mipspmu.read_counter(n);
+ if (!(counter & mipspmu.overflow))
+ continue;
+
+ handle_associated_event(cpuc, n, &data, regs);
+ handled = IRQ_HANDLED;
}
#ifdef CONFIG_MIPS_PERF_SHARED_TC_COUNTERS
432 n32 fsmount sys_fsmount
433 n32 fspick sys_fspick
434 n32 pidfd_open sys_pidfd_open
+# 435 reserved for clone3
432 n64 fsmount sys_fsmount
433 n64 fspick sys_fspick
434 n64 pidfd_open sys_pidfd_open
+# 435 reserved for clone3
432 o32 fsmount sys_fsmount
433 o32 fspick sys_fspick
434 o32 pidfd_open sys_pidfd_open
+# 435 reserved for clone3
#include <irq.h>
/* register definitions - internal irqs */
-#define LTQ_ICU_IM0_ISR 0x0000
-#define LTQ_ICU_IM0_IER 0x0008
-#define LTQ_ICU_IM0_IOSR 0x0010
-#define LTQ_ICU_IM0_IRSR 0x0018
-#define LTQ_ICU_IM0_IMR 0x0020
-#define LTQ_ICU_IM1_ISR 0x0028
-#define LTQ_ICU_OFFSET (LTQ_ICU_IM1_ISR - LTQ_ICU_IM0_ISR)
+#define LTQ_ICU_ISR 0x0000
+#define LTQ_ICU_IER 0x0008
+#define LTQ_ICU_IOSR 0x0010
+#define LTQ_ICU_IRSR 0x0018
+#define LTQ_ICU_IMR 0x0020
+
+#define LTQ_ICU_IM_SIZE 0x28
/* register definitions - external irqs */
#define LTQ_EIU_EXIN_C 0x0000
*/
#define LTQ_ICU_EBU_IRQ 22
-#define ltq_icu_w32(m, x, y) ltq_w32((x), ltq_icu_membase[m] + (y))
-#define ltq_icu_r32(m, x) ltq_r32(ltq_icu_membase[m] + (x))
+#define ltq_icu_w32(vpe, m, x, y) \
+ ltq_w32((x), ltq_icu_membase[vpe] + m*LTQ_ICU_IM_SIZE + (y))
+
+#define ltq_icu_r32(vpe, m, x) \
+ ltq_r32(ltq_icu_membase[vpe] + m*LTQ_ICU_IM_SIZE + (x))
#define ltq_eiu_w32(x, y) ltq_w32((x), ltq_eiu_membase + (y))
#define ltq_eiu_r32(x) ltq_r32(ltq_eiu_membase + (x))
-/* our 2 ipi interrupts for VSMP */
-#define MIPS_CPU_IPI_RESCHED_IRQ 0
-#define MIPS_CPU_IPI_CALL_IRQ 1
-
/* we have a cascade of 8 irqs */
#define MIPS_CPU_IRQ_CASCADE 8
static int exin_avail;
static u32 ltq_eiu_irq[MAX_EIU];
-static void __iomem *ltq_icu_membase[MAX_IM];
+static void __iomem *ltq_icu_membase[NR_CPUS];
static void __iomem *ltq_eiu_membase;
static struct irq_domain *ltq_domain;
+static DEFINE_SPINLOCK(ltq_eiu_lock);
+static DEFINE_RAW_SPINLOCK(ltq_icu_lock);
static int ltq_perfcount_irq;
int ltq_eiu_get_irq(int exin)
void ltq_disable_irq(struct irq_data *d)
{
- u32 ier = LTQ_ICU_IM0_IER;
- int offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
- int im = offset / INT_NUM_IM_OFFSET;
+ unsigned long offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
+ unsigned long im = offset / INT_NUM_IM_OFFSET;
+ unsigned long flags;
+ int vpe;
offset %= INT_NUM_IM_OFFSET;
- ltq_icu_w32(im, ltq_icu_r32(im, ier) & ~BIT(offset), ier);
+
+ raw_spin_lock_irqsave(<q_icu_lock, flags);
+ for_each_present_cpu(vpe) {
+ ltq_icu_w32(vpe, im,
+ ltq_icu_r32(vpe, im, LTQ_ICU_IER) & ~BIT(offset),
+ LTQ_ICU_IER);
+ }
+ raw_spin_unlock_irqrestore(<q_icu_lock, flags);
}
void ltq_mask_and_ack_irq(struct irq_data *d)
{
- u32 ier = LTQ_ICU_IM0_IER;
- u32 isr = LTQ_ICU_IM0_ISR;
- int offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
- int im = offset / INT_NUM_IM_OFFSET;
+ unsigned long offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
+ unsigned long im = offset / INT_NUM_IM_OFFSET;
+ unsigned long flags;
+ int vpe;
offset %= INT_NUM_IM_OFFSET;
- ltq_icu_w32(im, ltq_icu_r32(im, ier) & ~BIT(offset), ier);
- ltq_icu_w32(im, BIT(offset), isr);
+
+ raw_spin_lock_irqsave(<q_icu_lock, flags);
+ for_each_present_cpu(vpe) {
+ ltq_icu_w32(vpe, im,
+ ltq_icu_r32(vpe, im, LTQ_ICU_IER) & ~BIT(offset),
+ LTQ_ICU_IER);
+ ltq_icu_w32(vpe, im, BIT(offset), LTQ_ICU_ISR);
+ }
+ raw_spin_unlock_irqrestore(<q_icu_lock, flags);
}
static void ltq_ack_irq(struct irq_data *d)
{
- u32 isr = LTQ_ICU_IM0_ISR;
- int offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
- int im = offset / INT_NUM_IM_OFFSET;
+ unsigned long offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
+ unsigned long im = offset / INT_NUM_IM_OFFSET;
+ unsigned long flags;
+ int vpe;
offset %= INT_NUM_IM_OFFSET;
- ltq_icu_w32(im, BIT(offset), isr);
+
+ raw_spin_lock_irqsave(<q_icu_lock, flags);
+ for_each_present_cpu(vpe) {
+ ltq_icu_w32(vpe, im, BIT(offset), LTQ_ICU_ISR);
+ }
+ raw_spin_unlock_irqrestore(<q_icu_lock, flags);
}
void ltq_enable_irq(struct irq_data *d)
{
- u32 ier = LTQ_ICU_IM0_IER;
- int offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
- int im = offset / INT_NUM_IM_OFFSET;
+ unsigned long offset = d->hwirq - MIPS_CPU_IRQ_CASCADE;
+ unsigned long im = offset / INT_NUM_IM_OFFSET;
+ unsigned long flags;
+ int vpe;
offset %= INT_NUM_IM_OFFSET;
- ltq_icu_w32(im, ltq_icu_r32(im, ier) | BIT(offset), ier);
+
+ vpe = cpumask_first(irq_data_get_effective_affinity_mask(d));
+
+ /* This shouldn't be even possible, maybe during CPU hotplug spam */
+ if (unlikely(vpe >= nr_cpu_ids))
+ vpe = smp_processor_id();
+
+ raw_spin_lock_irqsave(<q_icu_lock, flags);
+
+ ltq_icu_w32(vpe, im, ltq_icu_r32(vpe, im, LTQ_ICU_IER) | BIT(offset),
+ LTQ_ICU_IER);
+
+ raw_spin_unlock_irqrestore(<q_icu_lock, flags);
}
static int ltq_eiu_settype(struct irq_data *d, unsigned int type)
{
int i;
+ unsigned long flags;
for (i = 0; i < exin_avail; i++) {
if (d->hwirq == ltq_eiu_irq[i]) {
if (edge)
irq_set_handler(d->hwirq, handle_edge_irq);
- ltq_eiu_w32(ltq_eiu_r32(LTQ_EIU_EXIN_C) |
- (val << (i * 4)), LTQ_EIU_EXIN_C);
+ spin_lock_irqsave(<q_eiu_lock, flags);
+ ltq_eiu_w32((ltq_eiu_r32(LTQ_EIU_EXIN_C) &
+ (~(7 << (i * 4)))) | (val << (i * 4)),
+ LTQ_EIU_EXIN_C);
+ spin_unlock_irqrestore(<q_eiu_lock, flags);
}
}
}
}
+#if defined(CONFIG_SMP)
+static int ltq_icu_irq_set_affinity(struct irq_data *d,
+ const struct cpumask *cpumask, bool force)
+{
+ struct cpumask tmask;
+
+ if (!cpumask_and(&tmask, cpumask, cpu_online_mask))
+ return -EINVAL;
+
+ irq_data_update_effective_affinity(d, &tmask);
+
+ return IRQ_SET_MASK_OK;
+}
+#endif
+
static struct irq_chip ltq_irq_type = {
.name = "icu",
.irq_enable = ltq_enable_irq,
.irq_ack = ltq_ack_irq,
.irq_mask = ltq_disable_irq,
.irq_mask_ack = ltq_mask_and_ack_irq,
+#if defined(CONFIG_SMP)
+ .irq_set_affinity = ltq_icu_irq_set_affinity,
+#endif
};
static struct irq_chip ltq_eiu_type = {
.irq_mask = ltq_disable_irq,
.irq_mask_ack = ltq_mask_and_ack_irq,
.irq_set_type = ltq_eiu_settype,
+#if defined(CONFIG_SMP)
+ .irq_set_affinity = ltq_icu_irq_set_affinity,
+#endif
};
static void ltq_hw_irq_handler(struct irq_desc *desc)
{
- int module = irq_desc_get_irq(desc) - 2;
+ unsigned int module = irq_desc_get_irq(desc) - 2;
u32 irq;
- int hwirq;
+ irq_hw_number_t hwirq;
+ int vpe = smp_processor_id();
- irq = ltq_icu_r32(module, LTQ_ICU_IM0_IOSR);
+ irq = ltq_icu_r32(vpe, module, LTQ_ICU_IOSR);
if (irq == 0)
return;
static int icu_map(struct irq_domain *d, unsigned int irq, irq_hw_number_t hw)
{
struct irq_chip *chip = <q_irq_type;
+ struct irq_data *data;
int i;
if (hw < MIPS_CPU_IRQ_CASCADE)
if (hw == ltq_eiu_irq[i])
chip = <q_eiu_type;
+ data = irq_get_irq_data(irq);
+
+ irq_data_update_effective_affinity(data, cpumask_of(0));
+
irq_set_chip_and_handler(irq, chip, handle_level_irq);
return 0;
{
struct device_node *eiu_node;
struct resource res;
- int i, ret;
+ int i, ret, vpe;
- for (i = 0; i < MAX_IM; i++) {
- if (of_address_to_resource(node, i, &res))
- panic("Failed to get icu memory range");
+ /* load register regions of available ICUs */
+ for_each_possible_cpu(vpe) {
+ if (of_address_to_resource(node, vpe, &res))
+ panic("Failed to get icu%i memory range", vpe);
if (!request_mem_region(res.start, resource_size(&res),
res.name))
- pr_err("Failed to request icu memory");
+ pr_err("Failed to request icu%i memory\n", vpe);
- ltq_icu_membase[i] = ioremap_nocache(res.start,
+ ltq_icu_membase[vpe] = ioremap_nocache(res.start,
resource_size(&res));
- if (!ltq_icu_membase[i])
- panic("Failed to remap icu memory");
+
+ if (!ltq_icu_membase[vpe])
+ panic("Failed to remap icu%i memory", vpe);
}
/* turn off all irqs by default */
- for (i = 0; i < MAX_IM; i++) {
- /* make sure all irqs are turned off by default */
- ltq_icu_w32(i, 0, LTQ_ICU_IM0_IER);
- /* clear all possibly pending interrupts */
- ltq_icu_w32(i, ~0, LTQ_ICU_IM0_ISR);
+ for_each_possible_cpu(vpe) {
+ for (i = 0; i < MAX_IM; i++) {
+ /* make sure all irqs are turned off by default */
+ ltq_icu_w32(vpe, i, 0, LTQ_ICU_IER);
+
+ /* clear all possibly pending interrupts */
+ ltq_icu_w32(vpe, i, ~0, LTQ_ICU_ISR);
+ ltq_icu_w32(vpe, i, ~0, LTQ_ICU_IMR);
+
+ /* clear resend */
+ ltq_icu_w32(vpe, i, 0, LTQ_ICU_IRSR);
+ }
}
mips_cpu_irq_init();
return CP0_LEGACY_COMPARE_IRQ;
}
-static struct of_device_id __initdata of_irq_ids[] = {
+static const struct of_device_id of_irq_ids[] __initconst = {
{ .compatible = "lantiq,icu", .data = icu_of_init },
{},
};
*
* It's only valid to call this when @task is known to be blocked.
*/
-int syscall_get_nr(struct task_struct *task, struct pt_regs *regs)
+static inline int
+syscall_get_nr(struct task_struct *task, struct pt_regs *regs)
{
return regs->syscallno;
}
* system call instruction. This may not be the same as what the
* register state looked like at system call entry tracing.
*/
-void syscall_rollback(struct task_struct *task, struct pt_regs *regs)
+static inline void
+syscall_rollback(struct task_struct *task, struct pt_regs *regs)
{
regs->uregs[0] = regs->orig_r0;
}
* It's only valid to call this when @task is stopped for tracing on exit
* from a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
*/
-long syscall_get_error(struct task_struct *task, struct pt_regs *regs)
+static inline long
+syscall_get_error(struct task_struct *task, struct pt_regs *regs)
{
unsigned long error = regs->uregs[0];
return IS_ERR_VALUE(error) ? error : 0;
* It's only valid to call this when @task is stopped for tracing on exit
* from a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
*/
-long syscall_get_return_value(struct task_struct *task, struct pt_regs *regs)
+static inline long
+syscall_get_return_value(struct task_struct *task, struct pt_regs *regs)
{
return regs->uregs[0];
}
* It's only valid to call this when @task is stopped for tracing on exit
* from a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
*/
-void syscall_set_return_value(struct task_struct *task, struct pt_regs *regs,
- int error, long val)
+static inline void
+syscall_set_return_value(struct task_struct *task, struct pt_regs *regs,
+ int error, long val)
{
regs->uregs[0] = (long)error ? error : val;
}
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
*/
#define SYSCALL_MAX_ARGS 6
-void syscall_get_arguments(struct task_struct *task, struct pt_regs *regs,
- unsigned long *args)
+static inline void
+syscall_get_arguments(struct task_struct *task, struct pt_regs *regs,
+ unsigned long *args)
{
args[0] = regs->orig_r0;
args++;
* It's only valid to call this when @task is stopped for tracing on
* entry to a system call, due to %TIF_SYSCALL_TRACE or %TIF_SYSCALL_AUDIT.
*/
-void syscall_set_arguments(struct task_struct *task, struct pt_regs *regs,
- const unsigned long *args)
+static inline void
+syscall_set_arguments(struct task_struct *task, struct pt_regs *regs,
+ const unsigned long *args)
{
regs->orig_r0 = args[0];
args++;
machines, but will use only one CPU of a multiprocessor machine.
On a uniprocessor machine, the kernel will run faster if you say N.
- See also <file:Documentation/lockup-watchdogs.txt> and the SMP-HOWTO
+ See also <file:Documentation/admin-guide/lockup-watchdogs.rst> and the SMP-HOWTO
available at <http://www.tldp.org/docs.html#howto>.
If you don't know what to do here, say N.
args[0] = regs->gr[26];
}
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ unsigned long error = regs->gr[28];
+ return IS_ERR_VALUE(error) ? error : 0;
+}
+
static inline long syscall_get_return_value(struct task_struct *task,
struct pt_regs *regs)
{
#define __ARCH_WANT_SYS_FORK
#define __ARCH_WANT_SYS_VFORK
#define __ARCH_WANT_SYS_CLONE
+#define __ARCH_WANT_SYS_CLONE3
#define __ARCH_WANT_COMPAT_SYS_SENDFILE
#ifdef CONFIG_64BIT
.endm
fork_like clone
+fork_like clone3
fork_like fork
fork_like vfork
struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
struct kprobe *p = kprobe_running();
+ if (!p)
+ return 0;
+
if (regs->iaoq[0] != (unsigned long)p->ainsn.insn+4)
return 0;
if ((addr & (sizeof(unsigned long)-1)) ||
addr >= sizeof(struct pt_regs))
break;
+ if (addr == PT_IAOQ0 || addr == PT_IAOQ1) {
+ data |= 3; /* ensure userspace privilege */
+ }
if ((addr >= PT_GR1 && addr <= PT_GR31) ||
addr == PT_IAOQ0 || addr == PT_IAOQ1 ||
(addr >= PT_FR0 && addr <= PT_FR31 + 4) ||
static compat_ulong_t translate_usr_offset(compat_ulong_t offset)
{
- if (offset < 0)
- return sizeof(struct pt_regs);
- else if (offset <= 32*4) /* gr[0..31] */
- return offset * 2 + 4;
- else if (offset <= 32*4+32*8) /* gr[0..31] + fr[0..31] */
- return offset + 32*4;
- else if (offset < sizeof(struct pt_regs)/2 + 32*4)
- return offset * 2 + 4 - 32*8;
+ compat_ulong_t pos;
+
+ if (offset < 32*4) /* gr[0..31] */
+ pos = offset * 2 + 4;
+ else if (offset < 32*4+32*8) /* fr[0] ... fr[31] */
+ pos = (offset - 32*4) + PT_FR0;
+ else if (offset < sizeof(struct pt_regs)/2 + 32*4) /* sr[0] ... ipsw */
+ pos = (offset - 32*4 - 32*8) * 2 + PT_SR0 + 4;
else
- return sizeof(struct pt_regs);
+ pos = sizeof(struct pt_regs);
+
+ return pos;
}
long compat_arch_ptrace(struct task_struct *child, compat_long_t request,
addr = translate_usr_offset(addr);
if (addr >= sizeof(struct pt_regs))
break;
+ if (addr == PT_IAOQ0+4 || addr == PT_IAOQ1+4) {
+ data |= 3; /* ensure userspace privilege */
+ }
if (addr >= PT_FR0 && addr <= PT_FR31 + 4) {
/* Special case, fp regs are 64 bits anyway */
- *(__u64 *) ((char *) task_regs(child) + addr) = data;
+ *(__u32 *) ((char *) task_regs(child) + addr) = data;
ret = 0;
}
else if ((addr >= PT_GR1+4 && addr <= PT_GR31+4) ||
return;
case RI(iaoq[0]):
case RI(iaoq[1]):
- regs->iaoq[num - RI(iaoq[0])] = val;
+ /* set 2 lowest bits to ensure userspace privilege: */
+ regs->iaoq[num - RI(iaoq[0])] = val | 3;
return;
case RI(sar): regs->sar = val;
return;
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+435 common clone3 sys_clone3_wrapper
select ARCH_HAS_MMIOWB if PPC64
select ARCH_HAS_PHYS_TO_DMA
select ARCH_HAS_PMEM_API if PPC64
+ select ARCH_HAS_PTE_DEVMAP if PPC_BOOK3S_64
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_MEMBARRIER_CALLBACKS
select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE && PPC64
select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST
select ARCH_HAS_UACCESS_FLUSHCACHE if PPC64
select ARCH_HAS_UBSAN_SANITIZE_ALL
- select ARCH_HAS_ZONE_DEVICE if PPC_BOOK3S_64
select ARCH_HAVE_NMI_SAFE_CMPXCHG
select ARCH_KEEP_MEMBLOCK
select ARCH_MIGHT_HAVE_PC_PARPORT
#define _PAGE_SOFT_DIRTY _RPAGE_SW3 /* software: software dirty tracking */
#define _PAGE_SPECIAL _RPAGE_SW2 /* software: special page */
#define _PAGE_DEVMAP _RPAGE_SW1 /* software: ZONE_DEVICE page */
-#define __HAVE_ARCH_PTE_DEVMAP
/*
* Drivers request for cache inhibited pte mapping using _PAGE_NO_CACHE
regs->gpr[3] = regs->orig_gpr3;
}
+static inline long syscall_get_error(struct task_struct *task,
+ struct pt_regs *regs)
+{
+ /*
+ * If the system call failed,
+ * regs->gpr[3] contains a positive ERRORCODE.
+ */
+ return (regs->ccr & 0x10000000UL) ? -regs->gpr[3] : 0;
+}
+
static inline long syscall_get_return_value(struct task_struct *task,
struct pt_regs *regs)
{
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
+
#define MCL_CURRENT 0x2000 /* lock all currently mapped pages */
#define MCL_FUTURE 0x4000 /* lock all additions to address space */
#define MCL_ONFAULT 0x8000 /* lock all pages that are faulted in */
-#define MAP_POPULATE 0x8000 /* populate (prefault) pagetables */
-#define MAP_NONBLOCK 0x10000 /* do not block on IO */
-#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
-#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
-
/* Override any generic PKEY permission defines */
#define PKEY_DISABLE_EXECUTE 0x4
#undef PKEY_ACCESS_MASK
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+# 435 reserved for clone3
#include <linux/anon_inodes.h>
#include <linux/iommu.h>
#include <linux/file.h>
+#include <linux/mm.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
return tce_pages + ALIGN(stt_bytes, PAGE_SIZE) / PAGE_SIZE;
}
-static long kvmppc_account_memlimit(unsigned long stt_pages, bool inc)
-{
- long ret = 0;
-
- if (!current || !current->mm)
- return ret; /* process exited */
-
- down_write(¤t->mm->mmap_sem);
-
- if (inc) {
- unsigned long locked, lock_limit;
-
- locked = current->mm->locked_vm + stt_pages;
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
- if (locked > lock_limit && !capable(CAP_IPC_LOCK))
- ret = -ENOMEM;
- else
- current->mm->locked_vm += stt_pages;
- } else {
- if (WARN_ON_ONCE(stt_pages > current->mm->locked_vm))
- stt_pages = current->mm->locked_vm;
-
- current->mm->locked_vm -= stt_pages;
- }
-
- pr_debug("[%d] RLIMIT_MEMLOCK KVM %c%ld %ld/%ld%s\n", current->pid,
- inc ? '+' : '-',
- stt_pages << PAGE_SHIFT,
- current->mm->locked_vm << PAGE_SHIFT,
- rlimit(RLIMIT_MEMLOCK),
- ret ? " - exceeded" : "");
-
- up_write(¤t->mm->mmap_sem);
-
- return ret;
-}
-
static void kvm_spapr_tce_iommu_table_free(struct rcu_head *head)
{
struct kvmppc_spapr_tce_iommu_table *stit = container_of(head,
kvm_put_kvm(stt->kvm);
- kvmppc_account_memlimit(
+ account_locked_vm(current->mm,
kvmppc_stt_pages(kvmppc_tce_pages(stt->size)), false);
call_rcu(&stt->rcu, release_spapr_tce_table);
return -EINVAL;
npages = kvmppc_tce_pages(size);
- ret = kvmppc_account_memlimit(kvmppc_stt_pages(npages), true);
+ ret = account_locked_vm(current->mm, kvmppc_stt_pages(npages), true);
if (ret)
return ret;
kfree(stt);
fail_acct:
- kvmppc_account_memlimit(kvmppc_stt_pages(npages), false);
+ account_locked_vm(current->mm, kvmppc_stt_pages(npages), false);
return ret;
}
#include <linux/hugetlb.h>
#include <linux/swap.h>
#include <linux/sizes.h>
+#include <linux/mm.h>
#include <asm/mmu_context.h>
#include <asm/pte-walk.h>
#include <linux/mm_inline.h>
u64 dev_hpa; /* Device memory base address */
};
-static long mm_iommu_adjust_locked_vm(struct mm_struct *mm,
- unsigned long npages, bool incr)
-{
- long ret = 0, locked, lock_limit;
-
- if (!npages)
- return 0;
-
- down_write(&mm->mmap_sem);
-
- if (incr) {
- locked = mm->locked_vm + npages;
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
- if (locked > lock_limit && !capable(CAP_IPC_LOCK))
- ret = -ENOMEM;
- else
- mm->locked_vm += npages;
- } else {
- if (WARN_ON_ONCE(npages > mm->locked_vm))
- npages = mm->locked_vm;
- mm->locked_vm -= npages;
- }
-
- pr_debug("[%d] RLIMIT_MEMLOCK HASH64 %c%ld %ld/%ld\n",
- current ? current->pid : 0,
- incr ? '+' : '-',
- npages << PAGE_SHIFT,
- mm->locked_vm << PAGE_SHIFT,
- rlimit(RLIMIT_MEMLOCK));
- up_write(&mm->mmap_sem);
-
- return ret;
-}
-
bool mm_iommu_preregistered(struct mm_struct *mm)
{
return !list_empty(&mm->context.iommu_group_mem_list);
unsigned long entry, chunk;
if (dev_hpa == MM_IOMMU_TABLE_INVALID_HPA) {
- ret = mm_iommu_adjust_locked_vm(mm, entries, true);
+ ret = account_locked_vm(mm, entries, true);
if (ret)
return ret;
kfree(mem);
unlock_exit:
- mm_iommu_adjust_locked_vm(mm, locked_entries, false);
+ account_locked_vm(mm, locked_entries, false);
return ret;
}
unlock_exit:
mutex_unlock(&mem_list_mutex);
- mm_iommu_adjust_locked_vm(mm, unlock_entries, false);
+ account_locked_vm(mm, unlock_entries, false);
return ret;
}
return 0;
}
}
+
+int __init arch_ioremap_p4d_supported(void)
+{
+ return 0;
+}
#include <asm/debug.h>
#include <asm/kup.h>
-static inline bool notify_page_fault(struct pt_regs *regs)
-{
- bool ret = false;
-
-#ifdef CONFIG_KPROBES
- /* kprobe_running() needs smp_processor_id() */
- if (!user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, 11))
- ret = true;
- preempt_enable();
- }
-#endif /* CONFIG_KPROBES */
-
- if (unlikely(debugger_fault_handler(regs)))
- ret = true;
-
- return ret;
-}
-
/*
* Check whether the instruction inst is a store using
* an update addressing form which will update r1.
int is_write = page_fault_is_write(error_code);
vm_fault_t fault, major = 0;
bool must_retry = false;
+ bool kprobe_fault = kprobe_page_fault(regs, 11);
- if (notify_page_fault(regs))
+ if (unlikely(debugger_fault_handler(regs) || kprobe_fault))
return 0;
if (unlikely(page_fault_is_bad(error_code))) {
return __add_pages(nid, start_pfn, nr_pages, restrictions);
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
void __ref arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
pr_warn("Hash collision while resizing HPT\n");
}
#endif
-#endif /* CONFIG_MEMORY_HOTPLUG */
#ifndef CONFIG_NEED_MULTIPLE_NODES
void __init mem_topology_setup(void)
/* called with device_hotplug_lock held */
static bool memtrace_offline_pages(u32 nid, u64 start_pfn, u64 nr_pages)
{
- u64 end_pfn = start_pfn + nr_pages - 1;
+ const unsigned long start = PFN_PHYS(start_pfn);
+ const unsigned long size = PFN_PHYS(nr_pages);
- if (walk_memory_range(start_pfn, end_pfn, NULL,
- check_memblock_online))
+ if (walk_memory_blocks(start, size, NULL, check_memblock_online))
return false;
- walk_memory_range(start_pfn, end_pfn, (void *)MEM_GOING_OFFLINE,
- change_memblock_state);
+ walk_memory_blocks(start, size, (void *)MEM_GOING_OFFLINE,
+ change_memblock_state);
if (offline_pages(start_pfn, nr_pages)) {
- walk_memory_range(start_pfn, end_pfn, (void *)MEM_ONLINE,
- change_memblock_state);
+ walk_memory_blocks(start, size, (void *)MEM_ONLINE,
+ change_memblock_state);
return false;
}
- walk_memory_range(start_pfn, end_pfn, (void *)MEM_OFFLINE,
- change_memblock_state);
+ walk_memory_blocks(start, size, (void *)MEM_OFFLINE,
+ change_memblock_state);
return true;
*/
if (!memhp_auto_online) {
lock_device_hotplug();
- walk_memory_range(PFN_DOWN(ent->start),
- PFN_UP(ent->start + ent->size - 1),
- NULL, online_mem_block);
+ walk_memory_blocks(ent->start, ent->size, NULL,
+ online_mem_block);
unlock_device_hotplug();
}
select ARCH_HAS_MMIOWB
select HAVE_EBPF_JIT if 64BIT
select EDAC_SUPPORT
+ select ARCH_HAS_GIGANTIC_PAGE
+ select ARCH_WANT_HUGE_PMD_SHARE if 64BIT
config MMU
def_bool y
default 0xffffffff80000000 if 64BIT && MAXPHYSMEM_2GB
default 0xffffffe000000000 if 64BIT && MAXPHYSMEM_128GB
+config ARCH_WANT_GENERAL_HUGETLB
+ def_bool y
+
+config SYS_SUPPORTS_HUGETLBFS
+ def_bool y
+
config STACKTRACE_SUPPORT
def_bool y
default 3 if 64BIT
default 2
+source "arch/riscv/Kconfig.socs"
+
menu "Platform type"
choice
--- /dev/null
+menu "SoC selection"
+
+config SOC_SIFIVE
+ bool "SiFive SoCs"
+ select SERIAL_SIFIVE
+ select SERIAL_SIFIVE_CONSOLE
+ select CLK_SIFIVE
+ select CLK_SIFIVE_FU540_PRCI
+ select SIFIVE_PLIC
+ help
+ This enables support for SiFive SoC platform hardware.
+
+endmenu
# SPDX-License-Identifier: GPL-2.0
-dtb-y += hifive-unleashed-a00.dtb
+dtb-$(CONFIG_SOC_SIFIVE) += hifive-unleashed-a00.dtb
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
+CONFIG_NO_HZ_IDLE=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_CGROUPS=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_EXPERT=y
CONFIG_BPF_SYSCALL=y
+CONFIG_SOC_SIFIVE=y
CONFIG_SMP=y
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_SERIAL_EARLYCON_RISCV_SBI=y
-CONFIG_SERIAL_SIFIVE=y
-CONFIG_SERIAL_SIFIVE_CONSOLE=y
CONFIG_HVC_RISCV_SBI=y
# CONFIG_PTP_1588_CLOCK is not set
CONFIG_DRM=y
CONFIG_USB_STORAGE=y
CONFIG_USB_UAS=y
CONFIG_VIRTIO_MMIO=y
-CONFIG_CLK_SIFIVE=y
-CONFIG_CLK_SIFIVE_FU540_PRCI=y
-CONFIG_SIFIVE_PLIC=y
CONFIG_SPI_SIFIVE=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
+CONFIG_NO_HZ_IDLE=y
+CONFIG_HIGH_RES_TIMERS=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_CGROUPS=y
#ifndef _ASM_RISCV_CACHEFLUSH_H
#define _ASM_RISCV_CACHEFLUSH_H
-#include <asm-generic/cacheflush.h>
+#include <linux/mm.h>
-#undef flush_icache_range
-#undef flush_icache_user_range
-#undef flush_dcache_page
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
+/*
+ * The cache doesn't need to be flushed when TLB entries change when
+ * the cache is mapped to physical memory, not virtual memory
+ */
+static inline void flush_cache_all(void)
+{
+}
+
+static inline void flush_cache_mm(struct mm_struct *mm)
+{
+}
+
+static inline void flush_cache_dup_mm(struct mm_struct *mm)
+{
+}
+
+static inline void flush_cache_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end)
+{
+}
+
+static inline void flush_cache_page(struct vm_area_struct *vma,
+ unsigned long vmaddr,
+ unsigned long pfn)
+{
+}
+
+static inline void flush_dcache_mmap_lock(struct address_space *mapping)
+{
+}
+
+static inline void flush_dcache_mmap_unlock(struct address_space *mapping)
+{
+}
+
+static inline void flush_icache_page(struct vm_area_struct *vma,
+ struct page *page)
+{
+}
+
+static inline void flush_cache_vmap(unsigned long start, unsigned long end)
+{
+}
+
+static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
+{
+}
+
+#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
+ do { \
+ memcpy(dst, src, len); \
+ flush_icache_user_range(vma, page, vaddr, len); \
+ } while (0)
+#define copy_from_user_page(vma, page, vaddr, dst, src, len) \
+ memcpy(dst, src, len)
static inline void local_flush_icache_all(void)
{
*/
enum fixed_addresses {
FIX_HOLE,
+#define FIX_FDT_SIZE SZ_1M
+ FIX_FDT_END,
+ FIX_FDT = FIX_FDT_END + FIX_FDT_SIZE / PAGE_SIZE - 1,
+ FIX_PTE,
+ FIX_PMD,
FIX_EARLYCON_MEM_BASE,
__end_of_fixed_addresses
};
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_RISCV_HUGETLB_H
+#define _ASM_RISCV_HUGETLB_H
+
+#include <asm-generic/hugetlb.h>
+#include <asm/page.h>
+
+static inline int is_hugepage_only_range(struct mm_struct *mm,
+ unsigned long addr,
+ unsigned long len) {
+ return 0;
+}
+
+static inline void arch_clear_hugepage_flags(struct page *page)
+{
+}
+
+#endif /* _ASM_RISCV_HUGETLB_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_IMAGE_H
+#define __ASM_IMAGE_H
+
+#define RISCV_IMAGE_MAGIC "RISCV"
+
+#define RISCV_IMAGE_FLAG_BE_SHIFT 0
+#define RISCV_IMAGE_FLAG_BE_MASK 0x1
+
+#define RISCV_IMAGE_FLAG_LE 0
+#define RISCV_IMAGE_FLAG_BE 1
+
+#ifdef CONFIG_CPU_BIG_ENDIAN
+#error conversion of header fields to LE not yet implemented
+#else
+#define __HEAD_FLAG_BE RISCV_IMAGE_FLAG_LE
+#endif
+
+#define __HEAD_FLAG(field) (__HEAD_FLAG_##field << \
+ RISCV_IMAGE_FLAG_##field##_SHIFT)
+
+#define __HEAD_FLAGS (__HEAD_FLAG(BE))
+
+#define RISCV_HEADER_VERSION_MAJOR 0
+#define RISCV_HEADER_VERSION_MINOR 1
+
+#define RISCV_HEADER_VERSION (RISCV_HEADER_VERSION_MAJOR << 16 | \
+ RISCV_HEADER_VERSION_MINOR)
+
+#ifndef __ASSEMBLY__
+/**
+ * struct riscv_image_header - riscv kernel image header
+ * @code0: Executable code
+ * @code1: Executable code
+ * @text_offset: Image load offset (little endian)
+ * @image_size: Effective Image size (little endian)
+ * @flags: kernel flags (little endian)
+ * @version: version
+ * @res1: reserved
+ * @res2: reserved
+ * @magic: Magic number
+ * @res3: reserved (will be used for additional RISC-V specific
+ * header)
+ * @res4: reserved (will be used for PE COFF offset)
+ *
+ * The intention is for this header format to be shared between multiple
+ * architectures to avoid a proliferation of image header formats.
+ */
+
+struct riscv_image_header {
+ u32 code0;
+ u32 code1;
+ u64 text_offset;
+ u64 image_size;
+ u64 flags;
+ u32 version;
+ u32 res1;
+ u64 res2;
+ u64 magic;
+ u32 res3;
+ u32 res4;
+};
+#endif /* __ASSEMBLY__ */
+#endif /* __ASM_IMAGE_H */
#define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT)
#define PAGE_MASK (~(PAGE_SIZE - 1))
+#ifdef CONFIG_64BIT
+#define HUGE_MAX_HSTATE 2
+#else
+#define HUGE_MAX_HSTATE 1
+#endif
+#define HPAGE_SHIFT PMD_SHIFT
+#define HPAGE_SIZE (_AC(1, UL) << HPAGE_SHIFT)
+#define HPAGE_MASK (~(HPAGE_SIZE - 1))
+#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT)
+
/*
* PAGE_OFFSET -- the first address of the first page of memory.
* When not using MMU this corresponds to the first free page in
#include <asm-generic/memory_model.h>
#include <asm-generic/getorder.h>
-/* vDSO support */
-/* We do define AT_SYSINFO_EHDR but don't use the gate mechanism */
-#define __HAVE_ARCH_GATE_AREA
-
#endif /* _ASM_RISCV_PAGE_H */
return __pmd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
}
+static inline unsigned long _pmd_pfn(pmd_t pmd)
+{
+ return pmd_val(pmd) >> _PAGE_PFN_SHIFT;
+}
+
#define pmd_ERROR(e) \
pr_err("%s:%d: bad pmd %016lx.\n", __FILE__, __LINE__, pmd_val(e))
#define PAGE_KERNEL __pgprot(_PAGE_KERNEL)
#define PAGE_KERNEL_EXEC __pgprot(_PAGE_KERNEL | _PAGE_EXEC)
+#define PAGE_TABLE __pgprot(_PAGE_TABLE)
+
extern pgd_t swapper_pg_dir[];
/* MAP_PRIVATE permissions: xwr (copy-on-write) */
set_pmd(pmdp, __pmd(0));
}
-
static inline pgd_t pfn_pgd(unsigned long pfn, pgprot_t prot)
{
return __pgd((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot));
}
+static inline unsigned long _pgd_pfn(pgd_t pgd)
+{
+ return pgd_val(pgd) >> _PAGE_PFN_SHIFT;
+}
+
#define pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1))
/* Locate an entry in the page global directory */
return __pte(pte_val(pte) | _PAGE_SPECIAL);
}
+static inline pte_t pte_mkhuge(pte_t pte)
+{
+ return pte;
+}
+
/* Modify page protection bits */
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
#define kern_addr_valid(addr) (1) /* FIXME */
#endif
+extern void *dtb_early_va;
extern void setup_bootmem(void);
extern void paging_init(void);
#define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE)
/*
- * Task size is 0x40000000000 for RV64 or 0xb800000 for RV32.
+ * Task size is 0x4000000000 for RV64 or 0xb800000 for RV32.
* Note that PGDIR_SIZE must evenly divide TASK_SIZE.
*/
#ifdef CONFIG_64BIT
#include <asm/thread_info.h>
#include <asm/page.h>
#include <asm/csr.h>
+#include <asm/image.h>
__INIT
ENTRY(_start)
+ /*
+ * Image header expected by Linux boot-loaders. The image header data
+ * structure is described in asm/image.h.
+ * Do not modify it without modifying the structure and all bootloaders
+ * that expects this header format!!
+ */
+ /* jump to start kernel */
+ j _start_kernel
+ /* reserved */
+ .word 0
+ .balign 8
+#if __riscv_xlen == 64
+ /* Image load offset(2MB) from start of RAM */
+ .dword 0x200000
+#else
+ /* Image load offset(4MB) from start of RAM */
+ .dword 0x400000
+#endif
+ /* Effective size of kernel image */
+ .dword _end - _start
+ .dword __HEAD_FLAGS
+ .word RISCV_HEADER_VERSION
+ .word 0
+ .dword 0
+ .asciz RISCV_IMAGE_MAGIC
+ .word 0
+ .balign 4
+ .word 0
+
+.global _start_kernel
+_start_kernel:
/* Mask all interrupts */
csrw CSR_SIE, zero
csrw CSR_SIP, zero
/* Initialize page tables and relocate to virtual addresses */
la sp, init_thread_union + THREAD_SIZE
+ mv a0, s1
call setup_vm
+ la a0, early_pg_dir
call relocate
/* Restore C environment */
la sp, init_thread_union + THREAD_SIZE
/* Start the kernel */
- mv a0, s1
call parse_dtb
tail start_kernel
relocate:
/* Relocate return address */
li a1, PAGE_OFFSET
- la a0, _start
- sub a1, a1, a0
+ la a2, _start
+ sub a1, a1, a2
add ra, ra, a1
/* Point stvec to virtual address of intruction after satp write */
- la a0, 1f
- add a0, a0, a1
- csrw CSR_STVEC, a0
+ la a2, 1f
+ add a2, a2, a1
+ csrw CSR_STVEC, a2
/* Compute satp for kernel page tables, but don't load it yet */
- la a2, swapper_pg_dir
- srl a2, a2, PAGE_SHIFT
+ srl a2, a0, PAGE_SHIFT
li a1, SATP_MODE
or a2, a2, a1
fence
/* Enable virtual memory and relocate to virtual address */
+ la a0, swapper_pg_dir
call relocate
tail smp_callin
atomic_t hart_lottery;
unsigned long boot_cpu_hartid;
-void __init parse_dtb(phys_addr_t dtb_phys)
+void __init parse_dtb(void)
{
- void *dtb = __va(dtb_phys);
-
- if (early_init_dt_scan(dtb))
+ if (early_init_dt_scan(dtb_early_va))
return;
pr_err("No DTB passed to the kernel\n");
return "[vdso]";
return NULL;
}
-
-/*
- * Function stubs to prevent linker errors when AT_SYSINFO_EHDR is defined
- */
-
-int in_gate_area_no_mm(unsigned long addr)
-{
- return 0;
-}
-
-int in_gate_area(struct mm_struct *mm, unsigned long addr)
-{
- return 0;
-}
-
-struct vm_area_struct *get_gate_vma(struct mm_struct *mm)
-{
- return NULL;
-}
obj-y += cacheflush.o
obj-y += context.o
obj-y += sifive_l2_cache.o
+
+obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+#include <linux/hugetlb.h>
+#include <linux/err.h>
+
+int pud_huge(pud_t pud)
+{
+ return pud_present(pud) &&
+ (pud_val(pud) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC));
+}
+
+int pmd_huge(pmd_t pmd)
+{
+ return pmd_present(pmd) &&
+ (pmd_val(pmd) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC));
+}
+
+static __init int setup_hugepagesz(char *opt)
+{
+ unsigned long ps = memparse(opt, &opt);
+
+ if (ps == HPAGE_SIZE) {
+ hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
+ } else if (IS_ENABLED(CONFIG_64BIT) && ps == PUD_SIZE) {
+ hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+ } else {
+ hugetlb_bad_size();
+ pr_err("hugepagesz: Unsupported page size %lu M\n", ps >> 20);
+ return 0;
+ }
+
+ return 1;
+}
+__setup("hugepagesz=", setup_hugepagesz);
+
+#ifdef CONFIG_CONTIG_ALLOC
+static __init int gigantic_pages_init(void)
+{
+ /* With CONTIG_ALLOC, we can allocate gigantic pages at runtime */
+ if (IS_ENABLED(CONFIG_64BIT) && !size_to_hstate(1UL << PUD_SHIFT))
+ hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+ return 0;
+}
+arch_initcall(gigantic_pages_init);
+#endif
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2019 Western Digital Corporation or its affiliates.
*/
#include <linux/init.h>
__page_aligned_bss;
EXPORT_SYMBOL(empty_zero_page);
+extern char _start[];
+
static void __init zone_sizes_init(void)
{
unsigned long max_zone_pfns[MAX_NR_ZONES] = { 0, };
memset((void *)empty_zero_page, 0, PAGE_SIZE);
}
-void __init paging_init(void)
-{
- setup_zero_page();
- local_flush_tlb_all();
- zone_sizes_init();
-}
-
void __init mem_init(void)
{
#ifdef CONFIG_FLATMEM
initrd_start = 0;
initrd_end = 0;
}
-
-void __init free_initrd_mem(unsigned long start, unsigned long end)
-{
- free_reserved_area((void *)start, (void *)end, -1, "initrd");
-}
#endif /* CONFIG_BLK_DEV_INITRD */
void __init setup_bootmem(void)
{
struct memblock_region *reg;
phys_addr_t mem_size = 0;
+ phys_addr_t vmlinux_end = __pa(&_end);
+ phys_addr_t vmlinux_start = __pa(&_start);
/* Find the memory region containing the kernel */
for_each_memblock(memory, reg) {
- phys_addr_t vmlinux_end = __pa(_end);
phys_addr_t end = reg->base + reg->size;
if (reg->base <= vmlinux_end && vmlinux_end <= end) {
- /*
- * Reserve from the start of the region to the end of
- * the kernel
- */
- memblock_reserve(reg->base, vmlinux_end - reg->base);
mem_size = min(reg->size, (phys_addr_t)-PAGE_OFFSET);
/*
}
BUG_ON(mem_size == 0);
+ /* Reserve from the start of the kernel to the end of the kernel */
+ memblock_reserve(vmlinux_start, vmlinux_end - vmlinux_start);
+
set_max_mapnr(PFN_DOWN(mem_size));
max_low_pfn = PFN_DOWN(memblock_end_of_DRAM());
unsigned long pfn_base;
EXPORT_SYMBOL(pfn_base);
+void *dtb_early_va;
pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
-pgd_t trampoline_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE);
+pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss;
+pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss;
+static bool mmu_enabled;
-#ifndef __PAGETABLE_PMD_FOLDED
-#define NUM_SWAPPER_PMDS ((uintptr_t)-PAGE_OFFSET >> PGDIR_SHIFT)
-pmd_t swapper_pmd[PTRS_PER_PMD*((-PAGE_OFFSET)/PGDIR_SIZE)] __page_aligned_bss;
-pmd_t trampoline_pmd[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE);
-pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss;
-#endif
+#define MAX_EARLY_MAPPING_SIZE SZ_128M
-pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss;
+pgd_t early_pg_dir[PTRS_PER_PGD] __initdata __aligned(PAGE_SIZE);
void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot)
{
}
}
+static pte_t *__init get_pte_virt(phys_addr_t pa)
+{
+ if (mmu_enabled) {
+ clear_fixmap(FIX_PTE);
+ return (pte_t *)set_fixmap_offset(FIX_PTE, pa);
+ } else {
+ return (pte_t *)((uintptr_t)pa);
+ }
+}
+
+static phys_addr_t __init alloc_pte(uintptr_t va)
+{
+ /*
+ * We only create PMD or PGD early mappings so we
+ * should never reach here with MMU disabled.
+ */
+ BUG_ON(!mmu_enabled);
+
+ return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE);
+}
+
+static void __init create_pte_mapping(pte_t *ptep,
+ uintptr_t va, phys_addr_t pa,
+ phys_addr_t sz, pgprot_t prot)
+{
+ uintptr_t pte_index = pte_index(va);
+
+ BUG_ON(sz != PAGE_SIZE);
+
+ if (pte_none(ptep[pte_index]))
+ ptep[pte_index] = pfn_pte(PFN_DOWN(pa), prot);
+}
+
+#ifndef __PAGETABLE_PMD_FOLDED
+
+pmd_t trampoline_pmd[PTRS_PER_PMD] __page_aligned_bss;
+pmd_t fixmap_pmd[PTRS_PER_PMD] __page_aligned_bss;
+
+#if MAX_EARLY_MAPPING_SIZE < PGDIR_SIZE
+#define NUM_EARLY_PMDS 1UL
+#else
+#define NUM_EARLY_PMDS (1UL + MAX_EARLY_MAPPING_SIZE / PGDIR_SIZE)
+#endif
+pmd_t early_pmd[PTRS_PER_PMD * NUM_EARLY_PMDS] __initdata __aligned(PAGE_SIZE);
+
+static pmd_t *__init get_pmd_virt(phys_addr_t pa)
+{
+ if (mmu_enabled) {
+ clear_fixmap(FIX_PMD);
+ return (pmd_t *)set_fixmap_offset(FIX_PMD, pa);
+ } else {
+ return (pmd_t *)((uintptr_t)pa);
+ }
+}
+
+static phys_addr_t __init alloc_pmd(uintptr_t va)
+{
+ uintptr_t pmd_num;
+
+ if (mmu_enabled)
+ return memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE);
+
+ pmd_num = (va - PAGE_OFFSET) >> PGDIR_SHIFT;
+ BUG_ON(pmd_num >= NUM_EARLY_PMDS);
+ return (uintptr_t)&early_pmd[pmd_num * PTRS_PER_PMD];
+}
+
+static void __init create_pmd_mapping(pmd_t *pmdp,
+ uintptr_t va, phys_addr_t pa,
+ phys_addr_t sz, pgprot_t prot)
+{
+ pte_t *ptep;
+ phys_addr_t pte_phys;
+ uintptr_t pmd_index = pmd_index(va);
+
+ if (sz == PMD_SIZE) {
+ if (pmd_none(pmdp[pmd_index]))
+ pmdp[pmd_index] = pfn_pmd(PFN_DOWN(pa), prot);
+ return;
+ }
+
+ if (pmd_none(pmdp[pmd_index])) {
+ pte_phys = alloc_pte(va);
+ pmdp[pmd_index] = pfn_pmd(PFN_DOWN(pte_phys), PAGE_TABLE);
+ ptep = get_pte_virt(pte_phys);
+ memset(ptep, 0, PAGE_SIZE);
+ } else {
+ pte_phys = PFN_PHYS(_pmd_pfn(pmdp[pmd_index]));
+ ptep = get_pte_virt(pte_phys);
+ }
+
+ create_pte_mapping(ptep, va, pa, sz, prot);
+}
+
+#define pgd_next_t pmd_t
+#define alloc_pgd_next(__va) alloc_pmd(__va)
+#define get_pgd_next_virt(__pa) get_pmd_virt(__pa)
+#define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \
+ create_pmd_mapping(__nextp, __va, __pa, __sz, __prot)
+#define PTE_PARENT_SIZE PMD_SIZE
+#define fixmap_pgd_next fixmap_pmd
+#else
+#define pgd_next_t pte_t
+#define alloc_pgd_next(__va) alloc_pte(__va)
+#define get_pgd_next_virt(__pa) get_pte_virt(__pa)
+#define create_pgd_next_mapping(__nextp, __va, __pa, __sz, __prot) \
+ create_pte_mapping(__nextp, __va, __pa, __sz, __prot)
+#define PTE_PARENT_SIZE PGDIR_SIZE
+#define fixmap_pgd_next fixmap_pte
+#endif
+
+static void __init create_pgd_mapping(pgd_t *pgdp,
+ uintptr_t va, phys_addr_t pa,
+ phys_addr_t sz, pgprot_t prot)
+{
+ pgd_next_t *nextp;
+ phys_addr_t next_phys;
+ uintptr_t pgd_index = pgd_index(va);
+
+ if (sz == PGDIR_SIZE) {
+ if (pgd_val(pgdp[pgd_index]) == 0)
+ pgdp[pgd_index] = pfn_pgd(PFN_DOWN(pa), prot);
+ return;
+ }
+
+ if (pgd_val(pgdp[pgd_index]) == 0) {
+ next_phys = alloc_pgd_next(va);
+ pgdp[pgd_index] = pfn_pgd(PFN_DOWN(next_phys), PAGE_TABLE);
+ nextp = get_pgd_next_virt(next_phys);
+ memset(nextp, 0, PAGE_SIZE);
+ } else {
+ next_phys = PFN_PHYS(_pgd_pfn(pgdp[pgd_index]));
+ nextp = get_pgd_next_virt(next_phys);
+ }
+
+ create_pgd_next_mapping(nextp, va, pa, sz, prot);
+}
+
+static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size)
+{
+ uintptr_t map_size = PAGE_SIZE;
+
+ /* Upgrade to PMD/PGDIR mappings whenever possible */
+ if (!(base & (PTE_PARENT_SIZE - 1)) &&
+ !(size & (PTE_PARENT_SIZE - 1)))
+ map_size = PTE_PARENT_SIZE;
+
+ return map_size;
+}
+
/*
* setup_vm() is called from head.S with MMU-off.
*
"not use absolute addressing."
#endif
-asmlinkage void __init setup_vm(void)
+asmlinkage void __init setup_vm(uintptr_t dtb_pa)
{
- extern char _start;
- uintptr_t i;
- uintptr_t pa = (uintptr_t) &_start;
- pgprot_t prot = __pgprot(pgprot_val(PAGE_KERNEL) | _PAGE_EXEC);
+ uintptr_t va, end_va;
+ uintptr_t load_pa = (uintptr_t)(&_start);
+ uintptr_t load_sz = (uintptr_t)(&_end) - load_pa;
+ uintptr_t map_size = best_map_size(load_pa, MAX_EARLY_MAPPING_SIZE);
- va_pa_offset = PAGE_OFFSET - pa;
- pfn_base = PFN_DOWN(pa);
+ va_pa_offset = PAGE_OFFSET - load_pa;
+ pfn_base = PFN_DOWN(load_pa);
+
+ /*
+ * Enforce boot alignment requirements of RV32 and
+ * RV64 by only allowing PMD or PGD mappings.
+ */
+ BUG_ON(map_size == PAGE_SIZE);
/* Sanity check alignment and size */
BUG_ON((PAGE_OFFSET % PGDIR_SIZE) != 0);
- BUG_ON((pa % (PAGE_SIZE * PTRS_PER_PTE)) != 0);
+ BUG_ON((load_pa % map_size) != 0);
+ BUG_ON(load_sz > MAX_EARLY_MAPPING_SIZE);
+
+ /* Setup early PGD for fixmap */
+ create_pgd_mapping(early_pg_dir, FIXADDR_START,
+ (uintptr_t)fixmap_pgd_next, PGDIR_SIZE, PAGE_TABLE);
#ifndef __PAGETABLE_PMD_FOLDED
- trampoline_pg_dir[(PAGE_OFFSET >> PGDIR_SHIFT) % PTRS_PER_PGD] =
- pfn_pgd(PFN_DOWN((uintptr_t)trampoline_pmd),
- __pgprot(_PAGE_TABLE));
- trampoline_pmd[0] = pfn_pmd(PFN_DOWN(pa), prot);
+ /* Setup fixmap PMD */
+ create_pmd_mapping(fixmap_pmd, FIXADDR_START,
+ (uintptr_t)fixmap_pte, PMD_SIZE, PAGE_TABLE);
+ /* Setup trampoline PGD and PMD */
+ create_pgd_mapping(trampoline_pg_dir, PAGE_OFFSET,
+ (uintptr_t)trampoline_pmd, PGDIR_SIZE, PAGE_TABLE);
+ create_pmd_mapping(trampoline_pmd, PAGE_OFFSET,
+ load_pa, PMD_SIZE, PAGE_KERNEL_EXEC);
+#else
+ /* Setup trampoline PGD */
+ create_pgd_mapping(trampoline_pg_dir, PAGE_OFFSET,
+ load_pa, PGDIR_SIZE, PAGE_KERNEL_EXEC);
+#endif
- for (i = 0; i < (-PAGE_OFFSET)/PGDIR_SIZE; ++i) {
- size_t o = (PAGE_OFFSET >> PGDIR_SHIFT) % PTRS_PER_PGD + i;
+ /*
+ * Setup early PGD covering entire kernel which will allows
+ * us to reach paging_init(). We map all memory banks later
+ * in setup_vm_final() below.
+ */
+ end_va = PAGE_OFFSET + load_sz;
+ for (va = PAGE_OFFSET; va < end_va; va += map_size)
+ create_pgd_mapping(early_pg_dir, va,
+ load_pa + (va - PAGE_OFFSET),
+ map_size, PAGE_KERNEL_EXEC);
+
+ /* Create fixed mapping for early FDT parsing */
+ end_va = __fix_to_virt(FIX_FDT) + FIX_FDT_SIZE;
+ for (va = __fix_to_virt(FIX_FDT); va < end_va; va += PAGE_SIZE)
+ create_pte_mapping(fixmap_pte, va,
+ dtb_pa + (va - __fix_to_virt(FIX_FDT)),
+ PAGE_SIZE, PAGE_KERNEL);
+
+ /* Save pointer to DTB for early FDT parsing */
+ dtb_early_va = (void *)fix_to_virt(FIX_FDT) + (dtb_pa & ~PAGE_MASK);
+}
- swapper_pg_dir[o] =
- pfn_pgd(PFN_DOWN((uintptr_t)swapper_pmd) + i,
- __pgprot(_PAGE_TABLE));
- }
- for (i = 0; i < ARRAY_SIZE(swapper_pmd); i++)
- swapper_pmd[i] = pfn_pmd(PFN_DOWN(pa + i * PMD_SIZE), prot);
-
- swapper_pg_dir[(FIXADDR_START >> PGDIR_SHIFT) % PTRS_PER_PGD] =
- pfn_pgd(PFN_DOWN((uintptr_t)fixmap_pmd),
- __pgprot(_PAGE_TABLE));
- fixmap_pmd[(FIXADDR_START >> PMD_SHIFT) % PTRS_PER_PMD] =
- pfn_pmd(PFN_DOWN((uintptr_t)fixmap_pte),
- __pgprot(_PAGE_TABLE));
-#else
- trampoline_pg_dir[(PAGE_OFFSET >> PGDIR_SHIFT) % PTRS_PER_PGD] =
- pfn_pgd(PFN_DOWN(pa), prot);
+static void __init setup_vm_final(void)
+{
+ uintptr_t va, map_size;
+ phys_addr_t pa, start, end;
+ struct memblock_region *reg;
- for (i = 0; i < (-PAGE_OFFSET)/PGDIR_SIZE; ++i) {
- size_t o = (PAGE_OFFSET >> PGDIR_SHIFT) % PTRS_PER_PGD + i;
+ /* Set mmu_enabled flag */
+ mmu_enabled = true;
- swapper_pg_dir[o] =
- pfn_pgd(PFN_DOWN(pa + i * PGDIR_SIZE), prot);
+ /* Setup swapper PGD for fixmap */
+ create_pgd_mapping(swapper_pg_dir, FIXADDR_START,
+ __pa(fixmap_pgd_next),
+ PGDIR_SIZE, PAGE_TABLE);
+
+ /* Map all memory banks */
+ for_each_memblock(memory, reg) {
+ start = reg->base;
+ end = start + reg->size;
+
+ if (start >= end)
+ break;
+ if (memblock_is_nomap(reg))
+ continue;
+ if (start <= __pa(PAGE_OFFSET) &&
+ __pa(PAGE_OFFSET) < end)
+ start = __pa(PAGE_OFFSET);
+
+ map_size = best_map_size(start, end - start);
+ for (pa = start; pa < end; pa += map_size) {
+ va = (uintptr_t)__va(pa);
+ create_pgd_mapping(swapper_pg_dir, va, pa,
+ map_size, PAGE_KERNEL_EXEC);
+ }
}
- swapper_pg_dir[(FIXADDR_START >> PGDIR_SHIFT) % PTRS_PER_PGD] =
- pfn_pgd(PFN_DOWN((uintptr_t)fixmap_pte),
- __pgprot(_PAGE_TABLE));
-#endif
+ /* Clear fixmap PTE and PMD mappings */
+ clear_fixmap(FIX_PTE);
+ clear_fixmap(FIX_PMD);
+
+ /* Move to swapper page table */
+ csr_write(sptbr, PFN_DOWN(__pa(swapper_pg_dir)) | SATP_MODE);
+ local_flush_tlb_all();
+}
+
+void __init paging_init(void)
+{
+ setup_vm_final();
+ setup_zero_page();
+ zone_sizes_init();
}
static irqreturn_t l2_int_handler(int irq, void *device)
{
- unsigned int regval, add_h, add_l;
+ unsigned int add_h, add_l;
if (irq == g_irq[DIR_CORR]) {
add_h = readl(l2_base + SIFIVE_L2_DIRECCFIX_HIGH);
add_l = readl(l2_base + SIFIVE_L2_DIRECCFIX_LOW);
pr_err("L2CACHE: DirError @ 0x%08X.%08X\n", add_h, add_l);
- regval = readl(l2_base + SIFIVE_L2_DIRECCFIX_COUNT);
+ /* Reading this register clears the DirError interrupt sig */
+ readl(l2_base + SIFIVE_L2_DIRECCFIX_COUNT);
atomic_notifier_call_chain(&l2_err_chain, SIFIVE_L2_ERR_TYPE_CE,
"DirECCFix");
}
add_h = readl(l2_base + SIFIVE_L2_DATECCFIX_HIGH);
add_l = readl(l2_base + SIFIVE_L2_DATECCFIX_LOW);
pr_err("L2CACHE: DataError @ 0x%08X.%08X\n", add_h, add_l);
- regval = readl(l2_base + SIFIVE_L2_DATECCFIX_COUNT);
+ /* Reading this register clears the DataError interrupt sig */
+ readl(l2_base + SIFIVE_L2_DATECCFIX_COUNT);
atomic_notifier_call_chain(&l2_err_chain, SIFIVE_L2_ERR_TYPE_CE,
"DatECCFix");
}
add_h = readl(l2_base + SIFIVE_L2_DATECCFAIL_HIGH);
add_l = readl(l2_base + SIFIVE_L2_DATECCFAIL_LOW);
pr_err("L2CACHE: DataFail @ 0x%08X.%08X\n", add_h, add_l);
- regval = readl(l2_base + SIFIVE_L2_DATECCFAIL_COUNT);
+ /* Reading this register clears the DataFail interrupt sig */
+ readl(l2_base + SIFIVE_L2_DATECCFAIL_COUNT);
atomic_notifier_call_chain(&l2_err_chain, SIFIVE_L2_ERR_TYPE_UE,
"DatECCFail");
}
void __user *buffer, size_t *lenp, loff_t *ppos)
{
int timer_active = appldata_timer_active;
- int zero = 0;
- int one = 1;
int rc;
struct ctl_table ctl_entry = {
.procname = ctl->procname,
.data = &timer_active,
.maxlen = sizeof(int),
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
};
rc = proc_douintvec_minmax(&ctl_entry, write, buffer, lenp, ppos);
void __user *buffer, size_t *lenp, loff_t *ppos)
{
int interval = appldata_interval;
- int one = 1;
int rc;
struct ctl_table ctl_entry = {
.procname = ctl->procname,
.data = &interval,
.maxlen = sizeof(int),
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
};
rc = proc_dointvec_minmax(&ctl_entry, write, buffer, lenp, ppos);
struct list_head *lh;
int rc, found;
int active;
- int zero = 0;
- int one = 1;
struct ctl_table ctl_entry = {
.data = &active,
.maxlen = sizeof(int),
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
};
found = 0;
#ifndef __ASM_CTL_REG_H
#define __ASM_CTL_REG_H
-#include <linux/const.h>
-
-#define CR0_CLOCK_COMPARATOR_SIGN _BITUL(63 - 10)
-#define CR0_EMERGENCY_SIGNAL_SUBMASK _BITUL(63 - 49)
-#define CR0_EXTERNAL_CALL_SUBMASK _BITUL(63 - 50)
-#define CR0_CLOCK_COMPARATOR_SUBMASK _BITUL(63 - 52)
-#define CR0_CPU_TIMER_SUBMASK _BITUL(63 - 53)
-#define CR0_SERVICE_SIGNAL_SUBMASK _BITUL(63 - 54)
-#define CR0_UNUSED_56 _BITUL(63 - 56)
-#define CR0_INTERRUPT_KEY_SUBMASK _BITUL(63 - 57)
-#define CR0_MEASUREMENT_ALERT_SUBMASK _BITUL(63 - 58)
-
-#define CR2_GUARDED_STORAGE _BITUL(63 - 59)
-
-#define CR14_UNUSED_32 _BITUL(63 - 32)
-#define CR14_UNUSED_33 _BITUL(63 - 33)
-#define CR14_CHANNEL_REPORT_SUBMASK _BITUL(63 - 35)
-#define CR14_RECOVERY_SUBMASK _BITUL(63 - 36)
-#define CR14_DEGRADATION_SUBMASK _BITUL(63 - 37)
-#define CR14_EXTERNAL_DAMAGE_SUBMASK _BITUL(63 - 38)
-#define CR14_WARNING_SUBMASK _BITUL(63 - 39)
+#include <linux/bits.h>
+
+#define CR0_CLOCK_COMPARATOR_SIGN BIT(63 - 10)
+#define CR0_EMERGENCY_SIGNAL_SUBMASK BIT(63 - 49)
+#define CR0_EXTERNAL_CALL_SUBMASK BIT(63 - 50)
+#define CR0_CLOCK_COMPARATOR_SUBMASK BIT(63 - 52)
+#define CR0_CPU_TIMER_SUBMASK BIT(63 - 53)
+#define CR0_SERVICE_SIGNAL_SUBMASK BIT(63 - 54)
+#define CR0_UNUSED_56 BIT(63 - 56)
+#define CR0_INTERRUPT_KEY_SUBMASK BIT(63 - 57)
+#define CR0_MEASUREMENT_ALERT_SUBMASK BIT(63 - 58)
+
+#define CR2_GUARDED_STORAGE BIT(63 - 59)
+
+#define CR14_UNUSED_32 BIT(63 - 32)
+#define CR14_UNUSED_33 BIT(63 - 33)
+#define CR14_CHANNEL_REPORT_SUBMASK BIT(63 - 35)
+#define CR14_RECOVERY_SUBMASK BIT(63 - 36)
+#define CR14_DEGRADATION_SUBMASK BIT(63 - 37)
+#define CR14_EXTERNAL_DAMAGE_SUBMASK BIT(63 - 38)
+#define CR14_WARNING_SUBMASK BIT(63 - 39)
#ifndef __ASSEMBLY__
#ifndef _ASM_S390_NMI_H
#define _ASM_S390_NMI_H
-#include <linux/const.h>
+#include <linux/bits.h>
#include <linux/types.h>
#define MCIC_SUBCLASS_MASK (1ULL<<63 | 1ULL<<62 | 1ULL<<61 | \
1ULL<<55 | 1ULL<<54 | 1ULL<<53 | \
1ULL<<52 | 1ULL<<47 | 1ULL<<46 | \
1ULL<<45 | 1ULL<<44)
-#define MCCK_CODE_SYSTEM_DAMAGE _BITUL(63)
-#define MCCK_CODE_EXT_DAMAGE _BITUL(63 - 5)
-#define MCCK_CODE_CP _BITUL(63 - 9)
-#define MCCK_CODE_CPU_TIMER_VALID _BITUL(63 - 46)
-#define MCCK_CODE_PSW_MWP_VALID _BITUL(63 - 20)
-#define MCCK_CODE_PSW_IA_VALID _BITUL(63 - 23)
-#define MCCK_CODE_CR_VALID _BITUL(63 - 29)
-#define MCCK_CODE_GS_VALID _BITUL(63 - 36)
-#define MCCK_CODE_FC_VALID _BITUL(63 - 43)
+#define MCCK_CODE_SYSTEM_DAMAGE BIT(63)
+#define MCCK_CODE_EXT_DAMAGE BIT(63 - 5)
+#define MCCK_CODE_CP BIT(63 - 9)
+#define MCCK_CODE_CPU_TIMER_VALID BIT(63 - 46)
+#define MCCK_CODE_PSW_MWP_VALID BIT(63 - 20)
+#define MCCK_CODE_PSW_IA_VALID BIT(63 - 23)
+#define MCCK_CODE_CR_VALID BIT(63 - 29)
+#define MCCK_CODE_GS_VALID BIT(63 - 36)
+#define MCCK_CODE_FC_VALID BIT(63 - 43)
#ifndef __ASSEMBLY__
#ifndef __ASM_S390_PROCESSOR_H
#define __ASM_S390_PROCESSOR_H
-#include <linux/const.h>
+#include <linux/bits.h>
#define CIF_MCCK_PENDING 0 /* machine check handling is pending */
#define CIF_ASCE_PRIMARY 1 /* primary asce needs fixup / uaccess */
#define CIF_MCCK_GUEST 7 /* machine check happening in guest */
#define CIF_DEDICATED_CPU 8 /* this CPU is dedicated */
-#define _CIF_MCCK_PENDING _BITUL(CIF_MCCK_PENDING)
-#define _CIF_ASCE_PRIMARY _BITUL(CIF_ASCE_PRIMARY)
-#define _CIF_ASCE_SECONDARY _BITUL(CIF_ASCE_SECONDARY)
-#define _CIF_NOHZ_DELAY _BITUL(CIF_NOHZ_DELAY)
-#define _CIF_FPU _BITUL(CIF_FPU)
-#define _CIF_IGNORE_IRQ _BITUL(CIF_IGNORE_IRQ)
-#define _CIF_ENABLED_WAIT _BITUL(CIF_ENABLED_WAIT)
-#define _CIF_MCCK_GUEST _BITUL(CIF_MCCK_GUEST)
-#define _CIF_DEDICATED_CPU _BITUL(CIF_DEDICATED_CPU)
+#define _CIF_MCCK_PENDING BIT(CIF_MCCK_PENDING)
+#define _CIF_ASCE_PRIMARY BIT(CIF_ASCE_PRIMARY)
+#define _CIF_ASCE_SECONDARY BIT(CIF_ASCE_SECONDARY)
+#define _CIF_NOHZ_DELAY BIT(CIF_NOHZ_DELAY)
+#define _CIF_FPU BIT(CIF_FPU)
+#define _CIF_IGNORE_IRQ BIT(CIF_IGNORE_IRQ)
+#define _CIF_ENABLED_WAIT BIT(CIF_ENABLED_WAIT)
+#define _CIF_MCCK_GUEST BIT(CIF_MCCK_GUEST)
+#define _CIF_DEDICATED_CPU BIT(CIF_DEDICATED_CPU)
#ifndef __ASSEMBLY__
#ifndef _S390_PTRACE_H
#define _S390_PTRACE_H
-#include <linux/const.h>
+#include <linux/bits.h>
#include <uapi/asm/ptrace.h>
#define PIF_SYSCALL 0 /* inside a system call */
#define PIF_SYSCALL_RESTART 2 /* restart the current system call */
#define PIF_GUEST_FAULT 3 /* indicates program check in sie64a */
-#define _PIF_SYSCALL _BITUL(PIF_SYSCALL)
-#define _PIF_PER_TRAP _BITUL(PIF_PER_TRAP)
-#define _PIF_SYSCALL_RESTART _BITUL(PIF_SYSCALL_RESTART)
-#define _PIF_GUEST_FAULT _BITUL(PIF_GUEST_FAULT)
+#define _PIF_SYSCALL BIT(PIF_SYSCALL)
+#define _PIF_PER_TRAP BIT(PIF_PER_TRAP)
+#define _PIF_SYSCALL_RESTART BIT(PIF_SYSCALL_RESTART)
+#define _PIF_GUEST_FAULT BIT(PIF_GUEST_FAULT)
#ifndef __ASSEMBLY__
#ifndef _ASM_S390_SETUP_H
#define _ASM_S390_SETUP_H
-#include <linux/const.h>
+#include <linux/bits.h>
#include <uapi/asm/setup.h>
#define EP_OFFSET 0x10008
* Machine features detected in early.c
*/
-#define MACHINE_FLAG_VM _BITUL(0)
-#define MACHINE_FLAG_KVM _BITUL(1)
-#define MACHINE_FLAG_LPAR _BITUL(2)
-#define MACHINE_FLAG_DIAG9C _BITUL(3)
-#define MACHINE_FLAG_ESOP _BITUL(4)
-#define MACHINE_FLAG_IDTE _BITUL(5)
-#define MACHINE_FLAG_DIAG44 _BITUL(6)
-#define MACHINE_FLAG_EDAT1 _BITUL(7)
-#define MACHINE_FLAG_EDAT2 _BITUL(8)
-#define MACHINE_FLAG_TOPOLOGY _BITUL(10)
-#define MACHINE_FLAG_TE _BITUL(11)
-#define MACHINE_FLAG_TLB_LC _BITUL(12)
-#define MACHINE_FLAG_VX _BITUL(13)
-#define MACHINE_FLAG_TLB_GUEST _BITUL(14)
-#define MACHINE_FLAG_NX _BITUL(15)
-#define MACHINE_FLAG_GS _BITUL(16)
-#define MACHINE_FLAG_SCC _BITUL(17)
-
-#define LPP_MAGIC _BITUL(31)
+#define MACHINE_FLAG_VM BIT(0)
+#define MACHINE_FLAG_KVM BIT(1)
+#define MACHINE_FLAG_LPAR BIT(2)
+#define MACHINE_FLAG_DIAG9C BIT(3)
+#define MACHINE_FLAG_ESOP BIT(4)
+#define MACHINE_FLAG_IDTE BIT(5)
+#define MACHINE_FLAG_DIAG44 BIT(6)
+#define MACHINE_FLAG_EDAT1 BIT(7)
+#define MACHINE_FLAG_EDAT2 BIT(8)
+#define MACHINE_FLAG_TOPOLOGY BIT(10)
+#define MACHINE_FLAG_TE BIT(11)
+#define MACHINE_FLAG_TLB_LC BIT(12)
+#define MACHINE_FLAG_VX BIT(13)
+#define MACHINE_FLAG_TLB_GUEST BIT(14)
+#define MACHINE_FLAG_NX BIT(15)
+#define MACHINE_FLAG_GS BIT(16)
+#define MACHINE_FLAG_SCC BIT(17)
+
+#define LPP_MAGIC BIT(31)
#define LPP_PID_MASK _AC(0xffffffff, UL)
/* Offsets to entry points in kernel/head.S */
#ifndef _ASM_THREAD_INFO_H
#define _ASM_THREAD_INFO_H
-#include <linux/const.h>
+#include <linux/bits.h>
/*
* General size of kernel stacks
#define TIF_SECCOMP 26 /* secure computing */
#define TIF_SYSCALL_TRACEPOINT 27 /* syscall tracepoint instrumentation */
-#define _TIF_NOTIFY_RESUME _BITUL(TIF_NOTIFY_RESUME)
-#define _TIF_SIGPENDING _BITUL(TIF_SIGPENDING)
-#define _TIF_NEED_RESCHED _BITUL(TIF_NEED_RESCHED)
-#define _TIF_UPROBE _BITUL(TIF_UPROBE)
-#define _TIF_GUARDED_STORAGE _BITUL(TIF_GUARDED_STORAGE)
-#define _TIF_PATCH_PENDING _BITUL(TIF_PATCH_PENDING)
-#define _TIF_ISOLATE_BP _BITUL(TIF_ISOLATE_BP)
-#define _TIF_ISOLATE_BP_GUEST _BITUL(TIF_ISOLATE_BP_GUEST)
-
-#define _TIF_31BIT _BITUL(TIF_31BIT)
-#define _TIF_SINGLE_STEP _BITUL(TIF_SINGLE_STEP)
-
-#define _TIF_SYSCALL_TRACE _BITUL(TIF_SYSCALL_TRACE)
-#define _TIF_SYSCALL_AUDIT _BITUL(TIF_SYSCALL_AUDIT)
-#define _TIF_SECCOMP _BITUL(TIF_SECCOMP)
-#define _TIF_SYSCALL_TRACEPOINT _BITUL(TIF_SYSCALL_TRACEPOINT)
+#define _TIF_NOTIFY_RESUME BIT(TIF_NOTIFY_RESUME)
+#define _TIF_SIGPENDING BIT(TIF_SIGPENDING)
+#define _TIF_NEED_RESCHED BIT(TIF_NEED_RESCHED)
+#define _TIF_UPROBE BIT(TIF_UPROBE)
+#define _TIF_GUARDED_STORAGE BIT(TIF_GUARDED_STORAGE)
+#define _TIF_PATCH_PENDING BIT(TIF_PATCH_PENDING)
+#define _TIF_ISOLATE_BP BIT(TIF_ISOLATE_BP)
+#define _TIF_ISOLATE_BP_GUEST BIT(TIF_ISOLATE_BP_GUEST)
+
+#define _TIF_31BIT BIT(TIF_31BIT)
+#define _TIF_SINGLE_STEP BIT(TIF_SINGLE_STEP)
+
+#define _TIF_SYSCALL_TRACE BIT(TIF_SYSCALL_TRACE)
+#define _TIF_SYSCALL_AUDIT BIT(TIF_SYSCALL_AUDIT)
+#define _TIF_SECCOMP BIT(TIF_SECCOMP)
+#define _TIF_SYSCALL_TRACEPOINT BIT(TIF_SYSCALL_TRACEPOINT)
#endif /* _ASM_THREAD_INFO_H */
432 common fsmount sys_fsmount sys_fsmount
433 common fspick sys_fspick sys_fspick
434 common pidfd_open sys_pidfd_open sys_pidfd_open
+# 435 reserved for clone3
{
int enabled = topology_is_enabled();
int new_mode;
- int zero = 0;
- int one = 1;
int rc;
struct ctl_table ctl_entry = {
.procname = ctl->procname,
.data = &enabled,
.maxlen = sizeof(int),
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
};
rc = proc_douintvec_minmax(&ctl_entry, write, buffer, lenp, ppos);
}
early_initcall(fault_init);
-static inline int notify_page_fault(struct pt_regs *regs)
-{
- int ret = 0;
-
- /* kprobe_running() needs smp_processor_id() */
- if (kprobes_built_in() && !user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, 14))
- ret = 1;
- preempt_enable();
- }
- return ret;
-}
-
/*
* Find out which address space caused the exception.
*/
*/
clear_pt_regs_flag(regs, PIF_PER_TRAP);
- if (notify_page_fault(regs))
+ if (kprobe_page_fault(regs, 14))
return 0;
mm = tsk->mm;
unsigned long size_pages = PFN_DOWN(size);
int rc;
+ if (WARN_ON_ONCE(restrictions->altmap))
+ return -EINVAL;
+
rc = vmem_add_mapping(start, size);
if (rc)
return rc;
return rc;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
- /*
- * There is no hardware or firmware interface which could trigger a
- * hot memory remove on s390. So there is nothing that needs to be
- * implemented.
- */
- BUG();
+ unsigned long start_pfn = start >> PAGE_SHIFT;
+ unsigned long nr_pages = size >> PAGE_SHIFT;
+ struct zone *zone;
+
+ zone = page_zone(pfn_to_page(start_pfn));
+ __remove_pages(zone, start_pfn, nr_pages, altmap);
+ vmem_remove_mapping(start, size);
}
-#endif
#endif /* CONFIG_MEMORY_HOTPLUG */
to a memory address not used by the main kernel using
PHYSICAL_START.
- For more details see Documentation/kdump/kdump.rst
+ For more details see Documentation/admin-guide/kdump/kdump.rst
config KEXEC_JUMP
bool "kexec jump (EXPERIMENTAL)"
People using multiprocessor machines who say Y here should also say
Y to "Enhanced Real Time Clock Support", below.
- See also <file:Documentation/lockup-watchdogs.txt> and the SMP-HOWTO
+ See also <file:Documentation/admin-guide/lockup-watchdogs.rst> and the SMP-HOWTO
available at <http://www.tldp.org/docs.html#howto>.
If you don't know what to do here, say N.
bool
config SH_DEVICE_TREE
- bool "Board Described by Device Tree"
+ bool
select OF
select OF_EARLY_FLATTREE
select TIMER_OF
select COMMON_CLK
select GENERIC_CALIBRATE_DELAY
- help
- Select Board Described by Device Tree to build a kernel that
- does not hard-code any board-specific knowledge but instead uses
- a device tree blob provided by the boot-loader. You must enable
- drivers for any hardware you want to use separately. At this
- time, only boards based on the open-hardware J-Core processors
- have sufficient driver coverage to use this option; do not
- select it if you are using original SuperH hardware.
config SH_JCORE_SOC
bool "J-Core SoC"
- depends on SH_DEVICE_TREE && (CPU_SH2 || CPU_J2)
+ select SH_DEVICE_TREE
select CLKSRC_JCORE_PIT
select JCORE_AIC
- default y if CPU_J2
+ depends on CPU_J2
help
Select this option to include drivers core components of the
J-Core SoC, including interrupt controllers and timers.
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_CFI=y
CONFIG_INET=y
CONFIG_IP_PNP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_UNIX=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_FW_LOADER is not set
CONFIG_GDROM=y
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_IPV6 is not set
CONFIG_IRDA=y
CONFIG_SH_SIR=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_DEBUG_DRIVER=y
CONFIG_DEBUG_DEVRES=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_PCCARD=y
CONFIG_PM=y
CONFIG_APM_EMULATION=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
CONFIG_BLK_DEV_SD=y
CONFIG_ATA=y
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_KEYBOARD is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
CONFIG_ATALK=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_IDE=y
CONFIG_IP_PNP=y
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_SD=y
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_MTD=y
CONFIG_INET=y
CONFIG_IP_PNP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_IDE=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
CONFIG_BRIDGE=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_MTD=y
CONFIG_MTD_CFI=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
CONFIG_BRIDGE=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_SD=y
CONFIG_BINFMT_SHARED_FLAT=y
CONFIG_PM=y
CONFIG_CPU_IDLE=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_FW_LOADER is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_FW_LOADER is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_UNIX=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_SD=y
CONFIG_UNIX=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_IPV6=y
# CONFIG_INET6_XFRM_MODE_BEET is not set
CONFIG_NET_SCHED=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_PARPORT=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_FW_LOADER is not set
CONFIG_SYN_COOKIES=y
# CONFIG_INET_DIAG is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_NET_CLS_TCINDEX=y
CONFIG_NET_CLS_ROUTE4=y
CONFIG_NET_CLS_FW=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_NET_CLS_TCINDEX=y
CONFIG_NET_CLS_ROUTE4=y
CONFIG_NET_CLS_FW=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_UNIX=y
CONFIG_INET=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_SD=y
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_STANDALONE is not set
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_FW_LOADER is not set
# CONFIG_IPV6 is not set
CONFIG_NETWORK_SECMARK=y
CONFIG_NET_PKTGEN=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_RAM=y
CONFIG_CDROM_PKTCDVD=y
CONFIG_NET_CLS_TCINDEX=y
CONFIG_NET_CLS_ROUTE4=y
CONFIG_NET_CLS_U32=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_KEXEC_JUMP=y
CONFIG_HIBERNATION=y
CONFIG_CPU_IDLE=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_INPUT is not set
# CONFIG_SERIO is not set
CONFIG_IP_PNP_DHCP=y
CONFIG_IPV6=y
# CONFIG_WIRELESS is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLKDEVS=y
CONFIG_PM=y
CONFIG_HIBERNATION=y
CONFIG_CPU_IDLE=y
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
# CONFIG_INPUT is not set
# CONFIG_SERIO is not set
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_CAN_RAW=m
CONFIG_CAN_BCM=m
CONFIG_CAN_VCAN=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_SD=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_IPT=m
CONFIG_NET_ACT_PEDIT=m
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_FW_LOADER=m
CONFIG_CONNECTOR=m
CONFIG_MTD=m
CONFIG_MAC80211=y
CONFIG_MAC80211_RC_PID=y
# CONFIG_MAC80211_RC_MINSTREL is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
CONFIG_MTD_CFI=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IPV6 is not set
-CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_FW_LOADER is not set
CONFIG_MTD=y
CONFIG_MTD_BLOCK=y
struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
addr = (kprobe_opcode_t *) (args->regs->pc);
- if (val == DIE_TRAP) {
+ if (val == DIE_TRAP &&
+ args->trapnr == (BREAKPOINT_INSTRUCTION & 0xff)) {
if (!kprobe_running()) {
if (kprobe_handler(args->regs)) {
ret = NOTIFY_STOP;
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+# 435 reserved for clone3
#include <asm/tlbflush.h>
#include <asm/traps.h>
-static inline int notify_page_fault(struct pt_regs *regs, int trap)
-{
- int ret = 0;
-
- if (kprobes_built_in() && !user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, trap))
- ret = 1;
- preempt_enable();
- }
-
- return ret;
-}
-
static void
force_sig_info_fault(int si_signo, int si_code, unsigned long address)
{
if (unlikely(fault_in_kernel_space(address))) {
if (vmalloc_fault(address) >= 0)
return;
- if (notify_page_fault(regs, vec))
+ if (kprobe_page_fault(regs, vec))
return;
bad_area_nosemaphore(regs, error_code, address);
return;
}
- if (unlikely(notify_page_fault(regs, vec)))
+ if (unlikely(kprobe_page_fault(regs, vec)))
return;
/* Only enable interrupts if they were on before the fault */
EXPORT_SYMBOL_GPL(memory_add_physaddr_to_nid);
#endif
-#ifdef CONFIG_MEMORY_HOTREMOVE
void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
zone = page_zone(pfn_to_page(start_pfn));
__remove_pages(zone, start_pfn, nr_pages, altmap);
}
-#endif
#endif /* CONFIG_MEMORY_HOTPLUG */
Y to "Enhanced Real Time Clock Support", below. The "Advanced Power
Management" code will be disabled if you say Y here.
- See also <file:Documentation/lockup-watchdogs.txt> and the SMP-HOWTO
+ See also <file:Documentation/admin-guide/lockup-watchdogs.rst> and the SMP-HOWTO
available at <http://www.tldp.org/docs.html#howto>.
If you don't know what to do here, say N.
#define MCL_FUTURE 0x4000 /* lock all additions to address space */
#define MCL_ONFAULT 0x8000 /* lock all pages that are faulted in */
-#define MAP_POPULATE 0x8000 /* populate (prefault) pagetables */
-#define MAP_NONBLOCK 0x10000 /* do not block on IO */
-#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
-#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
-
-
#endif /* _UAPI__SPARC_MMAN_H__ */
432 common fsmount sys_fsmount
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
+# 435 reserved for clone3
int show_unhandled_signals = 1;
-static inline __kprobes int notify_page_fault(struct pt_regs *regs)
-{
- int ret = 0;
-
- /* kprobe_running() needs smp_processor_id() */
- if (kprobes_built_in() && !user_mode(regs)) {
- preempt_disable();
- if (kprobe_running() && kprobe_fault_handler(regs, 0))
- ret = 1;
- preempt_enable();
- }
- return ret;
-}
-
static void __kprobes unhandled_fault(unsigned long address,
struct task_struct *tsk,
struct pt_regs *regs)
fault_code = get_thread_fault_code();
- if (notify_page_fault(regs))
+ if (kprobe_page_fault(regs, 0))
goto exit_exception;
si_code = SEGV_MAPERR;
select ARCH_HAS_KCOV if X86_64
select ARCH_HAS_MEMBARRIER_SYNC_CORE
select ARCH_HAS_PMEM_API if X86_64
+ select ARCH_HAS_PTE_DEVMAP if X86_64
select ARCH_HAS_PTE_SPECIAL
select ARCH_HAS_REFCOUNT
select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64
select ARCH_HAS_STRICT_MODULE_RWX
select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
select ARCH_HAS_UBSAN_SANITIZE_ALL
- select ARCH_HAS_ZONE_DEVICE if X86_64
select ARCH_HAVE_NMI_SAFE_CMPXCHG
select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI
select ARCH_MIGHT_HAVE_PC_PARPORT
select ARCH_USE_QUEUED_SPINLOCKS
select ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH
select ARCH_WANTS_DYNAMIC_TASK_STRUCT
+ select ARCH_WANT_HUGE_PMD_SHARE
select ARCH_WANTS_THP_SWAP if X86_64
select BUILDTIME_EXTABLE_SORT
select CLKEVT_I8253
config ARCH_SUSPEND_POSSIBLE
def_bool y
-config ARCH_WANT_HUGE_PMD_SHARE
- def_bool y
-
config ARCH_WANT_GENERAL_HUGETLB
def_bool y
Management" code will be disabled if you say Y here.
See also <file:Documentation/x86/i386/IO-APIC.rst>,
- <file:Documentation/lockup-watchdogs.txt> and the SMP-HOWTO available at
+ <file:Documentation/admin-guide/lockup-watchdogs.rst> and the SMP-HOWTO available at
<http://www.tldp.org/docs.html#howto>.
If you don't know what to do here, say N.
This kernel feature allows a bzImage to be loaded directly
by EFI firmware without the use of a bootloader.
- See Documentation/efi-stub.txt for more information.
+ See Documentation/admin-guide/efi-stub.rst for more information.
config EFI_MIXED
bool "EFI mixed-mode support"
to a memory address not used by the main kernel or BIOS using
PHYSICAL_START, or it must be built as a relocatable image
(CONFIG_RELOCATABLE=y).
- For more details see Documentation/kdump/kdump.rst
+ For more details see Documentation/admin-guide/kdump/kdump.rst
config KEXEC_JUMP
bool "kexec jump"
the reserved region. In other words, it can be set based on
the "X" value as specified in the "crashkernel=YM@XM"
command line boot parameter passed to the panic-ed
- kernel. Please take a look at Documentation/kdump/kdump.rst
+ kernel. Please take a look at Documentation/admin-guide/kdump/kdump.rst
for more details about crash dumps.
Usage of bzImage for capturing the crash dump is recommended as
/* Register vsyscall32 into the ABI table */
#include <linux/sysctl.h>
-static const int zero;
-static const int one = 1;
-
static struct ctl_table abi_table2[] = {
{
.procname = "vsyscall32",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (int *)&zero,
- .extra2 = (int *)&one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{}
};
{
struct kernel_clone_args args = {
.flags = (clone_flags & ~CSIGNAL),
+ .pidfd = parent_tidptr,
.child_tid = child_tidptr,
.parent_tid = parent_tidptr,
.exit_signal = (clone_flags & CSIGNAL),
.tls = tls_val,
};
+ if (!legacy_clone_args_valid(&args))
+ return -EINVAL;
+
return _do_fork(&args);
}
{
return (unsigned int)virt_to_phys(address);
}
-#define isa_page_to_bus(page) ((unsigned int)page_to_phys(page))
#define isa_bus_to_virt phys_to_virt
/*
return boot_cpu_has(X86_FEATURE_PSE);
}
-#ifdef __HAVE_ARCH_PTE_DEVMAP
+#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
static inline int pmd_devmap(pmd_t pmd)
{
return !!(pmd_val(pmd) & _PAGE_DEVMAP);
return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
}
-#ifdef __HAVE_ARCH_PTE_DEVMAP
+#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
static inline int pte_devmap(pte_t a)
{
return (pte_flags(a) & _PAGE_DEVMAP) == _PAGE_DEVMAP;
#if defined(CONFIG_X86_64) || defined(CONFIG_X86_PAE)
#define _PAGE_NX (_AT(pteval_t, 1) << _PAGE_BIT_NX)
#define _PAGE_DEVMAP (_AT(u64, 1) << _PAGE_BIT_DEVMAP)
-#define __HAVE_ARCH_PTE_DEVMAP
#else
#define _PAGE_NX (_AT(pteval_t, 0))
#define _PAGE_DEVMAP (_AT(pteval_t, 0))
})
#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
-# define WARN_ON_IN_IRQ() WARN_ON_ONCE(!in_task())
+static inline bool pagefault_disabled(void);
+# define WARN_ON_IN_IRQ() \
+ WARN_ON_ONCE(!in_task() && !pagefault_disabled())
#else
# define WARN_ON_IN_IRQ()
#endif
ctx->kfc.magic = RDTGROUP_SUPER_MAGIC;
fc->fs_private = &ctx->kfc;
fc->ops = &rdt_fs_context_ops;
- if (fc->user_ns)
- put_user_ns(fc->user_ns);
+ put_user_ns(fc->user_ns);
fc->user_ns = get_user_ns(&init_user_ns);
fc->global = true;
return 0;
return add_break(rec->ip, old);
}
-static int add_breakpoints(struct dyn_ftrace *rec, int enable)
+static int add_breakpoints(struct dyn_ftrace *rec, bool enable)
{
unsigned long ftrace_addr;
int ret;
return add_update_code(ip, new);
}
-static int add_update(struct dyn_ftrace *rec, int enable)
+static int add_update(struct dyn_ftrace *rec, bool enable)
{
unsigned long ftrace_addr;
int ret;
return ftrace_write(ip, new, 1);
}
-static int finish_update(struct dyn_ftrace *rec, int enable)
+static int finish_update(struct dyn_ftrace *rec, bool enable)
{
unsigned long ftrace_addr;
int ret;
return ret;
}
-static unsigned int zero;
-static unsigned int one = 1;
static struct ctl_table itmt_kern_table[] = {
{
.procname = "sched_itmt_enabled",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = sched_itmt_update_handler,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{}
};
return 0;
}
-static nokprobe_inline int kprobes_fault(struct pt_regs *regs)
-{
- if (!kprobes_built_in())
- return 0;
- if (user_mode(regs))
- return 0;
- /*
- * To be potentially processing a kprobe fault and to be allowed to call
- * kprobe_running(), we have to be non-preemptible.
- */
- if (preemptible())
- return 0;
- if (!kprobe_running())
- return 0;
- return kprobe_fault_handler(regs, X86_TRAP_PF);
-}
-
/*
* Prefetch quirks:
*
return;
/* kprobes don't want to hook the spurious faults: */
- if (kprobes_fault(regs))
+ if (kprobe_page_fault(regs, X86_TRAP_PF))
return;
/*
mm = tsk->mm;
/* kprobes don't want to hook the spurious faults: */
- if (unlikely(kprobes_fault(regs)))
+ if (unlikely(kprobe_page_fault(regs, X86_TRAP_PF)))
return;
/*
return __add_pages(nid, start_pfn, nr_pages, restrictions);
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap)
{
__remove_pages(zone, start_pfn, nr_pages, altmap);
}
#endif
-#endif
int kernel_set_to_readonly __read_mostly;
remove_pagetable(start, end, false, altmap);
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
static void __meminit
kernel_physical_mapping_remove(unsigned long start, unsigned long end)
{
__remove_pages(zone, start_pfn, nr_pages, altmap);
kernel_physical_mapping_remove(start, start + size);
}
-#endif
#endif /* CONFIG_MEMORY_HOTPLUG */
static struct kcore_list kcore_vsyscall;
{
int err;
- if (boot_cpu_has(X86_FEATURE_PSE))
+ if (end - start < PAGES_PER_SECTION * sizeof(struct page))
+ err = vmemmap_populate_basepages(start, end, node);
+ else if (boot_cpu_has(X86_FEATURE_PSE))
err = vmemmap_populate_hugepages(start, end, node, altmap);
else if (altmap) {
pr_err_once("%s: no cpu support for altmap allocations\n",
}
EXPORT_SYMBOL(iounmap);
+int __init arch_ioremap_p4d_supported(void)
+{
+ return 0;
+}
+
int __init arch_ioremap_pud_supported(void)
{
#ifdef CONFIG_X86_64
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/dts-v1/;
+
+/ {
+ compatible = "cdns,xtensa-iss";
+ #address-cells = <1>;
+ #size-cells = <1>;
+ interrupt-parent = <&pic>;
+
+ chosen {
+ bootargs = "console=ttyS0,115200n8 debug";
+ };
+
+ memory@0 {
+ device_type = "memory";
+ reg = <0x00000000 0x80000000>;
+ };
+
+ cpus {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ cpu@0 {
+ compatible = "cdns,xtensa-cpu";
+ reg = <0>;
+ clocks = <&osc>;
+ };
+ };
+
+ clocks {
+ osc: osc {
+ #clock-cells = <0>;
+ compatible = "fixed-clock";
+ clock-frequency = <40000000>;
+ };
+ };
+
+ pic: pic {
+ compatible = "cdns,xtensa-pic";
+ /* one cell: internal irq number,
+ * two cells: second cell == 0: internal irq number
+ * second cell == 1: external irq number
+ */
+ #address-cells = <0>;
+ #interrupt-cells = <2>;
+ interrupt-controller;
+ };
+
+ pci {
+ compatible = "pci-host-ecam-generic";
+ device_type = "pci";
+ #address-cells = <3>;
+ #size-cells = <2>;
+ #interrupt-cells = <0x1>;
+
+ bus-range = <0x0 0x3f>;
+ reg = <0xc0000000 0x04000000>;
+
+ // BUS_ADDRESS(3) CPU_PHYSICAL(1) SIZE(2)
+ ranges = <0x01000000 0x0 0xc4000000 0xc4000000 0x0 0x04000000>,
+ <0x02000000 0x0 0xc8000000 0xc8000000 0x0 0x18000000>;
+
+ // PCI_DEVICE(3) INT#(1) CONTROLLER(PHANDLE) CONTROLLER_DATA(2)
+ interrupt-map = <
+ 0x0000 0x0 0x0 0x1 &pic 0x0 0x1
+ 0x0800 0x0 0x0 0x1 &pic 0x1 0x1
+ 0x1000 0x0 0x0 0x1 &pic 0x2 0x1
+ 0x1800 0x0 0x0 0x1 &pic 0x3 0x1
+ >;
+
+ interrupt-map-mask = <0x1800 0x0 0x0 0x7>;
+ };
+};
--- /dev/null
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+CONFIG_NO_HZ_IDLE=y
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_PREEMPT=y
+CONFIG_IRQ_TIME_ACCOUNTING=y
+CONFIG_BSD_PROCESS_ACCT=y
+CONFIG_MEMCG=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
+CONFIG_CGROUP_DEBUG=y
+CONFIG_NAMESPACES=y
+CONFIG_SCHED_AUTOGROUP=y
+CONFIG_RELAY=y
+CONFIG_BLK_DEV_INITRD=y
+CONFIG_EXPERT=y
+CONFIG_SYSCTL_SYSCALL=y
+CONFIG_KALLSYMS_ALL=y
+CONFIG_PERF_EVENTS=y
+CONFIG_XTENSA_VARIANT_DC233C=y
+CONFIG_XTENSA_UNALIGNED_USER=y
+CONFIG_VECTORS_OFFSET=0x00002000
+CONFIG_XTENSA_KSEG_512M=y
+CONFIG_HIGHMEM=y
+CONFIG_CMDLINE_BOOL=y
+CONFIG_CMDLINE="console=ttyS0,115200n8 ip=dhcp root=/dev/nfs rw debug memmap=0x80000000@0"
+CONFIG_USE_OF=y
+CONFIG_BUILTIN_DTB_SOURCE="virt"
+# CONFIG_PARSE_BOOTPARAM is not set
+CONFIG_JUMP_LABEL=y
+CONFIG_MODULES=y
+CONFIG_MODULE_UNLOAD=y
+# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
+# CONFIG_COMPACTION is not set
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_IP_PNP_BOOTP=y
+CONFIG_IP_PNP_RARP=y
+# CONFIG_IPV6 is not set
+CONFIG_NETFILTER=y
+# CONFIG_WIRELESS is not set
+CONFIG_PCI=y
+CONFIG_PCI_HOST_GENERIC=y
+CONFIG_UEVENT_HELPER=y
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+# CONFIG_STANDALONE is not set
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_BLK_DEV_RAM=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_SCSI=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_NETDEVICES=y
+CONFIG_VIRTIO_NET=y
+# CONFIG_ETHERNET is not set
+# CONFIG_WLAN is not set
+CONFIG_INPUT_MOUSEDEV=y
+CONFIG_INPUT_EVDEV=y
+# CONFIG_INPUT_KEYBOARD is not set
+# CONFIG_LEGACY_PTYS is not set
+CONFIG_HW_RANDOM=y
+CONFIG_HW_RANDOM_VIRTIO=y
+# CONFIG_HWMON is not set
+CONFIG_DRM=y
+CONFIG_DRM_VGEM=y
+CONFIG_DRM_VIRTIO_GPU=y
+CONFIG_FB_MODE_HELPERS=y
+# CONFIG_VGA_CONSOLE is not set
+CONFIG_FRAMEBUFFER_CONSOLE=y
+CONFIG_LOGO=y
+# CONFIG_USB_SUPPORT is not set
+CONFIG_VIRTIO_PCI=y
+CONFIG_VIRTIO_INPUT=y
+# CONFIG_IOMMU_SUPPORT is not set
+CONFIG_EXT3_FS=y
+CONFIG_FANOTIFY=y
+CONFIG_VFAT_FS=y
+CONFIG_PROC_KCORE=y
+CONFIG_TMPFS=y
+CONFIG_TMPFS_POSIX_ACL=y
+CONFIG_NFS_FS=y
+CONFIG_NFS_V4=y
+CONFIG_NFS_SWAP=y
+CONFIG_ROOT_NFS=y
+CONFIG_SUNRPC_DEBUG=y
+CONFIG_NLS_CODEPAGE_437=y
+CONFIG_NLS_ISO8859_1=y
+CONFIG_CRYPTO_ECHAINIV=y
+CONFIG_CRYPTO_DEFLATE=y
+CONFIG_CRYPTO_LZO=y
+CONFIG_CRYPTO_ANSI_CPRNG=y
+CONFIG_CRYPTO_DEV_VIRTIO=y
+CONFIG_FONTS=y
+CONFIG_PRINTK_TIME=y
+CONFIG_DYNAMIC_DEBUG=y
+CONFIG_DEBUG_INFO=y
+CONFIG_MAGIC_SYSRQ=y
+# CONFIG_SCHED_DEBUG is not set
+CONFIG_SCHEDSTATS=y
+CONFIG_DEBUG_RT_MUTEXES=y
+CONFIG_DEBUG_SPINLOCK=y
+CONFIG_DEBUG_MUTEXES=y
+CONFIG_DEBUG_ATOMIC_SLEEP=y
+CONFIG_STACKTRACE=y
+CONFIG_RCU_TRACE=y
+# CONFIG_FTRACE is not set
+# CONFIG_S32C1I_SELFTEST is not set
#endif
.endm
+#define XTENSA_STACK_ALIGNMENT 16
+
+#if defined(__XTENSA_WINDOWED_ABI__)
+#define XTENSA_FRAME_SIZE_RESERVE 16
+#define XTENSA_SPILL_STACK_RESERVE 32
+
+#define abi_entry(frame_size) \
+ entry sp, (XTENSA_FRAME_SIZE_RESERVE + \
+ (((frame_size) + XTENSA_STACK_ALIGNMENT - 1) & \
+ -XTENSA_STACK_ALIGNMENT))
+#define abi_entry_default abi_entry(0)
+
+#define abi_ret(frame_size) retw
+#define abi_ret_default retw
+
+#elif defined(__XTENSA_CALL0_ABI__)
+
+#define XTENSA_SPILL_STACK_RESERVE 0
+
+#define abi_entry(frame_size) __abi_entry (frame_size)
+
+ .macro __abi_entry frame_size
+ .ifgt \frame_size
+ addi sp, sp, -(((\frame_size) + XTENSA_STACK_ALIGNMENT - 1) & \
+ -XTENSA_STACK_ALIGNMENT)
+ .endif
+ .endm
+
+#define abi_entry_default
+
+#define abi_ret(frame_size) __abi_ret (frame_size)
+
+ .macro __abi_ret frame_size
+ .ifgt \frame_size
+ addi sp, sp, (((\frame_size) + XTENSA_STACK_ALIGNMENT - 1) & \
+ -XTENSA_STACK_ALIGNMENT)
+ .endif
+ ret
+ .endm
+
+#define abi_ret_default ret
+
+#else
+#error Unsupported Xtensa ABI
+#endif
+
#endif /* _XTENSA_ASMMACRO_H */
#if XCHAL_HAVE_S32C1I && (XCHAL_HW_MIN_VERSION >= XTENSA_HWVERSION_RC_2009_0)
/*
* We Have Atomic Operation Control (ATOMCTL) Register; Initialize it.
- * For details see Documentation/xtensa/atomctl.txt
+ * For details see Documentation/xtensa/atomctl.rst
*/
#if XCHAL_DCACHE_IS_COHERENT
movi a3, 0x25 /* For SMP/MX -- internal for writeback,
extern void platform_heartbeat (void);
/*
- * platform_pcibios_init is called to allow the platform to setup the pci bus.
- */
-extern void platform_pcibios_init (void);
-
-/*
- * platform_pcibios_fixup allows to modify the PCI configuration.
- */
-extern int platform_pcibios_fixup (void);
-
-/*
* platform_calibrate_ccount calibrates cpu clock freq (CONFIG_XTENSA_CALIBRATE)
*/
extern void platform_calibrate_ccount (void);
+++ /dev/null
-/*
- * include/asm-xtensa/types.h
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- *
- * Copyright (C) 2001 - 2005 Tensilica Inc.
- */
-#ifndef _XTENSA_TYPES_H
-#define _XTENSA_TYPES_H
-
-#include <uapi/asm/types.h>
-
-#ifndef __ASSEMBLY__
-/*
- * These aren't exported outside the kernel to avoid name space clashes
- */
-
-#define BITS_PER_LONG 32
-
-#endif
-#endif /* _XTENSA_TYPES_H */
#define MAP_STACK 0x40000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x80000 /* create a huge page mapping */
#define MAP_FIXED_NOREPLACE 0x100000 /* MAP_FIXED which doesn't unmap underlying mapping */
-#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED
-# define MAP_UNINITIALIZED 0x4000000 /* For anonymous mmap, memory could be
+#define MAP_UNINITIALIZED 0x4000000 /* For anonymous mmap, memory could be
* uninitialized */
-#else
-# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
-#endif
/*
* Flags for msync
ENTRY(coprocessor_flush)
- entry a1, 32
+ /* reserve 4 bytes on stack to save a0 */
+ abi_entry(4)
+
s32i a0, a1, 0
movi a0, .Lsave_cp_regs_jump_table
addx8 a3, a3, a0
beqz a3, 1f
callx0 a3
1: l32i a0, a1, 0
- retw
+
+ abi_ret(4)
ENDPROC(coprocessor_flush)
ENTRY(system_call)
- entry a1, 32
+ /* reserve 4 bytes on stack for function parameter */
+ abi_entry(4)
/* regs->syscall = regs->areg[2] */
s32i a6, a2, PT_AREG2
bnez a3, 1f
- retw
+ abi_ret(4)
1:
l32i a4, a1, 4
mov a6, a2
call4 do_syscall_trace_leave
s32i a3, a2, PT_SYSCALL
- retw
+ abi_ret(4)
ENDPROC(system_call)
ENTRY(_switch_to)
- entry a1, 48
+ abi_entry(XTENSA_SPILL_STACK_RESERVE)
mov a11, a3 # and 'next' (a3)
wsr a14, ps
rsync
- retw
+ abi_ret(XTENSA_SPILL_STACK_RESERVE)
ENDPROC(_switch_to)
*/
#include <linux/linkage.h>
+#include <asm/asmmacro.h>
#include <asm/ftrace.h>
/*
ENTRY(_mcount)
- entry a1, 16
+ abi_entry_default
movi a4, ftrace_trace_function
l32i a4, a4, 0
movi a3, ftrace_stub
bne a3, a4, 1f
- retw
+ abi_ret_default
1: xor a7, a2, a1
movi a3, 0x3fffffff
addi a6, a6, -MCOUNT_INSN_SIZE
callx4 a4
- retw
+ abi_ret_default
ENDPROC(_mcount)
ENTRY(ftrace_stub)
- entry a1, 16
- retw
+ abi_entry_default
+ abi_ret_default
ENDPROC(ftrace_stub)
#include <asm/pci-bridge.h>
#include <asm/platform.h>
-/* PCI Controller */
-
-
-/*
- * pcibios_alloc_controller
- * pcibios_enable_device
- * pcibios_fixups
- * pcibios_align_resource
- * pcibios_fixup_bus
- * pci_bus_add_device
- */
-
-static struct pci_controller *pci_ctrl_head;
-static struct pci_controller **pci_ctrl_tail = &pci_ctrl_head;
-
-static int pci_bus_count;
-
/*
* We need to avoid collisions with `mirrored' VGA ports
* and other strange ISA hardware, so we always want the
return start;
}
-static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
- struct list_head *resources)
-{
- struct resource *res;
- unsigned long io_offset;
- int i;
-
- io_offset = (unsigned long)pci_ctrl->io_space.base;
- res = &pci_ctrl->io_resource;
- if (!res->flags) {
- if (io_offset)
- pr_err("I/O resource not set for host bridge %d\n",
- pci_ctrl->index);
- res->start = 0;
- res->end = IO_SPACE_LIMIT;
- res->flags = IORESOURCE_IO;
- }
- res->start += io_offset;
- res->end += io_offset;
- pci_add_resource_offset(resources, res, io_offset);
-
- for (i = 0; i < 3; i++) {
- res = &pci_ctrl->mem_resources[i];
- if (!res->flags) {
- if (i > 0)
- continue;
- pr_err("Memory resource not set for host bridge %d\n",
- pci_ctrl->index);
- res->start = 0;
- res->end = ~0U;
- res->flags = IORESOURCE_MEM;
- }
- pci_add_resource(resources, res);
- }
-}
-
-static int __init pcibios_init(void)
-{
- struct pci_controller *pci_ctrl;
- struct list_head resources;
- struct pci_bus *bus;
- int next_busno = 0, ret;
-
- pr_info("PCI: Probing PCI hardware\n");
-
- /* Scan all of the recorded PCI controllers. */
- for (pci_ctrl = pci_ctrl_head; pci_ctrl; pci_ctrl = pci_ctrl->next) {
- pci_ctrl->last_busno = 0xff;
- INIT_LIST_HEAD(&resources);
- pci_controller_apertures(pci_ctrl, &resources);
- bus = pci_scan_root_bus(NULL, pci_ctrl->first_busno,
- pci_ctrl->ops, pci_ctrl, &resources);
- if (!bus)
- continue;
-
- pci_ctrl->bus = bus;
- pci_ctrl->last_busno = bus->busn_res.end;
- if (next_busno <= pci_ctrl->last_busno)
- next_busno = pci_ctrl->last_busno+1;
- }
- pci_bus_count = next_busno;
- ret = platform_pcibios_fixup();
- if (ret)
- return ret;
-
- for (pci_ctrl = pci_ctrl_head; pci_ctrl; pci_ctrl = pci_ctrl->next) {
- if (pci_ctrl->bus)
- pci_bus_add_devices(pci_ctrl->bus);
- }
-
- return 0;
-}
-
-subsys_initcall(pcibios_init);
-
void pcibios_fixup_bus(struct pci_bus *bus)
{
if (bus->parent) {
}
}
-void pcibios_set_master(struct pci_dev *dev)
-{
- /* No special bus mastering setup handling */
-}
-
-int pcibios_enable_device(struct pci_dev *dev, int mask)
-{
- u16 cmd, old_cmd;
- int idx;
- struct resource *r;
-
- pci_read_config_word(dev, PCI_COMMAND, &cmd);
- old_cmd = cmd;
- for (idx=0; idx<6; idx++) {
- r = &dev->resource[idx];
- if (!r->start && r->end) {
- pci_err(dev, "can't enable device: resource collisions\n");
- return -EINVAL;
- }
- if (r->flags & IORESOURCE_IO)
- cmd |= PCI_COMMAND_IO;
- if (r->flags & IORESOURCE_MEM)
- cmd |= PCI_COMMAND_MEMORY;
- }
- if (cmd != old_cmd) {
- pci_info(dev, "enabling device (%04x -> %04x)\n", old_cmd, cmd);
- pci_write_config_word(dev, PCI_COMMAND, cmd);
- }
-
- return 0;
-}
-
/*
* Platform support for /proc/bus/pci/X/Y mmap()s.
* -- paulus.
_F(void, power_off, (void), { while(1); });
_F(void, idle, (void), { __asm__ __volatile__ ("waiti 0" ::: "memory"); });
_F(void, heartbeat, (void), { });
-_F(int, pcibios_fixup, (void), { return 0; });
-_F(void, pcibios_init, (void), { });
#ifdef CONFIG_XTENSA_CALIBRATE_CCOUNT
_F(void, calibrate_ccount, (void),
conswitchp = &dummy_con;
# endif
#endif
-
-#ifdef CONFIG_PCI
- platform_pcibios_init();
-#endif
}
static DEFINE_PER_CPU(struct cpu, cpu_data);
* Experiments with Ethernet and SLIP connections show that buf
* is aligned on either a 2-byte or 4-byte boundary.
*/
- entry sp, 32
+ abi_entry_default
extui a5, a2, 0, 2
bnez a5, 8f /* branch if 2-byte aligned */
/* Fall-through on common case, 4-byte alignment */
ONES_ADD(a4, a6)
7:
mov a2, a4
- retw
+ abi_ret_default
/* uncommon case, buf is 2-byte aligned */
8:
ENTRY(csum_partial_copy_generic)
- entry sp, 32
+ abi_entry_default
mov a12, a3
mov a11, a4
or a10, a2, a3
ONES_ADD(a5, a9)
8:
mov a2, a5
- retw
+ abi_ret_default
5:
/* Control branch to here when either src or dst is odd. We
blt a12, a11, .Leloop
#endif
2:
- retw
+ abi_ret_default
11:
movi a2, -EFAULT
s32i a2, a7, 0 /* dst_err_ptr */
movi a2, 0
- retw
+ abi_ret_default
.previous
bne a3, a7, .Lnextbyte # continue loop if $a3:src != $a7:src_end
#endif /* !XCHAL_HAVE_LOOPS */
.Lbytecopydone:
- retw
+ abi_ret_default
/*
* Destination is unaligned
ENTRY(__memcpy)
WEAK(memcpy)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ dst, a3/ src, a4/ len
mov a5, a2 # copy dst so that a2 is return value
.Lcommon:
bbsi.l a4, 2, .L3
bbsi.l a4, 1, .L4
bbsi.l a4, 0, .L5
- retw
+ abi_ret_default
.L3:
# copy 4 bytes
l32i a6, a3, 0
addi a5, a5, 4
bbsi.l a4, 1, .L4
bbsi.l a4, 0, .L5
- retw
+ abi_ret_default
.L4:
# copy 2 bytes
l16ui a6, a3, 0
s16i a6, a5, 0
addi a5, a5, 2
bbsi.l a4, 0, .L5
- retw
+ abi_ret_default
.L5:
# copy 1 byte
l8ui a6, a3, 0
s8i a6, a5, 0
- retw
+ abi_ret_default
/*
* Destination is aligned, Source is unaligned
#endif
bbsi.l a4, 1, .L14
bbsi.l a4, 0, .L15
-.Ldone: retw
+.Ldone: abi_ret_default
.L14:
# copy 2 bytes
l8ui a6, a3, 0
s8i a7, a5, 1
addi a5, a5, 2
bbsi.l a4, 0, .L15
- retw
+ abi_ret_default
.L15:
# copy 1 byte
l8ui a6, a3, 0
s8i a6, a5, 0
- retw
+ abi_ret_default
ENDPROC(__memcpy)
ENTRY(bcopy)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2=src, a3=dst, a4=len
mov a5, a3
mov a3, a2
# $a3:src != $a7:src_start
#endif /* !XCHAL_HAVE_LOOPS */
.Lbackbytecopydone:
- retw
+ abi_ret_default
/*
* Destination is unaligned
ENTRY(__memmove)
WEAK(memmove)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ dst, a3/ src, a4/ len
mov a5, a2 # copy dst so that a2 is return value
.Lmovecommon:
bbsi.l a4, 2, .Lback3
bbsi.l a4, 1, .Lback4
bbsi.l a4, 0, .Lback5
- retw
+ abi_ret_default
.Lback3:
# copy 4 bytes
addi a3, a3, -4
s32i a6, a5, 0
bbsi.l a4, 1, .Lback4
bbsi.l a4, 0, .Lback5
- retw
+ abi_ret_default
.Lback4:
# copy 2 bytes
addi a3, a3, -2
addi a5, a5, -2
s16i a6, a5, 0
bbsi.l a4, 0, .Lback5
- retw
+ abi_ret_default
.Lback5:
# copy 1 byte
addi a3, a3, -1
l8ui a6, a3, 0
addi a5, a5, -1
s8i a6, a5, 0
- retw
+ abi_ret_default
/*
* Destination is aligned, Source is unaligned
bbsi.l a4, 1, .Lback14
bbsi.l a4, 0, .Lback15
.Lbackdone:
- retw
+ abi_ret_default
.Lback14:
# copy 2 bytes
addi a3, a3, -2
s8i a6, a5, 0
s8i a7, a5, 1
bbsi.l a4, 0, .Lback15
- retw
+ abi_ret_default
.Lback15:
# copy 1 byte
addi a3, a3, -1
addi a5, a5, -1
l8ui a6, a3, 0
s8i a6, a5, 0
- retw
+ abi_ret_default
ENDPROC(__memmove)
ENTRY(__memset)
WEAK(memset)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ dst, a3/ c, a4/ length
extui a3, a3, 0, 8 # mask to just 8 bits
slli a7, a3, 8 # duplicate character in all bytes of word
srli a7, a4, 4 # number of loop iterations with 16B
# per iteration
bnez a4, .Laligned
- retw
+ abi_ret_default
/*
* Destination is word-aligned.
EX(10f) s8i a3, a5, 0
.L5:
.Lret1:
- retw
+ abi_ret_default
/*
* Destination is unaligned
blt a5, a6, .Lbyteloop
#endif /* !XCHAL_HAVE_LOOPS */
.Lbytesetdone:
- retw
+ abi_ret_default
ENDPROC(__memset)
10:
movi a2, 0
- retw
+ abi_ret_default
.text
ENTRY(__strncpy_user)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ dst, a3/ src, a4/ len
mov a11, a2 # leave dst in return value register
beqz a4, .Lret # if len is zero
bnez a4, .Lsrcaligned # if len is nonzero
.Lret:
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
/*
* dst is word-aligned, src is word-aligned
.Lz3: # byte 3 is zero
addi a11, a11, 3 # advance dst pointer
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
.Lz0: # byte 0 is zero
#ifdef __XTENSA_EB__
movi a9, 0
#endif /* __XTENSA_EB__ */
EX(10f) s8i a9, a11, 0
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
.Lz1: # byte 1 is zero
#ifdef __XTENSA_EB__
extui a9, a9, 16, 16
EX(10f) s16i a9, a11, 0
addi a11, a11, 1 # advance dst pointer
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
.Lz2: # byte 2 is zero
#ifdef __XTENSA_EB__
extui a9, a9, 16, 16
EX(10f) s8i a9, a11, 2
addi a11, a11, 2 # advance dst pointer
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
.align 4 # 1 mod 4 alignment for LOOPNEZ
.byte 0 # (0 mod 4 alignment for LBEG)
.Lunalignedend:
sub a2, a11, a2 # compute strlen
- retw
+ abi_ret_default
ENDPROC(__strncpy_user)
10:
11:
movi a2, -EFAULT
- retw
+ abi_ret_default
.text
ENTRY(__strnlen_user)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ s, a3/ len
addi a4, a2, -4 # because we overincrement at the end;
# we compensate with load offsets of 4
addi a4, a4, 1 # advance string pointer
.L101:
sub a2, a4, a2 # compute length
- retw
+ abi_ret_default
# NOTE that in several places below, we point to the byte just after
# the zero byte in order to include the NULL terminator in the count.
.Lz0: # byte 0 is zero
addi a4, a4, 1 # point just beyond zero byte
sub a2, a4, a2 # subtract to get length
- retw
+ abi_ret_default
.Lz1: # byte 1 is zero
addi a4, a4, 1+1 # point just beyond zero byte
sub a2, a4, a2 # subtract to get length
- retw
+ abi_ret_default
.Lz2: # byte 2 is zero
addi a4, a4, 2+1 # point just beyond zero byte
sub a2, a4, a2 # subtract to get length
- retw
+ abi_ret_default
.L1mod2: # address is odd
EX(10f) l8ui a9, a4, 4 # get byte 0
# byte 3 is zero
addi a4, a4, 3+1 # point just beyond zero byte
sub a2, a4, a2 # subtract to get length
- retw
+ abi_ret_default
ENDPROC(__strnlen_user)
.align 4
10:
movi a2, 0
- retw
+ abi_ret_default
.text
ENTRY(__xtensa_copy_user)
- entry sp, 16 # minimal stack frame
+ abi_entry_default
# a2/ dst, a3/ src, a4/ len
mov a5, a2 # copy dst so that a2 is return value
mov a11, a4 # preserve original len for error case
__ssa8 a3 # set shift amount from byte offset
bnez a4, .Lsrcunaligned
movi a2, 0 # return success for len==0
- retw
+ abi_ret_default
/*
* Destination is unaligned
#endif /* !XCHAL_HAVE_LOOPS */
.Lbytecopydone:
movi a2, 0 # return success for len bytes copied
- retw
+ abi_ret_default
/*
* Destination and source are word-aligned.
EX(10f) s8i a6, a5, 0
.L5:
movi a2, 0 # return success for len bytes copied
- retw
+ abi_ret_default
/*
* Destination is aligned, Source is unaligned
EX(10f) s8i a6, a5, 0
.L15:
movi a2, 0 # return success for len bytes copied
- retw
+ abi_ret_default
ENDPROC(__xtensa_copy_user)
10:
sub a2, a5, a2 /* a2 <-- bytes copied */
sub a2, a11, a2 /* a2 <-- bytes not copied */
- retw
+ abi_ret_default
* If PHYS_OFFSET is zero reserve page at address 0:
* successfull allocations should never return NULL.
*/
- if (PHYS_OFFSET)
- memblock_reserve(0, PHYS_OFFSET);
- else
- memblock_reserve(0, 1);
+ memblock_reserve(0, PHYS_OFFSET ? PHYS_OFFSET : 1);
early_init_fdt_scan_reserved_mem();
ENTRY(clear_page)
- entry a1, 16
+ abi_entry_default
movi a3, 0
__loopi a2, a7, PAGE_SIZE, 32
s32i a3, a2, 28
__endla a2, a7, 32
- retw
+ abi_ret_default
ENDPROC(clear_page)
ENTRY(copy_page)
- entry a1, 16
+ abi_entry_default
__loopi a2, a4, PAGE_SIZE, 32
__endl a2, a4
- retw
+ abi_ret_default
ENDPROC(copy_page)
ENTRY(clear_page_alias)
- entry a1, 32
+ abi_entry_default
/* Skip setting up a temporary DTLB if not aliased low page. */
__endla a2, a7, 32
bnez a6, 1f
- retw
+ abi_ret_default
/* We need to invalidate the temporary idtlb entry, if any. */
1: idtlb a4
dsync
- retw
+ abi_ret_default
ENDPROC(clear_page_alias)
ENTRY(copy_page_alias)
- entry a1, 32
+ abi_entry_default
/* Skip setting up a temporary DTLB for destination if not aliased. */
bnez a6, 1f
bnez a7, 2f
- retw
+ abi_ret_default
1: addi a2, a2, -PAGE_SIZE
idtlb a2
dsync
bnez a7, 2f
- retw
+ abi_ret_default
2: addi a3, a3, -PAGE_SIZE+1
idtlb a3
dsync
- retw
+ abi_ret_default
ENDPROC(copy_page_alias)
ENTRY(__flush_invalidate_dcache_page_alias)
- entry sp, 16
+ abi_entry_default
movi a7, 0 # required for exception handler
addi a6, a3, (PAGE_KERNEL | _PAGE_HW_WRITE)
idtlb a4
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_invalidate_dcache_page_alias)
ENTRY(__invalidate_dcache_page_alias)
- entry sp, 16
+ abi_entry_default
movi a7, 0 # required for exception handler
addi a6, a3, (PAGE_KERNEL | _PAGE_HW_WRITE)
idtlb a4
dsync
- retw
+ abi_ret_default
ENDPROC(__invalidate_dcache_page_alias)
#endif
ENTRY(__invalidate_icache_page_alias)
- entry sp, 16
+ abi_entry_default
addi a6, a3, (PAGE_KERNEL_EXEC | _PAGE_HW_WRITE)
mov a4, a2
iitlb a4
isync
- retw
+ abi_ret_default
ENDPROC(__invalidate_icache_page_alias)
ENTRY(__invalidate_icache_page)
- entry sp, 16
+ abi_entry_default
___invalidate_icache_page a2 a3
isync
- retw
+ abi_ret_default
ENDPROC(__invalidate_icache_page)
ENTRY(__invalidate_dcache_page)
- entry sp, 16
+ abi_entry_default
___invalidate_dcache_page a2 a3
dsync
- retw
+ abi_ret_default
ENDPROC(__invalidate_dcache_page)
ENTRY(__flush_invalidate_dcache_page)
- entry sp, 16
+ abi_entry_default
___flush_invalidate_dcache_page a2 a3
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_invalidate_dcache_page)
ENTRY(__flush_dcache_page)
- entry sp, 16
+ abi_entry_default
___flush_dcache_page a2 a3
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_dcache_page)
ENTRY(__invalidate_icache_range)
- entry sp, 16
+ abi_entry_default
___invalidate_icache_range a2 a3 a4
isync
- retw
+ abi_ret_default
ENDPROC(__invalidate_icache_range)
ENTRY(__flush_invalidate_dcache_range)
- entry sp, 16
+ abi_entry_default
___flush_invalidate_dcache_range a2 a3 a4
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_invalidate_dcache_range)
ENTRY(__flush_dcache_range)
- entry sp, 16
+ abi_entry_default
___flush_dcache_range a2 a3 a4
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_dcache_range)
ENTRY(__invalidate_dcache_range)
- entry sp, 16
+ abi_entry_default
___invalidate_dcache_range a2 a3 a4
- retw
+ abi_ret_default
ENDPROC(__invalidate_dcache_range)
ENTRY(__invalidate_icache_all)
- entry sp, 16
+ abi_entry_default
___invalidate_icache_all a2 a3
isync
- retw
+ abi_ret_default
ENDPROC(__invalidate_icache_all)
ENTRY(__flush_invalidate_dcache_all)
- entry sp, 16
+ abi_entry_default
___flush_invalidate_dcache_all a2 a3
dsync
- retw
+ abi_ret_default
ENDPROC(__flush_invalidate_dcache_all)
ENTRY(__invalidate_dcache_all)
- entry sp, 16
+ abi_entry_default
___invalidate_dcache_all a2 a3
dsync
- retw
+ abi_ret_default
ENDPROC(__invalidate_dcache_all)
one needs to mount and use blkio cgroup controller for creating
cgroups and specifying per device IO rate policies.
- See Documentation/cgroup-v1/blkio-controller.rst for more information.
+ See Documentation/admin-guide/cgroup-v1/blkio-controller.rst for more information.
config BLK_DEV_THROTTLING_LOW
bool "Block throttling .low limit interface support (EXPERIMENTAL)"
which don't otherwise have any standardized method for listing the
partitions on a block device.
- See Documentation/block/cmdline-partition.txt for more information.
+ See Documentation/block/cmdline-partition.rst for more information.
config BLK_WBT
bool "Enable support for block device writeback throttling"
regardless of the device parameters and with any workload. It
also guarantees a low latency to interactive and soft
real-time applications. Details in
- Documentation/block/bfq-iosched.txt
+ Documentation/block/bfq-iosched.rst
config BFQ_GROUP_IOSCHED
bool "BFQ hierarchical scheduling support"
* low-latency capabilities. BFQ also supports full hierarchical
* scheduling through cgroups. Next paragraphs provide an introduction
* on BFQ inner workings. Details on BFQ benefits, usage and
- * limitations can be found in Documentation/block/bfq-iosched.txt.
+ * limitations can be found in Documentation/block/bfq-iosched.rst.
*
* BFQ is a proportional-share storage-I/O scheduling algorithm based
* on the slice-by-slice service scheme of CFQ. But BFQ assigns
* send/receive integrity metadata it must use this function to register
* the capability with the block layer. The template is a blk_integrity
* struct with values appropriate for the underlying hardware. See
- * Documentation/block/data-integrity.txt.
+ * Documentation/block/data-integrity.rst.
*/
void blk_integrity_register(struct gendisk *disk, struct blk_integrity *template)
{
*
* ioprio_set(PRIO_PROCESS, pid, prio);
*
- * See also Documentation/block/ioprio.txt
+ * See also Documentation/block/ioprio.rst
*
*/
#include <linux/gfp.h>
#include "blk-mq-sched.h"
/*
- * See Documentation/block/deadline-iosched.txt
+ * See Documentation/block/deadline-iosched.rst
*/
static const int read_expire = HZ / 2; /* max time before a read is submitted. */
static const int write_expire = 5 * HZ; /* ditto for writes, these limits are SOFT! */
Normal partitions are now called Basic Disks under Windows 2000, XP,
and Vista.
- For a fuller description read <file:Documentation/ldm.txt>.
+ For a fuller description read <file:Documentation/admin-guide/ldm.rst>.
If unsure, say N.
*
* The format for the command line is just like mtdparts.
*
- * For further information, see "Documentation/block/cmdline-partition.txt"
+ * For further information, see "Documentation/block/cmdline-partition.rst"
*
*/
return 0;
}
-static unsigned long acpi_meminfo_start_pfn(struct acpi_memory_info *info)
-{
- return PFN_DOWN(info->start_addr);
-}
-
-static unsigned long acpi_meminfo_end_pfn(struct acpi_memory_info *info)
-{
- return PFN_UP(info->start_addr + info->length-1);
-}
-
static int acpi_bind_memblk(struct memory_block *mem, void *arg)
{
return acpi_bind_one(&mem->dev, arg);
static int acpi_bind_memory_blocks(struct acpi_memory_info *info,
struct acpi_device *adev)
{
- return walk_memory_range(acpi_meminfo_start_pfn(info),
- acpi_meminfo_end_pfn(info), adev,
- acpi_bind_memblk);
+ return walk_memory_blocks(info->start_addr, info->length, adev,
+ acpi_bind_memblk);
}
static int acpi_unbind_memblk(struct memory_block *mem, void *arg)
static void acpi_unbind_memory_blocks(struct acpi_memory_info *info)
{
- walk_memory_range(acpi_meminfo_start_pfn(info),
- acpi_meminfo_end_pfn(info), NULL, acpi_unbind_memblk);
+ walk_memory_blocks(info->start_addr, info->length, NULL,
+ acpi_unbind_memblk);
}
static int acpi_memory_enable_device(struct acpi_memory_device *mem_device)
MODULE_PARM_DESC(report_key_events,
"0: none, 1: output changes, 2: brightness changes, 3: all");
+static int hw_changes_brightness = -1;
+module_param(hw_changes_brightness, int, 0644);
+MODULE_PARM_DESC(hw_changes_brightness,
+ "Set this to 1 on buggy hw which changes the brightness itself when "
+ "a hotkey is pressed: -1: auto, 0: normal 1: hw-changes-brightness");
+
/*
* Whether the struct acpi_video_device_attrib::device_id_scheme bit should be
* assumed even if not actually set.
return 0;
}
+static int video_hw_changes_brightness(
+ const struct dmi_system_id *d)
+{
+ if (hw_changes_brightness == -1)
+ hw_changes_brightness = 1;
+ return 0;
+}
+
static const struct dmi_system_id video_dmi_table[] = {
/*
* Broken _BQC workaround http://bugzilla.kernel.org/show_bug.cgi?id=13121
DMI_MATCH(DMI_PRODUCT_NAME, "Vostro V131"),
},
},
+ /*
+ * Some machines change the brightness themselves when a brightness
+ * hotkey gets pressed, despite us telling them not to. In this case
+ * acpi_video_device_notify() should only call backlight_force_update(
+ * BACKLIGHT_UPDATE_HOTKEY) and not do anything else.
+ */
+ {
+ /* https://bugzilla.kernel.org/show_bug.cgi?id=204077 */
+ .callback = video_hw_changes_brightness,
+ .ident = "Packard Bell EasyNote MZ35",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Packard Bell"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "EasyNote MZ35"),
+ },
+ },
{}
};
bus = video_device->video;
input = bus->input;
+ if (hw_changes_brightness > 0) {
+ if (video_device->backlight)
+ backlight_force_update(video_device->backlight,
+ BACKLIGHT_UPDATE_HOTKEY);
+ acpi_notifier_call_chain(device, event, 0);
+ return;
+ }
+
switch (event) {
case ACPI_VIDEO_NOTIFY_CYCLE_BRIGHTNESS: /* Cycle brightness */
brightness_switch_event(video_device, event);
return_ACPI_STATUS(status);
}
- /* Complete the initialization/resolution of package objects */
+ /* Complete the initialization/resolution of new objects */
- status = acpi_ns_walk_namespace(ACPI_TYPE_PACKAGE, ACPI_ROOT_OBJECT,
- ACPI_UINT32_MAX, 0,
- acpi_ns_init_one_package, NULL, NULL,
- NULL);
+ acpi_ex_exit_interpreter();
+ acpi_ns_initialize_objects();
+ acpi_ex_enter_interpreter();
/* Parameter Data (optional) */
return_ACPI_STATUS(status);
}
- /* Complete the initialization/resolution of package objects */
+ /* Complete the initialization/resolution of new objects */
- status = acpi_ns_walk_namespace(ACPI_TYPE_PACKAGE, ACPI_ROOT_OBJECT,
- ACPI_UINT32_MAX, 0,
- acpi_ns_init_one_package, NULL, NULL,
- NULL);
+ acpi_ex_exit_interpreter();
+ acpi_ns_initialize_objects();
+ acpi_ex_enter_interpreter();
/* Store the ddb_handle into the Target operand */
status = acpi_tb_install_and_load_table(ACPI_PTR_TO_PHYSADDR(table),
ACPI_TABLE_ORIGIN_EXTERNAL_VIRTUAL,
FALSE, &table_index);
-
if (ACPI_SUCCESS(status)) {
- /* Complete the initialization/resolution of package objects */
- status = acpi_ns_walk_namespace(ACPI_TYPE_PACKAGE,
- ACPI_ROOT_OBJECT,
- ACPI_UINT32_MAX, 0,
- acpi_ns_init_one_package,
- NULL, NULL, NULL);
+ /* Complete the initialization/resolution of new objects */
+
+ acpi_ns_initialize_objects();
}
return_ACPI_STATUS(status);
#include "internal.h"
+#ifdef CONFIG_DMI
static const struct dmi_system_id acpi_rev_dmi_table[] __initconst;
+#endif
/*
* POLICY: If *anything* doesn't work, put it on the blacklist.
}
(void)early_acpi_osi_init();
+#ifdef CONFIG_DMI
dmi_check_system(acpi_rev_dmi_table);
+#endif
return blacklisted;
}
offset = to_interleave_offset(offset, mmio);
writeq(cmd, mmio->addr.base + offset);
- nvdimm_flush(nfit_blk->nd_region);
+ nvdimm_flush(nfit_blk->nd_region, NULL);
if (nfit_blk->dimm_flags & NFIT_BLK_DCR_LATCH)
readq(mmio->addr.base + offset);
}
if (rw)
- nvdimm_flush(nfit_blk->nd_region);
+ nvdimm_flush(nfit_blk->nd_region, NULL);
rc = read_blk_stat(nfit_blk, lane) ? -EIO : 0;
return rc;
printk("%s: can't allocate TSQ.\n", card->name);
return -1;
}
- memset(card->tsq.base, 0, TSQSIZE);
card->tsq.last = card->tsq.base + TSQ_NUM_ENTRIES - 1;
card->tsq.next = card->tsq.last;
if (!thread)
return 0;
- err = ksys_mount("devtmpfs", (char *)mntdir, "devtmpfs", MS_SILENT,
- NULL);
+ err = ksys_mount("devtmpfs", mntdir, "devtmpfs", MS_SILENT, NULL);
if (err)
printk(KERN_INFO "devtmpfs: error mounting %i\n", err);
else
* firmware fallback configuration table
*/
-static unsigned int zero;
-static unsigned int one = 1;
-
struct firmware_fallback_config fw_fallback_config = {
.force_sysfs_fallback = IS_ENABLED(CONFIG_FW_LOADER_USER_HELPER_FALLBACK),
.loading_timeout = 60,
};
EXPORT_SYMBOL_GPL(fw_fallback_config);
+#ifdef CONFIG_SYSCTL
struct ctl_table firmware_config_table[] = {
{
.procname = "force_sysfs_fallback",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_douintvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "ignore_sysfs_fallback",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_douintvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{ }
};
EXPORT_SYMBOL_GPL(firmware_config_table);
+#endif
static int sections_per_block;
-static inline int base_memory_block_id(int section_nr)
+static inline unsigned long base_memory_block_id(unsigned long section_nr)
{
return section_nr / sections_per_block;
}
+static inline unsigned long pfn_to_block_id(unsigned long pfn)
+{
+ return base_memory_block_id(pfn_to_section_nr(pfn));
+}
+
+static inline unsigned long phys_to_block_id(unsigned long phys)
+{
+ return pfn_to_block_id(PFN_DOWN(phys));
+}
+
static int memory_subsys_online(struct device *dev);
static int memory_subsys_offline(struct device *dev);
static ssize_t removable_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
- unsigned long i, pfn;
- int ret = 1;
struct memory_block *mem = to_memory_block(dev);
+ unsigned long pfn;
+ int ret = 1, i;
if (mem->state != MEM_ONLINE)
goto out;
return 0;
}
-/*
- * A reference for the returned object is held and the reference for the
- * hinted object is released.
- */
-struct memory_block *find_memory_block_hinted(struct mem_section *section,
- struct memory_block *hint)
+/* A reference for the returned memory block device is acquired. */
+static struct memory_block *find_memory_block_by_id(unsigned long block_id)
{
- int block_id = base_memory_block_id(__section_nr(section));
- struct device *hintdev = hint ? &hint->dev : NULL;
struct device *dev;
- dev = subsys_find_device_by_id(&memory_subsys, block_id, hintdev);
- if (hint)
- put_device(&hint->dev);
- if (!dev)
- return NULL;
- return to_memory_block(dev);
+ dev = subsys_find_device_by_id(&memory_subsys, block_id, NULL);
+ return dev ? to_memory_block(dev) : NULL;
}
/*
*/
struct memory_block *find_memory_block(struct mem_section *section)
{
- return find_memory_block_hinted(section, NULL);
+ unsigned long block_id = base_memory_block_id(__section_nr(section));
+
+ return find_memory_block_by_id(block_id);
}
static struct attribute *memory_memblk_attrs[] = {
}
static int init_memory_block(struct memory_block **memory,
- struct mem_section *section, unsigned long state)
+ unsigned long block_id, unsigned long state)
{
struct memory_block *mem;
unsigned long start_pfn;
- int scn_nr;
int ret = 0;
+ mem = find_memory_block_by_id(block_id);
+ if (mem) {
+ put_device(&mem->dev);
+ return -EEXIST;
+ }
mem = kzalloc(sizeof(*mem), GFP_KERNEL);
if (!mem)
return -ENOMEM;
- scn_nr = __section_nr(section);
- mem->start_section_nr =
- base_memory_block_id(scn_nr) * sections_per_block;
+ mem->start_section_nr = block_id * sections_per_block;
mem->end_section_nr = mem->start_section_nr + sections_per_block - 1;
mem->state = state;
start_pfn = section_nr_to_pfn(mem->start_section_nr);
return ret;
}
-static int add_memory_block(int base_section_nr)
+static int add_memory_block(unsigned long base_section_nr)
{
+ int ret, section_count = 0;
struct memory_block *mem;
- int i, ret, section_count = 0, section_nr;
+ unsigned long nr;
- for (i = base_section_nr;
- i < base_section_nr + sections_per_block;
- i++) {
- if (!present_section_nr(i))
- continue;
- if (section_count == 0)
- section_nr = i;
- section_count++;
- }
+ for (nr = base_section_nr; nr < base_section_nr + sections_per_block;
+ nr++)
+ if (present_section_nr(nr))
+ section_count++;
if (section_count == 0)
return 0;
- ret = init_memory_block(&mem, __nr_to_section(section_nr), MEM_ONLINE);
+ ret = init_memory_block(&mem, base_memory_block_id(base_section_nr),
+ MEM_ONLINE);
if (ret)
return ret;
mem->section_count = section_count;
return 0;
}
+static void unregister_memory(struct memory_block *memory)
+{
+ if (WARN_ON_ONCE(memory->dev.bus != &memory_subsys))
+ return;
+
+ /* drop the ref. we got via find_memory_block() */
+ put_device(&memory->dev);
+ device_unregister(&memory->dev);
+}
+
/*
- * need an interface for the VM to add new memory regions,
- * but without onlining it.
+ * Create memory block devices for the given memory area. Start and size
+ * have to be aligned to memory block granularity. Memory block devices
+ * will be initialized as offline.
*/
-int hotplug_memory_register(int nid, struct mem_section *section)
+int create_memory_block_devices(unsigned long start, unsigned long size)
{
- int ret = 0;
+ const unsigned long start_block_id = pfn_to_block_id(PFN_DOWN(start));
+ unsigned long end_block_id = pfn_to_block_id(PFN_DOWN(start + size));
struct memory_block *mem;
+ unsigned long block_id;
+ int ret = 0;
- mutex_lock(&mem_sysfs_mutex);
+ if (WARN_ON_ONCE(!IS_ALIGNED(start, memory_block_size_bytes()) ||
+ !IS_ALIGNED(size, memory_block_size_bytes())))
+ return -EINVAL;
- mem = find_memory_block(section);
- if (mem) {
- mem->section_count++;
- put_device(&mem->dev);
- } else {
- ret = init_memory_block(&mem, section, MEM_OFFLINE);
+ mutex_lock(&mem_sysfs_mutex);
+ for (block_id = start_block_id; block_id != end_block_id; block_id++) {
+ ret = init_memory_block(&mem, block_id, MEM_OFFLINE);
if (ret)
- goto out;
- mem->section_count++;
+ break;
+ mem->section_count = sections_per_block;
+ }
+ if (ret) {
+ end_block_id = block_id;
+ for (block_id = start_block_id; block_id != end_block_id;
+ block_id++) {
+ mem = find_memory_block_by_id(block_id);
+ mem->section_count = 0;
+ unregister_memory(mem);
+ }
}
-
-out:
mutex_unlock(&mem_sysfs_mutex);
return ret;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
-static void
-unregister_memory(struct memory_block *memory)
-{
- BUG_ON(memory->dev.bus != &memory_subsys);
-
- /* drop the ref. we got via find_memory_block() */
- put_device(&memory->dev);
- device_unregister(&memory->dev);
-}
-
-void unregister_memory_section(struct mem_section *section)
+/*
+ * Remove memory block devices for the given memory area. Start and size
+ * have to be aligned to memory block granularity. Memory block devices
+ * have to be offline.
+ */
+void remove_memory_block_devices(unsigned long start, unsigned long size)
{
+ const unsigned long start_block_id = pfn_to_block_id(PFN_DOWN(start));
+ const unsigned long end_block_id = pfn_to_block_id(PFN_DOWN(start + size));
struct memory_block *mem;
+ unsigned long block_id;
- if (WARN_ON_ONCE(!present_section(section)))
+ if (WARN_ON_ONCE(!IS_ALIGNED(start, memory_block_size_bytes()) ||
+ !IS_ALIGNED(size, memory_block_size_bytes())))
return;
mutex_lock(&mem_sysfs_mutex);
-
- /*
- * Some users of the memory hotplug do not want/need memblock to
- * track all sections. Skip over those.
- */
- mem = find_memory_block(section);
- if (!mem)
- goto out_unlock;
-
- unregister_mem_sect_under_nodes(mem, __section_nr(section));
-
- mem->section_count--;
- if (mem->section_count == 0)
+ for (block_id = start_block_id; block_id != end_block_id; block_id++) {
+ mem = find_memory_block_by_id(block_id);
+ if (WARN_ON_ONCE(!mem))
+ continue;
+ mem->section_count = 0;
+ unregister_memory_block_under_nodes(mem);
unregister_memory(mem);
- else
- put_device(&mem->dev);
-
-out_unlock:
+ }
mutex_unlock(&mem_sysfs_mutex);
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
/* return true if the memory block is offlined, otherwise, return false */
bool is_memblock_offlined(struct memory_block *mem)
*/
int __init memory_dev_init(void)
{
- unsigned int i;
int ret;
int err;
- unsigned long block_sz;
+ unsigned long block_sz, nr;
ret = subsys_system_register(&memory_subsys, memory_root_attr_groups);
if (ret)
* during boot and have been initialized
*/
mutex_lock(&mem_sysfs_mutex);
- for (i = 0; i <= __highest_present_section_nr;
- i += sections_per_block) {
- err = add_memory_block(i);
+ for (nr = 0; nr <= __highest_present_section_nr;
+ nr += sections_per_block) {
+ err = add_memory_block(nr);
if (!ret)
ret = err;
}
printk(KERN_ERR "%s() failed: %d\n", __func__, ret);
return ret;
}
+
+/**
+ * walk_memory_blocks - walk through all present memory blocks overlapped
+ * by the range [start, start + size)
+ *
+ * @start: start address of the memory range
+ * @size: size of the memory range
+ * @arg: argument passed to func
+ * @func: callback for each memory section walked
+ *
+ * This function walks through all present memory blocks overlapped by the
+ * range [start, start + size), calling func on each memory block.
+ *
+ * In case func() returns an error, walking is aborted and the error is
+ * returned.
+ */
+int walk_memory_blocks(unsigned long start, unsigned long size,
+ void *arg, walk_memory_blocks_func_t func)
+{
+ const unsigned long start_block_id = phys_to_block_id(start);
+ const unsigned long end_block_id = phys_to_block_id(start + size - 1);
+ struct memory_block *mem;
+ unsigned long block_id;
+ int ret = 0;
+
+ if (!size)
+ return 0;
+
+ for (block_id = start_block_id; block_id <= end_block_id; block_id++) {
+ mem = find_memory_block_by_id(block_id);
+ if (!mem)
+ continue;
+
+ ret = func(mem, arg);
+ put_device(&mem->dev);
+ if (ret)
+ break;
+ }
+ return ret;
+}
}
/* register memory section under specified node if it spans that node */
-int register_mem_sect_under_node(struct memory_block *mem_blk, void *arg)
+static int register_mem_sect_under_node(struct memory_block *mem_blk,
+ void *arg)
{
int ret, nid = *(int *)arg;
unsigned long pfn, sect_start_pfn, sect_end_pfn;
return 0;
}
-/* unregister memory section under all nodes that it spans */
-int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,
- unsigned long phys_index)
+/*
+ * Unregister memory block device under all nodes that it spans.
+ * Has to be called with mem_sysfs_mutex held (due to unlinked_nodes).
+ */
+void unregister_memory_block_under_nodes(struct memory_block *mem_blk)
{
- NODEMASK_ALLOC(nodemask_t, unlinked_nodes, GFP_KERNEL);
unsigned long pfn, sect_start_pfn, sect_end_pfn;
+ static nodemask_t unlinked_nodes;
- if (!mem_blk) {
- NODEMASK_FREE(unlinked_nodes);
- return -EFAULT;
- }
- if (!unlinked_nodes)
- return -ENOMEM;
- nodes_clear(*unlinked_nodes);
-
- sect_start_pfn = section_nr_to_pfn(phys_index);
- sect_end_pfn = sect_start_pfn + PAGES_PER_SECTION - 1;
+ nodes_clear(unlinked_nodes);
+ sect_start_pfn = section_nr_to_pfn(mem_blk->start_section_nr);
+ sect_end_pfn = section_nr_to_pfn(mem_blk->end_section_nr);
for (pfn = sect_start_pfn; pfn <= sect_end_pfn; pfn++) {
int nid;
continue;
if (!node_online(nid))
continue;
- if (node_test_and_set(nid, *unlinked_nodes))
+ if (node_test_and_set(nid, unlinked_nodes))
continue;
sysfs_remove_link(&node_devices[nid]->dev.kobj,
kobject_name(&mem_blk->dev.kobj));
sysfs_remove_link(&mem_blk->dev.kobj,
kobject_name(&node_devices[nid]->dev.kobj));
}
- NODEMASK_FREE(unlinked_nodes);
- return 0;
}
int link_mem_sections(int nid, unsigned long start_pfn, unsigned long end_pfn)
{
- return walk_memory_range(start_pfn, end_pfn, (void *)&nid,
- register_mem_sect_under_node);
+ return walk_memory_blocks(PFN_PHYS(start_pfn),
+ PFN_PHYS(end_pfn - start_pfn), (void *)&nid,
+ register_mem_sect_under_node);
}
#ifdef CONFIG_HUGETLBFS
* Copyright (c) 2002-3 Patrick Mochel
* Copyright (c) 2002-3 Open Source Development Labs
*
- * Please see Documentation/driver-model/platform.rst for more
+ * Please see Documentation/driver-api/driver-model/platform.rst for more
* information.
*/
if (ret)
genpd_free_dev_data(dev, gpd_data);
else
- dev_pm_qos_add_notifier(dev, &gpd_data->nb);
+ dev_pm_qos_add_notifier(dev, &gpd_data->nb,
+ DEV_PM_QOS_RESUME_LATENCY);
return ret;
}
pdd = dev->power.subsys_data->domain_data;
gpd_data = to_gpd_data(pdd);
- dev_pm_qos_remove_notifier(dev, &gpd_data->nb);
+ dev_pm_qos_remove_notifier(dev, &gpd_data->nb,
+ DEV_PM_QOS_RESUME_LATENCY);
genpd_lock(genpd);
out:
genpd_unlock(genpd);
- dev_pm_qos_add_notifier(dev, &gpd_data->nb);
+ dev_pm_qos_add_notifier(dev, &gpd_data->nb, DEV_PM_QOS_RESUME_LATENCY);
return ret;
}
* take its current PM QoS constraint (that's the only thing
* known at this point anyway).
*/
- constraint_ns = dev_pm_qos_read_value(dev);
+ constraint_ns = dev_pm_qos_read_value(dev, DEV_PM_QOS_RESUME_LATENCY);
constraint_ns *= NSEC_PER_USEC;
}
td->constraint_changed = false;
td->cached_suspend_ok = false;
td->effective_constraint_ns = 0;
- constraint_ns = __dev_pm_qos_read_value(dev);
+ constraint_ns = __dev_pm_qos_resume_latency(dev);
spin_unlock_irqrestore(&dev->power.lock, flags);
EXPORT_SYMBOL_GPL(dev_pm_qos_flags);
/**
- * __dev_pm_qos_read_value - Get PM QoS constraint for a given device.
+ * __dev_pm_qos_resume_latency - Get resume latency constraint for a given device.
* @dev: Device to get the PM QoS constraint value for.
*
* This routine must be called with dev->power.lock held.
*/
-s32 __dev_pm_qos_read_value(struct device *dev)
+s32 __dev_pm_qos_resume_latency(struct device *dev)
{
lockdep_assert_held(&dev->power.lock);
- return dev_pm_qos_raw_read_value(dev);
+ return dev_pm_qos_raw_resume_latency(dev);
}
/**
* dev_pm_qos_read_value - Get PM QoS constraint for a given device (locked).
* @dev: Device to get the PM QoS constraint value for.
+ * @type: QoS request type.
*/
-s32 dev_pm_qos_read_value(struct device *dev)
+s32 dev_pm_qos_read_value(struct device *dev, enum dev_pm_qos_req_type type)
{
+ struct dev_pm_qos *qos = dev->power.qos;
unsigned long flags;
s32 ret;
spin_lock_irqsave(&dev->power.lock, flags);
- ret = __dev_pm_qos_read_value(dev);
+
+ switch (type) {
+ case DEV_PM_QOS_RESUME_LATENCY:
+ ret = IS_ERR_OR_NULL(qos) ? PM_QOS_RESUME_LATENCY_NO_CONSTRAINT
+ : pm_qos_read_value(&qos->resume_latency);
+ break;
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ ret = IS_ERR_OR_NULL(qos) ? PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE
+ : pm_qos_read_value(&qos->min_frequency);
+ break;
+ case DEV_PM_QOS_MAX_FREQUENCY:
+ ret = IS_ERR_OR_NULL(qos) ? PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE
+ : pm_qos_read_value(&qos->max_frequency);
+ break;
+ default:
+ WARN_ON(1);
+ ret = 0;
+ }
+
spin_unlock_irqrestore(&dev->power.lock, flags);
return ret;
req->dev->power.set_latency_tolerance(req->dev, value);
}
break;
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ ret = pm_qos_update_target(&qos->min_frequency,
+ &req->data.pnode, action, value);
+ break;
+ case DEV_PM_QOS_MAX_FREQUENCY:
+ ret = pm_qos_update_target(&qos->max_frequency,
+ &req->data.pnode, action, value);
+ break;
case DEV_PM_QOS_FLAGS:
ret = pm_qos_update_flags(&qos->flags, &req->data.flr,
action, value);
if (!qos)
return -ENOMEM;
- n = kzalloc(sizeof(*n), GFP_KERNEL);
+ n = kzalloc(3 * sizeof(*n), GFP_KERNEL);
if (!n) {
kfree(qos);
return -ENOMEM;
}
- BLOCKING_INIT_NOTIFIER_HEAD(n);
c = &qos->resume_latency;
plist_head_init(&c->list);
c->no_constraint_value = PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
c->type = PM_QOS_MIN;
c->notifiers = n;
+ BLOCKING_INIT_NOTIFIER_HEAD(n);
c = &qos->latency_tolerance;
plist_head_init(&c->list);
c->no_constraint_value = PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT;
c->type = PM_QOS_MIN;
+ c = &qos->min_frequency;
+ plist_head_init(&c->list);
+ c->target_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE;
+ c->default_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE;
+ c->no_constraint_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE;
+ c->type = PM_QOS_MAX;
+ c->notifiers = ++n;
+ BLOCKING_INIT_NOTIFIER_HEAD(n);
+
+ c = &qos->max_frequency;
+ plist_head_init(&c->list);
+ c->target_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE;
+ c->default_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE;
+ c->no_constraint_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE;
+ c->type = PM_QOS_MIN;
+ c->notifiers = ++n;
+ BLOCKING_INIT_NOTIFIER_HEAD(n);
+
INIT_LIST_HEAD(&qos->flags.list);
spin_lock_irq(&dev->power.lock);
apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE);
memset(req, 0, sizeof(*req));
}
+
c = &qos->latency_tolerance;
plist_for_each_entry_safe(req, tmp, &c->list, data.pnode) {
apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE);
memset(req, 0, sizeof(*req));
}
+
+ c = &qos->min_frequency;
+ plist_for_each_entry_safe(req, tmp, &c->list, data.pnode) {
+ apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE);
+ memset(req, 0, sizeof(*req));
+ }
+
+ c = &qos->max_frequency;
+ plist_for_each_entry_safe(req, tmp, &c->list, data.pnode) {
+ apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE);
+ memset(req, 0, sizeof(*req));
+ }
+
f = &qos->flags;
list_for_each_entry_safe(req, tmp, &f->list, data.flr.node) {
apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE);
switch(req->type) {
case DEV_PM_QOS_RESUME_LATENCY:
case DEV_PM_QOS_LATENCY_TOLERANCE:
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ case DEV_PM_QOS_MAX_FREQUENCY:
curr_value = req->data.pnode.prio;
break;
case DEV_PM_QOS_FLAGS:
*
* @dev: target device for the constraint
* @notifier: notifier block managed by caller.
+ * @type: request type.
*
* Will register the notifier into a notification chain that gets called
* upon changes to the target value for the device.
* If the device's constraints object doesn't exist when this routine is called,
* it will be created (or error code will be returned if that fails).
*/
-int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier)
+int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type)
{
int ret = 0;
else if (!dev->power.qos)
ret = dev_pm_qos_constraints_allocate(dev);
- if (!ret)
+ if (ret)
+ goto unlock;
+
+ switch (type) {
+ case DEV_PM_QOS_RESUME_LATENCY:
ret = blocking_notifier_chain_register(dev->power.qos->resume_latency.notifiers,
notifier);
+ break;
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ ret = blocking_notifier_chain_register(dev->power.qos->min_frequency.notifiers,
+ notifier);
+ break;
+ case DEV_PM_QOS_MAX_FREQUENCY:
+ ret = blocking_notifier_chain_register(dev->power.qos->max_frequency.notifiers,
+ notifier);
+ break;
+ default:
+ WARN_ON(1);
+ ret = -EINVAL;
+ }
+unlock:
mutex_unlock(&dev_pm_qos_mtx);
return ret;
}
*
* @dev: target device for the constraint
* @notifier: notifier block to be removed.
+ * @type: request type.
*
* Will remove the notifier from the notification chain that gets called
* upon changes to the target value.
*/
int dev_pm_qos_remove_notifier(struct device *dev,
- struct notifier_block *notifier)
+ struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type)
{
- int retval = 0;
+ int ret = 0;
mutex_lock(&dev_pm_qos_mtx);
/* Silently return if the constraints object is not present. */
- if (!IS_ERR_OR_NULL(dev->power.qos))
- retval = blocking_notifier_chain_unregister(dev->power.qos->resume_latency.notifiers,
- notifier);
+ if (IS_ERR_OR_NULL(dev->power.qos))
+ goto unlock;
+
+ switch (type) {
+ case DEV_PM_QOS_RESUME_LATENCY:
+ ret = blocking_notifier_chain_unregister(dev->power.qos->resume_latency.notifiers,
+ notifier);
+ break;
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ ret = blocking_notifier_chain_unregister(dev->power.qos->min_frequency.notifiers,
+ notifier);
+ break;
+ case DEV_PM_QOS_MAX_FREQUENCY:
+ ret = blocking_notifier_chain_unregister(dev->power.qos->max_frequency.notifiers,
+ notifier);
+ break;
+ default:
+ WARN_ON(1);
+ ret = -EINVAL;
+ }
+unlock:
mutex_unlock(&dev_pm_qos_mtx);
- return retval;
+ return ret;
}
EXPORT_SYMBOL_GPL(dev_pm_qos_remove_notifier);
req = dev->power.qos->flags_req;
dev->power.qos->flags_req = NULL;
break;
+ default:
+ WARN_ON(1);
+ return;
}
__dev_pm_qos_remove_request(req);
kfree(req);
|| (dev->power.request_pending
&& dev->power.request == RPM_REQ_RESUME))
retval = -EAGAIN;
- else if (__dev_pm_qos_read_value(dev) == 0)
+ else if (__dev_pm_qos_resume_latency(dev) == 0)
retval = -EPERM;
else if (dev->power.runtime_status == RPM_SUSPENDED)
retval = 1;
If you want to use the floppy disk drive(s) of your PC under Linux,
say Y. Information about this driver, especially important for IBM
Thinkpad users, is contained in
- <file:Documentation/blockdev/floppy.txt>.
+ <file:Documentation/admin-guide/blockdev/floppy.rst>.
That file also contains the location of the Floppy driver FAQ as
well as location of the fdutils package used to configure additional
parameters of the driver at run time.
your computer's parallel port. Most of them are actually IDE devices
using a parallel port IDE adapter. This option enables the PARIDE
subsystem which contains drivers for many of these external drives.
- Read <file:Documentation/blockdev/paride.txt> for more information.
+ Read <file:Documentation/admin-guide/blockdev/paride.rst> for more information.
If you have said Y to the "Parallel-port support" configuration
option, you may share a single port between your printer and other
userland (making server and client physically the same computer,
communicating using the loopback network device).
- Read <file:Documentation/blockdev/nbd.txt> for more information,
+ Read <file:Documentation/admin-guide/blockdev/nbd.rst> for more information,
especially about where to find the server code, which runs in user
space and does not need special kernel support.
during the initial install of Linux.
Note that the kernel command line option "ramdisk=XX" is now obsolete.
- For details, read <file:Documentation/blockdev/ramdisk.txt>.
+ For details, read <file:Documentation/admin-guide/blockdev/ramdisk.rst>.
To compile this driver as a module, choose M here: the
module will be called brd. An alias "rd" has been defined
raw_cmd->kernel_data = floppy_track_buffer;
raw_cmd->length = 4 * F_SECT_PER_TRACK;
+ if (!F_SECT_PER_TRACK)
+ return;
+
/* allow for about 30ms for data transport per track */
head_shift = (F_SECT_PER_TRACK + 5) / 6;
int cnt;
/* sanity checking for parameters. */
- if (g->sect <= 0 ||
- g->head <= 0 ||
+ if ((int)g->sect <= 0 ||
+ (int)g->head <= 0 ||
+ /* check for overflow in max_sector */
+ (int)(g->sect * g->head) <= 0 ||
+ /* check for zero in F_SECT_PER_TRACK */
+ (unsigned char)((g->sect << 2) >> FD_SIZECODE(g)) == 0 ||
g->track <= 0 || g->track > UDP->tracks >> STRETCH(g) ||
/* check if reserved bits are set */
(g->stretch & ~(FD_STRETCH | FD_SWAPSIDES | FD_SECTBASEMASK)) != 0)
return 0;
}
+static bool valid_floppy_drive_params(const short autodetect[8],
+ int native_format)
+{
+ size_t floppy_type_size = ARRAY_SIZE(floppy_type);
+ size_t i = 0;
+
+ for (i = 0; i < 8; ++i) {
+ if (autodetect[i] < 0 ||
+ autodetect[i] >= floppy_type_size)
+ return false;
+ }
+
+ if (native_format < 0 || native_format >= floppy_type_size)
+ return false;
+
+ return true;
+}
+
static int fd_locked_ioctl(struct block_device *bdev, fmode_t mode, unsigned int cmd,
unsigned long param)
{
SUPBOUND(size, strlen((const char *)outparam) + 1);
break;
case FDSETDRVPRM:
+ if (!valid_floppy_drive_params(inparam.dp.autodetect,
+ inparam.dp.native_format))
+ return -EINVAL;
*UDP = inparam.dp;
break;
case FDGETDRVPRM:
return -EPERM;
if (copy_from_user(&v, arg, sizeof(struct compat_floppy_drive_params)))
return -EFAULT;
+ if (!valid_floppy_drive_params(v.autodetect, v.native_format))
+ return -EINVAL;
mutex_lock(&floppy_mutex);
UDP->cmos = v.cmos;
UDP->max_dtr = v.max_dtr;
pr_cont("\n");
} else
DPRINT("botched floppy option\n");
- DPRINT("Read Documentation/blockdev/floppy.txt\n");
+ DPRINT("Read Documentation/admin-guide/blockdev/floppy.rst\n");
return 0;
}
#define RBD_FEATURE_LAYERING (1ULL<<0)
#define RBD_FEATURE_STRIPINGV2 (1ULL<<1)
#define RBD_FEATURE_EXCLUSIVE_LOCK (1ULL<<2)
+#define RBD_FEATURE_OBJECT_MAP (1ULL<<3)
+#define RBD_FEATURE_FAST_DIFF (1ULL<<4)
#define RBD_FEATURE_DEEP_FLATTEN (1ULL<<5)
#define RBD_FEATURE_DATA_POOL (1ULL<<7)
#define RBD_FEATURE_OPERATIONS (1ULL<<8)
#define RBD_FEATURES_ALL (RBD_FEATURE_LAYERING | \
RBD_FEATURE_STRIPINGV2 | \
RBD_FEATURE_EXCLUSIVE_LOCK | \
+ RBD_FEATURE_OBJECT_MAP | \
+ RBD_FEATURE_FAST_DIFF | \
RBD_FEATURE_DEEP_FLATTEN | \
RBD_FEATURE_DATA_POOL | \
RBD_FEATURE_OPERATIONS)
struct list_head node;
};
+struct pending_result {
+ int result; /* first nonzero result */
+ int num_pending;
+};
+
struct rbd_img_request;
enum obj_request_type {
OBJ_OP_ZEROOUT,
};
+#define RBD_OBJ_FLAG_DELETION (1U << 0)
+#define RBD_OBJ_FLAG_COPYUP_ENABLED (1U << 1)
+#define RBD_OBJ_FLAG_COPYUP_ZEROS (1U << 2)
+#define RBD_OBJ_FLAG_MAY_EXIST (1U << 3)
+#define RBD_OBJ_FLAG_NOOP_FOR_NONEXISTENT (1U << 4)
+
+enum rbd_obj_read_state {
+ RBD_OBJ_READ_START = 1,
+ RBD_OBJ_READ_OBJECT,
+ RBD_OBJ_READ_PARENT,
+};
+
/*
* Writes go through the following state machine to deal with
* layering:
* even if there is a parent).
*/
enum rbd_obj_write_state {
- RBD_OBJ_WRITE_FLAT = 1,
- RBD_OBJ_WRITE_GUARD,
- RBD_OBJ_WRITE_READ_FROM_PARENT,
- RBD_OBJ_WRITE_COPYUP_EMPTY_SNAPC,
- RBD_OBJ_WRITE_COPYUP_OPS,
+ RBD_OBJ_WRITE_START = 1,
+ RBD_OBJ_WRITE_PRE_OBJECT_MAP,
+ RBD_OBJ_WRITE_OBJECT,
+ __RBD_OBJ_WRITE_COPYUP,
+ RBD_OBJ_WRITE_COPYUP,
+ RBD_OBJ_WRITE_POST_OBJECT_MAP,
+};
+
+enum rbd_obj_copyup_state {
+ RBD_OBJ_COPYUP_START = 1,
+ RBD_OBJ_COPYUP_READ_PARENT,
+ __RBD_OBJ_COPYUP_OBJECT_MAPS,
+ RBD_OBJ_COPYUP_OBJECT_MAPS,
+ __RBD_OBJ_COPYUP_WRITE_OBJECT,
+ RBD_OBJ_COPYUP_WRITE_OBJECT,
};
struct rbd_obj_request {
struct ceph_object_extent ex;
+ unsigned int flags; /* RBD_OBJ_FLAG_* */
union {
- bool tried_parent; /* for reads */
+ enum rbd_obj_read_state read_state; /* for reads */
enum rbd_obj_write_state write_state; /* for writes */
};
u32 bvec_idx;
};
};
+
+ enum rbd_obj_copyup_state copyup_state;
struct bio_vec *copyup_bvecs;
u32 copyup_bvec_count;
- struct ceph_osd_request *osd_req;
-
- u64 xferred; /* bytes transferred */
- int result;
+ struct list_head osd_reqs; /* w/ r_private_item */
+ struct mutex state_mutex;
+ struct pending_result pending;
struct kref kref;
};
IMG_REQ_LAYERED, /* ENOENT handling: normal = 0, layered = 1 */
};
+enum rbd_img_state {
+ RBD_IMG_START = 1,
+ RBD_IMG_EXCLUSIVE_LOCK,
+ __RBD_IMG_OBJECT_REQUESTS,
+ RBD_IMG_OBJECT_REQUESTS,
+};
+
struct rbd_img_request {
struct rbd_device *rbd_dev;
enum obj_operation_type op_type;
enum obj_request_type data_type;
unsigned long flags;
+ enum rbd_img_state state;
union {
u64 snap_id; /* for reads */
struct ceph_snap_context *snapc; /* for writes */
struct request *rq; /* block request */
struct rbd_obj_request *obj_request; /* obj req initiator */
};
- spinlock_t completion_lock;
- u64 xferred;/* aggregate bytes transferred */
- int result; /* first nonzero obj_request result */
+ struct list_head lock_item;
struct list_head object_extents; /* obj_req.ex structs */
- u32 pending_count;
+ struct mutex state_mutex;
+ struct pending_result pending;
+ struct work_struct work;
+ int work_result;
struct kref kref;
};
struct work_struct released_lock_work;
struct delayed_work lock_dwork;
struct work_struct unlock_work;
- wait_queue_head_t lock_waitq;
+ spinlock_t lock_lists_lock;
+ struct list_head acquiring_list;
+ struct list_head running_list;
+ struct completion acquire_wait;
+ int acquire_err;
+ struct completion releasing_wait;
+
+ spinlock_t object_map_lock;
+ u8 *object_map;
+ u64 object_map_size; /* in objects */
+ u64 object_map_flags;
struct workqueue_struct *task_wq;
* Flag bits for rbd_dev->flags:
* - REMOVING (which is coupled with rbd_dev->open_count) is protected
* by rbd_dev->lock
- * - BLACKLISTED is protected by rbd_dev->lock_rwsem
*/
enum rbd_dev_flags {
RBD_DEV_FLAG_EXISTS, /* mapped snapshot has not been deleted */
RBD_DEV_FLAG_REMOVING, /* this mapping is being removed */
- RBD_DEV_FLAG_BLACKLISTED, /* our ceph_client is blacklisted */
};
static DEFINE_MUTEX(client_mutex); /* Serialize client creation */
static bool __rbd_is_lock_owner(struct rbd_device *rbd_dev)
{
+ lockdep_assert_held(&rbd_dev->lock_rwsem);
+
return rbd_dev->lock_state == RBD_LOCK_STATE_LOCKED ||
rbd_dev->lock_state == RBD_LOCK_STATE_RELEASING;
}
u8 *order, u64 *snap_size);
static int _rbd_dev_v2_snap_features(struct rbd_device *rbd_dev, u64 snap_id,
u64 *snap_features);
+static int rbd_dev_v2_get_flags(struct rbd_device *rbd_dev);
+
+static void rbd_obj_handle_request(struct rbd_obj_request *obj_req, int result);
+static void rbd_img_handle_request(struct rbd_img_request *img_req, int result);
+
+/*
+ * Return true if nothing else is pending.
+ */
+static bool pending_result_dec(struct pending_result *pending, int *result)
+{
+ rbd_assert(pending->num_pending > 0);
+
+ if (*result && !pending->result)
+ pending->result = *result;
+ if (--pending->num_pending)
+ return false;
+
+ *result = pending->result;
+ return true;
+}
static int rbd_open(struct block_device *bdev, fmode_t mode)
{
static void rbd_obj_zero_range(struct rbd_obj_request *obj_req, u32 off,
u32 bytes)
{
+ dout("%s %p data buf %u~%u\n", __func__, obj_req, off, bytes);
+
switch (obj_req->img_request->data_type) {
case OBJ_REQUEST_BIO:
zero_bios(&obj_req->bio_pos, off, bytes);
kref_put(&obj_request->kref, rbd_obj_request_destroy);
}
-static void rbd_img_request_get(struct rbd_img_request *img_request)
-{
- dout("%s: img %p (was %d)\n", __func__, img_request,
- kref_read(&img_request->kref));
- kref_get(&img_request->kref);
-}
-
static void rbd_img_request_destroy(struct kref *kref);
static void rbd_img_request_put(struct rbd_img_request *img_request)
{
/* Image request now owns object's original reference */
obj_request->img_request = img_request;
- img_request->pending_count++;
dout("%s: img %p obj %p\n", __func__, img_request, obj_request);
}
rbd_obj_request_put(obj_request);
}
-static void rbd_obj_request_submit(struct rbd_obj_request *obj_request)
+static void rbd_osd_submit(struct ceph_osd_request *osd_req)
{
- struct ceph_osd_request *osd_req = obj_request->osd_req;
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
- dout("%s %p object_no %016llx %llu~%llu osd_req %p\n", __func__,
- obj_request, obj_request->ex.oe_objno, obj_request->ex.oe_off,
- obj_request->ex.oe_len, osd_req);
+ dout("%s osd_req %p for obj_req %p objno %llu %llu~%llu\n",
+ __func__, osd_req, obj_req, obj_req->ex.oe_objno,
+ obj_req->ex.oe_off, obj_req->ex.oe_len);
ceph_osdc_start_request(osd_req->r_osdc, osd_req, false);
}
}
}
-static void rbd_obj_handle_request(struct rbd_obj_request *obj_req);
-
static void rbd_osd_req_callback(struct ceph_osd_request *osd_req)
{
struct rbd_obj_request *obj_req = osd_req->r_priv;
+ int result;
dout("%s osd_req %p result %d for obj_req %p\n", __func__, osd_req,
osd_req->r_result, obj_req);
- rbd_assert(osd_req == obj_req->osd_req);
- obj_req->result = osd_req->r_result < 0 ? osd_req->r_result : 0;
- if (!obj_req->result && !rbd_img_is_write(obj_req->img_request))
- obj_req->xferred = osd_req->r_result;
+ /*
+ * Writes aren't allowed to return a data payload. In some
+ * guarded write cases (e.g. stat + zero on an empty object)
+ * a stat response makes it through, but we don't care.
+ */
+ if (osd_req->r_result > 0 && rbd_img_is_write(obj_req->img_request))
+ result = 0;
else
- /*
- * Writes aren't allowed to return a data payload. In some
- * guarded write cases (e.g. stat + zero on an empty object)
- * a stat response makes it through, but we don't care.
- */
- obj_req->xferred = 0;
+ result = osd_req->r_result;
- rbd_obj_handle_request(obj_req);
+ rbd_obj_handle_request(obj_req, result);
}
-static void rbd_osd_req_format_read(struct rbd_obj_request *obj_request)
+static void rbd_osd_format_read(struct ceph_osd_request *osd_req)
{
- struct ceph_osd_request *osd_req = obj_request->osd_req;
+ struct rbd_obj_request *obj_request = osd_req->r_priv;
osd_req->r_flags = CEPH_OSD_FLAG_READ;
osd_req->r_snapid = obj_request->img_request->snap_id;
}
-static void rbd_osd_req_format_write(struct rbd_obj_request *obj_request)
+static void rbd_osd_format_write(struct ceph_osd_request *osd_req)
{
- struct ceph_osd_request *osd_req = obj_request->osd_req;
+ struct rbd_obj_request *obj_request = osd_req->r_priv;
osd_req->r_flags = CEPH_OSD_FLAG_WRITE;
ktime_get_real_ts64(&osd_req->r_mtime);
}
static struct ceph_osd_request *
-__rbd_osd_req_create(struct rbd_obj_request *obj_req,
- struct ceph_snap_context *snapc, unsigned int num_ops)
+__rbd_obj_add_osd_request(struct rbd_obj_request *obj_req,
+ struct ceph_snap_context *snapc, int num_ops)
{
struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc;
struct ceph_osd_request *req;
const char *name_format = rbd_dev->image_format == 1 ?
RBD_V1_DATA_FORMAT : RBD_V2_DATA_FORMAT;
+ int ret;
req = ceph_osdc_alloc_request(osdc, snapc, num_ops, false, GFP_NOIO);
if (!req)
- return NULL;
+ return ERR_PTR(-ENOMEM);
+ list_add_tail(&req->r_private_item, &obj_req->osd_reqs);
req->r_callback = rbd_osd_req_callback;
req->r_priv = obj_req;
ceph_oloc_copy(&req->r_base_oloc, &rbd_dev->header_oloc);
req->r_base_oloc.pool = rbd_dev->layout.pool_id;
- if (ceph_oid_aprintf(&req->r_base_oid, GFP_NOIO, name_format,
- rbd_dev->header.object_prefix, obj_req->ex.oe_objno))
- goto err_req;
+ ret = ceph_oid_aprintf(&req->r_base_oid, GFP_NOIO, name_format,
+ rbd_dev->header.object_prefix,
+ obj_req->ex.oe_objno);
+ if (ret)
+ return ERR_PTR(ret);
return req;
-
-err_req:
- ceph_osdc_put_request(req);
- return NULL;
}
static struct ceph_osd_request *
-rbd_osd_req_create(struct rbd_obj_request *obj_req, unsigned int num_ops)
-{
- return __rbd_osd_req_create(obj_req, obj_req->img_request->snapc,
- num_ops);
-}
-
-static void rbd_osd_req_destroy(struct ceph_osd_request *osd_req)
+rbd_obj_add_osd_request(struct rbd_obj_request *obj_req, int num_ops)
{
- ceph_osdc_put_request(osd_req);
+ return __rbd_obj_add_osd_request(obj_req, obj_req->img_request->snapc,
+ num_ops);
}
static struct rbd_obj_request *rbd_obj_request_create(void)
return NULL;
ceph_object_extent_init(&obj_request->ex);
+ INIT_LIST_HEAD(&obj_request->osd_reqs);
+ mutex_init(&obj_request->state_mutex);
kref_init(&obj_request->kref);
dout("%s %p\n", __func__, obj_request);
static void rbd_obj_request_destroy(struct kref *kref)
{
struct rbd_obj_request *obj_request;
+ struct ceph_osd_request *osd_req;
u32 i;
obj_request = container_of(kref, struct rbd_obj_request, kref);
dout("%s: obj %p\n", __func__, obj_request);
- if (obj_request->osd_req)
- rbd_osd_req_destroy(obj_request->osd_req);
+ while (!list_empty(&obj_request->osd_reqs)) {
+ osd_req = list_first_entry(&obj_request->osd_reqs,
+ struct ceph_osd_request, r_private_item);
+ list_del_init(&osd_req->r_private_item);
+ ceph_osdc_put_request(osd_req);
+ }
switch (obj_request->img_request->data_type) {
case OBJ_REQUEST_NODATA:
if (rbd_dev_parent_get(rbd_dev))
img_request_layered_set(img_request);
- spin_lock_init(&img_request->completion_lock);
+ INIT_LIST_HEAD(&img_request->lock_item);
INIT_LIST_HEAD(&img_request->object_extents);
+ mutex_init(&img_request->state_mutex);
kref_init(&img_request->kref);
dout("%s: rbd_dev %p %s -> img %p\n", __func__, rbd_dev,
dout("%s: img %p\n", __func__, img_request);
+ WARN_ON(!list_empty(&img_request->lock_item));
for_each_obj_request_safe(img_request, obj_request, next_obj_request)
rbd_img_obj_request_del(img_request, obj_request);
kmem_cache_free(rbd_img_request_cache, img_request);
}
-static void prune_extents(struct ceph_file_extent *img_extents,
- u32 *num_img_extents, u64 overlap)
-{
- u32 cnt = *num_img_extents;
+#define BITS_PER_OBJ 2
+#define OBJS_PER_BYTE (BITS_PER_BYTE / BITS_PER_OBJ)
+#define OBJ_MASK ((1 << BITS_PER_OBJ) - 1)
- /* drop extents completely beyond the overlap */
- while (cnt && img_extents[cnt - 1].fe_off >= overlap)
- cnt--;
+static void __rbd_object_map_index(struct rbd_device *rbd_dev, u64 objno,
+ u64 *index, u8 *shift)
+{
+ u32 off;
- if (cnt) {
- struct ceph_file_extent *ex = &img_extents[cnt - 1];
+ rbd_assert(objno < rbd_dev->object_map_size);
+ *index = div_u64_rem(objno, OBJS_PER_BYTE, &off);
+ *shift = (OBJS_PER_BYTE - off - 1) * BITS_PER_OBJ;
+}
- /* trim final overlapping extent */
- if (ex->fe_off + ex->fe_len > overlap)
- ex->fe_len = overlap - ex->fe_off;
- }
+static u8 __rbd_object_map_get(struct rbd_device *rbd_dev, u64 objno)
+{
+ u64 index;
+ u8 shift;
- *num_img_extents = cnt;
+ lockdep_assert_held(&rbd_dev->object_map_lock);
+ __rbd_object_map_index(rbd_dev, objno, &index, &shift);
+ return (rbd_dev->object_map[index] >> shift) & OBJ_MASK;
}
-/*
- * Determine the byte range(s) covered by either just the object extent
- * or the entire object in the parent image.
- */
-static int rbd_obj_calc_img_extents(struct rbd_obj_request *obj_req,
- bool entire)
+static void __rbd_object_map_set(struct rbd_device *rbd_dev, u64 objno, u8 val)
{
- struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
- int ret;
-
- if (!rbd_dev->parent_overlap)
- return 0;
+ u64 index;
+ u8 shift;
+ u8 *p;
- ret = ceph_extent_to_file(&rbd_dev->layout, obj_req->ex.oe_objno,
- entire ? 0 : obj_req->ex.oe_off,
- entire ? rbd_dev->layout.object_size :
- obj_req->ex.oe_len,
- &obj_req->img_extents,
- &obj_req->num_img_extents);
- if (ret)
- return ret;
+ lockdep_assert_held(&rbd_dev->object_map_lock);
+ rbd_assert(!(val & ~OBJ_MASK));
- prune_extents(obj_req->img_extents, &obj_req->num_img_extents,
- rbd_dev->parent_overlap);
- return 0;
+ __rbd_object_map_index(rbd_dev, objno, &index, &shift);
+ p = &rbd_dev->object_map[index];
+ *p = (*p & ~(OBJ_MASK << shift)) | (val << shift);
}
-static void rbd_osd_req_setup_data(struct rbd_obj_request *obj_req, u32 which)
+static u8 rbd_object_map_get(struct rbd_device *rbd_dev, u64 objno)
{
- switch (obj_req->img_request->data_type) {
- case OBJ_REQUEST_BIO:
- osd_req_op_extent_osd_data_bio(obj_req->osd_req, which,
- &obj_req->bio_pos,
- obj_req->ex.oe_len);
- break;
- case OBJ_REQUEST_BVECS:
- case OBJ_REQUEST_OWN_BVECS:
- rbd_assert(obj_req->bvec_pos.iter.bi_size ==
- obj_req->ex.oe_len);
- rbd_assert(obj_req->bvec_idx == obj_req->bvec_count);
- osd_req_op_extent_osd_data_bvec_pos(obj_req->osd_req, which,
- &obj_req->bvec_pos);
- break;
- default:
- BUG();
- }
+ u8 state;
+
+ spin_lock(&rbd_dev->object_map_lock);
+ state = __rbd_object_map_get(rbd_dev, objno);
+ spin_unlock(&rbd_dev->object_map_lock);
+ return state;
}
-static int rbd_obj_setup_read(struct rbd_obj_request *obj_req)
+static bool use_object_map(struct rbd_device *rbd_dev)
{
- obj_req->osd_req = __rbd_osd_req_create(obj_req, NULL, 1);
- if (!obj_req->osd_req)
- return -ENOMEM;
-
- osd_req_op_extent_init(obj_req->osd_req, 0, CEPH_OSD_OP_READ,
- obj_req->ex.oe_off, obj_req->ex.oe_len, 0, 0);
- rbd_osd_req_setup_data(obj_req, 0);
-
- rbd_osd_req_format_read(obj_req);
- return 0;
+ return ((rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP) &&
+ !(rbd_dev->object_map_flags & RBD_FLAG_OBJECT_MAP_INVALID));
}
-static int __rbd_obj_setup_stat(struct rbd_obj_request *obj_req,
- unsigned int which)
+static bool rbd_object_map_may_exist(struct rbd_device *rbd_dev, u64 objno)
{
- struct page **pages;
+ u8 state;
- /*
- * The response data for a STAT call consists of:
- * le64 length;
- * struct {
- * le32 tv_sec;
- * le32 tv_nsec;
- * } mtime;
- */
- pages = ceph_alloc_page_vector(1, GFP_NOIO);
- if (IS_ERR(pages))
- return PTR_ERR(pages);
+ /* fall back to default logic if object map is disabled or invalid */
+ if (!use_object_map(rbd_dev))
+ return true;
- osd_req_op_init(obj_req->osd_req, which, CEPH_OSD_OP_STAT, 0);
- osd_req_op_raw_data_in_pages(obj_req->osd_req, which, pages,
- 8 + sizeof(struct ceph_timespec),
- 0, false, true);
- return 0;
+ state = rbd_object_map_get(rbd_dev, objno);
+ return state != OBJECT_NONEXISTENT;
}
-static int count_write_ops(struct rbd_obj_request *obj_req)
+static void rbd_object_map_name(struct rbd_device *rbd_dev, u64 snap_id,
+ struct ceph_object_id *oid)
{
- return 2; /* setallochint + write/writefull */
+ if (snap_id == CEPH_NOSNAP)
+ ceph_oid_printf(oid, "%s%s", RBD_OBJECT_MAP_PREFIX,
+ rbd_dev->spec->image_id);
+ else
+ ceph_oid_printf(oid, "%s%s.%016llx", RBD_OBJECT_MAP_PREFIX,
+ rbd_dev->spec->image_id, snap_id);
}
-static void __rbd_obj_setup_write(struct rbd_obj_request *obj_req,
- unsigned int which)
+static int rbd_object_map_lock(struct rbd_device *rbd_dev)
{
- struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
- u16 opcode;
-
- osd_req_op_alloc_hint_init(obj_req->osd_req, which++,
- rbd_dev->layout.object_size,
- rbd_dev->layout.object_size);
-
- if (rbd_obj_is_entire(obj_req))
- opcode = CEPH_OSD_OP_WRITEFULL;
- else
- opcode = CEPH_OSD_OP_WRITE;
+ struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc;
+ CEPH_DEFINE_OID_ONSTACK(oid);
+ u8 lock_type;
+ char *lock_tag;
+ struct ceph_locker *lockers;
+ u32 num_lockers;
+ bool broke_lock = false;
+ int ret;
- osd_req_op_extent_init(obj_req->osd_req, which, opcode,
- obj_req->ex.oe_off, obj_req->ex.oe_len, 0, 0);
- rbd_osd_req_setup_data(obj_req, which++);
+ rbd_object_map_name(rbd_dev, CEPH_NOSNAP, &oid);
- rbd_assert(which == obj_req->osd_req->r_num_ops);
- rbd_osd_req_format_write(obj_req);
-}
+again:
+ ret = ceph_cls_lock(osdc, &oid, &rbd_dev->header_oloc, RBD_LOCK_NAME,
+ CEPH_CLS_LOCK_EXCLUSIVE, "", "", "", 0);
+ if (ret != -EBUSY || broke_lock) {
+ if (ret == -EEXIST)
+ ret = 0; /* already locked by myself */
+ if (ret)
+ rbd_warn(rbd_dev, "failed to lock object map: %d", ret);
+ return ret;
+ }
-static int rbd_obj_setup_write(struct rbd_obj_request *obj_req)
-{
- unsigned int num_osd_ops, which = 0;
- bool need_guard;
- int ret;
+ ret = ceph_cls_lock_info(osdc, &oid, &rbd_dev->header_oloc,
+ RBD_LOCK_NAME, &lock_type, &lock_tag,
+ &lockers, &num_lockers);
+ if (ret) {
+ if (ret == -ENOENT)
+ goto again;
- /* reverse map the entire object onto the parent */
- ret = rbd_obj_calc_img_extents(obj_req, true);
- if (ret)
+ rbd_warn(rbd_dev, "failed to get object map lockers: %d", ret);
return ret;
+ }
- need_guard = rbd_obj_copyup_enabled(obj_req);
- num_osd_ops = need_guard + count_write_ops(obj_req);
+ kfree(lock_tag);
+ if (num_lockers == 0)
+ goto again;
- obj_req->osd_req = rbd_osd_req_create(obj_req, num_osd_ops);
- if (!obj_req->osd_req)
- return -ENOMEM;
+ rbd_warn(rbd_dev, "breaking object map lock owned by %s%llu",
+ ENTITY_NAME(lockers[0].id.name));
- if (need_guard) {
- ret = __rbd_obj_setup_stat(obj_req, which++);
- if (ret)
- return ret;
+ ret = ceph_cls_break_lock(osdc, &oid, &rbd_dev->header_oloc,
+ RBD_LOCK_NAME, lockers[0].id.cookie,
+ &lockers[0].id.name);
+ ceph_free_lockers(lockers, num_lockers);
+ if (ret) {
+ if (ret == -ENOENT)
+ goto again;
- obj_req->write_state = RBD_OBJ_WRITE_GUARD;
- } else {
- obj_req->write_state = RBD_OBJ_WRITE_FLAT;
+ rbd_warn(rbd_dev, "failed to break object map lock: %d", ret);
+ return ret;
}
- __rbd_obj_setup_write(obj_req, which);
- return 0;
+ broke_lock = true;
+ goto again;
}
-static u16 truncate_or_zero_opcode(struct rbd_obj_request *obj_req)
+static void rbd_object_map_unlock(struct rbd_device *rbd_dev)
{
- return rbd_obj_is_tail(obj_req) ? CEPH_OSD_OP_TRUNCATE :
- CEPH_OSD_OP_ZERO;
+ struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc;
+ CEPH_DEFINE_OID_ONSTACK(oid);
+ int ret;
+
+ rbd_object_map_name(rbd_dev, CEPH_NOSNAP, &oid);
+
+ ret = ceph_cls_unlock(osdc, &oid, &rbd_dev->header_oloc, RBD_LOCK_NAME,
+ "");
+ if (ret && ret != -ENOENT)
+ rbd_warn(rbd_dev, "failed to unlock object map: %d", ret);
}
-static int rbd_obj_setup_discard(struct rbd_obj_request *obj_req)
+static int decode_object_map_header(void **p, void *end, u64 *object_map_size)
{
- struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
- u64 off = obj_req->ex.oe_off;
- u64 next_off = obj_req->ex.oe_off + obj_req->ex.oe_len;
+ u8 struct_v;
+ u32 struct_len;
+ u32 header_len;
+ void *header_end;
int ret;
- /*
- * Align the range to alloc_size boundary and punt on discards
- * that are too small to free up any space.
- *
- * alloc_size == object_size && is_tail() is a special case for
- * filestore with filestore_punch_hole = false, needed to allow
- * truncate (in addition to delete).
- */
- if (rbd_dev->opts->alloc_size != rbd_dev->layout.object_size ||
- !rbd_obj_is_tail(obj_req)) {
- off = round_up(off, rbd_dev->opts->alloc_size);
- next_off = round_down(next_off, rbd_dev->opts->alloc_size);
- if (off >= next_off)
- return 1;
- }
+ ceph_decode_32_safe(p, end, header_len, e_inval);
+ header_end = *p + header_len;
- /* reverse map the entire object onto the parent */
- ret = rbd_obj_calc_img_extents(obj_req, true);
+ ret = ceph_start_decoding(p, end, 1, "BitVector header", &struct_v,
+ &struct_len);
if (ret)
return ret;
- obj_req->osd_req = rbd_osd_req_create(obj_req, 1);
- if (!obj_req->osd_req)
- return -ENOMEM;
-
- if (rbd_obj_is_entire(obj_req) && !obj_req->num_img_extents) {
- osd_req_op_init(obj_req->osd_req, 0, CEPH_OSD_OP_DELETE, 0);
- } else {
- dout("%s %p %llu~%llu -> %llu~%llu\n", __func__,
- obj_req, obj_req->ex.oe_off, obj_req->ex.oe_len,
- off, next_off - off);
- osd_req_op_extent_init(obj_req->osd_req, 0,
- truncate_or_zero_opcode(obj_req),
- off, next_off - off, 0, 0);
- }
+ ceph_decode_64_safe(p, end, *object_map_size, e_inval);
- obj_req->write_state = RBD_OBJ_WRITE_FLAT;
- rbd_osd_req_format_write(obj_req);
+ *p = header_end;
return 0;
+
+e_inval:
+ return -EINVAL;
}
-static int count_zeroout_ops(struct rbd_obj_request *obj_req)
+static int __rbd_object_map_load(struct rbd_device *rbd_dev)
{
- int num_osd_ops;
+ struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc;
+ CEPH_DEFINE_OID_ONSTACK(oid);
+ struct page **pages;
+ void *p, *end;
+ size_t reply_len;
+ u64 num_objects;
+ u64 object_map_bytes;
+ u64 object_map_size;
+ int num_pages;
+ int ret;
- if (rbd_obj_is_entire(obj_req) && obj_req->num_img_extents &&
- !rbd_obj_copyup_enabled(obj_req))
- num_osd_ops = 2; /* create + truncate */
- else
- num_osd_ops = 1; /* delete/truncate/zero */
+ rbd_assert(!rbd_dev->object_map && !rbd_dev->object_map_size);
- return num_osd_ops;
-}
+ num_objects = ceph_get_num_objects(&rbd_dev->layout,
+ rbd_dev->mapping.size);
+ object_map_bytes = DIV_ROUND_UP_ULL(num_objects * BITS_PER_OBJ,
+ BITS_PER_BYTE);
+ num_pages = calc_pages_for(0, object_map_bytes) + 1;
+ pages = ceph_alloc_page_vector(num_pages, GFP_KERNEL);
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
-static void __rbd_obj_setup_zeroout(struct rbd_obj_request *obj_req,
- unsigned int which)
-{
- u16 opcode;
+ reply_len = num_pages * PAGE_SIZE;
+ rbd_object_map_name(rbd_dev, rbd_dev->spec->snap_id, &oid);
+ ret = ceph_osdc_call(osdc, &oid, &rbd_dev->header_oloc,
+ "rbd", "object_map_load", CEPH_OSD_FLAG_READ,
+ NULL, 0, pages, &reply_len);
+ if (ret)
+ goto out;
- if (rbd_obj_is_entire(obj_req)) {
- if (obj_req->num_img_extents) {
- if (!rbd_obj_copyup_enabled(obj_req))
- osd_req_op_init(obj_req->osd_req, which++,
- CEPH_OSD_OP_CREATE, 0);
- opcode = CEPH_OSD_OP_TRUNCATE;
- } else {
- osd_req_op_init(obj_req->osd_req, which++,
- CEPH_OSD_OP_DELETE, 0);
- opcode = 0;
- }
- } else {
- opcode = truncate_or_zero_opcode(obj_req);
+ p = page_address(pages[0]);
+ end = p + min(reply_len, (size_t)PAGE_SIZE);
+ ret = decode_object_map_header(&p, end, &object_map_size);
+ if (ret)
+ goto out;
+
+ if (object_map_size != num_objects) {
+ rbd_warn(rbd_dev, "object map size mismatch: %llu vs %llu",
+ object_map_size, num_objects);
+ ret = -EINVAL;
+ goto out;
}
- if (opcode)
- osd_req_op_extent_init(obj_req->osd_req, which++, opcode,
- obj_req->ex.oe_off, obj_req->ex.oe_len,
- 0, 0);
+ if (offset_in_page(p) + object_map_bytes > reply_len) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ rbd_dev->object_map = kvmalloc(object_map_bytes, GFP_KERNEL);
+ if (!rbd_dev->object_map) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ rbd_dev->object_map_size = object_map_size;
+ ceph_copy_from_page_vector(pages, rbd_dev->object_map,
+ offset_in_page(p), object_map_bytes);
+
+out:
+ ceph_release_page_vector(pages, num_pages);
+ return ret;
+}
- rbd_assert(which == obj_req->osd_req->r_num_ops);
- rbd_osd_req_format_write(obj_req);
+static void rbd_object_map_free(struct rbd_device *rbd_dev)
+{
+ kvfree(rbd_dev->object_map);
+ rbd_dev->object_map = NULL;
+ rbd_dev->object_map_size = 0;
}
-static int rbd_obj_setup_zeroout(struct rbd_obj_request *obj_req)
+static int rbd_object_map_load(struct rbd_device *rbd_dev)
{
- unsigned int num_osd_ops, which = 0;
- bool need_guard;
int ret;
- /* reverse map the entire object onto the parent */
- ret = rbd_obj_calc_img_extents(obj_req, true);
+ ret = __rbd_object_map_load(rbd_dev);
if (ret)
return ret;
- need_guard = rbd_obj_copyup_enabled(obj_req);
- num_osd_ops = need_guard + count_zeroout_ops(obj_req);
-
- obj_req->osd_req = rbd_osd_req_create(obj_req, num_osd_ops);
- if (!obj_req->osd_req)
- return -ENOMEM;
+ ret = rbd_dev_v2_get_flags(rbd_dev);
+ if (ret) {
+ rbd_object_map_free(rbd_dev);
+ return ret;
+ }
+
+ if (rbd_dev->object_map_flags & RBD_FLAG_OBJECT_MAP_INVALID)
+ rbd_warn(rbd_dev, "object map is invalid");
+
+ return 0;
+}
+
+static int rbd_object_map_open(struct rbd_device *rbd_dev)
+{
+ int ret;
+
+ ret = rbd_object_map_lock(rbd_dev);
+ if (ret)
+ return ret;
+
+ ret = rbd_object_map_load(rbd_dev);
+ if (ret) {
+ rbd_object_map_unlock(rbd_dev);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void rbd_object_map_close(struct rbd_device *rbd_dev)
+{
+ rbd_object_map_free(rbd_dev);
+ rbd_object_map_unlock(rbd_dev);
+}
+
+/*
+ * This function needs snap_id (or more precisely just something to
+ * distinguish between HEAD and snapshot object maps), new_state and
+ * current_state that were passed to rbd_object_map_update().
+ *
+ * To avoid allocating and stashing a context we piggyback on the OSD
+ * request. A HEAD update has two ops (assert_locked). For new_state
+ * and current_state we decode our own object_map_update op, encoded in
+ * rbd_cls_object_map_update().
+ */
+static int rbd_object_map_update_finish(struct rbd_obj_request *obj_req,
+ struct ceph_osd_request *osd_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ struct ceph_osd_data *osd_data;
+ u64 objno;
+ u8 state, new_state, current_state;
+ bool has_current_state;
+ void *p;
+
+ if (osd_req->r_result)
+ return osd_req->r_result;
+
+ /*
+ * Nothing to do for a snapshot object map.
+ */
+ if (osd_req->r_num_ops == 1)
+ return 0;
+
+ /*
+ * Update in-memory HEAD object map.
+ */
+ rbd_assert(osd_req->r_num_ops == 2);
+ osd_data = osd_req_op_data(osd_req, 1, cls, request_data);
+ rbd_assert(osd_data->type == CEPH_OSD_DATA_TYPE_PAGES);
+
+ p = page_address(osd_data->pages[0]);
+ objno = ceph_decode_64(&p);
+ rbd_assert(objno == obj_req->ex.oe_objno);
+ rbd_assert(ceph_decode_64(&p) == objno + 1);
+ new_state = ceph_decode_8(&p);
+ has_current_state = ceph_decode_8(&p);
+ if (has_current_state)
+ current_state = ceph_decode_8(&p);
+
+ spin_lock(&rbd_dev->object_map_lock);
+ state = __rbd_object_map_get(rbd_dev, objno);
+ if (!has_current_state || current_state == state ||
+ (current_state == OBJECT_EXISTS && state == OBJECT_EXISTS_CLEAN))
+ __rbd_object_map_set(rbd_dev, objno, new_state);
+ spin_unlock(&rbd_dev->object_map_lock);
+
+ return 0;
+}
+
+static void rbd_object_map_callback(struct ceph_osd_request *osd_req)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+ int result;
+
+ dout("%s osd_req %p result %d for obj_req %p\n", __func__, osd_req,
+ osd_req->r_result, obj_req);
+
+ result = rbd_object_map_update_finish(obj_req, osd_req);
+ rbd_obj_handle_request(obj_req, result);
+}
+
+static bool update_needed(struct rbd_device *rbd_dev, u64 objno, u8 new_state)
+{
+ u8 state = rbd_object_map_get(rbd_dev, objno);
- if (need_guard) {
- ret = __rbd_obj_setup_stat(obj_req, which++);
+ if (state == new_state ||
+ (new_state == OBJECT_PENDING && state == OBJECT_NONEXISTENT) ||
+ (new_state == OBJECT_NONEXISTENT && state != OBJECT_PENDING))
+ return false;
+
+ return true;
+}
+
+static int rbd_cls_object_map_update(struct ceph_osd_request *req,
+ int which, u64 objno, u8 new_state,
+ const u8 *current_state)
+{
+ struct page **pages;
+ void *p, *start;
+ int ret;
+
+ ret = osd_req_op_cls_init(req, which, "rbd", "object_map_update");
+ if (ret)
+ return ret;
+
+ pages = ceph_alloc_page_vector(1, GFP_NOIO);
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
+
+ p = start = page_address(pages[0]);
+ ceph_encode_64(&p, objno);
+ ceph_encode_64(&p, objno + 1);
+ ceph_encode_8(&p, new_state);
+ if (current_state) {
+ ceph_encode_8(&p, 1);
+ ceph_encode_8(&p, *current_state);
+ } else {
+ ceph_encode_8(&p, 0);
+ }
+
+ osd_req_op_cls_request_data_pages(req, which, pages, p - start, 0,
+ false, true);
+ return 0;
+}
+
+/*
+ * Return:
+ * 0 - object map update sent
+ * 1 - object map update isn't needed
+ * <0 - error
+ */
+static int rbd_object_map_update(struct rbd_obj_request *obj_req, u64 snap_id,
+ u8 new_state, const u8 *current_state)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc;
+ struct ceph_osd_request *req;
+ int num_ops = 1;
+ int which = 0;
+ int ret;
+
+ if (snap_id == CEPH_NOSNAP) {
+ if (!update_needed(rbd_dev, obj_req->ex.oe_objno, new_state))
+ return 1;
+
+ num_ops++; /* assert_locked */
+ }
+
+ req = ceph_osdc_alloc_request(osdc, NULL, num_ops, false, GFP_NOIO);
+ if (!req)
+ return -ENOMEM;
+
+ list_add_tail(&req->r_private_item, &obj_req->osd_reqs);
+ req->r_callback = rbd_object_map_callback;
+ req->r_priv = obj_req;
+
+ rbd_object_map_name(rbd_dev, snap_id, &req->r_base_oid);
+ ceph_oloc_copy(&req->r_base_oloc, &rbd_dev->header_oloc);
+ req->r_flags = CEPH_OSD_FLAG_WRITE;
+ ktime_get_real_ts64(&req->r_mtime);
+
+ if (snap_id == CEPH_NOSNAP) {
+ /*
+ * Protect against possible race conditions during lock
+ * ownership transitions.
+ */
+ ret = ceph_cls_assert_locked(req, which++, RBD_LOCK_NAME,
+ CEPH_CLS_LOCK_EXCLUSIVE, "", "");
if (ret)
return ret;
+ }
+
+ ret = rbd_cls_object_map_update(req, which, obj_req->ex.oe_objno,
+ new_state, current_state);
+ if (ret)
+ return ret;
+
+ ret = ceph_osdc_alloc_messages(req, GFP_NOIO);
+ if (ret)
+ return ret;
- obj_req->write_state = RBD_OBJ_WRITE_GUARD;
+ ceph_osdc_start_request(osdc, req, false);
+ return 0;
+}
+
+static void prune_extents(struct ceph_file_extent *img_extents,
+ u32 *num_img_extents, u64 overlap)
+{
+ u32 cnt = *num_img_extents;
+
+ /* drop extents completely beyond the overlap */
+ while (cnt && img_extents[cnt - 1].fe_off >= overlap)
+ cnt--;
+
+ if (cnt) {
+ struct ceph_file_extent *ex = &img_extents[cnt - 1];
+
+ /* trim final overlapping extent */
+ if (ex->fe_off + ex->fe_len > overlap)
+ ex->fe_len = overlap - ex->fe_off;
+ }
+
+ *num_img_extents = cnt;
+}
+
+/*
+ * Determine the byte range(s) covered by either just the object extent
+ * or the entire object in the parent image.
+ */
+static int rbd_obj_calc_img_extents(struct rbd_obj_request *obj_req,
+ bool entire)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ int ret;
+
+ if (!rbd_dev->parent_overlap)
+ return 0;
+
+ ret = ceph_extent_to_file(&rbd_dev->layout, obj_req->ex.oe_objno,
+ entire ? 0 : obj_req->ex.oe_off,
+ entire ? rbd_dev->layout.object_size :
+ obj_req->ex.oe_len,
+ &obj_req->img_extents,
+ &obj_req->num_img_extents);
+ if (ret)
+ return ret;
+
+ prune_extents(obj_req->img_extents, &obj_req->num_img_extents,
+ rbd_dev->parent_overlap);
+ return 0;
+}
+
+static void rbd_osd_setup_data(struct ceph_osd_request *osd_req, int which)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+
+ switch (obj_req->img_request->data_type) {
+ case OBJ_REQUEST_BIO:
+ osd_req_op_extent_osd_data_bio(osd_req, which,
+ &obj_req->bio_pos,
+ obj_req->ex.oe_len);
+ break;
+ case OBJ_REQUEST_BVECS:
+ case OBJ_REQUEST_OWN_BVECS:
+ rbd_assert(obj_req->bvec_pos.iter.bi_size ==
+ obj_req->ex.oe_len);
+ rbd_assert(obj_req->bvec_idx == obj_req->bvec_count);
+ osd_req_op_extent_osd_data_bvec_pos(osd_req, which,
+ &obj_req->bvec_pos);
+ break;
+ default:
+ BUG();
+ }
+}
+
+static int rbd_osd_setup_stat(struct ceph_osd_request *osd_req, int which)
+{
+ struct page **pages;
+
+ /*
+ * The response data for a STAT call consists of:
+ * le64 length;
+ * struct {
+ * le32 tv_sec;
+ * le32 tv_nsec;
+ * } mtime;
+ */
+ pages = ceph_alloc_page_vector(1, GFP_NOIO);
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
+
+ osd_req_op_init(osd_req, which, CEPH_OSD_OP_STAT, 0);
+ osd_req_op_raw_data_in_pages(osd_req, which, pages,
+ 8 + sizeof(struct ceph_timespec),
+ 0, false, true);
+ return 0;
+}
+
+static int rbd_osd_setup_copyup(struct ceph_osd_request *osd_req, int which,
+ u32 bytes)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+ int ret;
+
+ ret = osd_req_op_cls_init(osd_req, which, "rbd", "copyup");
+ if (ret)
+ return ret;
+
+ osd_req_op_cls_request_data_bvecs(osd_req, which, obj_req->copyup_bvecs,
+ obj_req->copyup_bvec_count, bytes);
+ return 0;
+}
+
+static int rbd_obj_init_read(struct rbd_obj_request *obj_req)
+{
+ obj_req->read_state = RBD_OBJ_READ_START;
+ return 0;
+}
+
+static void __rbd_osd_setup_write_ops(struct ceph_osd_request *osd_req,
+ int which)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ u16 opcode;
+
+ if (!use_object_map(rbd_dev) ||
+ !(obj_req->flags & RBD_OBJ_FLAG_MAY_EXIST)) {
+ osd_req_op_alloc_hint_init(osd_req, which++,
+ rbd_dev->layout.object_size,
+ rbd_dev->layout.object_size);
+ }
+
+ if (rbd_obj_is_entire(obj_req))
+ opcode = CEPH_OSD_OP_WRITEFULL;
+ else
+ opcode = CEPH_OSD_OP_WRITE;
+
+ osd_req_op_extent_init(osd_req, which, opcode,
+ obj_req->ex.oe_off, obj_req->ex.oe_len, 0, 0);
+ rbd_osd_setup_data(osd_req, which);
+}
+
+static int rbd_obj_init_write(struct rbd_obj_request *obj_req)
+{
+ int ret;
+
+ /* reverse map the entire object onto the parent */
+ ret = rbd_obj_calc_img_extents(obj_req, true);
+ if (ret)
+ return ret;
+
+ if (rbd_obj_copyup_enabled(obj_req))
+ obj_req->flags |= RBD_OBJ_FLAG_COPYUP_ENABLED;
+
+ obj_req->write_state = RBD_OBJ_WRITE_START;
+ return 0;
+}
+
+static u16 truncate_or_zero_opcode(struct rbd_obj_request *obj_req)
+{
+ return rbd_obj_is_tail(obj_req) ? CEPH_OSD_OP_TRUNCATE :
+ CEPH_OSD_OP_ZERO;
+}
+
+static void __rbd_osd_setup_discard_ops(struct ceph_osd_request *osd_req,
+ int which)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+
+ if (rbd_obj_is_entire(obj_req) && !obj_req->num_img_extents) {
+ rbd_assert(obj_req->flags & RBD_OBJ_FLAG_DELETION);
+ osd_req_op_init(osd_req, which, CEPH_OSD_OP_DELETE, 0);
} else {
- obj_req->write_state = RBD_OBJ_WRITE_FLAT;
+ osd_req_op_extent_init(osd_req, which,
+ truncate_or_zero_opcode(obj_req),
+ obj_req->ex.oe_off, obj_req->ex.oe_len,
+ 0, 0);
+ }
+}
+
+static int rbd_obj_init_discard(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ u64 off, next_off;
+ int ret;
+
+ /*
+ * Align the range to alloc_size boundary and punt on discards
+ * that are too small to free up any space.
+ *
+ * alloc_size == object_size && is_tail() is a special case for
+ * filestore with filestore_punch_hole = false, needed to allow
+ * truncate (in addition to delete).
+ */
+ if (rbd_dev->opts->alloc_size != rbd_dev->layout.object_size ||
+ !rbd_obj_is_tail(obj_req)) {
+ off = round_up(obj_req->ex.oe_off, rbd_dev->opts->alloc_size);
+ next_off = round_down(obj_req->ex.oe_off + obj_req->ex.oe_len,
+ rbd_dev->opts->alloc_size);
+ if (off >= next_off)
+ return 1;
+
+ dout("%s %p %llu~%llu -> %llu~%llu\n", __func__,
+ obj_req, obj_req->ex.oe_off, obj_req->ex.oe_len,
+ off, next_off - off);
+ obj_req->ex.oe_off = off;
+ obj_req->ex.oe_len = next_off - off;
+ }
+
+ /* reverse map the entire object onto the parent */
+ ret = rbd_obj_calc_img_extents(obj_req, true);
+ if (ret)
+ return ret;
+
+ obj_req->flags |= RBD_OBJ_FLAG_NOOP_FOR_NONEXISTENT;
+ if (rbd_obj_is_entire(obj_req) && !obj_req->num_img_extents)
+ obj_req->flags |= RBD_OBJ_FLAG_DELETION;
+
+ obj_req->write_state = RBD_OBJ_WRITE_START;
+ return 0;
+}
+
+static void __rbd_osd_setup_zeroout_ops(struct ceph_osd_request *osd_req,
+ int which)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+ u16 opcode;
+
+ if (rbd_obj_is_entire(obj_req)) {
+ if (obj_req->num_img_extents) {
+ if (!(obj_req->flags & RBD_OBJ_FLAG_COPYUP_ENABLED))
+ osd_req_op_init(osd_req, which++,
+ CEPH_OSD_OP_CREATE, 0);
+ opcode = CEPH_OSD_OP_TRUNCATE;
+ } else {
+ rbd_assert(obj_req->flags & RBD_OBJ_FLAG_DELETION);
+ osd_req_op_init(osd_req, which++,
+ CEPH_OSD_OP_DELETE, 0);
+ opcode = 0;
+ }
+ } else {
+ opcode = truncate_or_zero_opcode(obj_req);
+ }
+
+ if (opcode)
+ osd_req_op_extent_init(osd_req, which, opcode,
+ obj_req->ex.oe_off, obj_req->ex.oe_len,
+ 0, 0);
+}
+
+static int rbd_obj_init_zeroout(struct rbd_obj_request *obj_req)
+{
+ int ret;
+
+ /* reverse map the entire object onto the parent */
+ ret = rbd_obj_calc_img_extents(obj_req, true);
+ if (ret)
+ return ret;
+
+ if (rbd_obj_copyup_enabled(obj_req))
+ obj_req->flags |= RBD_OBJ_FLAG_COPYUP_ENABLED;
+ if (!obj_req->num_img_extents) {
+ obj_req->flags |= RBD_OBJ_FLAG_NOOP_FOR_NONEXISTENT;
+ if (rbd_obj_is_entire(obj_req))
+ obj_req->flags |= RBD_OBJ_FLAG_DELETION;
}
- __rbd_obj_setup_zeroout(obj_req, which);
+ obj_req->write_state = RBD_OBJ_WRITE_START;
return 0;
}
+static int count_write_ops(struct rbd_obj_request *obj_req)
+{
+ struct rbd_img_request *img_req = obj_req->img_request;
+
+ switch (img_req->op_type) {
+ case OBJ_OP_WRITE:
+ if (!use_object_map(img_req->rbd_dev) ||
+ !(obj_req->flags & RBD_OBJ_FLAG_MAY_EXIST))
+ return 2; /* setallochint + write/writefull */
+
+ return 1; /* write/writefull */
+ case OBJ_OP_DISCARD:
+ return 1; /* delete/truncate/zero */
+ case OBJ_OP_ZEROOUT:
+ if (rbd_obj_is_entire(obj_req) && obj_req->num_img_extents &&
+ !(obj_req->flags & RBD_OBJ_FLAG_COPYUP_ENABLED))
+ return 2; /* create + truncate */
+
+ return 1; /* delete/truncate/zero */
+ default:
+ BUG();
+ }
+}
+
+static void rbd_osd_setup_write_ops(struct ceph_osd_request *osd_req,
+ int which)
+{
+ struct rbd_obj_request *obj_req = osd_req->r_priv;
+
+ switch (obj_req->img_request->op_type) {
+ case OBJ_OP_WRITE:
+ __rbd_osd_setup_write_ops(osd_req, which);
+ break;
+ case OBJ_OP_DISCARD:
+ __rbd_osd_setup_discard_ops(osd_req, which);
+ break;
+ case OBJ_OP_ZEROOUT:
+ __rbd_osd_setup_zeroout_ops(osd_req, which);
+ break;
+ default:
+ BUG();
+ }
+}
+
/*
- * For each object request in @img_req, allocate an OSD request, add
- * individual OSD ops and prepare them for submission. The number of
- * OSD ops depends on op_type and the overlap point (if any).
+ * Prune the list of object requests (adjust offset and/or length, drop
+ * redundant requests). Prepare object request state machines and image
+ * request state machine for execution.
*/
static int __rbd_img_fill_request(struct rbd_img_request *img_req)
{
for_each_obj_request_safe(img_req, obj_req, next_obj_req) {
switch (img_req->op_type) {
case OBJ_OP_READ:
- ret = rbd_obj_setup_read(obj_req);
+ ret = rbd_obj_init_read(obj_req);
break;
case OBJ_OP_WRITE:
- ret = rbd_obj_setup_write(obj_req);
+ ret = rbd_obj_init_write(obj_req);
break;
case OBJ_OP_DISCARD:
- ret = rbd_obj_setup_discard(obj_req);
+ ret = rbd_obj_init_discard(obj_req);
break;
case OBJ_OP_ZEROOUT:
- ret = rbd_obj_setup_zeroout(obj_req);
+ ret = rbd_obj_init_zeroout(obj_req);
break;
default:
BUG();
if (ret < 0)
return ret;
if (ret > 0) {
- img_req->xferred += obj_req->ex.oe_len;
- img_req->pending_count--;
rbd_img_obj_request_del(img_req, obj_req);
continue;
}
-
- ret = ceph_osdc_alloc_messages(obj_req->osd_req, GFP_NOIO);
- if (ret)
- return ret;
}
+ img_req->state = RBD_IMG_START;
return 0;
}
&it);
}
-static void rbd_img_request_submit(struct rbd_img_request *img_request)
+static void rbd_img_handle_request_work(struct work_struct *work)
{
- struct rbd_obj_request *obj_request;
+ struct rbd_img_request *img_req =
+ container_of(work, struct rbd_img_request, work);
- dout("%s: img %p\n", __func__, img_request);
+ rbd_img_handle_request(img_req, img_req->work_result);
+}
- rbd_img_request_get(img_request);
- for_each_obj_request(img_request, obj_request)
- rbd_obj_request_submit(obj_request);
+static void rbd_img_schedule(struct rbd_img_request *img_req, int result)
+{
+ INIT_WORK(&img_req->work, rbd_img_handle_request_work);
+ img_req->work_result = result;
+ queue_work(rbd_wq, &img_req->work);
+}
- rbd_img_request_put(img_request);
+static bool rbd_obj_may_exist(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+
+ if (rbd_object_map_may_exist(rbd_dev, obj_req->ex.oe_objno)) {
+ obj_req->flags |= RBD_OBJ_FLAG_MAY_EXIST;
+ return true;
+ }
+
+ dout("%s %p objno %llu assuming dne\n", __func__, obj_req,
+ obj_req->ex.oe_objno);
+ return false;
+}
+
+static int rbd_obj_read_object(struct rbd_obj_request *obj_req)
+{
+ struct ceph_osd_request *osd_req;
+ int ret;
+
+ osd_req = __rbd_obj_add_osd_request(obj_req, NULL, 1);
+ if (IS_ERR(osd_req))
+ return PTR_ERR(osd_req);
+
+ osd_req_op_extent_init(osd_req, 0, CEPH_OSD_OP_READ,
+ obj_req->ex.oe_off, obj_req->ex.oe_len, 0, 0);
+ rbd_osd_setup_data(osd_req, 0);
+ rbd_osd_format_read(osd_req);
+
+ ret = ceph_osdc_alloc_messages(osd_req, GFP_NOIO);
+ if (ret)
+ return ret;
+
+ rbd_osd_submit(osd_req);
+ return 0;
}
static int rbd_obj_read_from_parent(struct rbd_obj_request *obj_req)
return ret;
}
- rbd_img_request_submit(child_img_req);
+ /* avoid parent chain recursion */
+ rbd_img_schedule(child_img_req, 0);
return 0;
}
-static bool rbd_obj_handle_read(struct rbd_obj_request *obj_req)
+static bool rbd_obj_advance_read(struct rbd_obj_request *obj_req, int *result)
{
struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
int ret;
- if (obj_req->result == -ENOENT &&
- rbd_dev->parent_overlap && !obj_req->tried_parent) {
- /* reverse map this object extent onto the parent */
- ret = rbd_obj_calc_img_extents(obj_req, false);
+again:
+ switch (obj_req->read_state) {
+ case RBD_OBJ_READ_START:
+ rbd_assert(!*result);
+
+ if (!rbd_obj_may_exist(obj_req)) {
+ *result = -ENOENT;
+ obj_req->read_state = RBD_OBJ_READ_OBJECT;
+ goto again;
+ }
+
+ ret = rbd_obj_read_object(obj_req);
if (ret) {
- obj_req->result = ret;
+ *result = ret;
return true;
}
-
- if (obj_req->num_img_extents) {
- obj_req->tried_parent = true;
- ret = rbd_obj_read_from_parent(obj_req);
+ obj_req->read_state = RBD_OBJ_READ_OBJECT;
+ return false;
+ case RBD_OBJ_READ_OBJECT:
+ if (*result == -ENOENT && rbd_dev->parent_overlap) {
+ /* reverse map this object extent onto the parent */
+ ret = rbd_obj_calc_img_extents(obj_req, false);
if (ret) {
- obj_req->result = ret;
+ *result = ret;
return true;
}
- return false;
+ if (obj_req->num_img_extents) {
+ ret = rbd_obj_read_from_parent(obj_req);
+ if (ret) {
+ *result = ret;
+ return true;
+ }
+ obj_req->read_state = RBD_OBJ_READ_PARENT;
+ return false;
+ }
}
+
+ /*
+ * -ENOENT means a hole in the image -- zero-fill the entire
+ * length of the request. A short read also implies zero-fill
+ * to the end of the request.
+ */
+ if (*result == -ENOENT) {
+ rbd_obj_zero_range(obj_req, 0, obj_req->ex.oe_len);
+ *result = 0;
+ } else if (*result >= 0) {
+ if (*result < obj_req->ex.oe_len)
+ rbd_obj_zero_range(obj_req, *result,
+ obj_req->ex.oe_len - *result);
+ else
+ rbd_assert(*result == obj_req->ex.oe_len);
+ *result = 0;
+ }
+ return true;
+ case RBD_OBJ_READ_PARENT:
+ return true;
+ default:
+ BUG();
}
+}
+
+static bool rbd_obj_write_is_noop(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+
+ if (rbd_object_map_may_exist(rbd_dev, obj_req->ex.oe_objno))
+ obj_req->flags |= RBD_OBJ_FLAG_MAY_EXIST;
+
+ if (!(obj_req->flags & RBD_OBJ_FLAG_MAY_EXIST) &&
+ (obj_req->flags & RBD_OBJ_FLAG_NOOP_FOR_NONEXISTENT)) {
+ dout("%s %p noop for nonexistent\n", __func__, obj_req);
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * Return:
+ * 0 - object map update sent
+ * 1 - object map update isn't needed
+ * <0 - error
+ */
+static int rbd_obj_write_pre_object_map(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ u8 new_state;
+
+ if (!(rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP))
+ return 1;
+
+ if (obj_req->flags & RBD_OBJ_FLAG_DELETION)
+ new_state = OBJECT_PENDING;
+ else
+ new_state = OBJECT_EXISTS;
+
+ return rbd_object_map_update(obj_req, CEPH_NOSNAP, new_state, NULL);
+}
+
+static int rbd_obj_write_object(struct rbd_obj_request *obj_req)
+{
+ struct ceph_osd_request *osd_req;
+ int num_ops = count_write_ops(obj_req);
+ int which = 0;
+ int ret;
- /*
- * -ENOENT means a hole in the image -- zero-fill the entire
- * length of the request. A short read also implies zero-fill
- * to the end of the request. In both cases we update xferred
- * count to indicate the whole request was satisfied.
- */
- if (obj_req->result == -ENOENT ||
- (!obj_req->result && obj_req->xferred < obj_req->ex.oe_len)) {
- rbd_assert(!obj_req->xferred || !obj_req->result);
- rbd_obj_zero_range(obj_req, obj_req->xferred,
- obj_req->ex.oe_len - obj_req->xferred);
- obj_req->result = 0;
- obj_req->xferred = obj_req->ex.oe_len;
+ if (obj_req->flags & RBD_OBJ_FLAG_COPYUP_ENABLED)
+ num_ops++; /* stat */
+
+ osd_req = rbd_obj_add_osd_request(obj_req, num_ops);
+ if (IS_ERR(osd_req))
+ return PTR_ERR(osd_req);
+
+ if (obj_req->flags & RBD_OBJ_FLAG_COPYUP_ENABLED) {
+ ret = rbd_osd_setup_stat(osd_req, which++);
+ if (ret)
+ return ret;
}
- return true;
+ rbd_osd_setup_write_ops(osd_req, which);
+ rbd_osd_format_write(osd_req);
+
+ ret = ceph_osdc_alloc_messages(osd_req, GFP_NOIO);
+ if (ret)
+ return ret;
+
+ rbd_osd_submit(osd_req);
+ return 0;
}
/*
#define MODS_ONLY U32_MAX
-static int rbd_obj_issue_copyup_empty_snapc(struct rbd_obj_request *obj_req,
- u32 bytes)
+static int rbd_obj_copyup_empty_snapc(struct rbd_obj_request *obj_req,
+ u32 bytes)
{
+ struct ceph_osd_request *osd_req;
int ret;
dout("%s obj_req %p bytes %u\n", __func__, obj_req, bytes);
- rbd_assert(obj_req->osd_req->r_ops[0].op == CEPH_OSD_OP_STAT);
rbd_assert(bytes > 0 && bytes != MODS_ONLY);
- rbd_osd_req_destroy(obj_req->osd_req);
- obj_req->osd_req = __rbd_osd_req_create(obj_req, &rbd_empty_snapc, 1);
- if (!obj_req->osd_req)
- return -ENOMEM;
+ osd_req = __rbd_obj_add_osd_request(obj_req, &rbd_empty_snapc, 1);
+ if (IS_ERR(osd_req))
+ return PTR_ERR(osd_req);
- ret = osd_req_op_cls_init(obj_req->osd_req, 0, "rbd", "copyup");
+ ret = rbd_osd_setup_copyup(osd_req, 0, bytes);
if (ret)
return ret;
- osd_req_op_cls_request_data_bvecs(obj_req->osd_req, 0,
- obj_req->copyup_bvecs,
- obj_req->copyup_bvec_count,
- bytes);
- rbd_osd_req_format_write(obj_req);
+ rbd_osd_format_write(osd_req);
- ret = ceph_osdc_alloc_messages(obj_req->osd_req, GFP_NOIO);
+ ret = ceph_osdc_alloc_messages(osd_req, GFP_NOIO);
if (ret)
return ret;
- rbd_obj_request_submit(obj_req);
+ rbd_osd_submit(osd_req);
return 0;
}
-static int rbd_obj_issue_copyup_ops(struct rbd_obj_request *obj_req, u32 bytes)
+static int rbd_obj_copyup_current_snapc(struct rbd_obj_request *obj_req,
+ u32 bytes)
{
- struct rbd_img_request *img_req = obj_req->img_request;
- unsigned int num_osd_ops = (bytes != MODS_ONLY);
- unsigned int which = 0;
+ struct ceph_osd_request *osd_req;
+ int num_ops = count_write_ops(obj_req);
+ int which = 0;
int ret;
dout("%s obj_req %p bytes %u\n", __func__, obj_req, bytes);
- rbd_assert(obj_req->osd_req->r_ops[0].op == CEPH_OSD_OP_STAT ||
- obj_req->osd_req->r_ops[0].op == CEPH_OSD_OP_CALL);
- rbd_osd_req_destroy(obj_req->osd_req);
- switch (img_req->op_type) {
- case OBJ_OP_WRITE:
- num_osd_ops += count_write_ops(obj_req);
- break;
- case OBJ_OP_ZEROOUT:
- num_osd_ops += count_zeroout_ops(obj_req);
- break;
- default:
- BUG();
- }
+ if (bytes != MODS_ONLY)
+ num_ops++; /* copyup */
- obj_req->osd_req = rbd_osd_req_create(obj_req, num_osd_ops);
- if (!obj_req->osd_req)
- return -ENOMEM;
+ osd_req = rbd_obj_add_osd_request(obj_req, num_ops);
+ if (IS_ERR(osd_req))
+ return PTR_ERR(osd_req);
if (bytes != MODS_ONLY) {
- ret = osd_req_op_cls_init(obj_req->osd_req, which, "rbd",
- "copyup");
+ ret = rbd_osd_setup_copyup(osd_req, which++, bytes);
if (ret)
return ret;
-
- osd_req_op_cls_request_data_bvecs(obj_req->osd_req, which++,
- obj_req->copyup_bvecs,
- obj_req->copyup_bvec_count,
- bytes);
}
- switch (img_req->op_type) {
- case OBJ_OP_WRITE:
- __rbd_obj_setup_write(obj_req, which);
- break;
- case OBJ_OP_ZEROOUT:
- __rbd_obj_setup_zeroout(obj_req, which);
- break;
- default:
- BUG();
- }
+ rbd_osd_setup_write_ops(osd_req, which);
+ rbd_osd_format_write(osd_req);
- ret = ceph_osdc_alloc_messages(obj_req->osd_req, GFP_NOIO);
+ ret = ceph_osdc_alloc_messages(osd_req, GFP_NOIO);
if (ret)
return ret;
- rbd_obj_request_submit(obj_req);
+ rbd_osd_submit(osd_req);
return 0;
}
-static int rbd_obj_issue_copyup(struct rbd_obj_request *obj_req, u32 bytes)
-{
- /*
- * Only send non-zero copyup data to save some I/O and network
- * bandwidth -- zero copyup data is equivalent to the object not
- * existing.
- */
- if (is_zero_bvecs(obj_req->copyup_bvecs, bytes)) {
- dout("%s obj_req %p detected zeroes\n", __func__, obj_req);
- bytes = 0;
- }
-
- if (obj_req->img_request->snapc->num_snaps && bytes > 0) {
- /*
- * Send a copyup request with an empty snapshot context to
- * deep-copyup the object through all existing snapshots.
- * A second request with the current snapshot context will be
- * sent for the actual modification.
- */
- obj_req->write_state = RBD_OBJ_WRITE_COPYUP_EMPTY_SNAPC;
- return rbd_obj_issue_copyup_empty_snapc(obj_req, bytes);
- }
-
- obj_req->write_state = RBD_OBJ_WRITE_COPYUP_OPS;
- return rbd_obj_issue_copyup_ops(obj_req, bytes);
-}
-
static int setup_copyup_bvecs(struct rbd_obj_request *obj_req, u64 obj_overlap)
{
u32 i;
return 0;
}
-static int rbd_obj_handle_write_guard(struct rbd_obj_request *obj_req)
+/*
+ * The target object doesn't exist. Read the data for the entire
+ * target object up to the overlap point (if any) from the parent,
+ * so we can use it for a copyup.
+ */
+static int rbd_obj_copyup_read_parent(struct rbd_obj_request *obj_req)
{
struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
int ret;
* request -- pass MODS_ONLY since the copyup isn't needed
* anymore.
*/
- obj_req->write_state = RBD_OBJ_WRITE_COPYUP_OPS;
- return rbd_obj_issue_copyup_ops(obj_req, MODS_ONLY);
+ return rbd_obj_copyup_current_snapc(obj_req, MODS_ONLY);
}
ret = setup_copyup_bvecs(obj_req, rbd_obj_img_extents_bytes(obj_req));
if (ret)
return ret;
- obj_req->write_state = RBD_OBJ_WRITE_READ_FROM_PARENT;
return rbd_obj_read_from_parent(obj_req);
}
-static bool rbd_obj_handle_write(struct rbd_obj_request *obj_req)
+static void rbd_obj_copyup_object_maps(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ struct ceph_snap_context *snapc = obj_req->img_request->snapc;
+ u8 new_state;
+ u32 i;
+ int ret;
+
+ rbd_assert(!obj_req->pending.result && !obj_req->pending.num_pending);
+
+ if (!(rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP))
+ return;
+
+ if (obj_req->flags & RBD_OBJ_FLAG_COPYUP_ZEROS)
+ return;
+
+ for (i = 0; i < snapc->num_snaps; i++) {
+ if ((rbd_dev->header.features & RBD_FEATURE_FAST_DIFF) &&
+ i + 1 < snapc->num_snaps)
+ new_state = OBJECT_EXISTS_CLEAN;
+ else
+ new_state = OBJECT_EXISTS;
+
+ ret = rbd_object_map_update(obj_req, snapc->snaps[i],
+ new_state, NULL);
+ if (ret < 0) {
+ obj_req->pending.result = ret;
+ return;
+ }
+
+ rbd_assert(!ret);
+ obj_req->pending.num_pending++;
+ }
+}
+
+static void rbd_obj_copyup_write_object(struct rbd_obj_request *obj_req)
+{
+ u32 bytes = rbd_obj_img_extents_bytes(obj_req);
+ int ret;
+
+ rbd_assert(!obj_req->pending.result && !obj_req->pending.num_pending);
+
+ /*
+ * Only send non-zero copyup data to save some I/O and network
+ * bandwidth -- zero copyup data is equivalent to the object not
+ * existing.
+ */
+ if (obj_req->flags & RBD_OBJ_FLAG_COPYUP_ZEROS)
+ bytes = 0;
+
+ if (obj_req->img_request->snapc->num_snaps && bytes > 0) {
+ /*
+ * Send a copyup request with an empty snapshot context to
+ * deep-copyup the object through all existing snapshots.
+ * A second request with the current snapshot context will be
+ * sent for the actual modification.
+ */
+ ret = rbd_obj_copyup_empty_snapc(obj_req, bytes);
+ if (ret) {
+ obj_req->pending.result = ret;
+ return;
+ }
+
+ obj_req->pending.num_pending++;
+ bytes = MODS_ONLY;
+ }
+
+ ret = rbd_obj_copyup_current_snapc(obj_req, bytes);
+ if (ret) {
+ obj_req->pending.result = ret;
+ return;
+ }
+
+ obj_req->pending.num_pending++;
+}
+
+static bool rbd_obj_advance_copyup(struct rbd_obj_request *obj_req, int *result)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ int ret;
+
+again:
+ switch (obj_req->copyup_state) {
+ case RBD_OBJ_COPYUP_START:
+ rbd_assert(!*result);
+
+ ret = rbd_obj_copyup_read_parent(obj_req);
+ if (ret) {
+ *result = ret;
+ return true;
+ }
+ if (obj_req->num_img_extents)
+ obj_req->copyup_state = RBD_OBJ_COPYUP_READ_PARENT;
+ else
+ obj_req->copyup_state = RBD_OBJ_COPYUP_WRITE_OBJECT;
+ return false;
+ case RBD_OBJ_COPYUP_READ_PARENT:
+ if (*result)
+ return true;
+
+ if (is_zero_bvecs(obj_req->copyup_bvecs,
+ rbd_obj_img_extents_bytes(obj_req))) {
+ dout("%s %p detected zeros\n", __func__, obj_req);
+ obj_req->flags |= RBD_OBJ_FLAG_COPYUP_ZEROS;
+ }
+
+ rbd_obj_copyup_object_maps(obj_req);
+ if (!obj_req->pending.num_pending) {
+ *result = obj_req->pending.result;
+ obj_req->copyup_state = RBD_OBJ_COPYUP_OBJECT_MAPS;
+ goto again;
+ }
+ obj_req->copyup_state = __RBD_OBJ_COPYUP_OBJECT_MAPS;
+ return false;
+ case __RBD_OBJ_COPYUP_OBJECT_MAPS:
+ if (!pending_result_dec(&obj_req->pending, result))
+ return false;
+ /* fall through */
+ case RBD_OBJ_COPYUP_OBJECT_MAPS:
+ if (*result) {
+ rbd_warn(rbd_dev, "snap object map update failed: %d",
+ *result);
+ return true;
+ }
+
+ rbd_obj_copyup_write_object(obj_req);
+ if (!obj_req->pending.num_pending) {
+ *result = obj_req->pending.result;
+ obj_req->copyup_state = RBD_OBJ_COPYUP_WRITE_OBJECT;
+ goto again;
+ }
+ obj_req->copyup_state = __RBD_OBJ_COPYUP_WRITE_OBJECT;
+ return false;
+ case __RBD_OBJ_COPYUP_WRITE_OBJECT:
+ if (!pending_result_dec(&obj_req->pending, result))
+ return false;
+ /* fall through */
+ case RBD_OBJ_COPYUP_WRITE_OBJECT:
+ return true;
+ default:
+ BUG();
+ }
+}
+
+/*
+ * Return:
+ * 0 - object map update sent
+ * 1 - object map update isn't needed
+ * <0 - error
+ */
+static int rbd_obj_write_post_object_map(struct rbd_obj_request *obj_req)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ u8 current_state = OBJECT_PENDING;
+
+ if (!(rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP))
+ return 1;
+
+ if (!(obj_req->flags & RBD_OBJ_FLAG_DELETION))
+ return 1;
+
+ return rbd_object_map_update(obj_req, CEPH_NOSNAP, OBJECT_NONEXISTENT,
+ ¤t_state);
+}
+
+static bool rbd_obj_advance_write(struct rbd_obj_request *obj_req, int *result)
+{
+ struct rbd_device *rbd_dev = obj_req->img_request->rbd_dev;
+ int ret;
+
+again:
+ switch (obj_req->write_state) {
+ case RBD_OBJ_WRITE_START:
+ rbd_assert(!*result);
+
+ if (rbd_obj_write_is_noop(obj_req))
+ return true;
+
+ ret = rbd_obj_write_pre_object_map(obj_req);
+ if (ret < 0) {
+ *result = ret;
+ return true;
+ }
+ obj_req->write_state = RBD_OBJ_WRITE_PRE_OBJECT_MAP;
+ if (ret > 0)
+ goto again;
+ return false;
+ case RBD_OBJ_WRITE_PRE_OBJECT_MAP:
+ if (*result) {
+ rbd_warn(rbd_dev, "pre object map update failed: %d",
+ *result);
+ return true;
+ }
+ ret = rbd_obj_write_object(obj_req);
+ if (ret) {
+ *result = ret;
+ return true;
+ }
+ obj_req->write_state = RBD_OBJ_WRITE_OBJECT;
+ return false;
+ case RBD_OBJ_WRITE_OBJECT:
+ if (*result == -ENOENT) {
+ if (obj_req->flags & RBD_OBJ_FLAG_COPYUP_ENABLED) {
+ *result = 0;
+ obj_req->copyup_state = RBD_OBJ_COPYUP_START;
+ obj_req->write_state = __RBD_OBJ_WRITE_COPYUP;
+ goto again;
+ }
+ /*
+ * On a non-existent object:
+ * delete - -ENOENT, truncate/zero - 0
+ */
+ if (obj_req->flags & RBD_OBJ_FLAG_DELETION)
+ *result = 0;
+ }
+ if (*result)
+ return true;
+
+ obj_req->write_state = RBD_OBJ_WRITE_COPYUP;
+ goto again;
+ case __RBD_OBJ_WRITE_COPYUP:
+ if (!rbd_obj_advance_copyup(obj_req, result))
+ return false;
+ /* fall through */
+ case RBD_OBJ_WRITE_COPYUP:
+ if (*result) {
+ rbd_warn(rbd_dev, "copyup failed: %d", *result);
+ return true;
+ }
+ ret = rbd_obj_write_post_object_map(obj_req);
+ if (ret < 0) {
+ *result = ret;
+ return true;
+ }
+ obj_req->write_state = RBD_OBJ_WRITE_POST_OBJECT_MAP;
+ if (ret > 0)
+ goto again;
+ return false;
+ case RBD_OBJ_WRITE_POST_OBJECT_MAP:
+ if (*result)
+ rbd_warn(rbd_dev, "post object map update failed: %d",
+ *result);
+ return true;
+ default:
+ BUG();
+ }
+}
+
+/*
+ * Return true if @obj_req is completed.
+ */
+static bool __rbd_obj_handle_request(struct rbd_obj_request *obj_req,
+ int *result)
+{
+ struct rbd_img_request *img_req = obj_req->img_request;
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+ bool done;
+
+ mutex_lock(&obj_req->state_mutex);
+ if (!rbd_img_is_write(img_req))
+ done = rbd_obj_advance_read(obj_req, result);
+ else
+ done = rbd_obj_advance_write(obj_req, result);
+ mutex_unlock(&obj_req->state_mutex);
+
+ if (done && *result) {
+ rbd_assert(*result < 0);
+ rbd_warn(rbd_dev, "%s at objno %llu %llu~%llu result %d",
+ obj_op_name(img_req->op_type), obj_req->ex.oe_objno,
+ obj_req->ex.oe_off, obj_req->ex.oe_len, *result);
+ }
+ return done;
+}
+
+/*
+ * This is open-coded in rbd_img_handle_request() to avoid parent chain
+ * recursion.
+ */
+static void rbd_obj_handle_request(struct rbd_obj_request *obj_req, int result)
+{
+ if (__rbd_obj_handle_request(obj_req, &result))
+ rbd_img_handle_request(obj_req->img_request, result);
+}
+
+static bool need_exclusive_lock(struct rbd_img_request *img_req)
+{
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+
+ if (!(rbd_dev->header.features & RBD_FEATURE_EXCLUSIVE_LOCK))
+ return false;
+
+ if (rbd_dev->spec->snap_id != CEPH_NOSNAP)
+ return false;
+
+ rbd_assert(!test_bit(IMG_REQ_CHILD, &img_req->flags));
+ if (rbd_dev->opts->lock_on_read ||
+ (rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP))
+ return true;
+
+ return rbd_img_is_write(img_req);
+}
+
+static bool rbd_lock_add_request(struct rbd_img_request *img_req)
+{
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+ bool locked;
+
+ lockdep_assert_held(&rbd_dev->lock_rwsem);
+ locked = rbd_dev->lock_state == RBD_LOCK_STATE_LOCKED;
+ spin_lock(&rbd_dev->lock_lists_lock);
+ rbd_assert(list_empty(&img_req->lock_item));
+ if (!locked)
+ list_add_tail(&img_req->lock_item, &rbd_dev->acquiring_list);
+ else
+ list_add_tail(&img_req->lock_item, &rbd_dev->running_list);
+ spin_unlock(&rbd_dev->lock_lists_lock);
+ return locked;
+}
+
+static void rbd_lock_del_request(struct rbd_img_request *img_req)
+{
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+ bool need_wakeup;
+
+ lockdep_assert_held(&rbd_dev->lock_rwsem);
+ spin_lock(&rbd_dev->lock_lists_lock);
+ rbd_assert(!list_empty(&img_req->lock_item));
+ list_del_init(&img_req->lock_item);
+ need_wakeup = (rbd_dev->lock_state == RBD_LOCK_STATE_RELEASING &&
+ list_empty(&rbd_dev->running_list));
+ spin_unlock(&rbd_dev->lock_lists_lock);
+ if (need_wakeup)
+ complete(&rbd_dev->releasing_wait);
+}
+
+static int rbd_img_exclusive_lock(struct rbd_img_request *img_req)
+{
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+
+ if (!need_exclusive_lock(img_req))
+ return 1;
+
+ if (rbd_lock_add_request(img_req))
+ return 1;
+
+ if (rbd_dev->opts->exclusive) {
+ WARN_ON(1); /* lock got released? */
+ return -EROFS;
+ }
+
+ /*
+ * Note the use of mod_delayed_work() in rbd_acquire_lock()
+ * and cancel_delayed_work() in wake_lock_waiters().
+ */
+ dout("%s rbd_dev %p queueing lock_dwork\n", __func__, rbd_dev);
+ queue_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0);
+ return 0;
+}
+
+static void rbd_img_object_requests(struct rbd_img_request *img_req)
+{
+ struct rbd_obj_request *obj_req;
+
+ rbd_assert(!img_req->pending.result && !img_req->pending.num_pending);
+
+ for_each_obj_request(img_req, obj_req) {
+ int result = 0;
+
+ if (__rbd_obj_handle_request(obj_req, &result)) {
+ if (result) {
+ img_req->pending.result = result;
+ return;
+ }
+ } else {
+ img_req->pending.num_pending++;
+ }
+ }
+}
+
+static bool rbd_img_advance(struct rbd_img_request *img_req, int *result)
{
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
int ret;
- switch (obj_req->write_state) {
- case RBD_OBJ_WRITE_GUARD:
- rbd_assert(!obj_req->xferred);
- if (obj_req->result == -ENOENT) {
- /*
- * The target object doesn't exist. Read the data for
- * the entire target object up to the overlap point (if
- * any) from the parent, so we can use it for a copyup.
- */
- ret = rbd_obj_handle_write_guard(obj_req);
- if (ret) {
- obj_req->result = ret;
- return true;
- }
- return false;
- }
- /* fall through */
- case RBD_OBJ_WRITE_FLAT:
- case RBD_OBJ_WRITE_COPYUP_OPS:
- if (!obj_req->result)
- /*
- * There is no such thing as a successful short
- * write -- indicate the whole request was satisfied.
- */
- obj_req->xferred = obj_req->ex.oe_len;
- return true;
- case RBD_OBJ_WRITE_READ_FROM_PARENT:
- if (obj_req->result)
- return true;
+again:
+ switch (img_req->state) {
+ case RBD_IMG_START:
+ rbd_assert(!*result);
- rbd_assert(obj_req->xferred);
- ret = rbd_obj_issue_copyup(obj_req, obj_req->xferred);
- if (ret) {
- obj_req->result = ret;
- obj_req->xferred = 0;
+ ret = rbd_img_exclusive_lock(img_req);
+ if (ret < 0) {
+ *result = ret;
return true;
}
+ img_req->state = RBD_IMG_EXCLUSIVE_LOCK;
+ if (ret > 0)
+ goto again;
return false;
- case RBD_OBJ_WRITE_COPYUP_EMPTY_SNAPC:
- if (obj_req->result)
+ case RBD_IMG_EXCLUSIVE_LOCK:
+ if (*result)
return true;
- obj_req->write_state = RBD_OBJ_WRITE_COPYUP_OPS;
- ret = rbd_obj_issue_copyup_ops(obj_req, MODS_ONLY);
- if (ret) {
- obj_req->result = ret;
- return true;
+ rbd_assert(!need_exclusive_lock(img_req) ||
+ __rbd_is_lock_owner(rbd_dev));
+
+ rbd_img_object_requests(img_req);
+ if (!img_req->pending.num_pending) {
+ *result = img_req->pending.result;
+ img_req->state = RBD_IMG_OBJECT_REQUESTS;
+ goto again;
}
+ img_req->state = __RBD_IMG_OBJECT_REQUESTS;
return false;
+ case __RBD_IMG_OBJECT_REQUESTS:
+ if (!pending_result_dec(&img_req->pending, result))
+ return false;
+ /* fall through */
+ case RBD_IMG_OBJECT_REQUESTS:
+ return true;
default:
BUG();
}
}
/*
- * Returns true if @obj_req is completed, or false otherwise.
+ * Return true if @img_req is completed.
*/
-static bool __rbd_obj_handle_request(struct rbd_obj_request *obj_req)
-{
- switch (obj_req->img_request->op_type) {
- case OBJ_OP_READ:
- return rbd_obj_handle_read(obj_req);
- case OBJ_OP_WRITE:
- return rbd_obj_handle_write(obj_req);
- case OBJ_OP_DISCARD:
- case OBJ_OP_ZEROOUT:
- if (rbd_obj_handle_write(obj_req)) {
- /*
- * Hide -ENOENT from delete/truncate/zero -- discarding
- * a non-existent object is not a problem.
- */
- if (obj_req->result == -ENOENT) {
- obj_req->result = 0;
- obj_req->xferred = obj_req->ex.oe_len;
- }
- return true;
- }
- return false;
- default:
- BUG();
- }
-}
-
-static void rbd_obj_end_request(struct rbd_obj_request *obj_req)
+static bool __rbd_img_handle_request(struct rbd_img_request *img_req,
+ int *result)
{
- struct rbd_img_request *img_req = obj_req->img_request;
+ struct rbd_device *rbd_dev = img_req->rbd_dev;
+ bool done;
- rbd_assert((!obj_req->result &&
- obj_req->xferred == obj_req->ex.oe_len) ||
- (obj_req->result < 0 && !obj_req->xferred));
- if (!obj_req->result) {
- img_req->xferred += obj_req->xferred;
- return;
+ if (need_exclusive_lock(img_req)) {
+ down_read(&rbd_dev->lock_rwsem);
+ mutex_lock(&img_req->state_mutex);
+ done = rbd_img_advance(img_req, result);
+ if (done)
+ rbd_lock_del_request(img_req);
+ mutex_unlock(&img_req->state_mutex);
+ up_read(&rbd_dev->lock_rwsem);
+ } else {
+ mutex_lock(&img_req->state_mutex);
+ done = rbd_img_advance(img_req, result);
+ mutex_unlock(&img_req->state_mutex);
}
- rbd_warn(img_req->rbd_dev,
- "%s at objno %llu %llu~%llu result %d xferred %llu",
- obj_op_name(img_req->op_type), obj_req->ex.oe_objno,
- obj_req->ex.oe_off, obj_req->ex.oe_len, obj_req->result,
- obj_req->xferred);
- if (!img_req->result) {
- img_req->result = obj_req->result;
- img_req->xferred = 0;
+ if (done && *result) {
+ rbd_assert(*result < 0);
+ rbd_warn(rbd_dev, "%s%s result %d",
+ test_bit(IMG_REQ_CHILD, &img_req->flags) ? "child " : "",
+ obj_op_name(img_req->op_type), *result);
}
+ return done;
}
-static void rbd_img_end_child_request(struct rbd_img_request *img_req)
-{
- struct rbd_obj_request *obj_req = img_req->obj_request;
-
- rbd_assert(test_bit(IMG_REQ_CHILD, &img_req->flags));
- rbd_assert((!img_req->result &&
- img_req->xferred == rbd_obj_img_extents_bytes(obj_req)) ||
- (img_req->result < 0 && !img_req->xferred));
-
- obj_req->result = img_req->result;
- obj_req->xferred = img_req->xferred;
- rbd_img_request_put(img_req);
-}
-
-static void rbd_img_end_request(struct rbd_img_request *img_req)
-{
- rbd_assert(!test_bit(IMG_REQ_CHILD, &img_req->flags));
- rbd_assert((!img_req->result &&
- img_req->xferred == blk_rq_bytes(img_req->rq)) ||
- (img_req->result < 0 && !img_req->xferred));
-
- blk_mq_end_request(img_req->rq,
- errno_to_blk_status(img_req->result));
- rbd_img_request_put(img_req);
-}
-
-static void rbd_obj_handle_request(struct rbd_obj_request *obj_req)
+static void rbd_img_handle_request(struct rbd_img_request *img_req, int result)
{
- struct rbd_img_request *img_req;
-
again:
- if (!__rbd_obj_handle_request(obj_req))
- return;
-
- img_req = obj_req->img_request;
- spin_lock(&img_req->completion_lock);
- rbd_obj_end_request(obj_req);
- rbd_assert(img_req->pending_count);
- if (--img_req->pending_count) {
- spin_unlock(&img_req->completion_lock);
+ if (!__rbd_img_handle_request(img_req, &result))
return;
- }
- spin_unlock(&img_req->completion_lock);
if (test_bit(IMG_REQ_CHILD, &img_req->flags)) {
- obj_req = img_req->obj_request;
- rbd_img_end_child_request(img_req);
- goto again;
+ struct rbd_obj_request *obj_req = img_req->obj_request;
+
+ rbd_img_request_put(img_req);
+ if (__rbd_obj_handle_request(obj_req, &result)) {
+ img_req = obj_req->img_request;
+ goto again;
+ }
+ } else {
+ struct request *rq = img_req->rq;
+
+ rbd_img_request_put(img_req);
+ blk_mq_end_request(rq, errno_to_blk_status(result));
}
- rbd_img_end_request(img_req);
}
static const struct rbd_client_id rbd_empty_cid;
{
struct rbd_client_id cid = rbd_get_cid(rbd_dev);
+ rbd_dev->lock_state = RBD_LOCK_STATE_LOCKED;
strcpy(rbd_dev->lock_cookie, cookie);
rbd_set_owner_cid(rbd_dev, &cid);
queue_work(rbd_dev->task_wq, &rbd_dev->acquired_lock_work);
if (ret)
return ret;
- rbd_dev->lock_state = RBD_LOCK_STATE_LOCKED;
__rbd_lock(rbd_dev, cookie);
return 0;
}
ret = ceph_cls_unlock(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc,
RBD_LOCK_NAME, rbd_dev->lock_cookie);
if (ret && ret != -ENOENT)
- rbd_warn(rbd_dev, "failed to unlock: %d", ret);
+ rbd_warn(rbd_dev, "failed to unlock header: %d", ret);
/* treat errors as the image is unlocked */
rbd_dev->lock_state = RBD_LOCK_STATE_UNLOCKED;
goto out;
}
-static void wake_requests(struct rbd_device *rbd_dev, bool wake_all)
+/*
+ * Either image request state machine(s) or rbd_add_acquire_lock()
+ * (i.e. "rbd map").
+ */
+static void wake_lock_waiters(struct rbd_device *rbd_dev, int result)
{
- dout("%s rbd_dev %p wake_all %d\n", __func__, rbd_dev, wake_all);
+ struct rbd_img_request *img_req;
+
+ dout("%s rbd_dev %p result %d\n", __func__, rbd_dev, result);
+ lockdep_assert_held_write(&rbd_dev->lock_rwsem);
cancel_delayed_work(&rbd_dev->lock_dwork);
- if (wake_all)
- wake_up_all(&rbd_dev->lock_waitq);
- else
- wake_up(&rbd_dev->lock_waitq);
+ if (!completion_done(&rbd_dev->acquire_wait)) {
+ rbd_assert(list_empty(&rbd_dev->acquiring_list) &&
+ list_empty(&rbd_dev->running_list));
+ rbd_dev->acquire_err = result;
+ complete_all(&rbd_dev->acquire_wait);
+ return;
+ }
+
+ list_for_each_entry(img_req, &rbd_dev->acquiring_list, lock_item) {
+ mutex_lock(&img_req->state_mutex);
+ rbd_assert(img_req->state == RBD_IMG_EXCLUSIVE_LOCK);
+ rbd_img_schedule(img_req, result);
+ mutex_unlock(&img_req->state_mutex);
+ }
+
+ list_splice_tail_init(&rbd_dev->acquiring_list, &rbd_dev->running_list);
}
static int get_lock_owner_info(struct rbd_device *rbd_dev,
goto again;
ret = find_watcher(rbd_dev, lockers);
- if (ret) {
- if (ret > 0)
- ret = 0; /* have to request lock */
- goto out;
- }
+ if (ret)
+ goto out; /* request lock or error */
- rbd_warn(rbd_dev, "%s%llu seems dead, breaking lock",
+ rbd_warn(rbd_dev, "breaking header lock owned by %s%llu",
ENTITY_NAME(lockers[0].id.name));
ret = ceph_monc_blacklist_add(&client->monc,
return ret;
}
+static int rbd_post_acquire_action(struct rbd_device *rbd_dev)
+{
+ int ret;
+
+ if (rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP) {
+ ret = rbd_object_map_open(rbd_dev);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
/*
- * ret is set only if lock_state is RBD_LOCK_STATE_UNLOCKED
+ * Return:
+ * 0 - lock acquired
+ * 1 - caller should call rbd_request_lock()
+ * <0 - error
*/
-static enum rbd_lock_state rbd_try_acquire_lock(struct rbd_device *rbd_dev,
- int *pret)
+static int rbd_try_acquire_lock(struct rbd_device *rbd_dev)
{
- enum rbd_lock_state lock_state;
+ int ret;
down_read(&rbd_dev->lock_rwsem);
dout("%s rbd_dev %p read lock_state %d\n", __func__, rbd_dev,
rbd_dev->lock_state);
if (__rbd_is_lock_owner(rbd_dev)) {
- lock_state = rbd_dev->lock_state;
up_read(&rbd_dev->lock_rwsem);
- return lock_state;
+ return 0;
}
up_read(&rbd_dev->lock_rwsem);
down_write(&rbd_dev->lock_rwsem);
dout("%s rbd_dev %p write lock_state %d\n", __func__, rbd_dev,
rbd_dev->lock_state);
- if (!__rbd_is_lock_owner(rbd_dev)) {
- *pret = rbd_try_lock(rbd_dev);
- if (*pret)
- rbd_warn(rbd_dev, "failed to acquire lock: %d", *pret);
+ if (__rbd_is_lock_owner(rbd_dev)) {
+ up_write(&rbd_dev->lock_rwsem);
+ return 0;
+ }
+
+ ret = rbd_try_lock(rbd_dev);
+ if (ret < 0) {
+ rbd_warn(rbd_dev, "failed to lock header: %d", ret);
+ if (ret == -EBLACKLISTED)
+ goto out;
+
+ ret = 1; /* request lock anyway */
+ }
+ if (ret > 0) {
+ up_write(&rbd_dev->lock_rwsem);
+ return ret;
+ }
+
+ rbd_assert(rbd_dev->lock_state == RBD_LOCK_STATE_LOCKED);
+ rbd_assert(list_empty(&rbd_dev->running_list));
+
+ ret = rbd_post_acquire_action(rbd_dev);
+ if (ret) {
+ rbd_warn(rbd_dev, "post-acquire action failed: %d", ret);
+ /*
+ * Can't stay in RBD_LOCK_STATE_LOCKED because
+ * rbd_lock_add_request() would let the request through,
+ * assuming that e.g. object map is locked and loaded.
+ */
+ rbd_unlock(rbd_dev);
}
- lock_state = rbd_dev->lock_state;
+out:
+ wake_lock_waiters(rbd_dev, ret);
up_write(&rbd_dev->lock_rwsem);
- return lock_state;
+ return ret;
}
static void rbd_acquire_lock(struct work_struct *work)
{
struct rbd_device *rbd_dev = container_of(to_delayed_work(work),
struct rbd_device, lock_dwork);
- enum rbd_lock_state lock_state;
- int ret = 0;
+ int ret;
dout("%s rbd_dev %p\n", __func__, rbd_dev);
again:
- lock_state = rbd_try_acquire_lock(rbd_dev, &ret);
- if (lock_state != RBD_LOCK_STATE_UNLOCKED || ret == -EBLACKLISTED) {
- if (lock_state == RBD_LOCK_STATE_LOCKED)
- wake_requests(rbd_dev, true);
- dout("%s rbd_dev %p lock_state %d ret %d - done\n", __func__,
- rbd_dev, lock_state, ret);
+ ret = rbd_try_acquire_lock(rbd_dev);
+ if (ret <= 0) {
+ dout("%s rbd_dev %p ret %d - done\n", __func__, rbd_dev, ret);
return;
}
goto again; /* treat this as a dead client */
} else if (ret == -EROFS) {
rbd_warn(rbd_dev, "peer will not release lock");
- /*
- * If this is rbd_add_acquire_lock(), we want to fail
- * immediately -- reuse BLACKLISTED flag. Otherwise we
- * want to block.
- */
- if (!(rbd_dev->disk->flags & GENHD_FL_UP)) {
- set_bit(RBD_DEV_FLAG_BLACKLISTED, &rbd_dev->flags);
- /* wake "rbd map --exclusive" process */
- wake_requests(rbd_dev, false);
- }
+ down_write(&rbd_dev->lock_rwsem);
+ wake_lock_waiters(rbd_dev, ret);
+ up_write(&rbd_dev->lock_rwsem);
} else if (ret < 0) {
rbd_warn(rbd_dev, "error requesting lock: %d", ret);
mod_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork,
}
}
-/*
- * lock_rwsem must be held for write
- */
-static bool rbd_release_lock(struct rbd_device *rbd_dev)
+static bool rbd_quiesce_lock(struct rbd_device *rbd_dev)
{
- dout("%s rbd_dev %p read lock_state %d\n", __func__, rbd_dev,
- rbd_dev->lock_state);
+ bool need_wait;
+
+ dout("%s rbd_dev %p\n", __func__, rbd_dev);
+ lockdep_assert_held_write(&rbd_dev->lock_rwsem);
+
if (rbd_dev->lock_state != RBD_LOCK_STATE_LOCKED)
return false;
- rbd_dev->lock_state = RBD_LOCK_STATE_RELEASING;
- downgrade_write(&rbd_dev->lock_rwsem);
/*
* Ensure that all in-flight IO is flushed.
- *
- * FIXME: ceph_osdc_sync() flushes the entire OSD client, which
- * may be shared with other devices.
*/
- ceph_osdc_sync(&rbd_dev->rbd_client->client->osdc);
+ rbd_dev->lock_state = RBD_LOCK_STATE_RELEASING;
+ rbd_assert(!completion_done(&rbd_dev->releasing_wait));
+ need_wait = !list_empty(&rbd_dev->running_list);
+ downgrade_write(&rbd_dev->lock_rwsem);
+ if (need_wait)
+ wait_for_completion(&rbd_dev->releasing_wait);
up_read(&rbd_dev->lock_rwsem);
down_write(&rbd_dev->lock_rwsem);
- dout("%s rbd_dev %p write lock_state %d\n", __func__, rbd_dev,
- rbd_dev->lock_state);
if (rbd_dev->lock_state != RBD_LOCK_STATE_RELEASING)
return false;
+ rbd_assert(list_empty(&rbd_dev->running_list));
+ return true;
+}
+
+static void rbd_pre_release_action(struct rbd_device *rbd_dev)
+{
+ if (rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP)
+ rbd_object_map_close(rbd_dev);
+}
+
+static void __rbd_release_lock(struct rbd_device *rbd_dev)
+{
+ rbd_assert(list_empty(&rbd_dev->running_list));
+
+ rbd_pre_release_action(rbd_dev);
rbd_unlock(rbd_dev);
+}
+
+/*
+ * lock_rwsem must be held for write
+ */
+static void rbd_release_lock(struct rbd_device *rbd_dev)
+{
+ if (!rbd_quiesce_lock(rbd_dev))
+ return;
+
+ __rbd_release_lock(rbd_dev);
+
/*
* Give others a chance to grab the lock - we would re-acquire
- * almost immediately if we got new IO during ceph_osdc_sync()
- * otherwise. We need to ack our own notifications, so this
- * lock_dwork will be requeued from rbd_wait_state_locked()
- * after wake_requests() in rbd_handle_released_lock().
+ * almost immediately if we got new IO while draining the running
+ * list otherwise. We need to ack our own notifications, so this
+ * lock_dwork will be requeued from rbd_handle_released_lock() by
+ * way of maybe_kick_acquire().
*/
cancel_delayed_work(&rbd_dev->lock_dwork);
- return true;
}
static void rbd_release_lock_work(struct work_struct *work)
up_write(&rbd_dev->lock_rwsem);
}
+static void maybe_kick_acquire(struct rbd_device *rbd_dev)
+{
+ bool have_requests;
+
+ dout("%s rbd_dev %p\n", __func__, rbd_dev);
+ if (__rbd_is_lock_owner(rbd_dev))
+ return;
+
+ spin_lock(&rbd_dev->lock_lists_lock);
+ have_requests = !list_empty(&rbd_dev->acquiring_list);
+ spin_unlock(&rbd_dev->lock_lists_lock);
+ if (have_requests || delayed_work_pending(&rbd_dev->lock_dwork)) {
+ dout("%s rbd_dev %p kicking lock_dwork\n", __func__, rbd_dev);
+ mod_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0);
+ }
+}
+
static void rbd_handle_acquired_lock(struct rbd_device *rbd_dev, u8 struct_v,
void **p)
{
down_read(&rbd_dev->lock_rwsem);
}
- if (!__rbd_is_lock_owner(rbd_dev))
- wake_requests(rbd_dev, false);
+ maybe_kick_acquire(rbd_dev);
up_read(&rbd_dev->lock_rwsem);
}
down_read(&rbd_dev->lock_rwsem);
}
- if (!__rbd_is_lock_owner(rbd_dev))
- wake_requests(rbd_dev, false);
+ maybe_kick_acquire(rbd_dev);
up_read(&rbd_dev->lock_rwsem);
}
static void rbd_unregister_watch(struct rbd_device *rbd_dev)
{
- WARN_ON(waitqueue_active(&rbd_dev->lock_waitq));
cancel_tasks_sync(rbd_dev);
mutex_lock(&rbd_dev->watch_mutex);
char cookie[32];
int ret;
- WARN_ON(rbd_dev->lock_state != RBD_LOCK_STATE_LOCKED);
+ if (!rbd_quiesce_lock(rbd_dev))
+ return;
format_lock_cookie(rbd_dev, cookie);
ret = ceph_cls_set_cookie(osdc, &rbd_dev->header_oid,
* Lock cookie cannot be updated on older OSDs, so do
* a manual release and queue an acquire.
*/
- if (rbd_release_lock(rbd_dev))
- queue_delayed_work(rbd_dev->task_wq,
- &rbd_dev->lock_dwork, 0);
+ __rbd_release_lock(rbd_dev);
+ queue_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0);
} else {
__rbd_lock(rbd_dev, cookie);
+ wake_lock_waiters(rbd_dev, 0);
}
}
ret = __rbd_register_watch(rbd_dev);
if (ret) {
rbd_warn(rbd_dev, "failed to reregister watch: %d", ret);
- if (ret == -EBLACKLISTED || ret == -ENOENT) {
- set_bit(RBD_DEV_FLAG_BLACKLISTED, &rbd_dev->flags);
- wake_requests(rbd_dev, true);
- } else {
+ if (ret != -EBLACKLISTED && ret != -ENOENT) {
queue_delayed_work(rbd_dev->task_wq,
&rbd_dev->watch_dwork,
RBD_RETRY_DELAY);
+ mutex_unlock(&rbd_dev->watch_mutex);
+ return;
}
+
mutex_unlock(&rbd_dev->watch_mutex);
+ down_write(&rbd_dev->lock_rwsem);
+ wake_lock_waiters(rbd_dev, ret);
+ up_write(&rbd_dev->lock_rwsem);
return;
}
ret = ceph_osdc_call(osdc, oid, oloc, RBD_DRV_NAME, method_name,
CEPH_OSD_FLAG_READ, req_page, outbound_size,
- reply_page, &inbound_size);
+ &reply_page, &inbound_size);
if (!ret) {
memcpy(inbound, page_address(reply_page), inbound_size);
ret = inbound_size;
return ret;
}
-/*
- * lock_rwsem must be held for read
- */
-static int rbd_wait_state_locked(struct rbd_device *rbd_dev, bool may_acquire)
-{
- DEFINE_WAIT(wait);
- unsigned long timeout;
- int ret = 0;
-
- if (test_bit(RBD_DEV_FLAG_BLACKLISTED, &rbd_dev->flags))
- return -EBLACKLISTED;
-
- if (rbd_dev->lock_state == RBD_LOCK_STATE_LOCKED)
- return 0;
-
- if (!may_acquire) {
- rbd_warn(rbd_dev, "exclusive lock required");
- return -EROFS;
- }
-
- do {
- /*
- * Note the use of mod_delayed_work() in rbd_acquire_lock()
- * and cancel_delayed_work() in wake_requests().
- */
- dout("%s rbd_dev %p queueing lock_dwork\n", __func__, rbd_dev);
- queue_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0);
- prepare_to_wait_exclusive(&rbd_dev->lock_waitq, &wait,
- TASK_UNINTERRUPTIBLE);
- up_read(&rbd_dev->lock_rwsem);
- timeout = schedule_timeout(ceph_timeout_jiffies(
- rbd_dev->opts->lock_timeout));
- down_read(&rbd_dev->lock_rwsem);
- if (test_bit(RBD_DEV_FLAG_BLACKLISTED, &rbd_dev->flags)) {
- ret = -EBLACKLISTED;
- break;
- }
- if (!timeout) {
- rbd_warn(rbd_dev, "timed out waiting for lock");
- ret = -ETIMEDOUT;
- break;
- }
- } while (rbd_dev->lock_state != RBD_LOCK_STATE_LOCKED);
-
- finish_wait(&rbd_dev->lock_waitq, &wait);
- return ret;
-}
-
static void rbd_queue_workfn(struct work_struct *work)
{
struct request *rq = blk_mq_rq_from_pdu(work);
u64 length = blk_rq_bytes(rq);
enum obj_operation_type op_type;
u64 mapping_size;
- bool must_be_locked;
int result;
switch (req_op(rq)) {
goto err_rq;
}
- must_be_locked =
- (rbd_dev->header.features & RBD_FEATURE_EXCLUSIVE_LOCK) &&
- (op_type != OBJ_OP_READ || rbd_dev->opts->lock_on_read);
- if (must_be_locked) {
- down_read(&rbd_dev->lock_rwsem);
- result = rbd_wait_state_locked(rbd_dev,
- !rbd_dev->opts->exclusive);
- if (result)
- goto err_unlock;
- }
-
img_request = rbd_img_request_create(rbd_dev, op_type, snapc);
if (!img_request) {
result = -ENOMEM;
- goto err_unlock;
+ goto err_rq;
}
img_request->rq = rq;
snapc = NULL; /* img_request consumes a ref */
else
result = rbd_img_fill_from_bio(img_request, offset, length,
rq->bio);
- if (result || !img_request->pending_count)
+ if (result)
goto err_img_request;
- rbd_img_request_submit(img_request);
- if (must_be_locked)
- up_read(&rbd_dev->lock_rwsem);
+ rbd_img_handle_request(img_request, 0);
return;
err_img_request:
rbd_img_request_put(img_request);
-err_unlock:
- if (must_be_locked)
- up_read(&rbd_dev->lock_rwsem);
err_rq:
if (result)
rbd_warn(rbd_dev, "%s %llx at %llx result %d",
INIT_WORK(&rbd_dev->released_lock_work, rbd_notify_released_lock);
INIT_DELAYED_WORK(&rbd_dev->lock_dwork, rbd_acquire_lock);
INIT_WORK(&rbd_dev->unlock_work, rbd_release_lock_work);
- init_waitqueue_head(&rbd_dev->lock_waitq);
+ spin_lock_init(&rbd_dev->lock_lists_lock);
+ INIT_LIST_HEAD(&rbd_dev->acquiring_list);
+ INIT_LIST_HEAD(&rbd_dev->running_list);
+ init_completion(&rbd_dev->acquire_wait);
+ init_completion(&rbd_dev->releasing_wait);
+
+ spin_lock_init(&rbd_dev->object_map_lock);
rbd_dev->dev.bus = &rbd_bus_type;
rbd_dev->dev.type = &rbd_device_type;
&rbd_dev->header.features);
}
+/*
+ * These are generic image flags, but since they are used only for
+ * object map, store them in rbd_dev->object_map_flags.
+ *
+ * For the same reason, this function is called only on object map
+ * (re)load and not on header refresh.
+ */
+static int rbd_dev_v2_get_flags(struct rbd_device *rbd_dev)
+{
+ __le64 snapid = cpu_to_le64(rbd_dev->spec->snap_id);
+ __le64 flags;
+ int ret;
+
+ ret = rbd_obj_method_sync(rbd_dev, &rbd_dev->header_oid,
+ &rbd_dev->header_oloc, "get_flags",
+ &snapid, sizeof(snapid),
+ &flags, sizeof(flags));
+ if (ret < 0)
+ return ret;
+ if (ret < sizeof(flags))
+ return -EBADMSG;
+
+ rbd_dev->object_map_flags = le64_to_cpu(flags);
+ return 0;
+}
+
struct parent_image_info {
u64 pool_id;
const char *pool_ns;
ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc,
"rbd", "parent_get", CEPH_OSD_FLAG_READ,
- req_page, sizeof(u64), reply_page, &reply_len);
+ req_page, sizeof(u64), &reply_page, &reply_len);
if (ret)
return ret == -EOPNOTSUPP ? 1 : ret;
ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc,
"rbd", "parent_overlap_get", CEPH_OSD_FLAG_READ,
- req_page, sizeof(u64), reply_page, &reply_len);
+ req_page, sizeof(u64), &reply_page, &reply_len);
if (ret)
return ret;
ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc,
"rbd", "get_parent", CEPH_OSD_FLAG_READ,
- req_page, sizeof(u64), reply_page, &reply_len);
+ req_page, sizeof(u64), &reply_page, &reply_len);
if (ret)
return ret;
{
down_write(&rbd_dev->lock_rwsem);
if (__rbd_is_lock_owner(rbd_dev))
- rbd_unlock(rbd_dev);
+ __rbd_release_lock(rbd_dev);
up_write(&rbd_dev->lock_rwsem);
}
+/*
+ * If the wait is interrupted, an error is returned even if the lock
+ * was successfully acquired. rbd_dev_image_unlock() will release it
+ * if needed.
+ */
static int rbd_add_acquire_lock(struct rbd_device *rbd_dev)
{
- int ret;
+ long ret;
if (!(rbd_dev->header.features & RBD_FEATURE_EXCLUSIVE_LOCK)) {
+ if (!rbd_dev->opts->exclusive && !rbd_dev->opts->lock_on_read)
+ return 0;
+
rbd_warn(rbd_dev, "exclusive-lock feature is not enabled");
return -EINVAL;
}
- /* FIXME: "rbd map --exclusive" should be in interruptible */
- down_read(&rbd_dev->lock_rwsem);
- ret = rbd_wait_state_locked(rbd_dev, true);
- up_read(&rbd_dev->lock_rwsem);
+ if (rbd_dev->spec->snap_id != CEPH_NOSNAP)
+ return 0;
+
+ rbd_assert(!rbd_is_lock_owner(rbd_dev));
+ queue_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0);
+ ret = wait_for_completion_killable_timeout(&rbd_dev->acquire_wait,
+ ceph_timeout_jiffies(rbd_dev->opts->lock_timeout));
+ if (ret > 0)
+ ret = rbd_dev->acquire_err;
+ else if (!ret)
+ ret = -ETIMEDOUT;
+
if (ret) {
- rbd_warn(rbd_dev, "failed to acquire exclusive lock");
- return -EROFS;
+ rbd_warn(rbd_dev, "failed to acquire exclusive lock: %ld", ret);
+ return ret;
}
+ /*
+ * The lock may have been released by now, unless automatic lock
+ * transitions are disabled.
+ */
+ rbd_assert(!rbd_dev->opts->exclusive || rbd_is_lock_owner(rbd_dev));
return 0;
}
struct rbd_image_header *header;
rbd_dev_parent_put(rbd_dev);
+ rbd_object_map_free(rbd_dev);
+ rbd_dev_mapping_clear(rbd_dev);
/* Free dynamic fields from the header, then zero it out */
static void rbd_dev_device_release(struct rbd_device *rbd_dev)
{
clear_bit(RBD_DEV_FLAG_EXISTS, &rbd_dev->flags);
- rbd_dev_mapping_clear(rbd_dev);
rbd_free_disk(rbd_dev);
if (!single_major)
unregister_blkdev(rbd_dev->major, rbd_dev->name);
if (ret)
goto err_out_blkdev;
- ret = rbd_dev_mapping_set(rbd_dev);
- if (ret)
- goto err_out_disk;
-
set_capacity(rbd_dev->disk, rbd_dev->mapping.size / SECTOR_SIZE);
set_disk_ro(rbd_dev->disk, rbd_dev->opts->read_only);
ret = dev_set_name(&rbd_dev->dev, "%d", rbd_dev->dev_id);
if (ret)
- goto err_out_mapping;
+ goto err_out_disk;
set_bit(RBD_DEV_FLAG_EXISTS, &rbd_dev->flags);
up_write(&rbd_dev->header_rwsem);
return 0;
-err_out_mapping:
- rbd_dev_mapping_clear(rbd_dev);
err_out_disk:
rbd_free_disk(rbd_dev);
err_out_blkdev:
goto err_out_probe;
}
+ ret = rbd_dev_mapping_set(rbd_dev);
+ if (ret)
+ goto err_out_probe;
+
+ if (rbd_dev->spec->snap_id != CEPH_NOSNAP &&
+ (rbd_dev->header.features & RBD_FEATURE_OBJECT_MAP)) {
+ ret = rbd_object_map_load(rbd_dev);
+ if (ret)
+ goto err_out_probe;
+ }
+
if (rbd_dev->header.features & RBD_FEATURE_LAYERING) {
ret = rbd_dev_v2_parent_info(rbd_dev);
if (ret)
if (rc)
goto err_out_image_probe;
- if (rbd_dev->opts->exclusive) {
- rc = rbd_add_acquire_lock(rbd_dev);
- if (rc)
- goto err_out_device_setup;
- }
+ rc = rbd_add_acquire_lock(rbd_dev);
+ if (rc)
+ goto err_out_image_lock;
/* Everything's ready. Announce the disk to the world. */
err_out_image_lock:
rbd_dev_image_unlock(rbd_dev);
-err_out_device_setup:
rbd_dev_device_release(rbd_dev);
err_out_image_probe:
rbd_dev_image_release(rbd_dev);
/* For format version 2, rbd image 'foo' consists of objects
* rbd_id.foo - id of image
* rbd_header.<id> - image metadata
+ * rbd_object_map.<id> - optional image object map
* rbd_data.<id>.0000000000000000
* rbd_data.<id>.0000000000000001
* ... - data
*/
#define RBD_HEADER_PREFIX "rbd_header."
+#define RBD_OBJECT_MAP_PREFIX "rbd_object_map."
#define RBD_ID_PREFIX "rbd_id."
#define RBD_V2_DATA_FORMAT "%s.%016llx"
RBD_NOTIFY_OP_HEADER_UPDATE = 3,
};
+#define OBJECT_NONEXISTENT 0
+#define OBJECT_EXISTS 1
+#define OBJECT_PENDING 2
+#define OBJECT_EXISTS_CLEAN 3
+
+#define RBD_FLAG_OBJECT_MAP_INVALID (1ULL << 0)
+#define RBD_FLAG_FAST_DIFF_INVALID (1ULL << 1)
+
/*
* For format version 1, rbd image 'foo' consists of objects
* foo.rbd - image metadata
It has several use cases, for example: /tmp storage, use as swap
disks and maybe many more.
- See Documentation/blockdev/zram.txt for more information.
+ See Documentation/admin-guide/blockdev/zram.rst for more information.
config ZRAM_WRITEBACK
bool "Write back incompressible or idle page to backing device"
With /sys/block/zramX/{idle,writeback}, application could ask
idle page's writeback to the backing device to save in memory.
- See Documentation/blockdev/zram.txt for more information.
+ See Documentation/admin-guide/blockdev/zram.rst for more information.
config ZRAM_MEMORY_TRACKING
bool "Track zRam block status"
of zRAM. Admin could see the information via
/sys/kernel/debug/zram/zramX/block_state.
- See Documentation/blockdev/zram.txt for more information.
+ See Documentation/admin-guide/blockdev/zram.rst for more information.
and set the RTC in an SMP compatible fashion.
If you think you have a use for such a device (such as periodic data
- sampling), then say Y here, and read <file:Documentation/rtc.txt>
+ sampling), then say Y here, and read <file:Documentation/admin-guide/rtc.rst>
for details.
To compile this driver as a module, choose M here: the
/dev/rtc.
If you think you have a use for such a device (such as periodic data
- sampling), then say Y here, and read <file:Documentation/rtc.txt>
+ sampling), then say Y here, and read <file:Documentation/admin-guide/rtc.rst>
for details.
To compile this driver as a module, choose M here: the
Device which can be found in many (all ?) Sony Vaio laptops.
If you have one of those laptops, read
- <file:Documentation/laptops/sonypi.txt>, and say Y or M here.
+ <file:Documentation/admin-guide/laptops/sonypi.rst>, and say Y or M here.
To compile this driver as a module, choose M here: the
module will be called sonypi.
* Copyright 2006 Michael Buesch <m@bues.ch>
* Copyright 2005 (c) MontaVista Software, Inc.
*
- * Please read Documentation/hw_random.txt for details on use.
+ * Please read Documentation/admin-guide/hw_random.rst for details on use.
*
* This software may be used and distributed according to the terms
* of the GNU General Public License, incorporated herein by reference.
This driver uses SCPI Message Protocol to interact with the
firmware providing all the clock controls.
+config COMMON_CLK_SI5341
+ tristate "Clock driver for SiLabs 5341 and 5340 A/B/C/D devices"
+ depends on I2C
+ select REGMAP_I2C
+ help
+ This driver supports Silicon Labs Si5341 and Si5340 programmable clock
+ generators. Not all features of these chips are currently supported
+ by the driver, in particular it only supports XTAL input. The chip can
+ be pre-programmed to support other configurations and features not yet
+ implemented in the driver.
+
config COMMON_CLK_SI5351
tristate "Clock driver for SiLabs 5351A/B/C"
depends on I2C
config COMMON_CLK_XGENE
bool "Clock driver for APM XGene SoC"
- default y
+ default ARCH_XGENE
depends on ARM64 || COMPILE_TEST
---help---
Sypport for the APM X-Gene SoC reference, PLL, and device clocks.
obj-$(CONFIG_COMMON_CLK_S2MPS11) += clk-s2mps11.o
obj-$(CONFIG_COMMON_CLK_SCMI) += clk-scmi.o
obj-$(CONFIG_COMMON_CLK_SCPI) += clk-scpi.o
+obj-$(CONFIG_COMMON_CLK_SI5341) += clk-si5341.o
obj-$(CONFIG_COMMON_CLK_SI5351) += clk-si5351.o
obj-$(CONFIG_COMMON_CLK_SI514) += clk-si514.o
obj-$(CONFIG_COMMON_CLK_SI544) += clk-si544.o
SLOW_CLOCK_FREQ)
#define AT91_SCKC_CR 0x00
-#define AT91_SCKC_RCEN (1 << 0)
-#define AT91_SCKC_OSC32EN (1 << 1)
-#define AT91_SCKC_OSC32BYP (1 << 2)
-#define AT91_SCKC_OSCSEL (1 << 3)
+
+struct clk_slow_bits {
+ u32 cr_rcen;
+ u32 cr_osc32en;
+ u32 cr_osc32byp;
+ u32 cr_oscsel;
+};
struct clk_slow_osc {
struct clk_hw hw;
void __iomem *sckcr;
+ const struct clk_slow_bits *bits;
unsigned long startup_usec;
};
struct clk_sama5d4_slow_osc {
struct clk_hw hw;
void __iomem *sckcr;
+ const struct clk_slow_bits *bits;
unsigned long startup_usec;
bool prepared;
};
struct clk_slow_rc_osc {
struct clk_hw hw;
void __iomem *sckcr;
+ const struct clk_slow_bits *bits;
unsigned long frequency;
unsigned long accuracy;
unsigned long startup_usec;
struct clk_sam9x5_slow {
struct clk_hw hw;
void __iomem *sckcr;
+ const struct clk_slow_bits *bits;
u8 parent;
};
void __iomem *sckcr = osc->sckcr;
u32 tmp = readl(sckcr);
- if (tmp & (AT91_SCKC_OSC32BYP | AT91_SCKC_OSC32EN))
+ if (tmp & (osc->bits->cr_osc32byp | osc->bits->cr_osc32en))
return 0;
- writel(tmp | AT91_SCKC_OSC32EN, sckcr);
+ writel(tmp | osc->bits->cr_osc32en, sckcr);
usleep_range(osc->startup_usec, osc->startup_usec + 1);
void __iomem *sckcr = osc->sckcr;
u32 tmp = readl(sckcr);
- if (tmp & AT91_SCKC_OSC32BYP)
+ if (tmp & osc->bits->cr_osc32byp)
return;
- writel(tmp & ~AT91_SCKC_OSC32EN, sckcr);
+ writel(tmp & ~osc->bits->cr_osc32en, sckcr);
}
static int clk_slow_osc_is_prepared(struct clk_hw *hw)
void __iomem *sckcr = osc->sckcr;
u32 tmp = readl(sckcr);
- if (tmp & AT91_SCKC_OSC32BYP)
+ if (tmp & osc->bits->cr_osc32byp)
return 1;
- return !!(tmp & AT91_SCKC_OSC32EN);
+ return !!(tmp & osc->bits->cr_osc32en);
}
static const struct clk_ops slow_osc_ops = {
const char *name,
const char *parent_name,
unsigned long startup,
- bool bypass)
+ bool bypass,
+ const struct clk_slow_bits *bits)
{
struct clk_slow_osc *osc;
struct clk_hw *hw;
osc->hw.init = &init;
osc->sckcr = sckcr;
osc->startup_usec = startup;
+ osc->bits = bits;
if (bypass)
- writel((readl(sckcr) & ~AT91_SCKC_OSC32EN) | AT91_SCKC_OSC32BYP,
- sckcr);
+ writel((readl(sckcr) & ~osc->bits->cr_osc32en) |
+ osc->bits->cr_osc32byp, sckcr);
hw = &osc->hw;
ret = clk_hw_register(NULL, &osc->hw);
return hw;
}
+static void at91_clk_unregister_slow_osc(struct clk_hw *hw)
+{
+ struct clk_slow_osc *osc = to_clk_slow_osc(hw);
+
+ clk_hw_unregister(hw);
+ kfree(osc);
+}
+
static unsigned long clk_slow_rc_osc_recalc_rate(struct clk_hw *hw,
unsigned long parent_rate)
{
struct clk_slow_rc_osc *osc = to_clk_slow_rc_osc(hw);
void __iomem *sckcr = osc->sckcr;
- writel(readl(sckcr) | AT91_SCKC_RCEN, sckcr);
+ writel(readl(sckcr) | osc->bits->cr_rcen, sckcr);
usleep_range(osc->startup_usec, osc->startup_usec + 1);
struct clk_slow_rc_osc *osc = to_clk_slow_rc_osc(hw);
void __iomem *sckcr = osc->sckcr;
- writel(readl(sckcr) & ~AT91_SCKC_RCEN, sckcr);
+ writel(readl(sckcr) & ~osc->bits->cr_rcen, sckcr);
}
static int clk_slow_rc_osc_is_prepared(struct clk_hw *hw)
{
struct clk_slow_rc_osc *osc = to_clk_slow_rc_osc(hw);
- return !!(readl(osc->sckcr) & AT91_SCKC_RCEN);
+ return !!(readl(osc->sckcr) & osc->bits->cr_rcen);
}
static const struct clk_ops slow_rc_osc_ops = {
const char *name,
unsigned long frequency,
unsigned long accuracy,
- unsigned long startup)
+ unsigned long startup,
+ const struct clk_slow_bits *bits)
{
struct clk_slow_rc_osc *osc;
struct clk_hw *hw;
osc->hw.init = &init;
osc->sckcr = sckcr;
+ osc->bits = bits;
osc->frequency = frequency;
osc->accuracy = accuracy;
osc->startup_usec = startup;
return hw;
}
+static void at91_clk_unregister_slow_rc_osc(struct clk_hw *hw)
+{
+ struct clk_slow_rc_osc *osc = to_clk_slow_rc_osc(hw);
+
+ clk_hw_unregister(hw);
+ kfree(osc);
+}
+
static int clk_sam9x5_slow_set_parent(struct clk_hw *hw, u8 index)
{
struct clk_sam9x5_slow *slowck = to_clk_sam9x5_slow(hw);
tmp = readl(sckcr);
- if ((!index && !(tmp & AT91_SCKC_OSCSEL)) ||
- (index && (tmp & AT91_SCKC_OSCSEL)))
+ if ((!index && !(tmp & slowck->bits->cr_oscsel)) ||
+ (index && (tmp & slowck->bits->cr_oscsel)))
return 0;
if (index)
- tmp |= AT91_SCKC_OSCSEL;
+ tmp |= slowck->bits->cr_oscsel;
else
- tmp &= ~AT91_SCKC_OSCSEL;
+ tmp &= ~slowck->bits->cr_oscsel;
writel(tmp, sckcr);
{
struct clk_sam9x5_slow *slowck = to_clk_sam9x5_slow(hw);
- return !!(readl(slowck->sckcr) & AT91_SCKC_OSCSEL);
+ return !!(readl(slowck->sckcr) & slowck->bits->cr_oscsel);
}
static const struct clk_ops sam9x5_slow_ops = {
at91_clk_register_sam9x5_slow(void __iomem *sckcr,
const char *name,
const char **parent_names,
- int num_parents)
+ int num_parents,
+ const struct clk_slow_bits *bits)
{
struct clk_sam9x5_slow *slowck;
struct clk_hw *hw;
slowck->hw.init = &init;
slowck->sckcr = sckcr;
- slowck->parent = !!(readl(sckcr) & AT91_SCKC_OSCSEL);
+ slowck->bits = bits;
+ slowck->parent = !!(readl(sckcr) & slowck->bits->cr_oscsel);
hw = &slowck->hw;
ret = clk_hw_register(NULL, &slowck->hw);
return hw;
}
+static void at91_clk_unregister_sam9x5_slow(struct clk_hw *hw)
+{
+ struct clk_sam9x5_slow *slowck = to_clk_sam9x5_slow(hw);
+
+ clk_hw_unregister(hw);
+ kfree(slowck);
+}
+
static void __init at91sam9x5_sckc_register(struct device_node *np,
- unsigned int rc_osc_startup_us)
+ unsigned int rc_osc_startup_us,
+ const struct clk_slow_bits *bits)
{
const char *parent_names[2] = { "slow_rc_osc", "slow_osc" };
void __iomem *regbase = of_iomap(np, 0);
struct device_node *child = NULL;
const char *xtal_name;
- struct clk_hw *hw;
+ struct clk_hw *slow_rc, *slow_osc, *slowck;
bool bypass;
+ int ret;
if (!regbase)
return;
- hw = at91_clk_register_slow_rc_osc(regbase, parent_names[0], 32768,
- 50000000, rc_osc_startup_us);
- if (IS_ERR(hw))
+ slow_rc = at91_clk_register_slow_rc_osc(regbase, parent_names[0],
+ 32768, 50000000,
+ rc_osc_startup_us, bits);
+ if (IS_ERR(slow_rc))
return;
xtal_name = of_clk_get_parent_name(np, 0);
/* DT backward compatibility */
child = of_get_compatible_child(np, "atmel,at91sam9x5-clk-slow-osc");
if (!child)
- return;
+ goto unregister_slow_rc;
xtal_name = of_clk_get_parent_name(child, 0);
bypass = of_property_read_bool(child, "atmel,osc-bypass");
}
if (!xtal_name)
- return;
-
- hw = at91_clk_register_slow_osc(regbase, parent_names[1], xtal_name,
- 1200000, bypass);
- if (IS_ERR(hw))
- return;
+ goto unregister_slow_rc;
- hw = at91_clk_register_sam9x5_slow(regbase, "slowck", parent_names, 2);
- if (IS_ERR(hw))
- return;
+ slow_osc = at91_clk_register_slow_osc(regbase, parent_names[1],
+ xtal_name, 1200000, bypass, bits);
+ if (IS_ERR(slow_osc))
+ goto unregister_slow_rc;
- of_clk_add_hw_provider(np, of_clk_hw_simple_get, hw);
+ slowck = at91_clk_register_sam9x5_slow(regbase, "slowck", parent_names,
+ 2, bits);
+ if (IS_ERR(slowck))
+ goto unregister_slow_osc;
/* DT backward compatibility */
if (child)
- of_clk_add_hw_provider(child, of_clk_hw_simple_get, hw);
+ ret = of_clk_add_hw_provider(child, of_clk_hw_simple_get,
+ slowck);
+ else
+ ret = of_clk_add_hw_provider(np, of_clk_hw_simple_get, slowck);
+
+ if (WARN_ON(ret))
+ goto unregister_slowck;
+
+ return;
+
+unregister_slowck:
+ at91_clk_unregister_sam9x5_slow(slowck);
+unregister_slow_osc:
+ at91_clk_unregister_slow_osc(slow_osc);
+unregister_slow_rc:
+ at91_clk_unregister_slow_rc_osc(slow_rc);
}
+static const struct clk_slow_bits at91sam9x5_bits = {
+ .cr_rcen = BIT(0),
+ .cr_osc32en = BIT(1),
+ .cr_osc32byp = BIT(2),
+ .cr_oscsel = BIT(3),
+};
+
static void __init of_at91sam9x5_sckc_setup(struct device_node *np)
{
- at91sam9x5_sckc_register(np, 75);
+ at91sam9x5_sckc_register(np, 75, &at91sam9x5_bits);
}
CLK_OF_DECLARE(at91sam9x5_clk_sckc, "atmel,at91sam9x5-sckc",
of_at91sam9x5_sckc_setup);
static void __init of_sama5d3_sckc_setup(struct device_node *np)
{
- at91sam9x5_sckc_register(np, 500);
+ at91sam9x5_sckc_register(np, 500, &at91sam9x5_bits);
}
CLK_OF_DECLARE(sama5d3_clk_sckc, "atmel,sama5d3-sckc",
of_sama5d3_sckc_setup);
+static const struct clk_slow_bits at91sam9x60_bits = {
+ .cr_osc32en = BIT(1),
+ .cr_osc32byp = BIT(2),
+ .cr_oscsel = BIT(24),
+};
+
+static void __init of_sam9x60_sckc_setup(struct device_node *np)
+{
+ void __iomem *regbase = of_iomap(np, 0);
+ struct clk_hw_onecell_data *clk_data;
+ struct clk_hw *slow_rc, *slow_osc;
+ const char *xtal_name;
+ const char *parent_names[2] = { "slow_rc_osc", "slow_osc" };
+ bool bypass;
+ int ret;
+
+ if (!regbase)
+ return;
+
+ slow_rc = clk_hw_register_fixed_rate(NULL, parent_names[0], NULL, 0,
+ 32768);
+ if (IS_ERR(slow_rc))
+ return;
+
+ xtal_name = of_clk_get_parent_name(np, 0);
+ if (!xtal_name)
+ goto unregister_slow_rc;
+
+ bypass = of_property_read_bool(np, "atmel,osc-bypass");
+ slow_osc = at91_clk_register_slow_osc(regbase, parent_names[1],
+ xtal_name, 5000000, bypass,
+ &at91sam9x60_bits);
+ if (IS_ERR(slow_osc))
+ goto unregister_slow_rc;
+
+ clk_data = kzalloc(sizeof(*clk_data) + (2 * sizeof(struct clk_hw *)),
+ GFP_KERNEL);
+ if (!clk_data)
+ goto unregister_slow_osc;
+
+ /* MD_SLCK and TD_SLCK. */
+ clk_data->num = 2;
+ clk_data->hws[0] = clk_hw_register_fixed_rate(NULL, "md_slck",
+ parent_names[0],
+ 0, 32768);
+ if (IS_ERR(clk_data->hws[0]))
+ goto clk_data_free;
+
+ clk_data->hws[1] = at91_clk_register_sam9x5_slow(regbase, "td_slck",
+ parent_names, 2,
+ &at91sam9x60_bits);
+ if (IS_ERR(clk_data->hws[1]))
+ goto unregister_md_slck;
+
+ ret = of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_data);
+ if (WARN_ON(ret))
+ goto unregister_td_slck;
+
+ return;
+
+unregister_td_slck:
+ at91_clk_unregister_sam9x5_slow(clk_data->hws[1]);
+unregister_md_slck:
+ clk_hw_unregister(clk_data->hws[0]);
+clk_data_free:
+ kfree(clk_data);
+unregister_slow_osc:
+ at91_clk_unregister_slow_osc(slow_osc);
+unregister_slow_rc:
+ clk_hw_unregister(slow_rc);
+}
+CLK_OF_DECLARE(sam9x60_clk_sckc, "microchip,sam9x60-sckc",
+ of_sam9x60_sckc_setup);
+
static int clk_sama5d4_slow_osc_prepare(struct clk_hw *hw)
{
struct clk_sama5d4_slow_osc *osc = to_clk_sama5d4_slow_osc(hw);
* Assume that if it has already been selected (for example by the
* bootloader), enough time has aready passed.
*/
- if ((readl(osc->sckcr) & AT91_SCKC_OSCSEL)) {
+ if ((readl(osc->sckcr) & osc->bits->cr_oscsel)) {
osc->prepared = true;
return 0;
}
.is_prepared = clk_sama5d4_slow_osc_is_prepared,
};
+static const struct clk_slow_bits at91sama5d4_bits = {
+ .cr_oscsel = BIT(3),
+};
+
static void __init of_sama5d4_sckc_setup(struct device_node *np)
{
void __iomem *regbase = of_iomap(np, 0);
- struct clk_hw *hw;
+ struct clk_hw *slow_rc, *slowck;
struct clk_sama5d4_slow_osc *osc;
struct clk_init_data init;
const char *xtal_name;
const char *parent_names[2] = { "slow_rc_osc", "slow_osc" };
- bool bypass;
int ret;
if (!regbase)
return;
- hw = clk_hw_register_fixed_rate_with_accuracy(NULL, parent_names[0],
- NULL, 0, 32768,
- 250000000);
- if (IS_ERR(hw))
+ slow_rc = clk_hw_register_fixed_rate_with_accuracy(NULL,
+ parent_names[0],
+ NULL, 0, 32768,
+ 250000000);
+ if (IS_ERR(slow_rc))
return;
xtal_name = of_clk_get_parent_name(np, 0);
- bypass = of_property_read_bool(np, "atmel,osc-bypass");
-
osc = kzalloc(sizeof(*osc), GFP_KERNEL);
if (!osc)
- return;
+ goto unregister_slow_rc;
init.name = parent_names[1];
init.ops = &sama5d4_slow_osc_ops;
osc->hw.init = &init;
osc->sckcr = regbase;
osc->startup_usec = 1200000;
+ osc->bits = &at91sama5d4_bits;
- if (bypass)
- writel((readl(regbase) | AT91_SCKC_OSC32BYP), regbase);
-
- hw = &osc->hw;
ret = clk_hw_register(NULL, &osc->hw);
- if (ret) {
- kfree(osc);
- return;
- }
-
- hw = at91_clk_register_sam9x5_slow(regbase, "slowck", parent_names, 2);
- if (IS_ERR(hw))
- return;
-
- of_clk_add_hw_provider(np, of_clk_hw_simple_get, hw);
+ if (ret)
+ goto free_slow_osc_data;
+
+ slowck = at91_clk_register_sam9x5_slow(regbase, "slowck",
+ parent_names, 2,
+ &at91sama5d4_bits);
+ if (IS_ERR(slowck))
+ goto unregister_slow_osc;
+
+ ret = of_clk_add_hw_provider(np, of_clk_hw_simple_get, slowck);
+ if (WARN_ON(ret))
+ goto unregister_slowck;
+
+ return;
+
+unregister_slowck:
+ at91_clk_unregister_sam9x5_slow(slowck);
+unregister_slow_osc:
+ clk_hw_unregister(&osc->hw);
+free_slow_osc_data:
+ kfree(osc);
+unregister_slow_rc:
+ clk_hw_unregister(slow_rc);
}
CLK_OF_DECLARE(sama5d4_clk_sckc, "atmel,sama5d4-sckc",
of_sama5d4_sckc_setup);
# SPDX-License-Identifier: GPL-2.0-only
+config CLK_BCM2835
+ bool "Broadcom BCM2835 clock support"
+ depends on ARCH_BCM2835 || ARCH_BRCMSTB || COMPILE_TEST
+ depends on COMMON_CLK
+ default ARCH_BCM2835 || ARCH_BRCMSTB
+ help
+ Enable common clock framework support for Broadcom BCM2835
+ SoCs.
+
config CLK_BCM_63XX
bool "Broadcom BCM63xx clock support"
depends on ARCH_BCM_63XX || COMPILE_TEST
Enable common clock framework support for Broadcom BCM63xx DSL SoCs
based on the ARM architecture
+config CLK_BCM_63XX_GATE
+ bool "Broadcom BCM63xx gated clock support"
+ depends on BMIPS_GENERIC || COMPILE_TEST
+ default BMIPS_GENERIC
+ help
+ Enable common clock framework support for Broadcom BCM63xx DSL SoCs
+ based on the MIPS architecture
+
config CLK_BCM_KONA
bool "Broadcom Kona CCU clock support"
depends on ARCH_BCM_MOBILE || COMPILE_TEST
default ARCH_BCM_IPROC
help
Enable common clock framework support for the Broadcom Stingray SoC
+
+config CLK_RASPBERRYPI
+ tristate "Raspberry Pi firmware based clock support"
+ depends on RASPBERRYPI_FIRMWARE || (COMPILE_TEST && !RASPBERRYPI_FIRMWARE)
+ help
+ Enable common clock framework support for Raspberry Pi's firmware
+ dependent clocks
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_CLK_BCM_63XX) += clk-bcm63xx.o
+obj-$(CONFIG_CLK_BCM_63XX_GATE) += clk-bcm63xx-gate.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-kona.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-kona-setup.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-bcm281xx.o
obj-$(CONFIG_CLK_BCM_KONA) += clk-bcm21664.o
obj-$(CONFIG_COMMON_CLK_IPROC) += clk-iproc-armpll.o clk-iproc-pll.o clk-iproc-asiu.o
-obj-$(CONFIG_ARCH_BCM2835) += clk-bcm2835.o
-obj-$(CONFIG_ARCH_BCM2835) += clk-bcm2835-aux.o
+obj-$(CONFIG_CLK_BCM2835) += clk-bcm2835.o
+obj-$(CONFIG_CLK_BCM2835) += clk-bcm2835-aux.o
+obj-$(CONFIG_CLK_RASPBERRYPI) += clk-raspberrypi.o
obj-$(CONFIG_ARCH_BCM_53573) += clk-bcm53573-ilp.o
obj-$(CONFIG_CLK_BCM_CYGNUS) += clk-cygnus.o
obj-$(CONFIG_CLK_BCM_HR2) += clk-hr2.o
.fixed_divider = 1,
.flags = CLK_SET_RATE_PARENT),
- /* PLLB is used for the ARM's clock. */
- [BCM2835_PLLB] = REGISTER_PLL(
- .name = "pllb",
- .cm_ctrl_reg = CM_PLLB,
- .a2w_ctrl_reg = A2W_PLLB_CTRL,
- .frac_reg = A2W_PLLB_FRAC,
- .ana_reg_base = A2W_PLLB_ANA0,
- .reference_enable_mask = A2W_XOSC_CTRL_PLLB_ENABLE,
- .lock_mask = CM_LOCK_FLOCKB,
-
- .ana = &bcm2835_ana_default,
-
- .min_rate = 600000000u,
- .max_rate = 3000000000u,
- .max_fb_rate = BCM2835_MAX_FB_RATE),
- [BCM2835_PLLB_ARM] = REGISTER_PLL_DIV(
- .name = "pllb_arm",
- .source_pll = "pllb",
- .cm_reg = CM_PLLB,
- .a2w_reg = A2W_PLLB_ARM,
- .load_mask = CM_PLLB_LOADARM,
- .hold_mask = CM_PLLB_HOLDARM,
- .fixed_divider = 1,
- .flags = CLK_SET_RATE_PARENT),
+ /*
+ * PLLB is used for the ARM's clock. Controlled by firmware, see
+ * clk-raspberrypi.c.
+ */
/*
* PLLC is the core PLL, used to drive the core VPU clock.
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/clk-provider.h>
+#include <linux/init.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+struct clk_bcm63xx_table_entry {
+ const char * const name;
+ u8 bit;
+ unsigned long flags;
+};
+
+struct clk_bcm63xx_hw {
+ void __iomem *regs;
+ spinlock_t lock;
+
+ struct clk_hw_onecell_data data;
+};
+
+static const struct clk_bcm63xx_table_entry bcm3368_clocks[] = {
+ { .name = "mac", .bit = 3, },
+ { .name = "tc", .bit = 5, },
+ { .name = "us_top", .bit = 6, },
+ { .name = "ds_top", .bit = 7, },
+ { .name = "acm", .bit = 8, },
+ { .name = "spi", .bit = 9, },
+ { .name = "usbs", .bit = 10, },
+ { .name = "bmu", .bit = 11, },
+ { .name = "pcm", .bit = 12, },
+ { .name = "ntp", .bit = 13, },
+ { .name = "acp_b", .bit = 14, },
+ { .name = "acp_a", .bit = 15, },
+ { .name = "emusb", .bit = 17, },
+ { .name = "enet0", .bit = 18, },
+ { .name = "enet1", .bit = 19, },
+ { .name = "usbsu", .bit = 20, },
+ { .name = "ephy", .bit = 21, },
+ { },
+};
+
+static const struct clk_bcm63xx_table_entry bcm6328_clocks[] = {
+ { .name = "phy_mips", .bit = 0, },
+ { .name = "adsl_qproc", .bit = 1, },
+ { .name = "adsl_afe", .bit = 2, },
+ { .name = "adsl", .bit = 3, },
+ { .name = "mips", .bit = 4, .flags = CLK_IS_CRITICAL, },
+ { .name = "sar", .bit = 5, },
+ { .name = "pcm", .bit = 6, },
+ { .name = "usbd", .bit = 7, },
+ { .name = "usbh", .bit = 8, },
+ { .name = "hsspi", .bit = 9, },
+ { .name = "pcie", .bit = 10, },
+ { .name = "robosw", .bit = 11, },
+ { },
+};
+
+static const struct clk_bcm63xx_table_entry bcm6358_clocks[] = {
+ { .name = "enet", .bit = 4, },
+ { .name = "adslphy", .bit = 5, },
+ { .name = "pcm", .bit = 8, },
+ { .name = "spi", .bit = 9, },
+ { .name = "usbs", .bit = 10, },
+ { .name = "sar", .bit = 11, },
+ { .name = "emusb", .bit = 17, },
+ { .name = "enet0", .bit = 18, },
+ { .name = "enet1", .bit = 19, },
+ { .name = "usbsu", .bit = 20, },
+ { .name = "ephy", .bit = 21, },
+ { },
+};
+
+static const struct clk_bcm63xx_table_entry bcm6362_clocks[] = {
+ { .name = "adsl_qproc", .bit = 1, },
+ { .name = "adsl_afe", .bit = 2, },
+ { .name = "adsl", .bit = 3, },
+ { .name = "mips", .bit = 4, .flags = CLK_IS_CRITICAL, },
+ { .name = "wlan_ocp", .bit = 5, },
+ { .name = "swpkt_usb", .bit = 7, },
+ { .name = "swpkt_sar", .bit = 8, },
+ { .name = "sar", .bit = 9, },
+ { .name = "robosw", .bit = 10, },
+ { .name = "pcm", .bit = 11, },
+ { .name = "usbd", .bit = 12, },
+ { .name = "usbh", .bit = 13, },
+ { .name = "ipsec", .bit = 14, },
+ { .name = "spi", .bit = 15, },
+ { .name = "hsspi", .bit = 16, },
+ { .name = "pcie", .bit = 17, },
+ { .name = "fap", .bit = 18, },
+ { .name = "phymips", .bit = 19, },
+ { .name = "nand", .bit = 20, },
+ { },
+};
+
+static const struct clk_bcm63xx_table_entry bcm6368_clocks[] = {
+ { .name = "vdsl_qproc", .bit = 2, },
+ { .name = "vdsl_afe", .bit = 3, },
+ { .name = "vdsl_bonding", .bit = 4, },
+ { .name = "vdsl", .bit = 5, },
+ { .name = "phymips", .bit = 6, },
+ { .name = "swpkt_usb", .bit = 7, },
+ { .name = "swpkt_sar", .bit = 8, },
+ { .name = "spi", .bit = 9, },
+ { .name = "usbd", .bit = 10, },
+ { .name = "sar", .bit = 11, },
+ { .name = "robosw", .bit = 12, },
+ { .name = "utopia", .bit = 13, },
+ { .name = "pcm", .bit = 14, },
+ { .name = "usbh", .bit = 15, },
+ { .name = "disable_gless", .bit = 16, },
+ { .name = "nand", .bit = 17, },
+ { .name = "ipsec", .bit = 18, },
+ { },
+};
+
+static const struct clk_bcm63xx_table_entry bcm63268_clocks[] = {
+ { .name = "disable_gless", .bit = 0, },
+ { .name = "vdsl_qproc", .bit = 1, },
+ { .name = "vdsl_afe", .bit = 2, },
+ { .name = "vdsl", .bit = 3, },
+ { .name = "mips", .bit = 4, .flags = CLK_IS_CRITICAL, },
+ { .name = "wlan_ocp", .bit = 5, },
+ { .name = "dect", .bit = 6, },
+ { .name = "fap0", .bit = 7, },
+ { .name = "fap1", .bit = 8, },
+ { .name = "sar", .bit = 9, },
+ { .name = "robosw", .bit = 10, },
+ { .name = "pcm", .bit = 11, },
+ { .name = "usbd", .bit = 12, },
+ { .name = "usbh", .bit = 13, },
+ { .name = "ipsec", .bit = 14, },
+ { .name = "spi", .bit = 15, },
+ { .name = "hsspi", .bit = 16, },
+ { .name = "pcie", .bit = 17, },
+ { .name = "phymips", .bit = 18, },
+ { .name = "gmac", .bit = 19, },
+ { .name = "nand", .bit = 20, },
+ { .name = "tbus", .bit = 27, },
+ { .name = "robosw250", .bit = 31, },
+ { },
+};
+
+static int clk_bcm63xx_probe(struct platform_device *pdev)
+{
+ const struct clk_bcm63xx_table_entry *entry, *table;
+ struct clk_bcm63xx_hw *hw;
+ struct resource *r;
+ u8 maxbit = 0;
+ int i, ret;
+
+ table = of_device_get_match_data(&pdev->dev);
+ if (!table)
+ return -EINVAL;
+
+ for (entry = table; entry->name; entry++)
+ maxbit = max_t(u8, maxbit, entry->bit);
+
+ hw = devm_kzalloc(&pdev->dev, struct_size(hw, data.hws, maxbit),
+ GFP_KERNEL);
+ if (!hw)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, hw);
+
+ spin_lock_init(&hw->lock);
+
+ hw->data.num = maxbit;
+ for (i = 0; i < maxbit; i++)
+ hw->data.hws[i] = ERR_PTR(-ENODEV);
+
+ r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ hw->regs = devm_ioremap_resource(&pdev->dev, r);
+ if (IS_ERR(hw->regs))
+ return PTR_ERR(hw->regs);
+
+ for (entry = table; entry->name; entry++) {
+ struct clk_hw *clk;
+
+ clk = clk_hw_register_gate(&pdev->dev, entry->name, NULL,
+ entry->flags, hw->regs, entry->bit,
+ CLK_GATE_BIG_ENDIAN, &hw->lock);
+ if (IS_ERR(clk)) {
+ ret = PTR_ERR(clk);
+ goto out_err;
+ }
+
+ hw->data.hws[entry->bit] = clk;
+ }
+
+ ret = of_clk_add_hw_provider(pdev->dev.of_node, of_clk_hw_onecell_get,
+ &hw->data);
+ if (!ret)
+ return 0;
+out_err:
+ for (i = 0; i < hw->data.num; i++) {
+ if (!IS_ERR(hw->data.hws[i]))
+ clk_hw_unregister_gate(hw->data.hws[i]);
+ }
+
+ return ret;
+}
+
+static int clk_bcm63xx_remove(struct platform_device *pdev)
+{
+ struct clk_bcm63xx_hw *hw = platform_get_drvdata(pdev);
+ int i;
+
+ of_clk_del_provider(pdev->dev.of_node);
+
+ for (i = 0; i < hw->data.num; i++) {
+ if (!IS_ERR(hw->data.hws[i]))
+ clk_hw_unregister_gate(hw->data.hws[i]);
+ }
+
+ return 0;
+}
+
+static const struct of_device_id clk_bcm63xx_dt_ids[] = {
+ { .compatible = "brcm,bcm3368-clocks", .data = &bcm3368_clocks, },
+ { .compatible = "brcm,bcm6328-clocks", .data = &bcm6328_clocks, },
+ { .compatible = "brcm,bcm6358-clocks", .data = &bcm6358_clocks, },
+ { .compatible = "brcm,bcm6362-clocks", .data = &bcm6362_clocks, },
+ { .compatible = "brcm,bcm6368-clocks", .data = &bcm6368_clocks, },
+ { .compatible = "brcm,bcm63268-clocks", .data = &bcm63268_clocks, },
+ { }
+};
+
+static struct platform_driver clk_bcm63xx = {
+ .probe = clk_bcm63xx_probe,
+ .remove = clk_bcm63xx_remove,
+ .driver = {
+ .name = "bcm63xx-clock",
+ .of_match_table = clk_bcm63xx_dt_ids,
+ },
+};
+builtin_platform_driver(clk_bcm63xx);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Raspberry Pi driver for firmware controlled clocks
+ *
+ * Even though clk-bcm2835 provides an interface to the hardware registers for
+ * the system clocks we've had to factor out 'pllb' as the firmware 'owns' it.
+ * We're not allowed to change it directly as we might race with the
+ * over-temperature and under-voltage protections provided by the firmware.
+ *
+ * Copyright (C) 2019 Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
+ */
+
+#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <soc/bcm2835/raspberrypi-firmware.h>
+
+#define RPI_FIRMWARE_ARM_CLK_ID 0x00000003
+
+#define RPI_FIRMWARE_STATE_ENABLE_BIT BIT(0)
+#define RPI_FIRMWARE_STATE_WAIT_BIT BIT(1)
+
+/*
+ * Even though the firmware interface alters 'pllb' the frequencies are
+ * provided as per 'pllb_arm'. We need to scale before passing them trough.
+ */
+#define RPI_FIRMWARE_PLLB_ARM_DIV_RATE 2
+
+#define A2W_PLL_FRAC_BITS 20
+
+struct raspberrypi_clk {
+ struct device *dev;
+ struct rpi_firmware *firmware;
+ struct platform_device *cpufreq;
+
+ unsigned long min_rate;
+ unsigned long max_rate;
+
+ struct clk_hw pllb;
+ struct clk_hw *pllb_arm;
+ struct clk_lookup *pllb_arm_lookup;
+};
+
+/*
+ * Structure of the message passed to Raspberry Pi's firmware in order to
+ * change clock rates. The 'disable_turbo' option is only available to the ARM
+ * clock (pllb) which we enable by default as turbo mode will alter multiple
+ * clocks at once.
+ *
+ * Even though we're able to access the clock registers directly we're bound to
+ * use the firmware interface as the firmware ultimately takes care of
+ * mitigating overheating/undervoltage situations and we would be changing
+ * frequencies behind his back.
+ *
+ * For more information on the firmware interface check:
+ * https://github.com/raspberrypi/firmware/wiki/Mailbox-property-interface
+ */
+struct raspberrypi_firmware_prop {
+ __le32 id;
+ __le32 val;
+ __le32 disable_turbo;
+} __packed;
+
+static int raspberrypi_clock_property(struct rpi_firmware *firmware, u32 tag,
+ u32 clk, u32 *val)
+{
+ struct raspberrypi_firmware_prop msg = {
+ .id = cpu_to_le32(clk),
+ .val = cpu_to_le32(*val),
+ .disable_turbo = cpu_to_le32(1),
+ };
+ int ret;
+
+ ret = rpi_firmware_property(firmware, tag, &msg, sizeof(msg));
+ if (ret)
+ return ret;
+
+ *val = le32_to_cpu(msg.val);
+
+ return 0;
+}
+
+static int raspberrypi_fw_pll_is_on(struct clk_hw *hw)
+{
+ struct raspberrypi_clk *rpi = container_of(hw, struct raspberrypi_clk,
+ pllb);
+ u32 val = 0;
+ int ret;
+
+ ret = raspberrypi_clock_property(rpi->firmware,
+ RPI_FIRMWARE_GET_CLOCK_STATE,
+ RPI_FIRMWARE_ARM_CLK_ID, &val);
+ if (ret)
+ return 0;
+
+ return !!(val & RPI_FIRMWARE_STATE_ENABLE_BIT);
+}
+
+
+static unsigned long raspberrypi_fw_pll_get_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct raspberrypi_clk *rpi = container_of(hw, struct raspberrypi_clk,
+ pllb);
+ u32 val = 0;
+ int ret;
+
+ ret = raspberrypi_clock_property(rpi->firmware,
+ RPI_FIRMWARE_GET_CLOCK_RATE,
+ RPI_FIRMWARE_ARM_CLK_ID,
+ &val);
+ if (ret)
+ return ret;
+
+ return val * RPI_FIRMWARE_PLLB_ARM_DIV_RATE;
+}
+
+static int raspberrypi_fw_pll_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct raspberrypi_clk *rpi = container_of(hw, struct raspberrypi_clk,
+ pllb);
+ u32 new_rate = rate / RPI_FIRMWARE_PLLB_ARM_DIV_RATE;
+ int ret;
+
+ ret = raspberrypi_clock_property(rpi->firmware,
+ RPI_FIRMWARE_SET_CLOCK_RATE,
+ RPI_FIRMWARE_ARM_CLK_ID,
+ &new_rate);
+ if (ret)
+ dev_err_ratelimited(rpi->dev, "Failed to change %s frequency: %d",
+ clk_hw_get_name(hw), ret);
+
+ return ret;
+}
+
+/*
+ * Sadly there is no firmware rate rounding interface. We borrowed it from
+ * clk-bcm2835.
+ */
+static int raspberrypi_pll_determine_rate(struct clk_hw *hw,
+ struct clk_rate_request *req)
+{
+ struct raspberrypi_clk *rpi = container_of(hw, struct raspberrypi_clk,
+ pllb);
+ u64 div, final_rate;
+ u32 ndiv, fdiv;
+
+ /* We can't use req->rate directly as it would overflow */
+ final_rate = clamp(req->rate, rpi->min_rate, rpi->max_rate);
+
+ div = (u64)final_rate << A2W_PLL_FRAC_BITS;
+ do_div(div, req->best_parent_rate);
+
+ ndiv = div >> A2W_PLL_FRAC_BITS;
+ fdiv = div & ((1 << A2W_PLL_FRAC_BITS) - 1);
+
+ final_rate = ((u64)req->best_parent_rate *
+ ((ndiv << A2W_PLL_FRAC_BITS) + fdiv));
+
+ req->rate = final_rate >> A2W_PLL_FRAC_BITS;
+
+ return 0;
+}
+
+static const struct clk_ops raspberrypi_firmware_pll_clk_ops = {
+ .is_prepared = raspberrypi_fw_pll_is_on,
+ .recalc_rate = raspberrypi_fw_pll_get_rate,
+ .set_rate = raspberrypi_fw_pll_set_rate,
+ .determine_rate = raspberrypi_pll_determine_rate,
+};
+
+static int raspberrypi_register_pllb(struct raspberrypi_clk *rpi)
+{
+ u32 min_rate = 0, max_rate = 0;
+ struct clk_init_data init;
+ int ret;
+
+ memset(&init, 0, sizeof(init));
+
+ /* All of the PLLs derive from the external oscillator. */
+ init.parent_names = (const char *[]){ "osc" };
+ init.num_parents = 1;
+ init.name = "pllb";
+ init.ops = &raspberrypi_firmware_pll_clk_ops;
+ init.flags = CLK_GET_RATE_NOCACHE | CLK_IGNORE_UNUSED;
+
+ /* Get min & max rates set by the firmware */
+ ret = raspberrypi_clock_property(rpi->firmware,
+ RPI_FIRMWARE_GET_MIN_CLOCK_RATE,
+ RPI_FIRMWARE_ARM_CLK_ID,
+ &min_rate);
+ if (ret) {
+ dev_err(rpi->dev, "Failed to get %s min freq: %d\n",
+ init.name, ret);
+ return ret;
+ }
+
+ ret = raspberrypi_clock_property(rpi->firmware,
+ RPI_FIRMWARE_GET_MAX_CLOCK_RATE,
+ RPI_FIRMWARE_ARM_CLK_ID,
+ &max_rate);
+ if (ret) {
+ dev_err(rpi->dev, "Failed to get %s max freq: %d\n",
+ init.name, ret);
+ return ret;
+ }
+
+ if (!min_rate || !max_rate) {
+ dev_err(rpi->dev, "Unexpected frequency range: min %u, max %u\n",
+ min_rate, max_rate);
+ return -EINVAL;
+ }
+
+ dev_info(rpi->dev, "CPU frequency range: min %u, max %u\n",
+ min_rate, max_rate);
+
+ rpi->min_rate = min_rate * RPI_FIRMWARE_PLLB_ARM_DIV_RATE;
+ rpi->max_rate = max_rate * RPI_FIRMWARE_PLLB_ARM_DIV_RATE;
+
+ rpi->pllb.init = &init;
+
+ return devm_clk_hw_register(rpi->dev, &rpi->pllb);
+}
+
+static int raspberrypi_register_pllb_arm(struct raspberrypi_clk *rpi)
+{
+ rpi->pllb_arm = clk_hw_register_fixed_factor(rpi->dev,
+ "pllb_arm", "pllb",
+ CLK_SET_RATE_PARENT | CLK_GET_RATE_NOCACHE,
+ 1, 2);
+ if (IS_ERR(rpi->pllb_arm)) {
+ dev_err(rpi->dev, "Failed to initialize pllb_arm\n");
+ return PTR_ERR(rpi->pllb_arm);
+ }
+
+ rpi->pllb_arm_lookup = clkdev_hw_create(rpi->pllb_arm, NULL, "cpu0");
+ if (!rpi->pllb_arm_lookup) {
+ dev_err(rpi->dev, "Failed to initialize pllb_arm_lookup\n");
+ clk_hw_unregister_fixed_factor(rpi->pllb_arm);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int raspberrypi_clk_probe(struct platform_device *pdev)
+{
+ struct device_node *firmware_node;
+ struct device *dev = &pdev->dev;
+ struct rpi_firmware *firmware;
+ struct raspberrypi_clk *rpi;
+ int ret;
+
+ firmware_node = of_find_compatible_node(NULL, NULL,
+ "raspberrypi,bcm2835-firmware");
+ if (!firmware_node) {
+ dev_err(dev, "Missing firmware node\n");
+ return -ENOENT;
+ }
+
+ firmware = rpi_firmware_get(firmware_node);
+ of_node_put(firmware_node);
+ if (!firmware)
+ return -EPROBE_DEFER;
+
+ rpi = devm_kzalloc(dev, sizeof(*rpi), GFP_KERNEL);
+ if (!rpi)
+ return -ENOMEM;
+
+ rpi->dev = dev;
+ rpi->firmware = firmware;
+ platform_set_drvdata(pdev, rpi);
+
+ ret = raspberrypi_register_pllb(rpi);
+ if (ret) {
+ dev_err(dev, "Failed to initialize pllb, %d\n", ret);
+ return ret;
+ }
+
+ ret = raspberrypi_register_pllb_arm(rpi);
+ if (ret)
+ return ret;
+
+ rpi->cpufreq = platform_device_register_data(dev, "raspberrypi-cpufreq",
+ -1, NULL, 0);
+
+ return 0;
+}
+
+static int raspberrypi_clk_remove(struct platform_device *pdev)
+{
+ struct raspberrypi_clk *rpi = platform_get_drvdata(pdev);
+
+ platform_device_unregister(rpi->cpufreq);
+
+ return 0;
+}
+
+static struct platform_driver raspberrypi_clk_driver = {
+ .driver = {
+ .name = "raspberrypi-clk",
+ },
+ .probe = raspberrypi_clk_probe,
+ .remove = raspberrypi_clk_remove,
+};
+module_platform_driver(raspberrypi_clk_driver);
+
+MODULE_AUTHOR("Nicolas Saenz Julienne <nsaenzjulienne@suse.de>");
+MODULE_DESCRIPTION("Raspberry Pi firmware clock driver");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:raspberrypi-clk");
}
EXPORT_SYMBOL_GPL(clk_bulk_put);
-int __must_check clk_bulk_get(struct device *dev, int num_clks,
- struct clk_bulk_data *clks)
+static int __clk_bulk_get(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks, bool optional)
{
int ret;
int i;
clks[i].clk = clk_get(dev, clks[i].id);
if (IS_ERR(clks[i].clk)) {
ret = PTR_ERR(clks[i].clk);
+ clks[i].clk = NULL;
+
+ if (ret == -ENOENT && optional)
+ continue;
+
if (ret != -EPROBE_DEFER)
dev_err(dev, "Failed to get clk '%s': %d\n",
clks[i].id, ret);
- clks[i].clk = NULL;
goto err;
}
}
return ret;
}
+
+int __must_check clk_bulk_get(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks)
+{
+ return __clk_bulk_get(dev, num_clks, clks, false);
+}
EXPORT_SYMBOL(clk_bulk_get);
+int __must_check clk_bulk_get_optional(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks)
+{
+ return __clk_bulk_get(dev, num_clks, clks, true);
+}
+EXPORT_SYMBOL_GPL(clk_bulk_get_optional);
+
void clk_bulk_put_all(int num_clks, struct clk_bulk_data *clks)
{
if (IS_ERR_OR_NULL(clks))
static int cdce706_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
struct cdce706_dev_data *cdce;
int ret;
clk_bulk_put(devres->num_clks, devres->clks);
}
-int __must_check devm_clk_bulk_get(struct device *dev, int num_clks,
- struct clk_bulk_data *clks)
+static int __devm_clk_bulk_get(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks, bool optional)
{
struct clk_bulk_devres *devres;
int ret;
if (!devres)
return -ENOMEM;
- ret = clk_bulk_get(dev, num_clks, clks);
+ if (optional)
+ ret = clk_bulk_get_optional(dev, num_clks, clks);
+ else
+ ret = clk_bulk_get(dev, num_clks, clks);
if (!ret) {
devres->clks = clks;
devres->num_clks = num_clks;
return ret;
}
+
+int __must_check devm_clk_bulk_get(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks)
+{
+ return __devm_clk_bulk_get(dev, num_clks, clks, false);
+}
EXPORT_SYMBOL_GPL(devm_clk_bulk_get);
+int __must_check devm_clk_bulk_get_optional(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks)
+{
+ return __devm_clk_bulk_get(dev, num_clks, clks, true);
+}
+EXPORT_SYMBOL_GPL(devm_clk_bulk_get_optional);
+
int __must_check devm_clk_bulk_get_all(struct device *dev,
struct clk_bulk_data **clks)
{
#include <linux/platform_device.h>
#include <linux/regmap.h>
-#include <linux/mfd/lochnagar.h>
#include <linux/mfd/lochnagar1_regs.h>
#include <linux/mfd/lochnagar2_regs.h>
struct lochnagar_clk_priv {
struct device *dev;
struct regmap *regmap;
- enum lochnagar_type type;
-
- const char **parents;
- unsigned int nparents;
struct lochnagar_clk lclks[LOCHNAGAR_NUM_CLOCKS];
};
-static const char * const lochnagar1_clk_parents[] = {
- "ln-none",
- "ln-spdif-mclk",
- "ln-psia1-mclk",
- "ln-psia2-mclk",
- "ln-cdc-clkout",
- "ln-dsp-clkout",
- "ln-pmic-32k",
- "ln-gf-mclk1",
- "ln-gf-mclk3",
- "ln-gf-mclk2",
- "ln-gf-mclk4",
+#define LN_PARENT(NAME) { .name = NAME, .fw_name = NAME }
+
+static const struct clk_parent_data lochnagar1_clk_parents[] = {
+ LN_PARENT("ln-none"),
+ LN_PARENT("ln-spdif-mclk"),
+ LN_PARENT("ln-psia1-mclk"),
+ LN_PARENT("ln-psia2-mclk"),
+ LN_PARENT("ln-cdc-clkout"),
+ LN_PARENT("ln-dsp-clkout"),
+ LN_PARENT("ln-pmic-32k"),
+ LN_PARENT("ln-gf-mclk1"),
+ LN_PARENT("ln-gf-mclk3"),
+ LN_PARENT("ln-gf-mclk2"),
+ LN_PARENT("ln-gf-mclk4"),
};
-static const char * const lochnagar2_clk_parents[] = {
- "ln-none",
- "ln-cdc-clkout",
- "ln-dsp-clkout",
- "ln-pmic-32k",
- "ln-spdif-mclk",
- "ln-clk-12m",
- "ln-clk-11m",
- "ln-clk-24m",
- "ln-clk-22m",
- "ln-clk-8m",
- "ln-usb-clk-24m",
- "ln-gf-mclk1",
- "ln-gf-mclk3",
- "ln-gf-mclk2",
- "ln-psia1-mclk",
- "ln-psia2-mclk",
- "ln-spdif-clkout",
- "ln-adat-mclk",
- "ln-usb-clk-12m",
+static const struct clk_parent_data lochnagar2_clk_parents[] = {
+ LN_PARENT("ln-none"),
+ LN_PARENT("ln-cdc-clkout"),
+ LN_PARENT("ln-dsp-clkout"),
+ LN_PARENT("ln-pmic-32k"),
+ LN_PARENT("ln-spdif-mclk"),
+ LN_PARENT("ln-clk-12m"),
+ LN_PARENT("ln-clk-11m"),
+ LN_PARENT("ln-clk-24m"),
+ LN_PARENT("ln-clk-22m"),
+ LN_PARENT("ln-clk-8m"),
+ LN_PARENT("ln-usb-clk-24m"),
+ LN_PARENT("ln-gf-mclk1"),
+ LN_PARENT("ln-gf-mclk3"),
+ LN_PARENT("ln-gf-mclk2"),
+ LN_PARENT("ln-psia1-mclk"),
+ LN_PARENT("ln-psia2-mclk"),
+ LN_PARENT("ln-spdif-clkout"),
+ LN_PARENT("ln-adat-mclk"),
+ LN_PARENT("ln-usb-clk-12m"),
};
#define LN1_CLK(ID, NAME, REG) \
LN2_CLK(SOUNDCARD_MCLK, "ln-soundcard-mclk"),
};
+struct lochnagar_config {
+ const struct clk_parent_data *parents;
+ int nparents;
+ const struct lochnagar_clk *clks;
+};
+
+static const struct lochnagar_config lochnagar1_conf = {
+ .parents = lochnagar1_clk_parents,
+ .nparents = ARRAY_SIZE(lochnagar1_clk_parents),
+ .clks = lochnagar1_clks,
+};
+
+static const struct lochnagar_config lochnagar2_conf = {
+ .parents = lochnagar2_clk_parents,
+ .nparents = ARRAY_SIZE(lochnagar2_clk_parents),
+ .clks = lochnagar2_clks,
+};
+
static inline struct lochnagar_clk *lochnagar_hw_to_lclk(struct clk_hw *hw)
{
return container_of(hw, struct lochnagar_clk, hw);
if (ret < 0) {
dev_dbg(priv->dev, "Failed to read parent of %s: %d\n",
lclk->name, ret);
- return priv->nparents;
+ return hw->init->num_parents;
}
val &= lclk->src_mask;
.get_parent = lochnagar_clk_get_parent,
};
-static int lochnagar_init_parents(struct lochnagar_clk_priv *priv)
-{
- struct device_node *np = priv->dev->of_node;
- int i, j;
-
- switch (priv->type) {
- case LOCHNAGAR1:
- memcpy(priv->lclks, lochnagar1_clks, sizeof(lochnagar1_clks));
-
- priv->nparents = ARRAY_SIZE(lochnagar1_clk_parents);
- priv->parents = devm_kmemdup(priv->dev, lochnagar1_clk_parents,
- sizeof(lochnagar1_clk_parents),
- GFP_KERNEL);
- break;
- case LOCHNAGAR2:
- memcpy(priv->lclks, lochnagar2_clks, sizeof(lochnagar2_clks));
-
- priv->nparents = ARRAY_SIZE(lochnagar2_clk_parents);
- priv->parents = devm_kmemdup(priv->dev, lochnagar2_clk_parents,
- sizeof(lochnagar2_clk_parents),
- GFP_KERNEL);
- break;
- default:
- dev_err(priv->dev, "Unknown Lochnagar type: %d\n", priv->type);
- return -EINVAL;
- }
-
- if (!priv->parents)
- return -ENOMEM;
-
- for (i = 0; i < priv->nparents; i++) {
- j = of_property_match_string(np, "clock-names",
- priv->parents[i]);
- if (j >= 0)
- priv->parents[i] = of_clk_get_parent_name(np, j);
- }
-
- return 0;
-}
-
static struct clk_hw *
lochnagar_of_clk_hw_get(struct of_phandle_args *clkspec, void *data)
{
return &priv->lclks[idx].hw;
}
-static int lochnagar_init_clks(struct lochnagar_clk_priv *priv)
+static const struct of_device_id lochnagar_of_match[] = {
+ { .compatible = "cirrus,lochnagar1-clk", .data = &lochnagar1_conf },
+ { .compatible = "cirrus,lochnagar2-clk", .data = &lochnagar2_conf },
+ {}
+};
+MODULE_DEVICE_TABLE(of, lochnagar_of_match);
+
+static int lochnagar_clk_probe(struct platform_device *pdev)
{
struct clk_init_data clk_init = {
.ops = &lochnagar_clk_ops,
- .parent_names = priv->parents,
- .num_parents = priv->nparents,
};
+ struct device *dev = &pdev->dev;
+ struct lochnagar_clk_priv *priv;
+ const struct of_device_id *of_id;
struct lochnagar_clk *lclk;
+ struct lochnagar_config *conf;
int ret, i;
+ of_id = of_match_device(lochnagar_of_match, dev);
+ if (!of_id)
+ return -EINVAL;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->dev = dev;
+ priv->regmap = dev_get_regmap(dev->parent, NULL);
+ conf = (struct lochnagar_config *)of_id->data;
+
+ memcpy(priv->lclks, conf->clks, sizeof(priv->lclks));
+
+ clk_init.parent_data = conf->parents;
+ clk_init.num_parents = conf->nparents;
+
for (i = 0; i < ARRAY_SIZE(priv->lclks); i++) {
lclk = &priv->lclks[i];
lclk->priv = priv;
lclk->hw.init = &clk_init;
- ret = devm_clk_hw_register(priv->dev, &lclk->hw);
+ ret = devm_clk_hw_register(dev, &lclk->hw);
if (ret) {
- dev_err(priv->dev, "Failed to register %s: %d\n",
+ dev_err(dev, "Failed to register %s: %d\n",
lclk->name, ret);
return ret;
}
}
- ret = devm_of_clk_add_hw_provider(priv->dev, lochnagar_of_clk_hw_get,
- priv);
+ ret = devm_of_clk_add_hw_provider(dev, lochnagar_of_clk_hw_get, priv);
if (ret < 0)
- dev_err(priv->dev, "Failed to register provider: %d\n", ret);
+ dev_err(dev, "Failed to register provider: %d\n", ret);
return ret;
}
-static const struct of_device_id lochnagar_of_match[] = {
- { .compatible = "cirrus,lochnagar1-clk", .data = (void *)LOCHNAGAR1 },
- { .compatible = "cirrus,lochnagar2-clk", .data = (void *)LOCHNAGAR2 },
- {}
-};
-MODULE_DEVICE_TABLE(of, lochnagar_of_match);
-
-static int lochnagar_clk_probe(struct platform_device *pdev)
-{
- struct device *dev = &pdev->dev;
- struct lochnagar_clk_priv *priv;
- const struct of_device_id *of_id;
- int ret;
-
- of_id = of_match_device(lochnagar_of_match, dev);
- if (!of_id)
- return -EINVAL;
-
- priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
-
- priv->dev = dev;
- priv->regmap = dev_get_regmap(dev->parent, NULL);
- priv->type = (enum lochnagar_type)of_id->data;
-
- ret = lochnagar_init_parents(priv);
- if (ret)
- return ret;
-
- return lochnagar_init_clks(priv);
-}
-
static struct platform_driver lochnagar_clk_driver = {
.driver = {
.name = "lochnagar-clk",
return clk_pwm->fixed_rate;
}
+static int clk_pwm_get_duty_cycle(struct clk_hw *hw, struct clk_duty *duty)
+{
+ struct clk_pwm *clk_pwm = to_clk_pwm(hw);
+ struct pwm_state state;
+
+ pwm_get_state(clk_pwm->pwm, &state);
+
+ duty->num = state.duty_cycle;
+ duty->den = state.period;
+
+ return 0;
+}
+
static const struct clk_ops clk_pwm_ops = {
.prepare = clk_pwm_prepare,
.unprepare = clk_pwm_unprepare,
.recalc_rate = clk_pwm_recalc_rate,
+ .get_duty_cycle = clk_pwm_get_duty_cycle,
};
static int clk_pwm_probe(struct platform_device *pdev)
.flags = CG_VER3 | CG_LITTLE_ENDIAN,
},
{
+ .compat = "fsl,lx2160a-clockgen",
+ .cmux_groups = {
+ &clockgen2_cmux_cga12, &clockgen2_cmux_cgb
+ },
+ .cmux_to_group = {
+ 0, 0, 0, 0, 1, 1, 1, 1, -1
+ },
+ .pll_mask = 0x37,
+ .flags = CG_VER3 | CG_LITTLE_ENDIAN,
+ },
+ {
.compat = "fsl,p2041-clockgen",
.guts_compat = "fsl,qoriq-device-config-1.0",
.init_periph = p2041_init_periph,
CLK_OF_DECLARE(qoriq_clockgen_ls1046a, "fsl,ls1046a-clockgen", clockgen_init);
CLK_OF_DECLARE(qoriq_clockgen_ls1088a, "fsl,ls1088a-clockgen", clockgen_init);
CLK_OF_DECLARE(qoriq_clockgen_ls2080a, "fsl,ls2080a-clockgen", clockgen_init);
+CLK_OF_DECLARE(qoriq_clockgen_lx2160a, "fsl,lx2160a-clockgen", clockgen_init);
CLK_OF_DECLARE(qoriq_clockgen_p2041, "fsl,p2041-clockgen", clockgen_init);
CLK_OF_DECLARE(qoriq_clockgen_p3041, "fsl,p3041-clockgen", clockgen_init);
CLK_OF_DECLARE(qoriq_clockgen_p4080, "fsl,p4080-clockgen", clockgen_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Driver for Silicon Labs Si5341/Si5340 Clock generator
+ * Copyright (C) 2019 Topic Embedded Products
+ * Author: Mike Looijmans <mike.looijmans@topic.nl>
+ */
+
+#include <linux/clk.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
+#include <linux/gcd.h>
+#include <linux/math64.h>
+#include <linux/i2c.h>
+#include <linux/module.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+#include <asm/unaligned.h>
+
+#define SI5341_MAX_NUM_OUTPUTS 10
+#define SI5340_MAX_NUM_OUTPUTS 4
+
+#define SI5341_NUM_SYNTH 5
+#define SI5340_NUM_SYNTH 4
+
+/* Range of the synthesizer fractional divider */
+#define SI5341_SYNTH_N_MIN 10
+#define SI5341_SYNTH_N_MAX 4095
+
+/* The chip can get its input clock from 3 input pins or an XTAL */
+
+/* There is one PLL running at 13500–14256 MHz */
+#define SI5341_PLL_VCO_MIN 13500000000ull
+#define SI5341_PLL_VCO_MAX 14256000000ull
+
+/* The 5 frequency synthesizers obtain their input from the PLL */
+struct clk_si5341_synth {
+ struct clk_hw hw;
+ struct clk_si5341 *data;
+ u8 index;
+};
+#define to_clk_si5341_synth(_hw) \
+ container_of(_hw, struct clk_si5341_synth, hw)
+
+/* The output stages can be connected to any synth (full mux) */
+struct clk_si5341_output {
+ struct clk_hw hw;
+ struct clk_si5341 *data;
+ u8 index;
+};
+#define to_clk_si5341_output(_hw) \
+ container_of(_hw, struct clk_si5341_output, hw)
+
+struct clk_si5341 {
+ struct clk_hw hw;
+ struct regmap *regmap;
+ struct i2c_client *i2c_client;
+ struct clk_si5341_synth synth[SI5341_NUM_SYNTH];
+ struct clk_si5341_output clk[SI5341_MAX_NUM_OUTPUTS];
+ struct clk *pxtal;
+ const char *pxtal_name;
+ const u16 *reg_output_offset;
+ const u16 *reg_rdiv_offset;
+ u64 freq_vco; /* 13500–14256 MHz */
+ u8 num_outputs;
+ u8 num_synth;
+};
+#define to_clk_si5341(_hw) container_of(_hw, struct clk_si5341, hw)
+
+struct clk_si5341_output_config {
+ u8 out_format_drv_bits;
+ u8 out_cm_ampl_bits;
+ bool synth_master;
+ bool always_on;
+};
+
+#define SI5341_PAGE 0x0001
+#define SI5341_PN_BASE 0x0002
+#define SI5341_DEVICE_REV 0x0005
+#define SI5341_STATUS 0x000C
+#define SI5341_SOFT_RST 0x001C
+
+/* Input dividers (48-bit) */
+#define SI5341_IN_PDIV(x) (0x0208 + ((x) * 10))
+#define SI5341_IN_PSET(x) (0x020E + ((x) * 10))
+
+/* PLL configuration */
+#define SI5341_PLL_M_NUM 0x0235
+#define SI5341_PLL_M_DEN 0x023B
+
+/* Output configuration */
+#define SI5341_OUT_CONFIG(output) \
+ ((output)->data->reg_output_offset[(output)->index])
+#define SI5341_OUT_FORMAT(output) (SI5341_OUT_CONFIG(output) + 1)
+#define SI5341_OUT_CM(output) (SI5341_OUT_CONFIG(output) + 2)
+#define SI5341_OUT_MUX_SEL(output) (SI5341_OUT_CONFIG(output) + 3)
+#define SI5341_OUT_R_REG(output) \
+ ((output)->data->reg_rdiv_offset[(output)->index])
+
+/* Synthesize N divider */
+#define SI5341_SYNTH_N_NUM(x) (0x0302 + ((x) * 11))
+#define SI5341_SYNTH_N_DEN(x) (0x0308 + ((x) * 11))
+#define SI5341_SYNTH_N_UPD(x) (0x030C + ((x) * 11))
+
+/* Synthesizer output enable, phase bypass, power mode */
+#define SI5341_SYNTH_N_CLK_TO_OUTX_EN 0x0A03
+#define SI5341_SYNTH_N_PIBYP 0x0A04
+#define SI5341_SYNTH_N_PDNB 0x0A05
+#define SI5341_SYNTH_N_CLK_DIS 0x0B4A
+
+#define SI5341_REGISTER_MAX 0xBFF
+
+/* SI5341_OUT_CONFIG bits */
+#define SI5341_OUT_CFG_PDN BIT(0)
+#define SI5341_OUT_CFG_OE BIT(1)
+#define SI5341_OUT_CFG_RDIV_FORCE2 BIT(2)
+
+/* Static configuration (to be moved to firmware) */
+struct si5341_reg_default {
+ u16 address;
+ u8 value;
+};
+
+/* Output configuration registers 0..9 are not quite logically organized */
+static const u16 si5341_reg_output_offset[] = {
+ 0x0108,
+ 0x010D,
+ 0x0112,
+ 0x0117,
+ 0x011C,
+ 0x0121,
+ 0x0126,
+ 0x012B,
+ 0x0130,
+ 0x013A,
+};
+
+static const u16 si5340_reg_output_offset[] = {
+ 0x0112,
+ 0x0117,
+ 0x0126,
+ 0x012B,
+};
+
+/* The location of the R divider registers */
+static const u16 si5341_reg_rdiv_offset[] = {
+ 0x024A,
+ 0x024D,
+ 0x0250,
+ 0x0253,
+ 0x0256,
+ 0x0259,
+ 0x025C,
+ 0x025F,
+ 0x0262,
+ 0x0268,
+};
+static const u16 si5340_reg_rdiv_offset[] = {
+ 0x0250,
+ 0x0253,
+ 0x025C,
+ 0x025F,
+};
+
+/*
+ * Programming sequence from ClockBuilder, settings to initialize the system
+ * using only the XTAL input, without pre-divider.
+ * This also contains settings that aren't mentioned anywhere in the datasheet.
+ * The "known" settings like synth and output configuration are done later.
+ */
+static const struct si5341_reg_default si5341_reg_defaults[] = {
+ { 0x0017, 0x3A }, /* INT mask (disable interrupts) */
+ { 0x0018, 0xFF }, /* INT mask */
+ { 0x0021, 0x0F }, /* Select XTAL as input */
+ { 0x0022, 0x00 }, /* Not in datasheet */
+ { 0x002B, 0x02 }, /* SPI config */
+ { 0x002C, 0x20 }, /* LOS enable for XTAL */
+ { 0x002D, 0x00 }, /* LOS timing */
+ { 0x002E, 0x00 },
+ { 0x002F, 0x00 },
+ { 0x0030, 0x00 },
+ { 0x0031, 0x00 },
+ { 0x0032, 0x00 },
+ { 0x0033, 0x00 },
+ { 0x0034, 0x00 },
+ { 0x0035, 0x00 },
+ { 0x0036, 0x00 },
+ { 0x0037, 0x00 },
+ { 0x0038, 0x00 }, /* LOS setting (thresholds) */
+ { 0x0039, 0x00 },
+ { 0x003A, 0x00 },
+ { 0x003B, 0x00 },
+ { 0x003C, 0x00 },
+ { 0x003D, 0x00 }, /* LOS setting (thresholds) end */
+ { 0x0041, 0x00 }, /* LOS0_DIV_SEL */
+ { 0x0042, 0x00 }, /* LOS1_DIV_SEL */
+ { 0x0043, 0x00 }, /* LOS2_DIV_SEL */
+ { 0x0044, 0x00 }, /* LOS3_DIV_SEL */
+ { 0x009E, 0x00 }, /* Not in datasheet */
+ { 0x0102, 0x01 }, /* Enable outputs */
+ { 0x013F, 0x00 }, /* Not in datasheet */
+ { 0x0140, 0x00 }, /* Not in datasheet */
+ { 0x0141, 0x40 }, /* OUT LOS */
+ { 0x0202, 0x00 }, /* XAXB_FREQ_OFFSET (=0)*/
+ { 0x0203, 0x00 },
+ { 0x0204, 0x00 },
+ { 0x0205, 0x00 },
+ { 0x0206, 0x00 }, /* PXAXB (2^x) */
+ { 0x0208, 0x00 }, /* Px divider setting (usually 0) */
+ { 0x0209, 0x00 },
+ { 0x020A, 0x00 },
+ { 0x020B, 0x00 },
+ { 0x020C, 0x00 },
+ { 0x020D, 0x00 },
+ { 0x020E, 0x00 },
+ { 0x020F, 0x00 },
+ { 0x0210, 0x00 },
+ { 0x0211, 0x00 },
+ { 0x0212, 0x00 },
+ { 0x0213, 0x00 },
+ { 0x0214, 0x00 },
+ { 0x0215, 0x00 },
+ { 0x0216, 0x00 },
+ { 0x0217, 0x00 },
+ { 0x0218, 0x00 },
+ { 0x0219, 0x00 },
+ { 0x021A, 0x00 },
+ { 0x021B, 0x00 },
+ { 0x021C, 0x00 },
+ { 0x021D, 0x00 },
+ { 0x021E, 0x00 },
+ { 0x021F, 0x00 },
+ { 0x0220, 0x00 },
+ { 0x0221, 0x00 },
+ { 0x0222, 0x00 },
+ { 0x0223, 0x00 },
+ { 0x0224, 0x00 },
+ { 0x0225, 0x00 },
+ { 0x0226, 0x00 },
+ { 0x0227, 0x00 },
+ { 0x0228, 0x00 },
+ { 0x0229, 0x00 },
+ { 0x022A, 0x00 },
+ { 0x022B, 0x00 },
+ { 0x022C, 0x00 },
+ { 0x022D, 0x00 },
+ { 0x022E, 0x00 },
+ { 0x022F, 0x00 }, /* Px divider setting (usually 0) end */
+ { 0x026B, 0x00 }, /* DESIGN_ID (ASCII string) */
+ { 0x026C, 0x00 },
+ { 0x026D, 0x00 },
+ { 0x026E, 0x00 },
+ { 0x026F, 0x00 },
+ { 0x0270, 0x00 },
+ { 0x0271, 0x00 },
+ { 0x0272, 0x00 }, /* DESIGN_ID (ASCII string) end */
+ { 0x0339, 0x1F }, /* N_FSTEP_MSK */
+ { 0x033B, 0x00 }, /* Nx_FSTEPW (Frequency step) */
+ { 0x033C, 0x00 },
+ { 0x033D, 0x00 },
+ { 0x033E, 0x00 },
+ { 0x033F, 0x00 },
+ { 0x0340, 0x00 },
+ { 0x0341, 0x00 },
+ { 0x0342, 0x00 },
+ { 0x0343, 0x00 },
+ { 0x0344, 0x00 },
+ { 0x0345, 0x00 },
+ { 0x0346, 0x00 },
+ { 0x0347, 0x00 },
+ { 0x0348, 0x00 },
+ { 0x0349, 0x00 },
+ { 0x034A, 0x00 },
+ { 0x034B, 0x00 },
+ { 0x034C, 0x00 },
+ { 0x034D, 0x00 },
+ { 0x034E, 0x00 },
+ { 0x034F, 0x00 },
+ { 0x0350, 0x00 },
+ { 0x0351, 0x00 },
+ { 0x0352, 0x00 },
+ { 0x0353, 0x00 },
+ { 0x0354, 0x00 },
+ { 0x0355, 0x00 },
+ { 0x0356, 0x00 },
+ { 0x0357, 0x00 },
+ { 0x0358, 0x00 }, /* Nx_FSTEPW (Frequency step) end */
+ { 0x0359, 0x00 }, /* Nx_DELAY */
+ { 0x035A, 0x00 },
+ { 0x035B, 0x00 },
+ { 0x035C, 0x00 },
+ { 0x035D, 0x00 },
+ { 0x035E, 0x00 },
+ { 0x035F, 0x00 },
+ { 0x0360, 0x00 },
+ { 0x0361, 0x00 },
+ { 0x0362, 0x00 }, /* Nx_DELAY end */
+ { 0x0802, 0x00 }, /* Not in datasheet */
+ { 0x0803, 0x00 }, /* Not in datasheet */
+ { 0x0804, 0x00 }, /* Not in datasheet */
+ { 0x090E, 0x02 }, /* XAXB_EXTCLK_EN=0 XAXB_PDNB=1 (use XTAL) */
+ { 0x091C, 0x04 }, /* ZDM_EN=4 (Normal mode) */
+ { 0x0943, 0x00 }, /* IO_VDD_SEL=0 (0=1v8, use 1=3v3) */
+ { 0x0949, 0x00 }, /* IN_EN (disable input clocks) */
+ { 0x094A, 0x00 }, /* INx_TO_PFD_EN (disabled) */
+ { 0x0A02, 0x00 }, /* Not in datasheet */
+ { 0x0B44, 0x0F }, /* PDIV_ENB (datasheet does not mention what it is) */
+};
+
+/* Read and interpret a 44-bit followed by a 32-bit value in the regmap */
+static int si5341_decode_44_32(struct regmap *regmap, unsigned int reg,
+ u64 *val1, u32 *val2)
+{
+ int err;
+ u8 r[10];
+
+ err = regmap_bulk_read(regmap, reg, r, 10);
+ if (err < 0)
+ return err;
+
+ *val1 = ((u64)((r[5] & 0x0f) << 8 | r[4]) << 32) |
+ (get_unaligned_le32(r));
+ *val2 = get_unaligned_le32(&r[6]);
+
+ return 0;
+}
+
+static int si5341_encode_44_32(struct regmap *regmap, unsigned int reg,
+ u64 n_num, u32 n_den)
+{
+ u8 r[10];
+
+ /* Shift left as far as possible without overflowing */
+ while (!(n_num & BIT_ULL(43)) && !(n_den & BIT(31))) {
+ n_num <<= 1;
+ n_den <<= 1;
+ }
+
+ /* 44 bits (6 bytes) numerator */
+ put_unaligned_le32(n_num, r);
+ r[4] = (n_num >> 32) & 0xff;
+ r[5] = (n_num >> 40) & 0x0f;
+ /* 32 bits denominator */
+ put_unaligned_le32(n_den, &r[6]);
+
+ /* Program the fraction */
+ return regmap_bulk_write(regmap, reg, r, sizeof(r));
+}
+
+/* VCO, we assume it runs at a constant frequency */
+static unsigned long si5341_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_si5341 *data = to_clk_si5341(hw);
+ int err;
+ u64 res;
+ u64 m_num;
+ u32 m_den;
+ unsigned int shift;
+
+ /* Assume that PDIV is not being used, just read the PLL setting */
+ err = si5341_decode_44_32(data->regmap, SI5341_PLL_M_NUM,
+ &m_num, &m_den);
+ if (err < 0)
+ return 0;
+
+ if (!m_num || !m_den)
+ return 0;
+
+ /*
+ * Though m_num is 64-bit, only the upper bits are actually used. While
+ * calculating m_num and m_den, they are shifted as far as possible to
+ * the left. To avoid 96-bit division here, we just shift them back so
+ * we can do with just 64 bits.
+ */
+ shift = 0;
+ res = m_num;
+ while (res & 0xffff00000000ULL) {
+ ++shift;
+ res >>= 1;
+ }
+ res *= parent_rate;
+ do_div(res, (m_den >> shift));
+
+ /* We cannot return the actual frequency in 32 bit, store it locally */
+ data->freq_vco = res;
+
+ /* Report kHz since the value is out of range */
+ do_div(res, 1000);
+
+ return (unsigned long)res;
+}
+
+static const struct clk_ops si5341_clk_ops = {
+ .recalc_rate = si5341_clk_recalc_rate,
+};
+
+/* Synthesizers, there are 5 synthesizers that connect to any of the outputs */
+
+/* The synthesizer is on if all power and enable bits are set */
+static int si5341_synth_clk_is_on(struct clk_hw *hw)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ int err;
+ u32 val;
+ u8 index = synth->index;
+
+ err = regmap_read(synth->data->regmap,
+ SI5341_SYNTH_N_CLK_TO_OUTX_EN, &val);
+ if (err < 0)
+ return 0;
+
+ if (!(val & BIT(index)))
+ return 0;
+
+ err = regmap_read(synth->data->regmap, SI5341_SYNTH_N_PDNB, &val);
+ if (err < 0)
+ return 0;
+
+ if (!(val & BIT(index)))
+ return 0;
+
+ /* This bit must be 0 for the synthesizer to receive clock input */
+ err = regmap_read(synth->data->regmap, SI5341_SYNTH_N_CLK_DIS, &val);
+ if (err < 0)
+ return 0;
+
+ return !(val & BIT(index));
+}
+
+static void si5341_synth_clk_unprepare(struct clk_hw *hw)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ u8 index = synth->index; /* In range 0..5 */
+ u8 mask = BIT(index);
+
+ /* Disable output */
+ regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_CLK_TO_OUTX_EN, mask, 0);
+ /* Power down */
+ regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_PDNB, mask, 0);
+ /* Disable clock input to synth (set to 1 to disable) */
+ regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_CLK_DIS, mask, mask);
+}
+
+static int si5341_synth_clk_prepare(struct clk_hw *hw)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ int err;
+ u8 index = synth->index;
+ u8 mask = BIT(index);
+
+ /* Power up */
+ err = regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_PDNB, mask, mask);
+ if (err < 0)
+ return err;
+
+ /* Enable clock input to synth (set bit to 0 to enable) */
+ err = regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_CLK_DIS, mask, 0);
+ if (err < 0)
+ return err;
+
+ /* Enable output */
+ return regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_CLK_TO_OUTX_EN, mask, mask);
+}
+
+/* Synth clock frequency: Fvco * n_den / n_den, with Fvco in 13500-14256 MHz */
+static unsigned long si5341_synth_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ u64 f;
+ u64 n_num;
+ u32 n_den;
+ int err;
+
+ err = si5341_decode_44_32(synth->data->regmap,
+ SI5341_SYNTH_N_NUM(synth->index), &n_num, &n_den);
+ if (err < 0)
+ return err;
+
+ /*
+ * n_num and n_den are shifted left as much as possible, so to prevent
+ * overflow in 64-bit math, we shift n_den 4 bits to the right
+ */
+ f = synth->data->freq_vco;
+ f *= n_den >> 4;
+
+ /* Now we need to to 64-bit division: f/n_num */
+ /* And compensate for the 4 bits we dropped */
+ f = div64_u64(f, (n_num >> 4));
+
+ return f;
+}
+
+static long si5341_synth_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ u64 f;
+
+ /* The synthesizer accuracy is such that anything in range will work */
+ f = synth->data->freq_vco;
+ do_div(f, SI5341_SYNTH_N_MAX);
+ if (rate < f)
+ return f;
+
+ f = synth->data->freq_vco;
+ do_div(f, SI5341_SYNTH_N_MIN);
+ if (rate > f)
+ return f;
+
+ return rate;
+}
+
+static int si5341_synth_program(struct clk_si5341_synth *synth,
+ u64 n_num, u32 n_den, bool is_integer)
+{
+ int err;
+ u8 index = synth->index;
+
+ err = si5341_encode_44_32(synth->data->regmap,
+ SI5341_SYNTH_N_NUM(index), n_num, n_den);
+
+ err = regmap_update_bits(synth->data->regmap,
+ SI5341_SYNTH_N_PIBYP, BIT(index), is_integer ? BIT(index) : 0);
+ if (err < 0)
+ return err;
+
+ return regmap_write(synth->data->regmap,
+ SI5341_SYNTH_N_UPD(index), 0x01);
+}
+
+
+static int si5341_synth_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_si5341_synth *synth = to_clk_si5341_synth(hw);
+ u64 n_num;
+ u32 n_den;
+ u32 r;
+ u32 g;
+ bool is_integer;
+
+ n_num = synth->data->freq_vco;
+ n_den = rate;
+
+ /* see if there's an integer solution */
+ r = do_div(n_num, rate);
+ is_integer = (r == 0);
+ if (is_integer) {
+ /* Integer divider equal to n_num */
+ n_den = 1;
+ } else {
+ /* Calculate a fractional solution */
+ g = gcd(r, rate);
+ n_den = rate / g;
+ n_num *= n_den;
+ n_num += r / g;
+ }
+
+ dev_dbg(&synth->data->i2c_client->dev,
+ "%s(%u): n=0x%llx d=0x%x %s\n", __func__,
+ synth->index, n_num, n_den,
+ is_integer ? "int" : "frac");
+
+ return si5341_synth_program(synth, n_num, n_den, is_integer);
+}
+
+static const struct clk_ops si5341_synth_clk_ops = {
+ .is_prepared = si5341_synth_clk_is_on,
+ .prepare = si5341_synth_clk_prepare,
+ .unprepare = si5341_synth_clk_unprepare,
+ .recalc_rate = si5341_synth_clk_recalc_rate,
+ .round_rate = si5341_synth_clk_round_rate,
+ .set_rate = si5341_synth_clk_set_rate,
+};
+
+static int si5341_output_clk_is_on(struct clk_hw *hw)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+ int err;
+ u32 val;
+
+ err = regmap_read(output->data->regmap,
+ SI5341_OUT_CONFIG(output), &val);
+ if (err < 0)
+ return err;
+
+ /* Bit 0=PDN, 1=OE so only a value of 0x2 enables the output */
+ return (val & 0x03) == SI5341_OUT_CFG_OE;
+}
+
+/* Disables and then powers down the output */
+static void si5341_output_clk_unprepare(struct clk_hw *hw)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+
+ regmap_update_bits(output->data->regmap,
+ SI5341_OUT_CONFIG(output),
+ SI5341_OUT_CFG_OE, 0);
+ regmap_update_bits(output->data->regmap,
+ SI5341_OUT_CONFIG(output),
+ SI5341_OUT_CFG_PDN, SI5341_OUT_CFG_PDN);
+}
+
+/* Powers up and then enables the output */
+static int si5341_output_clk_prepare(struct clk_hw *hw)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+ int err;
+
+ err = regmap_update_bits(output->data->regmap,
+ SI5341_OUT_CONFIG(output),
+ SI5341_OUT_CFG_PDN, 0);
+ if (err < 0)
+ return err;
+
+ return regmap_update_bits(output->data->regmap,
+ SI5341_OUT_CONFIG(output),
+ SI5341_OUT_CFG_OE, SI5341_OUT_CFG_OE);
+}
+
+static unsigned long si5341_output_clk_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+ int err;
+ u32 val;
+ u32 r_divider;
+ u8 r[3];
+
+ err = regmap_bulk_read(output->data->regmap,
+ SI5341_OUT_R_REG(output), r, 3);
+ if (err < 0)
+ return err;
+
+ /* Calculate value as 24-bit integer*/
+ r_divider = r[2] << 16 | r[1] << 8 | r[0];
+
+ /* If Rx_REG is zero, the divider is disabled, so return a "0" rate */
+ if (!r_divider)
+ return 0;
+
+ /* Divider is 2*(Rx_REG+1) */
+ r_divider += 1;
+ r_divider <<= 1;
+
+ err = regmap_read(output->data->regmap,
+ SI5341_OUT_CONFIG(output), &val);
+ if (err < 0)
+ return err;
+
+ if (val & SI5341_OUT_CFG_RDIV_FORCE2)
+ r_divider = 2;
+
+ return parent_rate / r_divider;
+}
+
+static long si5341_output_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long *parent_rate)
+{
+ unsigned long r;
+
+ r = *parent_rate >> 1;
+
+ /* If rate is an even divisor, no changes to parent required */
+ if (r && !(r % rate))
+ return (long)rate;
+
+ if (clk_hw_get_flags(hw) & CLK_SET_RATE_PARENT) {
+ if (rate > 200000000) {
+ /* minimum r-divider is 2 */
+ r = 2;
+ } else {
+ /* Take a parent frequency near 400 MHz */
+ r = (400000000u / rate) & ~1;
+ }
+ *parent_rate = r * rate;
+ } else {
+ /* We cannot change our parent's rate, report what we can do */
+ r /= rate;
+ rate = *parent_rate / (r << 1);
+ }
+
+ return rate;
+}
+
+static int si5341_output_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+ unsigned long parent_rate)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+ /* Frequency divider is (r_div + 1) * 2 */
+ u32 r_div = (parent_rate / rate) >> 1;
+ int err;
+ u8 r[3];
+
+ if (r_div <= 1)
+ r_div = 0;
+ else if (r_div >= BIT(24))
+ r_div = BIT(24) - 1;
+ else
+ --r_div;
+
+ /* For a value of "2", we set the "OUT0_RDIV_FORCE2" bit */
+ err = regmap_update_bits(output->data->regmap,
+ SI5341_OUT_CONFIG(output),
+ SI5341_OUT_CFG_RDIV_FORCE2,
+ (r_div == 0) ? SI5341_OUT_CFG_RDIV_FORCE2 : 0);
+ if (err < 0)
+ return err;
+
+ /* Always write Rx_REG, because a zero value disables the divider */
+ r[0] = r_div ? (r_div & 0xff) : 1;
+ r[1] = (r_div >> 8) & 0xff;
+ r[2] = (r_div >> 16) & 0xff;
+ err = regmap_bulk_write(output->data->regmap,
+ SI5341_OUT_R_REG(output), r, 3);
+
+ return 0;
+}
+
+static int si5341_output_reparent(struct clk_si5341_output *output, u8 index)
+{
+ return regmap_update_bits(output->data->regmap,
+ SI5341_OUT_MUX_SEL(output), 0x07, index);
+}
+
+static int si5341_output_set_parent(struct clk_hw *hw, u8 index)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+
+ if (index >= output->data->num_synth)
+ return -EINVAL;
+
+ return si5341_output_reparent(output, index);
+}
+
+static u8 si5341_output_get_parent(struct clk_hw *hw)
+{
+ struct clk_si5341_output *output = to_clk_si5341_output(hw);
+ int err;
+ u32 val;
+
+ err = regmap_read(output->data->regmap,
+ SI5341_OUT_MUX_SEL(output), &val);
+
+ return val & 0x7;
+}
+
+static const struct clk_ops si5341_output_clk_ops = {
+ .is_prepared = si5341_output_clk_is_on,
+ .prepare = si5341_output_clk_prepare,
+ .unprepare = si5341_output_clk_unprepare,
+ .recalc_rate = si5341_output_clk_recalc_rate,
+ .round_rate = si5341_output_clk_round_rate,
+ .set_rate = si5341_output_clk_set_rate,
+ .set_parent = si5341_output_set_parent,
+ .get_parent = si5341_output_get_parent,
+};
+
+/*
+ * The chip can be bought in a pre-programmed version, or one can program the
+ * NVM in the chip to boot up in a preset mode. This routine tries to determine
+ * if that's the case, or if we need to reset and program everything from
+ * scratch. Returns negative error, or true/false.
+ */
+static int si5341_is_programmed_already(struct clk_si5341 *data)
+{
+ int err;
+ u8 r[4];
+
+ /* Read the PLL divider value, it must have a non-zero value */
+ err = regmap_bulk_read(data->regmap, SI5341_PLL_M_DEN,
+ r, ARRAY_SIZE(r));
+ if (err < 0)
+ return err;
+
+ return !!get_unaligned_le32(r);
+}
+
+static struct clk_hw *
+of_clk_si5341_get(struct of_phandle_args *clkspec, void *_data)
+{
+ struct clk_si5341 *data = _data;
+ unsigned int idx = clkspec->args[1];
+ unsigned int group = clkspec->args[0];
+
+ switch (group) {
+ case 0:
+ if (idx >= data->num_outputs) {
+ dev_err(&data->i2c_client->dev,
+ "invalid output index %u\n", idx);
+ return ERR_PTR(-EINVAL);
+ }
+ return &data->clk[idx].hw;
+ case 1:
+ if (idx >= data->num_synth) {
+ dev_err(&data->i2c_client->dev,
+ "invalid synthesizer index %u\n", idx);
+ return ERR_PTR(-EINVAL);
+ }
+ return &data->synth[idx].hw;
+ case 2:
+ if (idx > 0) {
+ dev_err(&data->i2c_client->dev,
+ "invalid PLL index %u\n", idx);
+ return ERR_PTR(-EINVAL);
+ }
+ return &data->hw;
+ default:
+ dev_err(&data->i2c_client->dev, "invalid group %u\n", group);
+ return ERR_PTR(-EINVAL);
+ }
+}
+
+static int si5341_probe_chip_id(struct clk_si5341 *data)
+{
+ int err;
+ u8 reg[4];
+ u16 model;
+
+ err = regmap_bulk_read(data->regmap, SI5341_PN_BASE, reg,
+ ARRAY_SIZE(reg));
+ if (err < 0) {
+ dev_err(&data->i2c_client->dev, "Failed to read chip ID\n");
+ return err;
+ }
+
+ model = get_unaligned_le16(reg);
+
+ dev_info(&data->i2c_client->dev, "Chip: %x Grade: %u Rev: %u\n",
+ model, reg[2], reg[3]);
+
+ switch (model) {
+ case 0x5340:
+ data->num_outputs = SI5340_MAX_NUM_OUTPUTS;
+ data->num_synth = SI5340_NUM_SYNTH;
+ data->reg_output_offset = si5340_reg_output_offset;
+ data->reg_rdiv_offset = si5340_reg_rdiv_offset;
+ break;
+ case 0x5341:
+ data->num_outputs = SI5341_MAX_NUM_OUTPUTS;
+ data->num_synth = SI5341_NUM_SYNTH;
+ data->reg_output_offset = si5341_reg_output_offset;
+ data->reg_rdiv_offset = si5341_reg_rdiv_offset;
+ break;
+ default:
+ dev_err(&data->i2c_client->dev, "Model '%x' not supported\n",
+ model);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Read active settings into the regmap cache for later reference */
+static int si5341_read_settings(struct clk_si5341 *data)
+{
+ int err;
+ u8 i;
+ u8 r[10];
+
+ err = regmap_bulk_read(data->regmap, SI5341_PLL_M_NUM, r, 10);
+ if (err < 0)
+ return err;
+
+ err = regmap_bulk_read(data->regmap,
+ SI5341_SYNTH_N_CLK_TO_OUTX_EN, r, 3);
+ if (err < 0)
+ return err;
+
+ err = regmap_bulk_read(data->regmap,
+ SI5341_SYNTH_N_CLK_DIS, r, 1);
+ if (err < 0)
+ return err;
+
+ for (i = 0; i < data->num_synth; ++i) {
+ err = regmap_bulk_read(data->regmap,
+ SI5341_SYNTH_N_NUM(i), r, 10);
+ if (err < 0)
+ return err;
+ }
+
+ for (i = 0; i < data->num_outputs; ++i) {
+ err = regmap_bulk_read(data->regmap,
+ data->reg_output_offset[i], r, 4);
+ if (err < 0)
+ return err;
+
+ err = regmap_bulk_read(data->regmap,
+ data->reg_rdiv_offset[i], r, 3);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
+static int si5341_write_multiple(struct clk_si5341 *data,
+ const struct si5341_reg_default *values, unsigned int num_values)
+{
+ unsigned int i;
+ int res;
+
+ for (i = 0; i < num_values; ++i) {
+ res = regmap_write(data->regmap,
+ values[i].address, values[i].value);
+ if (res < 0) {
+ dev_err(&data->i2c_client->dev,
+ "Failed to write %#x:%#x\n",
+ values[i].address, values[i].value);
+ return res;
+ }
+ }
+
+ return 0;
+}
+
+static const struct si5341_reg_default si5341_preamble[] = {
+ { 0x0B25, 0x00 },
+ { 0x0502, 0x01 },
+ { 0x0505, 0x03 },
+ { 0x0957, 0x1F },
+ { 0x0B4E, 0x1A },
+};
+
+static int si5341_send_preamble(struct clk_si5341 *data)
+{
+ int res;
+ u32 revision;
+
+ /* For revision 2 and up, the values are slightly different */
+ res = regmap_read(data->regmap, SI5341_DEVICE_REV, &revision);
+ if (res < 0)
+ return res;
+
+ /* Write "preamble" as specified by datasheet */
+ res = regmap_write(data->regmap, 0xB24, revision < 2 ? 0xD8 : 0xC0);
+ if (res < 0)
+ return res;
+ res = si5341_write_multiple(data,
+ si5341_preamble, ARRAY_SIZE(si5341_preamble));
+ if (res < 0)
+ return res;
+
+ /* Datasheet specifies a 300ms wait after sending the preamble */
+ msleep(300);
+
+ return 0;
+}
+
+/* Perform a soft reset and write post-amble */
+static int si5341_finalize_defaults(struct clk_si5341 *data)
+{
+ int res;
+ u32 revision;
+
+ res = regmap_read(data->regmap, SI5341_DEVICE_REV, &revision);
+ if (res < 0)
+ return res;
+
+ dev_dbg(&data->i2c_client->dev, "%s rev=%u\n", __func__, revision);
+
+ res = regmap_write(data->regmap, SI5341_SOFT_RST, 0x01);
+ if (res < 0)
+ return res;
+
+ /* Datasheet does not explain these nameless registers */
+ res = regmap_write(data->regmap, 0xB24, revision < 2 ? 0xDB : 0xC3);
+ if (res < 0)
+ return res;
+ res = regmap_write(data->regmap, 0x0B25, 0x02);
+ if (res < 0)
+ return res;
+
+ return 0;
+}
+
+
+static const struct regmap_range si5341_regmap_volatile_range[] = {
+ regmap_reg_range(0x000C, 0x0012), /* Status */
+ regmap_reg_range(0x001C, 0x001E), /* reset, finc/fdec */
+ regmap_reg_range(0x00E2, 0x00FE), /* NVM, interrupts, device ready */
+ /* Update bits for synth config */
+ regmap_reg_range(SI5341_SYNTH_N_UPD(0), SI5341_SYNTH_N_UPD(0)),
+ regmap_reg_range(SI5341_SYNTH_N_UPD(1), SI5341_SYNTH_N_UPD(1)),
+ regmap_reg_range(SI5341_SYNTH_N_UPD(2), SI5341_SYNTH_N_UPD(2)),
+ regmap_reg_range(SI5341_SYNTH_N_UPD(3), SI5341_SYNTH_N_UPD(3)),
+ regmap_reg_range(SI5341_SYNTH_N_UPD(4), SI5341_SYNTH_N_UPD(4)),
+};
+
+static const struct regmap_access_table si5341_regmap_volatile = {
+ .yes_ranges = si5341_regmap_volatile_range,
+ .n_yes_ranges = ARRAY_SIZE(si5341_regmap_volatile_range),
+};
+
+/* Pages 0, 1, 2, 3, 9, A, B are valid, so there are 12 pages */
+static const struct regmap_range_cfg si5341_regmap_ranges[] = {
+ {
+ .range_min = 0,
+ .range_max = SI5341_REGISTER_MAX,
+ .selector_reg = SI5341_PAGE,
+ .selector_mask = 0xff,
+ .selector_shift = 0,
+ .window_start = 0,
+ .window_len = 256,
+ },
+};
+
+static const struct regmap_config si5341_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .cache_type = REGCACHE_RBTREE,
+ .ranges = si5341_regmap_ranges,
+ .num_ranges = ARRAY_SIZE(si5341_regmap_ranges),
+ .max_register = SI5341_REGISTER_MAX,
+ .volatile_table = &si5341_regmap_volatile,
+};
+
+static int si5341_dt_parse_dt(struct i2c_client *client,
+ struct clk_si5341_output_config *config)
+{
+ struct device_node *child;
+ struct device_node *np = client->dev.of_node;
+ u32 num;
+ u32 val;
+
+ memset(config, 0, sizeof(struct clk_si5341_output_config) *
+ SI5341_MAX_NUM_OUTPUTS);
+
+ for_each_child_of_node(np, child) {
+ if (of_property_read_u32(child, "reg", &num)) {
+ dev_err(&client->dev, "missing reg property of %s\n",
+ child->name);
+ goto put_child;
+ }
+
+ if (num >= SI5341_MAX_NUM_OUTPUTS) {
+ dev_err(&client->dev, "invalid clkout %d\n", num);
+ goto put_child;
+ }
+
+ if (!of_property_read_u32(child, "silabs,format", &val)) {
+ /* Set cm and ampl conservatively to 3v3 settings */
+ switch (val) {
+ case 1: /* normal differential */
+ config[num].out_cm_ampl_bits = 0x33;
+ break;
+ case 2: /* low-power differential */
+ config[num].out_cm_ampl_bits = 0x13;
+ break;
+ case 4: /* LVCMOS */
+ config[num].out_cm_ampl_bits = 0x33;
+ /* Set SI recommended impedance for LVCMOS */
+ config[num].out_format_drv_bits |= 0xc0;
+ break;
+ default:
+ dev_err(&client->dev,
+ "invalid silabs,format %u for %u\n",
+ val, num);
+ goto put_child;
+ }
+ config[num].out_format_drv_bits &= ~0x07;
+ config[num].out_format_drv_bits |= val & 0x07;
+ /* Always enable the SYNC feature */
+ config[num].out_format_drv_bits |= 0x08;
+ }
+
+ if (!of_property_read_u32(child, "silabs,common-mode", &val)) {
+ if (val > 0xf) {
+ dev_err(&client->dev,
+ "invalid silabs,common-mode %u\n",
+ val);
+ goto put_child;
+ }
+ config[num].out_cm_ampl_bits &= 0xf0;
+ config[num].out_cm_ampl_bits |= val & 0x0f;
+ }
+
+ if (!of_property_read_u32(child, "silabs,amplitude", &val)) {
+ if (val > 0xf) {
+ dev_err(&client->dev,
+ "invalid silabs,amplitude %u\n",
+ val);
+ goto put_child;
+ }
+ config[num].out_cm_ampl_bits &= 0x0f;
+ config[num].out_cm_ampl_bits |= (val << 4) & 0xf0;
+ }
+
+ if (of_property_read_bool(child, "silabs,disable-high"))
+ config[num].out_format_drv_bits |= 0x10;
+
+ config[num].synth_master =
+ of_property_read_bool(child, "silabs,synth-master");
+
+ config[num].always_on =
+ of_property_read_bool(child, "always-on");
+ }
+
+ return 0;
+
+put_child:
+ of_node_put(child);
+ return -EINVAL;
+}
+
+/*
+ * If not pre-configured, calculate and set the PLL configuration manually.
+ * For low-jitter performance, the PLL should be set such that the synthesizers
+ * only need integer division.
+ * Without any user guidance, we'll set the PLL to 14GHz, which still allows
+ * the chip to generate any frequency on its outputs, but jitter performance
+ * may be sub-optimal.
+ */
+static int si5341_initialize_pll(struct clk_si5341 *data)
+{
+ struct device_node *np = data->i2c_client->dev.of_node;
+ u32 m_num = 0;
+ u32 m_den = 0;
+
+ if (of_property_read_u32(np, "silabs,pll-m-num", &m_num)) {
+ dev_err(&data->i2c_client->dev,
+ "PLL configuration requires silabs,pll-m-num\n");
+ }
+ if (of_property_read_u32(np, "silabs,pll-m-den", &m_den)) {
+ dev_err(&data->i2c_client->dev,
+ "PLL configuration requires silabs,pll-m-den\n");
+ }
+
+ if (!m_num || !m_den) {
+ dev_err(&data->i2c_client->dev,
+ "PLL configuration invalid, assume 14GHz\n");
+ m_den = clk_get_rate(data->pxtal) / 10;
+ m_num = 1400000000;
+ }
+
+ return si5341_encode_44_32(data->regmap,
+ SI5341_PLL_M_NUM, m_num, m_den);
+}
+
+static int si5341_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct clk_si5341 *data;
+ struct clk_init_data init;
+ const char *root_clock_name;
+ const char *synth_clock_names[SI5341_NUM_SYNTH];
+ int err;
+ unsigned int i;
+ struct clk_si5341_output_config config[SI5341_MAX_NUM_OUTPUTS];
+ bool initialization_required;
+
+ data = devm_kzalloc(&client->dev, sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ data->i2c_client = client;
+
+ data->pxtal = devm_clk_get(&client->dev, "xtal");
+ if (IS_ERR(data->pxtal)) {
+ if (PTR_ERR(data->pxtal) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+
+ dev_err(&client->dev, "Missing xtal clock input\n");
+ }
+
+ err = si5341_dt_parse_dt(client, config);
+ if (err)
+ return err;
+
+ if (of_property_read_string(client->dev.of_node, "clock-output-names",
+ &init.name))
+ init.name = client->dev.of_node->name;
+ root_clock_name = init.name;
+
+ data->regmap = devm_regmap_init_i2c(client, &si5341_regmap_config);
+ if (IS_ERR(data->regmap))
+ return PTR_ERR(data->regmap);
+
+ i2c_set_clientdata(client, data);
+
+ err = si5341_probe_chip_id(data);
+ if (err < 0)
+ return err;
+
+ /* "Activate" the xtal (usually a fixed clock) */
+ clk_prepare_enable(data->pxtal);
+
+ if (of_property_read_bool(client->dev.of_node, "silabs,reprogram")) {
+ initialization_required = true;
+ } else {
+ err = si5341_is_programmed_already(data);
+ if (err < 0)
+ return err;
+
+ initialization_required = !err;
+ }
+
+ if (initialization_required) {
+ /* Populate the regmap cache in preparation for "cache only" */
+ err = si5341_read_settings(data);
+ if (err < 0)
+ return err;
+
+ err = si5341_send_preamble(data);
+ if (err < 0)
+ return err;
+
+ /*
+ * We intend to send all 'final' register values in a single
+ * transaction. So cache all register writes until we're done
+ * configuring.
+ */
+ regcache_cache_only(data->regmap, true);
+
+ /* Write the configuration pairs from the firmware blob */
+ err = si5341_write_multiple(data, si5341_reg_defaults,
+ ARRAY_SIZE(si5341_reg_defaults));
+ if (err < 0)
+ return err;
+
+ /* PLL configuration is required */
+ err = si5341_initialize_pll(data);
+ if (err < 0)
+ return err;
+ }
+
+ /* Register the PLL */
+ data->pxtal_name = __clk_get_name(data->pxtal);
+ init.parent_names = &data->pxtal_name;
+ init.num_parents = 1; /* For now, only XTAL input supported */
+ init.ops = &si5341_clk_ops;
+ init.flags = 0;
+ data->hw.init = &init;
+
+ err = devm_clk_hw_register(&client->dev, &data->hw);
+ if (err) {
+ dev_err(&client->dev, "clock registration failed\n");
+ return err;
+ }
+
+ init.num_parents = 1;
+ init.parent_names = &root_clock_name;
+ init.ops = &si5341_synth_clk_ops;
+ for (i = 0; i < data->num_synth; ++i) {
+ synth_clock_names[i] = devm_kasprintf(&client->dev, GFP_KERNEL,
+ "%s.N%u", client->dev.of_node->name, i);
+ init.name = synth_clock_names[i];
+ data->synth[i].index = i;
+ data->synth[i].data = data;
+ data->synth[i].hw.init = &init;
+ err = devm_clk_hw_register(&client->dev, &data->synth[i].hw);
+ if (err) {
+ dev_err(&client->dev,
+ "synth N%u registration failed\n", i);
+ }
+ }
+
+ init.num_parents = data->num_synth;
+ init.parent_names = synth_clock_names;
+ init.ops = &si5341_output_clk_ops;
+ for (i = 0; i < data->num_outputs; ++i) {
+ init.name = kasprintf(GFP_KERNEL, "%s.%d",
+ client->dev.of_node->name, i);
+ init.flags = config[i].synth_master ? CLK_SET_RATE_PARENT : 0;
+ data->clk[i].index = i;
+ data->clk[i].data = data;
+ data->clk[i].hw.init = &init;
+ if (config[i].out_format_drv_bits & 0x07) {
+ regmap_write(data->regmap,
+ SI5341_OUT_FORMAT(&data->clk[i]),
+ config[i].out_format_drv_bits);
+ regmap_write(data->regmap,
+ SI5341_OUT_CM(&data->clk[i]),
+ config[i].out_cm_ampl_bits);
+ }
+ err = devm_clk_hw_register(&client->dev, &data->clk[i].hw);
+ kfree(init.name); /* clock framework made a copy of the name */
+ if (err) {
+ dev_err(&client->dev,
+ "output %u registration failed\n", i);
+ return err;
+ }
+ if (config[i].always_on)
+ clk_prepare(data->clk[i].hw.clk);
+ }
+
+ err = of_clk_add_hw_provider(client->dev.of_node, of_clk_si5341_get,
+ data);
+ if (err) {
+ dev_err(&client->dev, "unable to add clk provider\n");
+ return err;
+ }
+
+ if (initialization_required) {
+ /* Synchronize */
+ regcache_cache_only(data->regmap, false);
+ err = regcache_sync(data->regmap);
+ if (err < 0)
+ return err;
+
+ err = si5341_finalize_defaults(data);
+ if (err < 0)
+ return err;
+ }
+
+ /* Free the names, clk framework makes copies */
+ for (i = 0; i < data->num_synth; ++i)
+ devm_kfree(&client->dev, (void *)synth_clock_names[i]);
+
+ return 0;
+}
+
+static const struct i2c_device_id si5341_id[] = {
+ { "si5340", 0 },
+ { "si5341", 1 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, si5341_id);
+
+static const struct of_device_id clk_si5341_of_match[] = {
+ { .compatible = "silabs,si5340" },
+ { .compatible = "silabs,si5341" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, clk_si5341_of_match);
+
+static struct i2c_driver si5341_driver = {
+ .driver = {
+ .name = "si5341",
+ .of_match_table = clk_si5341_of_match,
+ },
+ .probe = si5341_probe,
+ .id_table = si5341_id,
+};
+module_i2c_driver(si5341_driver);
+
+MODULE_AUTHOR("Mike Looijmans <mike.looijmans@topic.nl>");
+MODULE_DESCRIPTION("Si5341 driver");
+MODULE_LICENSE("GPL");
#include <linux/clk-provider.h>
#include <linux/delay.h>
+#include <linux/math64.h>
#include <linux/module.h>
#include <linux/i2c.h>
#include <linux/regmap.h>
/* Lowest frequency synthesizeable using only the HS divider */
#define MIN_HSDIV_FREQ (FVCO_MIN / HS_DIV_MAX)
+/* Range and interpretation of the adjustment value */
+#define DELTA_M_MAX 8161512
+#define DELTA_M_FRAC_NUM 19
+#define DELTA_M_FRAC_DEN 20000
+
enum si544_speed_grade {
si544a,
si544b,
* @hs_div: 1st divider, 5..2046, must be even when >33
* @ls_div_bits: 2nd divider, as 2^x, range 0..5
* If ls_div_bits is non-zero, hs_div must be even
+ * @delta_m: Frequency shift for small -950..+950 ppm changes, 24 bit
*/
struct clk_si544_muldiv {
u32 fb_div_frac;
u16 fb_div_int;
u16 hs_div;
u8 ls_div_bits;
+ s32 delta_m;
};
/* Enables or disables the output driver */
settings->fb_div_int = reg[4] | (reg[5] & 0x07) << 8;
settings->fb_div_frac = reg[0] | reg[1] << 8 | reg[2] << 16 |
reg[3] << 24;
+
+ err = regmap_bulk_read(data->regmap, SI544_REG_ADPLL_DELTA_M0, reg, 3);
+ if (err)
+ return err;
+
+ /* Interpret as 24-bit signed number */
+ settings->delta_m = reg[0] << 8 | reg[1] << 16 | reg[2] << 24;
+ settings->delta_m >>= 8;
+
return 0;
}
+static int si544_set_delta_m(struct clk_si544 *data, s32 delta_m)
+{
+ u8 reg[3];
+
+ reg[0] = delta_m;
+ reg[1] = delta_m >> 8;
+ reg[2] = delta_m >> 16;
+
+ return regmap_bulk_write(data->regmap, SI544_REG_ADPLL_DELTA_M0,
+ reg, 3);
+}
+
static int si544_set_muldiv(struct clk_si544 *data,
struct clk_si544_muldiv *settings)
{
do_div(vco, FXO);
settings->fb_div_frac = vco;
+ /* Reset the frequency adjustment */
+ settings->delta_m = 0;
+
return 0;
}
/* Calculate resulting frequency given the register settings */
-static unsigned long si544_calc_rate(struct clk_si544_muldiv *settings)
+static unsigned long si544_calc_center_rate(
+ const struct clk_si544_muldiv *settings)
{
u32 d = settings->hs_div * BIT(settings->ls_div_bits);
u64 vco;
return vco;
}
+static unsigned long si544_calc_rate(const struct clk_si544_muldiv *settings)
+{
+ unsigned long rate = si544_calc_center_rate(settings);
+ s64 delta = (s64)rate * (DELTA_M_FRAC_NUM * settings->delta_m);
+
+ /*
+ * The clock adjustment is much smaller than 1 Hz, round to the
+ * nearest multiple. Apparently div64_s64 rounds towards zero, hence
+ * check the sign and adjust into the proper direction.
+ */
+ if (settings->delta_m < 0)
+ delta -= ((s64)DELTA_M_MAX * DELTA_M_FRAC_DEN) / 2;
+ else
+ delta += ((s64)DELTA_M_MAX * DELTA_M_FRAC_DEN) / 2;
+ delta = div64_s64(delta, ((s64)DELTA_M_MAX * DELTA_M_FRAC_DEN));
+
+ return rate + delta;
+}
+
static unsigned long si544_recalc_rate(struct clk_hw *hw,
unsigned long parent_rate)
{
unsigned long *parent_rate)
{
struct clk_si544 *data = to_clk_si544(hw);
- struct clk_si544_muldiv settings;
- int err;
if (!is_valid_frequency(data, rate))
return -EINVAL;
- err = si544_calc_muldiv(&settings, rate);
- if (err)
- return err;
+ /* The accuracy is less than 1 Hz, so any rate is possible */
+ return rate;
+}
- return si544_calc_rate(&settings);
+/* Calculates the maximum "small" change, 950 * rate / 1000000 */
+static unsigned long si544_max_delta(unsigned long rate)
+{
+ u64 num = rate;
+
+ num *= DELTA_M_FRAC_NUM;
+ do_div(num, DELTA_M_FRAC_DEN);
+
+ return num;
+}
+
+static s32 si544_calc_delta(s32 delta, s32 max_delta)
+{
+ s64 n = (s64)delta * DELTA_M_MAX;
+
+ return div_s64(n, max_delta);
}
-/*
- * Update output frequency for "big" frequency changes
- */
static int si544_set_rate(struct clk_hw *hw, unsigned long rate,
unsigned long parent_rate)
{
struct clk_si544 *data = to_clk_si544(hw);
struct clk_si544_muldiv settings;
+ unsigned long center;
+ long max_delta;
+ long delta;
unsigned int old_oe_state;
int err;
if (!is_valid_frequency(data, rate))
return -EINVAL;
+ /* Try using the frequency adjustment feature for a <= 950ppm change */
+ err = si544_get_muldiv(data, &settings);
+ if (err)
+ return err;
+
+ center = si544_calc_center_rate(&settings);
+ max_delta = si544_max_delta(center);
+ delta = rate - center;
+
+ if (abs(delta) <= max_delta)
+ return si544_set_delta_m(data,
+ si544_calc_delta(delta, max_delta));
+
+ /* Too big for the delta adjustment, need to reprogram */
err = si544_calc_muldiv(&settings, rate);
if (err)
return err;
if (err < 0)
return err;
+ err = si544_set_delta_m(data, settings.delta_m);
+ if (err < 0)
+ return err;
err = si544_set_muldiv(data, &settings);
if (err < 0)
static bool clk_core_can_round(struct clk_core * const core)
{
- if (core->ops->determine_rate || core->ops->round_rate)
- return true;
-
- return false;
+ return core->ops->determine_rate || core->ops->round_rate;
}
static int clk_core_round_rate_nolock(struct clk_core *core,
EXPORT_SYMBOL_GPL(clk_set_rate);
/**
- * clk_set_rate_exclusive - specify a new rate get exclusive control
+ * clk_set_rate_exclusive - specify a new rate and get exclusive control
* @clk: the clk whose rate is being changed
* @rate: the new rate for clk
*
* within a critical section
*
* This can be used initially to ensure that at least 1 consumer is
- * statisfied when several consumers are competing for exclusivity over the
+ * satisfied when several consumers are competing for exclusivity over the
* same clock provider.
*
* The exclusivity is not applied if setting the rate failed.
}
DEFINE_SHOW_ATTRIBUTE(clk_flags);
+static void possible_parent_show(struct seq_file *s, struct clk_core *core,
+ unsigned int i, char terminator)
+{
+ struct clk_core *parent;
+
+ /*
+ * Go through the following options to fetch a parent's name.
+ *
+ * 1. Fetch the registered parent clock and use its name
+ * 2. Use the global (fallback) name if specified
+ * 3. Use the local fw_name if provided
+ * 4. Fetch parent clock's clock-output-name if DT index was set
+ *
+ * This may still fail in some cases, such as when the parent is
+ * specified directly via a struct clk_hw pointer, but it isn't
+ * registered (yet).
+ */
+ parent = clk_core_get_parent_by_index(core, i);
+ if (parent)
+ seq_printf(s, "%s", parent->name);
+ else if (core->parents[i].name)
+ seq_printf(s, "%s", core->parents[i].name);
+ else if (core->parents[i].fw_name)
+ seq_printf(s, "<%s>(fw)", core->parents[i].fw_name);
+ else if (core->parents[i].index >= 0)
+ seq_printf(s, "%s",
+ of_clk_get_parent_name(core->of_node,
+ core->parents[i].index));
+ else
+ seq_puts(s, "(missing)");
+
+ seq_putc(s, terminator);
+}
+
static int possible_parents_show(struct seq_file *s, void *data)
{
struct clk_core *core = s->private;
int i;
for (i = 0; i < core->num_parents - 1; i++)
- seq_printf(s, "%s ", core->parents[i].name);
+ possible_parent_show(s, core, i, ' ');
- seq_printf(s, "%s\n", core->parents[i].name);
+ possible_parent_show(s, core, i, '\n');
return 0;
}
DEFINE_SHOW_ATTRIBUTE(possible_parents);
+static int current_parent_show(struct seq_file *s, void *data)
+{
+ struct clk_core *core = s->private;
+
+ if (core->parent)
+ seq_printf(s, "%s\n", core->parent->name);
+
+ return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(current_parent);
+
static int clk_duty_cycle_show(struct seq_file *s, void *data)
{
struct clk_core *core = s->private;
debugfs_create_file("clk_duty_cycle", 0444, root, core,
&clk_duty_cycle_fops);
+ if (core->num_parents > 0)
+ debugfs_create_file("clk_parent", 0444, root, core,
+ ¤t_parent_fops);
+
if (core->num_parents > 1)
debugfs_create_file("clk_possible_parents", 0444, root, core,
&possible_parents_fops);
void *data;
};
+extern struct of_device_id __clk_of_table;
static const struct of_device_id __clk_of_table_sentinel
__used __section(__clk_of_table_end);
{
return (struct clk *)hw;
}
-static struct clk_hw *__clk_get_hw(struct clk *clk)
-{
- return (struct clk_hw *)clk;
-}
static inline void __clk_put(struct clk *clk) { }
#endif
.set_rate = clk_busy_divider_set_rate,
};
-struct clk *imx_clk_busy_divider(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_busy_divider(const char *name, const char *parent_name,
void __iomem *reg, u8 shift, u8 width,
void __iomem *busy_reg, u8 busy_shift)
{
struct clk_busy_divider *busy;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
busy = kzalloc(sizeof(*busy), GFP_KERNEL);
if (!busy)
busy->div.hw.init = &init;
- clk = clk_register(NULL, &busy->div.hw);
- if (IS_ERR(clk))
+ hw = &busy->div.hw;
+
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(busy);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
struct clk_busy_mux {
.set_parent = clk_busy_mux_set_parent,
};
-struct clk *imx_clk_busy_mux(const char *name, void __iomem *reg, u8 shift,
+struct clk_hw *imx_clk_hw_busy_mux(const char *name, void __iomem *reg, u8 shift,
u8 width, void __iomem *busy_reg, u8 busy_shift,
const char * const *parent_names, int num_parents)
{
struct clk_busy_mux *busy;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
busy = kzalloc(sizeof(*busy), GFP_KERNEL);
if (!busy)
busy->mux.hw.init = &init;
- clk = clk_register(NULL, &busy->mux.hw);
- if (IS_ERR(clk))
+ hw = &busy->mux.hw;
+
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(busy);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
.set_rate = clk_cpu_set_rate,
};
-struct clk *imx_clk_cpu(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_cpu(const char *name, const char *parent_name,
struct clk *div, struct clk *mux, struct clk *pll,
struct clk *step)
{
struct clk_cpu *cpu;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
cpu = kzalloc(sizeof(*cpu), GFP_KERNEL);
if (!cpu)
init.num_parents = 1;
cpu->hw.init = &init;
+ hw = &cpu->hw;
- clk = clk_register(NULL, &cpu->hw);
- if (IS_ERR(clk))
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(cpu);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
.set_rate = clk_fixup_div_set_rate,
};
-struct clk *imx_clk_fixup_divider(const char *name, const char *parent,
+struct clk_hw *imx_clk_hw_fixup_divider(const char *name, const char *parent,
void __iomem *reg, u8 shift, u8 width,
void (*fixup)(u32 *val))
{
struct clk_fixup_div *fixup_div;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
if (!fixup)
return ERR_PTR(-EINVAL);
fixup_div->ops = &clk_divider_ops;
fixup_div->fixup = fixup;
- clk = clk_register(NULL, &fixup_div->divider.hw);
- if (IS_ERR(clk))
+ hw = &fixup_div->divider.hw;
+
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(fixup_div);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
.set_parent = clk_fixup_mux_set_parent,
};
-struct clk *imx_clk_fixup_mux(const char *name, void __iomem *reg,
+struct clk_hw *imx_clk_hw_fixup_mux(const char *name, void __iomem *reg,
u8 shift, u8 width, const char * const *parents,
int num_parents, void (*fixup)(u32 *val))
{
struct clk_fixup_mux *fixup_mux;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
if (!fixup)
return ERR_PTR(-EINVAL);
fixup_mux->ops = &clk_mux_ops;
fixup_mux->fixup = fixup;
- clk = clk_register(NULL, &fixup_mux->mux.hw);
- if (IS_ERR(clk))
+ hw = &fixup_mux->mux.hw;
+
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(fixup_mux);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
.is_enabled = clk_gate_exclusive_is_enabled,
};
-struct clk *imx_clk_gate_exclusive(const char *name, const char *parent,
+struct clk_hw *imx_clk_hw_gate_exclusive(const char *name, const char *parent,
void __iomem *reg, u8 shift, u32 exclusive_mask)
{
struct clk_gate_exclusive *exgate;
struct clk_gate *gate;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
if (exclusive_mask == 0)
return ERR_PTR(-EINVAL);
gate->hw.init = &init;
exgate->exclusive_mask = exclusive_mask;
- clk = clk_register(NULL, &gate->hw);
- if (IS_ERR(clk))
- kfree(exgate);
+ hw = &gate->hw;
- return clk;
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
+ kfree(gate);
+ return ERR_PTR(ret);
+ }
+
+ return hw;
}
.is_enabled = clk_gate2_is_enabled,
};
-struct clk *clk_register_gate2(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_gate2(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 bit_idx, u8 cgr_val,
u8 clk_gate2_flags, spinlock_t *lock,
unsigned int *share_count)
{
struct clk_gate2 *gate;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
gate = kzalloc(sizeof(struct clk_gate2), GFP_KERNEL);
if (!gate)
init.num_parents = parent_name ? 1 : 0;
gate->hw.init = &init;
+ hw = &gate->hw;
- clk = clk_register(dev, &gate->hw);
- if (IS_ERR(clk))
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(gate);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
#include <linux/types.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/of.h>
static const char *pll6_bypass_sels[] = { "pll6", "pll6_bypass_src", };
static const char *pll7_bypass_sels[] = { "pll7", "pll7_bypass_src", };
-static struct clk *clk[IMX6QDL_CLK_END];
-static struct clk_onecell_data clk_data;
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
static struct clk_div_table clk_enet_ref_table[] = {
{ .val = 0, .div = 20, },
return of_machine_is_compatible("fsl,imx6dl");
}
-static struct clk ** const uart_clks[] __initconst = {
- &clk[IMX6QDL_CLK_UART_IPG],
- &clk[IMX6QDL_CLK_UART_SERIAL],
- NULL
+static const int uart_clk_ids[] __initconst = {
+ IMX6QDL_CLK_UART_IPG,
+ IMX6QDL_CLK_UART_SERIAL,
};
+static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
+
static int ldb_di_sel_by_clock_id(int clock_id)
{
switch (clock_id) {
return false;
}
-#define CCM_CCDR 0x04
#define CCM_CCSR 0x0c
#define CCM_CS2CDR 0x2c
-#define CCDR_MMDC_CH1_MASK BIT(16)
#define CCSR_PLL3_SW_CLK_SEL BIT(0)
#define CS2CDR_LDB_DI0_CLK_SEL_SHIFT 9
#define CS2CDR_LDB_DI1_CLK_SEL_SHIFT 12
-static void __init imx6q_mmdc_ch1_mask_handshake(void __iomem *ccm_base)
-{
- unsigned int reg;
-
- reg = readl_relaxed(ccm_base + CCM_CCDR);
- reg |= CCDR_MMDC_CH1_MASK;
- writel_relaxed(reg, ccm_base + CCM_CCDR);
-}
-
/*
* The only way to disable the MMDC_CH1 clock is to move it to pll3_sw_clk
* via periph2_clk2_sel and then to disable pll3_sw_clk by selecting the
{
unsigned int reg;
- clk_set_parent(clk[IMX6QDL_CLK_PERIPH2_CLK2_SEL],
- clk[IMX6QDL_CLK_PLL3_USB_OTG]);
-
- /*
- * Handshake with mmdc_ch1 module must be masked when changing
- * periph2_clk_sel.
- */
- clk_set_parent(clk[IMX6QDL_CLK_PERIPH2], clk[IMX6QDL_CLK_PERIPH2_CLK2]);
+ clk_set_parent(hws[IMX6QDL_CLK_PERIPH2_CLK2_SEL]->clk,
+ hws[IMX6QDL_CLK_PLL3_USB_OTG]->clk);
/* Disable pll3_sw_clk by selecting the bypass clock source */
reg = readl_relaxed(ccm_base + CCM_CCSR);
reg = readl_relaxed(ccm_base + CCM_CCSR);
reg &= ~CCSR_PLL3_SW_CLK_SEL;
writel_relaxed(reg, ccm_base + CCM_CCSR);
-
- clk_set_parent(clk[IMX6QDL_CLK_PERIPH2], clk[IMX6QDL_CLK_PERIPH2_PRE]);
}
/*
/* Only switch to or from pll2_pfd2_396m if it is disabled */
if ((sel[i][0] == 2 || sel[i][3] == 2) &&
- (clk_get_parent(clk[IMX6QDL_CLK_PERIPH_PRE]) ==
- clk[IMX6QDL_CLK_PLL2_PFD2_396M])) {
+ (clk_get_parent(hws[IMX6QDL_CLK_PERIPH_PRE]->clk) ==
+ hws[IMX6QDL_CLK_PLL2_PFD2_396M]->clk)) {
pr_err("ccm: ldb_di%d_sel: couldn't disable pll2_pfd2_396m\n",
i);
sel[i][3] = sel[i][2] = sel[i][1] = sel[i][0];
/* Make sure PLL2 PFDs 0-2 are gated */
reg = readl_relaxed(anatop_base + CCM_ANALOG_PFD_528);
/* Cannot gate PFD2 if pll2_pfd2_396m is the parent of MMDC clock */
- if (clk_get_parent(clk[IMX6QDL_CLK_PERIPH_PRE]) ==
- clk[IMX6QDL_CLK_PLL2_PFD2_396M])
+ if (clk_get_parent(hws[IMX6QDL_CLK_PERIPH_PRE]->clk) ==
+ hws[IMX6QDL_CLK_PLL2_PFD2_396M]->clk)
reg |= PFD0_CLKGATE | PFD1_CLKGATE;
else
reg |= PFD0_CLKGATE | PFD1_CLKGATE | PFD2_CLKGATE;
writel_relaxed(reg, anatop_base + CCM_ANALOG_PLL_VIDEO);
}
+static struct clk_hw * __init imx6q_obtain_fixed_clk_hw(struct device_node *np,
+ const char *name,
+ unsigned long rate)
+{
+ struct clk *clk = of_clk_get_by_name(np, name);
+ struct clk_hw *hw;
+
+ if (IS_ERR(clk))
+ hw = imx_obtain_fixed_clock_hw(name, rate);
+ else
+ hw = __clk_get_hw(clk);
+
+ return hw;
+}
+
static void __init imx6q_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *anatop_base, *base;
int ret;
+ int i;
+
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX6QDL_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
- clk[IMX6QDL_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
- clk[IMX6QDL_CLK_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
- if (IS_ERR(clk[IMX6QDL_CLK_CKIL]))
- clk[IMX6QDL_CLK_CKIL] = imx_obtain_fixed_clock("ckil", 0);
- clk[IMX6QDL_CLK_CKIH] = of_clk_get_by_name(ccm_node, "ckih1");
- if (IS_ERR(clk[IMX6QDL_CLK_CKIH]))
- clk[IMX6QDL_CLK_CKIH] = imx_obtain_fixed_clock("ckih1", 0);
- clk[IMX6QDL_CLK_OSC] = of_clk_get_by_name(ccm_node, "osc");
- if (IS_ERR(clk[IMX6QDL_CLK_OSC]))
- clk[IMX6QDL_CLK_OSC] = imx_obtain_fixed_clock("osc", 0);
+ clk_hw_data->num = IMX6QDL_CLK_END;
+ hws = clk_hw_data->hws;
- /* Clock source from external clock via CLK1/2 PADs */
- clk[IMX6QDL_CLK_ANACLK1] = of_clk_get_by_name(ccm_node, "anaclk1");
- if (IS_ERR(clk[IMX6QDL_CLK_ANACLK1]))
- clk[IMX6QDL_CLK_ANACLK1] = imx_obtain_fixed_clock("anaclk1", 0);
+ hws[IMX6QDL_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
- clk[IMX6QDL_CLK_ANACLK2] = of_clk_get_by_name(ccm_node, "anaclk2");
- if (IS_ERR(clk[IMX6QDL_CLK_ANACLK2]))
- clk[IMX6QDL_CLK_ANACLK2] = imx_obtain_fixed_clock("anaclk2", 0);
+ hws[IMX6QDL_CLK_CKIL] = imx6q_obtain_fixed_clk_hw(ccm_node, "ckil", 0);
+ hws[IMX6QDL_CLK_CKIH] = imx6q_obtain_fixed_clk_hw(ccm_node, "ckih1", 0);
+ hws[IMX6QDL_CLK_OSC] = imx6q_obtain_fixed_clk_hw(ccm_node, "osc", 0);
+
+ /* Clock source from external clock via CLK1/2 PADs */
+ hws[IMX6QDL_CLK_ANACLK1] = imx6q_obtain_fixed_clk_hw(ccm_node, "anaclk1", 0);
+ hws[IMX6QDL_CLK_ANACLK2] = imx6q_obtain_fixed_clk_hw(ccm_node, "anaclk2", 0);
np = of_find_compatible_node(NULL, NULL, "fsl,imx6q-anatop");
anatop_base = base = of_iomap(np, 0);
video_div_table[3].div = 1;
}
- clk[IMX6QDL_PLL1_BYPASS_SRC] = imx_clk_mux("pll1_bypass_src", base + 0x00, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL2_BYPASS_SRC] = imx_clk_mux("pll2_bypass_src", base + 0x30, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL3_BYPASS_SRC] = imx_clk_mux("pll3_bypass_src", base + 0x10, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL4_BYPASS_SRC] = imx_clk_mux("pll4_bypass_src", base + 0x70, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL5_BYPASS_SRC] = imx_clk_mux("pll5_bypass_src", base + 0xa0, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL6_BYPASS_SRC] = imx_clk_mux("pll6_bypass_src", base + 0xe0, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clk[IMX6QDL_PLL7_BYPASS_SRC] = imx_clk_mux("pll7_bypass_src", base + 0x20, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL1_BYPASS_SRC] = imx_clk_hw_mux("pll1_bypass_src", base + 0x00, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL2_BYPASS_SRC] = imx_clk_hw_mux("pll2_bypass_src", base + 0x30, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL3_BYPASS_SRC] = imx_clk_hw_mux("pll3_bypass_src", base + 0x10, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL4_BYPASS_SRC] = imx_clk_hw_mux("pll4_bypass_src", base + 0x70, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL5_BYPASS_SRC] = imx_clk_hw_mux("pll5_bypass_src", base + 0xa0, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL6_BYPASS_SRC] = imx_clk_hw_mux("pll6_bypass_src", base + 0xe0, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6QDL_PLL7_BYPASS_SRC] = imx_clk_hw_mux("pll7_bypass_src", base + 0x20, 14, 2, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
/* type name parent_name base div_mask */
- clk[IMX6QDL_CLK_PLL1] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
- clk[IMX6QDL_CLK_PLL2] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
- clk[IMX6QDL_CLK_PLL3] = imx_clk_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
- clk[IMX6QDL_CLK_PLL4] = imx_clk_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
- clk[IMX6QDL_CLK_PLL5] = imx_clk_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
- clk[IMX6QDL_CLK_PLL6] = imx_clk_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
- clk[IMX6QDL_CLK_PLL7] = imx_clk_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
-
- clk[IMX6QDL_PLL1_BYPASS] = imx_clk_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL2_BYPASS] = imx_clk_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL3_BYPASS] = imx_clk_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL4_BYPASS] = imx_clk_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL5_BYPASS] = imx_clk_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL6_BYPASS] = imx_clk_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_PLL7_BYPASS] = imx_clk_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_PLL1] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
+ hws[IMX6QDL_CLK_PLL2] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
+ hws[IMX6QDL_CLK_PLL3] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
+ hws[IMX6QDL_CLK_PLL4] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
+ hws[IMX6QDL_CLK_PLL5] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
+ hws[IMX6QDL_CLK_PLL6] = imx_clk_hw_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
+ hws[IMX6QDL_CLK_PLL7] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
+
+ hws[IMX6QDL_PLL1_BYPASS] = imx_clk_hw_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL2_BYPASS] = imx_clk_hw_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL3_BYPASS] = imx_clk_hw_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL4_BYPASS] = imx_clk_hw_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL5_BYPASS] = imx_clk_hw_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL6_BYPASS] = imx_clk_hw_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_PLL7_BYPASS] = imx_clk_hw_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
/* Do not bypass PLLs initially */
- clk_set_parent(clk[IMX6QDL_PLL1_BYPASS], clk[IMX6QDL_CLK_PLL1]);
- clk_set_parent(clk[IMX6QDL_PLL2_BYPASS], clk[IMX6QDL_CLK_PLL2]);
- clk_set_parent(clk[IMX6QDL_PLL3_BYPASS], clk[IMX6QDL_CLK_PLL3]);
- clk_set_parent(clk[IMX6QDL_PLL4_BYPASS], clk[IMX6QDL_CLK_PLL4]);
- clk_set_parent(clk[IMX6QDL_PLL5_BYPASS], clk[IMX6QDL_CLK_PLL5]);
- clk_set_parent(clk[IMX6QDL_PLL6_BYPASS], clk[IMX6QDL_CLK_PLL6]);
- clk_set_parent(clk[IMX6QDL_PLL7_BYPASS], clk[IMX6QDL_CLK_PLL7]);
-
- clk[IMX6QDL_CLK_PLL1_SYS] = imx_clk_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
- clk[IMX6QDL_CLK_PLL2_BUS] = imx_clk_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
- clk[IMX6QDL_CLK_PLL3_USB_OTG] = imx_clk_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
- clk[IMX6QDL_CLK_PLL4_AUDIO] = imx_clk_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
- clk[IMX6QDL_CLK_PLL5_VIDEO] = imx_clk_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
- clk[IMX6QDL_CLK_PLL6_ENET] = imx_clk_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
- clk[IMX6QDL_CLK_PLL7_USB_HOST] = imx_clk_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
+ clk_set_parent(hws[IMX6QDL_PLL1_BYPASS]->clk, hws[IMX6QDL_CLK_PLL1]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL2_BYPASS]->clk, hws[IMX6QDL_CLK_PLL2]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL3_BYPASS]->clk, hws[IMX6QDL_CLK_PLL3]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL4_BYPASS]->clk, hws[IMX6QDL_CLK_PLL4]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL5_BYPASS]->clk, hws[IMX6QDL_CLK_PLL5]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL6_BYPASS]->clk, hws[IMX6QDL_CLK_PLL6]->clk);
+ clk_set_parent(hws[IMX6QDL_PLL7_BYPASS]->clk, hws[IMX6QDL_CLK_PLL7]->clk);
+
+ hws[IMX6QDL_CLK_PLL1_SYS] = imx_clk_hw_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
+ hws[IMX6QDL_CLK_PLL2_BUS] = imx_clk_hw_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
+ hws[IMX6QDL_CLK_PLL3_USB_OTG] = imx_clk_hw_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
+ hws[IMX6QDL_CLK_PLL4_AUDIO] = imx_clk_hw_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
+ hws[IMX6QDL_CLK_PLL5_VIDEO] = imx_clk_hw_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
+ hws[IMX6QDL_CLK_PLL6_ENET] = imx_clk_hw_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
+ hws[IMX6QDL_CLK_PLL7_USB_HOST] = imx_clk_hw_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
/*
* Bit 20 is the reserved and read-only bit, we do this only for:
* - Keep refcount when do usbphy clk_enable/disable, in that case,
* the clk framework may need to enable/disable usbphy's parent
*/
- clk[IMX6QDL_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
- clk[IMX6QDL_CLK_USBPHY2] = imx_clk_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
+ hws[IMX6QDL_CLK_USBPHY1] = imx_clk_hw_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
+ hws[IMX6QDL_CLK_USBPHY2] = imx_clk_hw_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
/*
* usbphy*_gate needs to be on after system boots up, and software
* never needs to control it anymore.
*/
- clk[IMX6QDL_CLK_USBPHY1_GATE] = imx_clk_gate("usbphy1_gate", "dummy", base + 0x10, 6);
- clk[IMX6QDL_CLK_USBPHY2_GATE] = imx_clk_gate("usbphy2_gate", "dummy", base + 0x20, 6);
+ hws[IMX6QDL_CLK_USBPHY1_GATE] = imx_clk_hw_gate("usbphy1_gate", "dummy", base + 0x10, 6);
+ hws[IMX6QDL_CLK_USBPHY2_GATE] = imx_clk_hw_gate("usbphy2_gate", "dummy", base + 0x20, 6);
/*
* The ENET PLL is special in that is has multiple outputs with
*
*/
if (!pll6_bypassed(ccm_node)) {
- clk[IMX6QDL_CLK_SATA_REF] = imx_clk_fixed_factor("sata_ref", "pll6_enet", 1, 5);
- clk[IMX6QDL_CLK_PCIE_REF] = imx_clk_fixed_factor("pcie_ref", "pll6_enet", 1, 4);
- clk[IMX6QDL_CLK_ENET_REF] = clk_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
+ hws[IMX6QDL_CLK_SATA_REF] = imx_clk_hw_fixed_factor("sata_ref", "pll6_enet", 1, 5);
+ hws[IMX6QDL_CLK_PCIE_REF] = imx_clk_hw_fixed_factor("pcie_ref", "pll6_enet", 1, 4);
+ hws[IMX6QDL_CLK_ENET_REF] = clk_hw_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
base + 0xe0, 0, 2, 0, clk_enet_ref_table,
&imx_ccm_lock);
} else {
- clk[IMX6QDL_CLK_SATA_REF] = imx_clk_fixed_factor("sata_ref", "pll6_enet", 1, 1);
- clk[IMX6QDL_CLK_PCIE_REF] = imx_clk_fixed_factor("pcie_ref", "pll6_enet", 1, 1);
- clk[IMX6QDL_CLK_ENET_REF] = imx_clk_fixed_factor("enet_ref", "pll6_enet", 1, 1);
+ hws[IMX6QDL_CLK_SATA_REF] = imx_clk_hw_fixed_factor("sata_ref", "pll6_enet", 1, 1);
+ hws[IMX6QDL_CLK_PCIE_REF] = imx_clk_hw_fixed_factor("pcie_ref", "pll6_enet", 1, 1);
+ hws[IMX6QDL_CLK_ENET_REF] = imx_clk_hw_fixed_factor("enet_ref", "pll6_enet", 1, 1);
}
- clk[IMX6QDL_CLK_SATA_REF_100M] = imx_clk_gate("sata_ref_100m", "sata_ref", base + 0xe0, 20);
- clk[IMX6QDL_CLK_PCIE_REF_125M] = imx_clk_gate("pcie_ref_125m", "pcie_ref", base + 0xe0, 19);
+ hws[IMX6QDL_CLK_SATA_REF_100M] = imx_clk_hw_gate("sata_ref_100m", "sata_ref", base + 0xe0, 20);
+ hws[IMX6QDL_CLK_PCIE_REF_125M] = imx_clk_hw_gate("pcie_ref_125m", "pcie_ref", base + 0xe0, 19);
- clk[IMX6QDL_CLK_LVDS1_SEL] = imx_clk_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
- clk[IMX6QDL_CLK_LVDS2_SEL] = imx_clk_mux("lvds2_sel", base + 0x160, 5, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
+ hws[IMX6QDL_CLK_LVDS1_SEL] = imx_clk_hw_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
+ hws[IMX6QDL_CLK_LVDS2_SEL] = imx_clk_hw_mux("lvds2_sel", base + 0x160, 5, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
/*
* lvds1_gate and lvds2_gate are pseudo-gates. Both can be
* it.
*/
writel(readl(base + 0x160) & ~0x3c00, base + 0x160);
- clk[IMX6QDL_CLK_LVDS1_GATE] = imx_clk_gate_exclusive("lvds1_gate", "lvds1_sel", base + 0x160, 10, BIT(12));
- clk[IMX6QDL_CLK_LVDS2_GATE] = imx_clk_gate_exclusive("lvds2_gate", "lvds2_sel", base + 0x160, 11, BIT(13));
+ hws[IMX6QDL_CLK_LVDS1_GATE] = imx_clk_hw_gate_exclusive("lvds1_gate", "lvds1_sel", base + 0x160, 10, BIT(12));
+ hws[IMX6QDL_CLK_LVDS2_GATE] = imx_clk_hw_gate_exclusive("lvds2_gate", "lvds2_sel", base + 0x160, 11, BIT(13));
- clk[IMX6QDL_CLK_LVDS1_IN] = imx_clk_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
- clk[IMX6QDL_CLK_LVDS2_IN] = imx_clk_gate_exclusive("lvds2_in", "anaclk2", base + 0x160, 13, BIT(11));
+ hws[IMX6QDL_CLK_LVDS1_IN] = imx_clk_hw_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
+ hws[IMX6QDL_CLK_LVDS2_IN] = imx_clk_hw_gate_exclusive("lvds2_in", "anaclk2", base + 0x160, 13, BIT(11));
/* name parent_name reg idx */
- clk[IMX6QDL_CLK_PLL2_PFD0_352M] = imx_clk_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
- clk[IMX6QDL_CLK_PLL2_PFD1_594M] = imx_clk_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
- clk[IMX6QDL_CLK_PLL2_PFD2_396M] = imx_clk_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
- clk[IMX6QDL_CLK_PLL3_PFD0_720M] = imx_clk_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
- clk[IMX6QDL_CLK_PLL3_PFD1_540M] = imx_clk_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
- clk[IMX6QDL_CLK_PLL3_PFD2_508M] = imx_clk_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
- clk[IMX6QDL_CLK_PLL3_PFD3_454M] = imx_clk_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
+ hws[IMX6QDL_CLK_PLL2_PFD0_352M] = imx_clk_hw_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
+ hws[IMX6QDL_CLK_PLL2_PFD1_594M] = imx_clk_hw_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
+ hws[IMX6QDL_CLK_PLL2_PFD2_396M] = imx_clk_hw_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
+ hws[IMX6QDL_CLK_PLL3_PFD0_720M] = imx_clk_hw_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
+ hws[IMX6QDL_CLK_PLL3_PFD1_540M] = imx_clk_hw_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
+ hws[IMX6QDL_CLK_PLL3_PFD2_508M] = imx_clk_hw_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
+ hws[IMX6QDL_CLK_PLL3_PFD3_454M] = imx_clk_hw_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
/* name parent_name mult div */
- clk[IMX6QDL_CLK_PLL2_198M] = imx_clk_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
- clk[IMX6QDL_CLK_PLL3_120M] = imx_clk_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
- clk[IMX6QDL_CLK_PLL3_80M] = imx_clk_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
- clk[IMX6QDL_CLK_PLL3_60M] = imx_clk_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
- clk[IMX6QDL_CLK_TWD] = imx_clk_fixed_factor("twd", "arm", 1, 2);
- clk[IMX6QDL_CLK_GPT_3M] = imx_clk_fixed_factor("gpt_3m", "osc", 1, 8);
- clk[IMX6QDL_CLK_VIDEO_27M] = imx_clk_fixed_factor("video_27m", "pll3_pfd1_540m", 1, 20);
+ hws[IMX6QDL_CLK_PLL2_198M] = imx_clk_hw_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
+ hws[IMX6QDL_CLK_PLL3_120M] = imx_clk_hw_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
+ hws[IMX6QDL_CLK_PLL3_80M] = imx_clk_hw_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
+ hws[IMX6QDL_CLK_PLL3_60M] = imx_clk_hw_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
+ hws[IMX6QDL_CLK_TWD] = imx_clk_hw_fixed_factor("twd", "arm", 1, 2);
+ hws[IMX6QDL_CLK_GPT_3M] = imx_clk_hw_fixed_factor("gpt_3m", "osc", 1, 8);
+ hws[IMX6QDL_CLK_VIDEO_27M] = imx_clk_hw_fixed_factor("video_27m", "pll3_pfd1_540m", 1, 20);
if (clk_on_imx6dl() || clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_GPU2D_AXI] = imx_clk_fixed_factor("gpu2d_axi", "mmdc_ch0_axi_podf", 1, 1);
- clk[IMX6QDL_CLK_GPU3D_AXI] = imx_clk_fixed_factor("gpu3d_axi", "mmdc_ch0_axi_podf", 1, 1);
+ hws[IMX6QDL_CLK_GPU2D_AXI] = imx_clk_hw_fixed_factor("gpu2d_axi", "mmdc_ch0_axi_podf", 1, 1);
+ hws[IMX6QDL_CLK_GPU3D_AXI] = imx_clk_hw_fixed_factor("gpu3d_axi", "mmdc_ch0_axi_podf", 1, 1);
}
- clk[IMX6QDL_CLK_PLL4_POST_DIV] = clk_register_divider_table(NULL, "pll4_post_div", "pll4_audio", CLK_SET_RATE_PARENT, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clk[IMX6QDL_CLK_PLL4_AUDIO_DIV] = clk_register_divider(NULL, "pll4_audio_div", "pll4_post_div", CLK_SET_RATE_PARENT, base + 0x170, 15, 1, 0, &imx_ccm_lock);
- clk[IMX6QDL_CLK_PLL5_POST_DIV] = clk_register_divider_table(NULL, "pll5_post_div", "pll5_video", CLK_SET_RATE_PARENT, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clk[IMX6QDL_CLK_PLL5_VIDEO_DIV] = clk_register_divider_table(NULL, "pll5_video_div", "pll5_post_div", CLK_SET_RATE_PARENT, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
+ hws[IMX6QDL_CLK_PLL4_POST_DIV] = clk_hw_register_divider_table(NULL, "pll4_post_div", "pll4_audio", CLK_SET_RATE_PARENT, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
+ hws[IMX6QDL_CLK_PLL4_AUDIO_DIV] = clk_hw_register_divider(NULL, "pll4_audio_div", "pll4_post_div", CLK_SET_RATE_PARENT, base + 0x170, 15, 1, 0, &imx_ccm_lock);
+ hws[IMX6QDL_CLK_PLL5_POST_DIV] = clk_hw_register_divider_table(NULL, "pll5_post_div", "pll5_video", CLK_SET_RATE_PARENT, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
+ hws[IMX6QDL_CLK_PLL5_VIDEO_DIV] = clk_hw_register_divider_table(NULL, "pll5_video_div", "pll5_post_div", CLK_SET_RATE_PARENT, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
np = ccm_node;
base = of_iomap(np, 0);
WARN_ON(!base);
/* name reg shift width parent_names num_parents */
- clk[IMX6QDL_CLK_STEP] = imx_clk_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
- clk[IMX6QDL_CLK_PLL1_SW] = imx_clk_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
- clk[IMX6QDL_CLK_PERIPH_PRE] = imx_clk_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
- clk[IMX6QDL_CLK_PERIPH2_PRE] = imx_clk_mux("periph2_pre", base + 0x18, 21, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
- clk[IMX6QDL_CLK_PERIPH_CLK2_SEL] = imx_clk_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
- clk[IMX6QDL_CLK_PERIPH2_CLK2_SEL] = imx_clk_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
- clk[IMX6QDL_CLK_AXI_SEL] = imx_clk_mux("axi_sel", base + 0x14, 6, 2, axi_sels, ARRAY_SIZE(axi_sels));
- clk[IMX6QDL_CLK_ESAI_SEL] = imx_clk_mux("esai_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clk[IMX6QDL_CLK_ASRC_SEL] = imx_clk_mux("asrc_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clk[IMX6QDL_CLK_SPDIF_SEL] = imx_clk_mux("spdif_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6QDL_CLK_STEP] = imx_clk_hw_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
+ hws[IMX6QDL_CLK_PLL1_SW] = imx_clk_hw_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
+ hws[IMX6QDL_CLK_PERIPH_PRE] = imx_clk_hw_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
+ hws[IMX6QDL_CLK_PERIPH2_PRE] = imx_clk_hw_mux("periph2_pre", base + 0x18, 21, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
+ hws[IMX6QDL_CLK_PERIPH_CLK2_SEL] = imx_clk_hw_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
+ hws[IMX6QDL_CLK_PERIPH2_CLK2_SEL] = imx_clk_hw_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
+ hws[IMX6QDL_CLK_AXI_SEL] = imx_clk_hw_mux("axi_sel", base + 0x14, 6, 2, axi_sels, ARRAY_SIZE(axi_sels));
+ hws[IMX6QDL_CLK_ESAI_SEL] = imx_clk_hw_mux("esai_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6QDL_CLK_ASRC_SEL] = imx_clk_hw_mux("asrc_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6QDL_CLK_SPDIF_SEL] = imx_clk_hw_mux("spdif_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
if (clk_on_imx6q()) {
- clk[IMX6QDL_CLK_GPU2D_AXI] = imx_clk_mux("gpu2d_axi", base + 0x18, 0, 1, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
- clk[IMX6QDL_CLK_GPU3D_AXI] = imx_clk_mux("gpu3d_axi", base + 0x18, 1, 1, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
+ hws[IMX6QDL_CLK_GPU2D_AXI] = imx_clk_hw_mux("gpu2d_axi", base + 0x18, 0, 1, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
+ hws[IMX6QDL_CLK_GPU3D_AXI] = imx_clk_hw_mux("gpu3d_axi", base + 0x18, 1, 1, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
}
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_CAN_SEL] = imx_clk_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
- clk[IMX6QDL_CLK_ECSPI_SEL] = imx_clk_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
- clk[IMX6QDL_CLK_IPG_PER_SEL] = imx_clk_mux("ipg_per_sel", base + 0x1c, 6, 1, ipg_per_sels, ARRAY_SIZE(ipg_per_sels));
- clk[IMX6QDL_CLK_UART_SEL] = imx_clk_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
- clk[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_mux("gpu2d_core_sel", base + 0x18, 16, 2, gpu2d_core_sels_2, ARRAY_SIZE(gpu2d_core_sels_2));
+ hws[IMX6QDL_CLK_CAN_SEL] = imx_clk_hw_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
+ hws[IMX6QDL_CLK_ECSPI_SEL] = imx_clk_hw_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
+ hws[IMX6QDL_CLK_IPG_PER_SEL] = imx_clk_hw_mux("ipg_per_sel", base + 0x1c, 6, 1, ipg_per_sels, ARRAY_SIZE(ipg_per_sels));
+ hws[IMX6QDL_CLK_UART_SEL] = imx_clk_hw_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
+ hws[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_hw_mux("gpu2d_core_sel", base + 0x18, 16, 2, gpu2d_core_sels_2, ARRAY_SIZE(gpu2d_core_sels_2));
} else if (clk_on_imx6dl()) {
- clk[IMX6QDL_CLK_MLB_SEL] = imx_clk_mux("mlb_sel", base + 0x18, 16, 2, gpu2d_core_sels, ARRAY_SIZE(gpu2d_core_sels));
+ hws[IMX6QDL_CLK_MLB_SEL] = imx_clk_hw_mux("mlb_sel", base + 0x18, 16, 2, gpu2d_core_sels, ARRAY_SIZE(gpu2d_core_sels));
} else {
- clk[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_mux("gpu2d_core_sel", base + 0x18, 16, 2, gpu2d_core_sels, ARRAY_SIZE(gpu2d_core_sels));
+ hws[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_hw_mux("gpu2d_core_sel", base + 0x18, 16, 2, gpu2d_core_sels, ARRAY_SIZE(gpu2d_core_sels));
}
- clk[IMX6QDL_CLK_GPU3D_CORE_SEL] = imx_clk_mux("gpu3d_core_sel", base + 0x18, 4, 2, gpu3d_core_sels, ARRAY_SIZE(gpu3d_core_sels));
+ hws[IMX6QDL_CLK_GPU3D_CORE_SEL] = imx_clk_hw_mux("gpu3d_core_sel", base + 0x18, 4, 2, gpu3d_core_sels, ARRAY_SIZE(gpu3d_core_sels));
if (clk_on_imx6dl())
- clk[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_mux("gpu2d_core_sel", base + 0x18, 8, 2, gpu3d_shader_sels, ARRAY_SIZE(gpu3d_shader_sels));
+ hws[IMX6QDL_CLK_GPU2D_CORE_SEL] = imx_clk_hw_mux("gpu2d_core_sel", base + 0x18, 8, 2, gpu3d_shader_sels, ARRAY_SIZE(gpu3d_shader_sels));
else
- clk[IMX6QDL_CLK_GPU3D_SHADER_SEL] = imx_clk_mux("gpu3d_shader_sel", base + 0x18, 8, 2, gpu3d_shader_sels, ARRAY_SIZE(gpu3d_shader_sels));
- clk[IMX6QDL_CLK_IPU1_SEL] = imx_clk_mux("ipu1_sel", base + 0x3c, 9, 2, ipu_sels, ARRAY_SIZE(ipu_sels));
- clk[IMX6QDL_CLK_IPU2_SEL] = imx_clk_mux("ipu2_sel", base + 0x3c, 14, 2, ipu_sels, ARRAY_SIZE(ipu_sels));
+ hws[IMX6QDL_CLK_GPU3D_SHADER_SEL] = imx_clk_hw_mux("gpu3d_shader_sel", base + 0x18, 8, 2, gpu3d_shader_sels, ARRAY_SIZE(gpu3d_shader_sels));
+ hws[IMX6QDL_CLK_IPU1_SEL] = imx_clk_hw_mux("ipu1_sel", base + 0x3c, 9, 2, ipu_sels, ARRAY_SIZE(ipu_sels));
+ hws[IMX6QDL_CLK_IPU2_SEL] = imx_clk_hw_mux("ipu2_sel", base + 0x3c, 14, 2, ipu_sels, ARRAY_SIZE(ipu_sels));
disable_anatop_clocks(anatop_base);
- imx6q_mmdc_ch1_mask_handshake(base);
+ imx_mmdc_mask_handshake(base, 1);
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_LDB_DI0_SEL] = imx_clk_mux_flags("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_LDB_DI1_SEL] = imx_clk_mux_flags("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_LDB_DI0_SEL] = imx_clk_hw_mux_flags("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_LDB_DI1_SEL] = imx_clk_hw_mux_flags("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels), CLK_SET_RATE_PARENT);
} else {
/*
* The LDB_DI0/1_SEL muxes are registered read-only due to a hardware
*/
init_ldb_clks(np, base);
- clk[IMX6QDL_CLK_LDB_DI0_SEL] = imx_clk_mux_ldb("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels));
- clk[IMX6QDL_CLK_LDB_DI1_SEL] = imx_clk_mux_ldb("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels));
+ hws[IMX6QDL_CLK_LDB_DI0_SEL] = imx_clk_hw_mux_ldb("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels));
+ hws[IMX6QDL_CLK_LDB_DI1_SEL] = imx_clk_hw_mux_ldb("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di_sels, ARRAY_SIZE(ldb_di_sels));
}
- clk[IMX6QDL_CLK_IPU1_DI0_PRE_SEL] = imx_clk_mux_flags("ipu1_di0_pre_sel", base + 0x34, 6, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU1_DI1_PRE_SEL] = imx_clk_mux_flags("ipu1_di1_pre_sel", base + 0x34, 15, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI0_PRE_SEL] = imx_clk_mux_flags("ipu2_di0_pre_sel", base + 0x38, 6, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI1_PRE_SEL] = imx_clk_mux_flags("ipu2_di1_pre_sel", base + 0x38, 15, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_HSI_TX_SEL] = imx_clk_mux("hsi_tx_sel", base + 0x30, 28, 1, hsi_tx_sels, ARRAY_SIZE(hsi_tx_sels));
- clk[IMX6QDL_CLK_PCIE_AXI_SEL] = imx_clk_mux("pcie_axi_sel", base + 0x18, 10, 1, pcie_axi_sels, ARRAY_SIZE(pcie_axi_sels));
+
+ hws[IMX6QDL_CLK_IPU1_DI0_PRE_SEL] = imx_clk_hw_mux_flags("ipu1_di0_pre_sel", base + 0x34, 6, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU1_DI1_PRE_SEL] = imx_clk_hw_mux_flags("ipu1_di1_pre_sel", base + 0x34, 15, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI0_PRE_SEL] = imx_clk_hw_mux_flags("ipu2_di0_pre_sel", base + 0x38, 6, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI1_PRE_SEL] = imx_clk_hw_mux_flags("ipu2_di1_pre_sel", base + 0x38, 15, 3, ipu_di_pre_sels, ARRAY_SIZE(ipu_di_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_HSI_TX_SEL] = imx_clk_hw_mux("hsi_tx_sel", base + 0x30, 28, 1, hsi_tx_sels, ARRAY_SIZE(hsi_tx_sels));
+ hws[IMX6QDL_CLK_PCIE_AXI_SEL] = imx_clk_hw_mux("pcie_axi_sel", base + 0x18, 10, 1, pcie_axi_sels, ARRAY_SIZE(pcie_axi_sels));
+
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_IPU1_DI0_SEL] = imx_clk_mux_flags("ipu1_di0_sel", base + 0x34, 0, 3, ipu1_di0_sels_2, ARRAY_SIZE(ipu1_di0_sels_2), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU1_DI1_SEL] = imx_clk_mux_flags("ipu1_di1_sel", base + 0x34, 9, 3, ipu1_di1_sels_2, ARRAY_SIZE(ipu1_di1_sels_2), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI0_SEL] = imx_clk_mux_flags("ipu2_di0_sel", base + 0x38, 0, 3, ipu2_di0_sels_2, ARRAY_SIZE(ipu2_di0_sels_2), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI1_SEL] = imx_clk_mux_flags("ipu2_di1_sel", base + 0x38, 9, 3, ipu2_di1_sels_2, ARRAY_SIZE(ipu2_di1_sels_2), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_SSI1_SEL] = imx_clk_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clk[IMX6QDL_CLK_SSI2_SEL] = imx_clk_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clk[IMX6QDL_CLK_SSI3_SEL] = imx_clk_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clk[IMX6QDL_CLK_USDHC1_SEL] = imx_clk_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clk[IMX6QDL_CLK_USDHC2_SEL] = imx_clk_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clk[IMX6QDL_CLK_USDHC3_SEL] = imx_clk_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clk[IMX6QDL_CLK_USDHC4_SEL] = imx_clk_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clk[IMX6QDL_CLK_ENFC_SEL] = imx_clk_mux("enfc_sel", base + 0x2c, 15, 3, enfc_sels_2, ARRAY_SIZE(enfc_sels_2));
- clk[IMX6QDL_CLK_EIM_SEL] = imx_clk_mux("eim_sel", base + 0x1c, 27, 2, eim_sels, ARRAY_SIZE(eim_sels));
- clk[IMX6QDL_CLK_EIM_SLOW_SEL] = imx_clk_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
- clk[IMX6QDL_CLK_PRE_AXI] = imx_clk_mux("pre_axi", base + 0x18, 1, 1, pre_axi_sels, ARRAY_SIZE(pre_axi_sels));
+ hws[IMX6QDL_CLK_IPU1_DI0_SEL] = imx_clk_hw_mux_flags("ipu1_di0_sel", base + 0x34, 0, 3, ipu1_di0_sels_2, ARRAY_SIZE(ipu1_di0_sels_2), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU1_DI1_SEL] = imx_clk_hw_mux_flags("ipu1_di1_sel", base + 0x34, 9, 3, ipu1_di1_sels_2, ARRAY_SIZE(ipu1_di1_sels_2), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI0_SEL] = imx_clk_hw_mux_flags("ipu2_di0_sel", base + 0x38, 0, 3, ipu2_di0_sels_2, ARRAY_SIZE(ipu2_di0_sels_2), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI1_SEL] = imx_clk_hw_mux_flags("ipu2_di1_sel", base + 0x38, 9, 3, ipu2_di1_sels_2, ARRAY_SIZE(ipu2_di1_sels_2), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_SSI1_SEL] = imx_clk_hw_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6QDL_CLK_SSI2_SEL] = imx_clk_hw_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6QDL_CLK_SSI3_SEL] = imx_clk_hw_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6QDL_CLK_USDHC1_SEL] = imx_clk_hw_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6QDL_CLK_USDHC2_SEL] = imx_clk_hw_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6QDL_CLK_USDHC3_SEL] = imx_clk_hw_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6QDL_CLK_USDHC4_SEL] = imx_clk_hw_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6QDL_CLK_ENFC_SEL] = imx_clk_hw_mux("enfc_sel", base + 0x2c, 15, 3, enfc_sels_2, ARRAY_SIZE(enfc_sels_2));
+ hws[IMX6QDL_CLK_EIM_SEL] = imx_clk_hw_mux("eim_sel", base + 0x1c, 27, 2, eim_sels, ARRAY_SIZE(eim_sels));
+ hws[IMX6QDL_CLK_EIM_SLOW_SEL] = imx_clk_hw_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
+ hws[IMX6QDL_CLK_PRE_AXI] = imx_clk_hw_mux("pre_axi", base + 0x18, 1, 1, pre_axi_sels, ARRAY_SIZE(pre_axi_sels));
} else {
- clk[IMX6QDL_CLK_IPU1_DI0_SEL] = imx_clk_mux_flags("ipu1_di0_sel", base + 0x34, 0, 3, ipu1_di0_sels, ARRAY_SIZE(ipu1_di0_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU1_DI1_SEL] = imx_clk_mux_flags("ipu1_di1_sel", base + 0x34, 9, 3, ipu1_di1_sels, ARRAY_SIZE(ipu1_di1_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI0_SEL] = imx_clk_mux_flags("ipu2_di0_sel", base + 0x38, 0, 3, ipu2_di0_sels, ARRAY_SIZE(ipu2_di0_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_IPU2_DI1_SEL] = imx_clk_mux_flags("ipu2_di1_sel", base + 0x38, 9, 3, ipu2_di1_sels, ARRAY_SIZE(ipu2_di1_sels), CLK_SET_RATE_PARENT);
- clk[IMX6QDL_CLK_SSI1_SEL] = imx_clk_fixup_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_SSI2_SEL] = imx_clk_fixup_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_SSI3_SEL] = imx_clk_fixup_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_USDHC1_SEL] = imx_clk_fixup_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_USDHC2_SEL] = imx_clk_fixup_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_USDHC3_SEL] = imx_clk_fixup_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_USDHC4_SEL] = imx_clk_fixup_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_ENFC_SEL] = imx_clk_mux("enfc_sel", base + 0x2c, 16, 2, enfc_sels, ARRAY_SIZE(enfc_sels));
- clk[IMX6QDL_CLK_EIM_SEL] = imx_clk_fixup_mux("eim_sel", base + 0x1c, 27, 2, eim_sels, ARRAY_SIZE(eim_sels), imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_EIM_SLOW_SEL] = imx_clk_fixup_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_IPU1_DI0_SEL] = imx_clk_hw_mux_flags("ipu1_di0_sel", base + 0x34, 0, 3, ipu1_di0_sels, ARRAY_SIZE(ipu1_di0_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU1_DI1_SEL] = imx_clk_hw_mux_flags("ipu1_di1_sel", base + 0x34, 9, 3, ipu1_di1_sels, ARRAY_SIZE(ipu1_di1_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI0_SEL] = imx_clk_hw_mux_flags("ipu2_di0_sel", base + 0x38, 0, 3, ipu2_di0_sels, ARRAY_SIZE(ipu2_di0_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_IPU2_DI1_SEL] = imx_clk_hw_mux_flags("ipu2_di1_sel", base + 0x38, 9, 3, ipu2_di1_sels, ARRAY_SIZE(ipu2_di1_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6QDL_CLK_SSI1_SEL] = imx_clk_hw_fixup_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_SSI2_SEL] = imx_clk_hw_fixup_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_SSI3_SEL] = imx_clk_hw_fixup_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_USDHC1_SEL] = imx_clk_hw_fixup_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_USDHC2_SEL] = imx_clk_hw_fixup_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_USDHC3_SEL] = imx_clk_hw_fixup_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_USDHC4_SEL] = imx_clk_hw_fixup_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_ENFC_SEL] = imx_clk_hw_mux("enfc_sel", base + 0x2c, 16, 2, enfc_sels, ARRAY_SIZE(enfc_sels));
+ hws[IMX6QDL_CLK_EIM_SEL] = imx_clk_hw_fixup_mux("eim_sel", base + 0x1c, 27, 2, eim_sels, ARRAY_SIZE(eim_sels), imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_EIM_SLOW_SEL] = imx_clk_hw_fixup_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels), imx_cscmr1_fixup);
}
- clk[IMX6QDL_CLK_VDO_AXI_SEL] = imx_clk_mux("vdo_axi_sel", base + 0x18, 11, 1, vdo_axi_sels, ARRAY_SIZE(vdo_axi_sels));
- clk[IMX6QDL_CLK_VPU_AXI_SEL] = imx_clk_mux("vpu_axi_sel", base + 0x18, 14, 2, vpu_axi_sels, ARRAY_SIZE(vpu_axi_sels));
- clk[IMX6QDL_CLK_CKO1_SEL] = imx_clk_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
- clk[IMX6QDL_CLK_CKO2_SEL] = imx_clk_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
- clk[IMX6QDL_CLK_CKO] = imx_clk_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
+
+ hws[IMX6QDL_CLK_VDO_AXI_SEL] = imx_clk_hw_mux("vdo_axi_sel", base + 0x18, 11, 1, vdo_axi_sels, ARRAY_SIZE(vdo_axi_sels));
+ hws[IMX6QDL_CLK_VPU_AXI_SEL] = imx_clk_hw_mux("vpu_axi_sel", base + 0x18, 14, 2, vpu_axi_sels, ARRAY_SIZE(vpu_axi_sels));
+ hws[IMX6QDL_CLK_CKO1_SEL] = imx_clk_hw_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
+ hws[IMX6QDL_CLK_CKO2_SEL] = imx_clk_hw_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
+ hws[IMX6QDL_CLK_CKO] = imx_clk_hw_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
/* name reg shift width busy: reg, shift parent_names num_parents */
- clk[IMX6QDL_CLK_PERIPH] = imx_clk_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
- clk[IMX6QDL_CLK_PERIPH2] = imx_clk_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
+ hws[IMX6QDL_CLK_PERIPH] = imx_clk_hw_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
+ hws[IMX6QDL_CLK_PERIPH2] = imx_clk_hw_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
/* name parent_name reg shift width */
- clk[IMX6QDL_CLK_PERIPH_CLK2] = imx_clk_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
- clk[IMX6QDL_CLK_PERIPH2_CLK2] = imx_clk_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
- clk[IMX6QDL_CLK_IPG] = imx_clk_divider("ipg", "ahb", base + 0x14, 8, 2);
- clk[IMX6QDL_CLK_ESAI_PRED] = imx_clk_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
- clk[IMX6QDL_CLK_ESAI_PODF] = imx_clk_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
- clk[IMX6QDL_CLK_ASRC_PRED] = imx_clk_divider("asrc_pred", "asrc_sel", base + 0x30, 12, 3);
- clk[IMX6QDL_CLK_ASRC_PODF] = imx_clk_divider("asrc_podf", "asrc_pred", base + 0x30, 9, 3);
- clk[IMX6QDL_CLK_SPDIF_PRED] = imx_clk_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
- clk[IMX6QDL_CLK_SPDIF_PODF] = imx_clk_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
+ hws[IMX6QDL_CLK_PERIPH_CLK2] = imx_clk_hw_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
+ hws[IMX6QDL_CLK_PERIPH2_CLK2] = imx_clk_hw_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
+ hws[IMX6QDL_CLK_IPG] = imx_clk_hw_divider("ipg", "ahb", base + 0x14, 8, 2);
+ hws[IMX6QDL_CLK_ESAI_PRED] = imx_clk_hw_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
+ hws[IMX6QDL_CLK_ESAI_PODF] = imx_clk_hw_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
+ hws[IMX6QDL_CLK_ASRC_PRED] = imx_clk_hw_divider("asrc_pred", "asrc_sel", base + 0x30, 12, 3);
+ hws[IMX6QDL_CLK_ASRC_PODF] = imx_clk_hw_divider("asrc_podf", "asrc_pred", base + 0x30, 9, 3);
+ hws[IMX6QDL_CLK_SPDIF_PRED] = imx_clk_hw_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
+ hws[IMX6QDL_CLK_SPDIF_PODF] = imx_clk_hw_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
+
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_IPG_PER] = imx_clk_divider("ipg_per", "ipg_per_sel", base + 0x1c, 0, 6);
- clk[IMX6QDL_CLK_ECSPI_ROOT] = imx_clk_divider("ecspi_root", "ecspi_sel", base + 0x38, 19, 6);
- clk[IMX6QDL_CLK_CAN_ROOT] = imx_clk_divider("can_root", "can_sel", base + 0x20, 2, 6);
- clk[IMX6QDL_CLK_UART_SERIAL_PODF] = imx_clk_divider("uart_serial_podf", "uart_sel", base + 0x24, 0, 6);
- clk[IMX6QDL_CLK_LDB_DI0_DIV_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0", 2, 7);
- clk[IMX6QDL_CLK_LDB_DI1_DIV_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "ldb_di1", 2, 7);
+ hws[IMX6QDL_CLK_IPG_PER] = imx_clk_hw_divider("ipg_per", "ipg_per_sel", base + 0x1c, 0, 6);
+ hws[IMX6QDL_CLK_ECSPI_ROOT] = imx_clk_hw_divider("ecspi_root", "ecspi_sel", base + 0x38, 19, 6);
+ hws[IMX6QDL_CLK_CAN_ROOT] = imx_clk_hw_divider("can_root", "can_sel", base + 0x20, 2, 6);
+ hws[IMX6QDL_CLK_UART_SERIAL_PODF] = imx_clk_hw_divider("uart_serial_podf", "uart_sel", base + 0x24, 0, 6);
+ hws[IMX6QDL_CLK_LDB_DI0_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di0_div_3_5", "ldb_di0", 2, 7);
+ hws[IMX6QDL_CLK_LDB_DI1_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di1_div_3_5", "ldb_di1", 2, 7);
} else {
- clk[IMX6QDL_CLK_ECSPI_ROOT] = imx_clk_divider("ecspi_root", "pll3_60m", base + 0x38, 19, 6);
- clk[IMX6QDL_CLK_CAN_ROOT] = imx_clk_divider("can_root", "pll3_60m", base + 0x20, 2, 6);
- clk[IMX6QDL_CLK_IPG_PER] = imx_clk_fixup_divider("ipg_per", "ipg", base + 0x1c, 0, 6, imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_UART_SERIAL_PODF] = imx_clk_divider("uart_serial_podf", "pll3_80m", base + 0x24, 0, 6);
- clk[IMX6QDL_CLK_LDB_DI0_DIV_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
- clk[IMX6QDL_CLK_LDB_DI1_DIV_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
+ hws[IMX6QDL_CLK_ECSPI_ROOT] = imx_clk_hw_divider("ecspi_root", "pll3_60m", base + 0x38, 19, 6);
+ hws[IMX6QDL_CLK_CAN_ROOT] = imx_clk_hw_divider("can_root", "pll3_60m", base + 0x20, 2, 6);
+ hws[IMX6QDL_CLK_IPG_PER] = imx_clk_hw_fixup_divider("ipg_per", "ipg", base + 0x1c, 0, 6, imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_UART_SERIAL_PODF] = imx_clk_hw_divider("uart_serial_podf", "pll3_80m", base + 0x24, 0, 6);
+ hws[IMX6QDL_CLK_LDB_DI0_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
+ hws[IMX6QDL_CLK_LDB_DI1_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
}
+
if (clk_on_imx6dl())
- clk[IMX6QDL_CLK_MLB_PODF] = imx_clk_divider("mlb_podf", "mlb_sel", base + 0x18, 23, 3);
+ hws[IMX6QDL_CLK_MLB_PODF] = imx_clk_hw_divider("mlb_podf", "mlb_sel", base + 0x18, 23, 3);
else
- clk[IMX6QDL_CLK_GPU2D_CORE_PODF] = imx_clk_divider("gpu2d_core_podf", "gpu2d_core_sel", base + 0x18, 23, 3);
- clk[IMX6QDL_CLK_GPU3D_CORE_PODF] = imx_clk_divider("gpu3d_core_podf", "gpu3d_core_sel", base + 0x18, 26, 3);
+ hws[IMX6QDL_CLK_GPU2D_CORE_PODF] = imx_clk_hw_divider("gpu2d_core_podf", "gpu2d_core_sel", base + 0x18, 23, 3);
+ hws[IMX6QDL_CLK_GPU3D_CORE_PODF] = imx_clk_hw_divider("gpu3d_core_podf", "gpu3d_core_sel", base + 0x18, 26, 3);
if (clk_on_imx6dl())
- clk[IMX6QDL_CLK_GPU2D_CORE_PODF] = imx_clk_divider("gpu2d_core_podf", "gpu2d_core_sel", base + 0x18, 29, 3);
+ hws[IMX6QDL_CLK_GPU2D_CORE_PODF] = imx_clk_hw_divider("gpu2d_core_podf", "gpu2d_core_sel", base + 0x18, 29, 3);
else
- clk[IMX6QDL_CLK_GPU3D_SHADER] = imx_clk_divider("gpu3d_shader", "gpu3d_shader_sel", base + 0x18, 29, 3);
- clk[IMX6QDL_CLK_IPU1_PODF] = imx_clk_divider("ipu1_podf", "ipu1_sel", base + 0x3c, 11, 3);
- clk[IMX6QDL_CLK_IPU2_PODF] = imx_clk_divider("ipu2_podf", "ipu2_sel", base + 0x3c, 16, 3);
- clk[IMX6QDL_CLK_LDB_DI0_PODF] = imx_clk_divider_flags("ldb_di0_podf", "ldb_di0_div_3_5", base + 0x20, 10, 1, 0);
- clk[IMX6QDL_CLK_LDB_DI1_PODF] = imx_clk_divider_flags("ldb_di1_podf", "ldb_di1_div_3_5", base + 0x20, 11, 1, 0);
- clk[IMX6QDL_CLK_IPU1_DI0_PRE] = imx_clk_divider("ipu1_di0_pre", "ipu1_di0_pre_sel", base + 0x34, 3, 3);
- clk[IMX6QDL_CLK_IPU1_DI1_PRE] = imx_clk_divider("ipu1_di1_pre", "ipu1_di1_pre_sel", base + 0x34, 12, 3);
- clk[IMX6QDL_CLK_IPU2_DI0_PRE] = imx_clk_divider("ipu2_di0_pre", "ipu2_di0_pre_sel", base + 0x38, 3, 3);
- clk[IMX6QDL_CLK_IPU2_DI1_PRE] = imx_clk_divider("ipu2_di1_pre", "ipu2_di1_pre_sel", base + 0x38, 12, 3);
- clk[IMX6QDL_CLK_HSI_TX_PODF] = imx_clk_divider("hsi_tx_podf", "hsi_tx_sel", base + 0x30, 29, 3);
- clk[IMX6QDL_CLK_SSI1_PRED] = imx_clk_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
- clk[IMX6QDL_CLK_SSI1_PODF] = imx_clk_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
- clk[IMX6QDL_CLK_SSI2_PRED] = imx_clk_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
- clk[IMX6QDL_CLK_SSI2_PODF] = imx_clk_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
- clk[IMX6QDL_CLK_SSI3_PRED] = imx_clk_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
- clk[IMX6QDL_CLK_SSI3_PODF] = imx_clk_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
- clk[IMX6QDL_CLK_USDHC1_PODF] = imx_clk_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
- clk[IMX6QDL_CLK_USDHC2_PODF] = imx_clk_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
- clk[IMX6QDL_CLK_USDHC3_PODF] = imx_clk_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
- clk[IMX6QDL_CLK_USDHC4_PODF] = imx_clk_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
- clk[IMX6QDL_CLK_ENFC_PRED] = imx_clk_divider("enfc_pred", "enfc_sel", base + 0x2c, 18, 3);
- clk[IMX6QDL_CLK_ENFC_PODF] = imx_clk_divider("enfc_podf", "enfc_pred", base + 0x2c, 21, 6);
+ hws[IMX6QDL_CLK_GPU3D_SHADER] = imx_clk_hw_divider("gpu3d_shader", "gpu3d_shader_sel", base + 0x18, 29, 3);
+ hws[IMX6QDL_CLK_IPU1_PODF] = imx_clk_hw_divider("ipu1_podf", "ipu1_sel", base + 0x3c, 11, 3);
+ hws[IMX6QDL_CLK_IPU2_PODF] = imx_clk_hw_divider("ipu2_podf", "ipu2_sel", base + 0x3c, 16, 3);
+ hws[IMX6QDL_CLK_LDB_DI0_PODF] = imx_clk_hw_divider_flags("ldb_di0_podf", "ldb_di0_div_3_5", base + 0x20, 10, 1, 0);
+ hws[IMX6QDL_CLK_LDB_DI1_PODF] = imx_clk_hw_divider_flags("ldb_di1_podf", "ldb_di1_div_3_5", base + 0x20, 11, 1, 0);
+ hws[IMX6QDL_CLK_IPU1_DI0_PRE] = imx_clk_hw_divider("ipu1_di0_pre", "ipu1_di0_pre_sel", base + 0x34, 3, 3);
+ hws[IMX6QDL_CLK_IPU1_DI1_PRE] = imx_clk_hw_divider("ipu1_di1_pre", "ipu1_di1_pre_sel", base + 0x34, 12, 3);
+ hws[IMX6QDL_CLK_IPU2_DI0_PRE] = imx_clk_hw_divider("ipu2_di0_pre", "ipu2_di0_pre_sel", base + 0x38, 3, 3);
+ hws[IMX6QDL_CLK_IPU2_DI1_PRE] = imx_clk_hw_divider("ipu2_di1_pre", "ipu2_di1_pre_sel", base + 0x38, 12, 3);
+ hws[IMX6QDL_CLK_HSI_TX_PODF] = imx_clk_hw_divider("hsi_tx_podf", "hsi_tx_sel", base + 0x30, 29, 3);
+ hws[IMX6QDL_CLK_SSI1_PRED] = imx_clk_hw_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
+ hws[IMX6QDL_CLK_SSI1_PODF] = imx_clk_hw_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
+ hws[IMX6QDL_CLK_SSI2_PRED] = imx_clk_hw_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
+ hws[IMX6QDL_CLK_SSI2_PODF] = imx_clk_hw_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
+ hws[IMX6QDL_CLK_SSI3_PRED] = imx_clk_hw_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
+ hws[IMX6QDL_CLK_SSI3_PODF] = imx_clk_hw_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
+ hws[IMX6QDL_CLK_USDHC1_PODF] = imx_clk_hw_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
+ hws[IMX6QDL_CLK_USDHC2_PODF] = imx_clk_hw_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
+ hws[IMX6QDL_CLK_USDHC3_PODF] = imx_clk_hw_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
+ hws[IMX6QDL_CLK_USDHC4_PODF] = imx_clk_hw_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
+ hws[IMX6QDL_CLK_ENFC_PRED] = imx_clk_hw_divider("enfc_pred", "enfc_sel", base + 0x2c, 18, 3);
+ hws[IMX6QDL_CLK_ENFC_PODF] = imx_clk_hw_divider("enfc_podf", "enfc_pred", base + 0x2c, 21, 6);
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_EIM_PODF] = imx_clk_divider("eim_podf", "eim_sel", base + 0x1c, 20, 3);
- clk[IMX6QDL_CLK_EIM_SLOW_PODF] = imx_clk_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
+ hws[IMX6QDL_CLK_EIM_PODF] = imx_clk_hw_divider("eim_podf", "eim_sel", base + 0x1c, 20, 3);
+ hws[IMX6QDL_CLK_EIM_SLOW_PODF] = imx_clk_hw_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
} else {
- clk[IMX6QDL_CLK_EIM_PODF] = imx_clk_fixup_divider("eim_podf", "eim_sel", base + 0x1c, 20, 3, imx_cscmr1_fixup);
- clk[IMX6QDL_CLK_EIM_SLOW_PODF] = imx_clk_fixup_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3, imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_EIM_PODF] = imx_clk_hw_fixup_divider("eim_podf", "eim_sel", base + 0x1c, 20, 3, imx_cscmr1_fixup);
+ hws[IMX6QDL_CLK_EIM_SLOW_PODF] = imx_clk_hw_fixup_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3, imx_cscmr1_fixup);
}
- clk[IMX6QDL_CLK_VPU_AXI_PODF] = imx_clk_divider("vpu_axi_podf", "vpu_axi_sel", base + 0x24, 25, 3);
- clk[IMX6QDL_CLK_CKO1_PODF] = imx_clk_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
- clk[IMX6QDL_CLK_CKO2_PODF] = imx_clk_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
+
+ hws[IMX6QDL_CLK_VPU_AXI_PODF] = imx_clk_hw_divider("vpu_axi_podf", "vpu_axi_sel", base + 0x24, 25, 3);
+ hws[IMX6QDL_CLK_CKO1_PODF] = imx_clk_hw_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
+ hws[IMX6QDL_CLK_CKO2_PODF] = imx_clk_hw_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
/* name parent_name reg shift width busy: reg, shift */
- clk[IMX6QDL_CLK_AXI] = imx_clk_busy_divider("axi", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
- clk[IMX6QDL_CLK_MMDC_CH0_AXI_PODF] = imx_clk_busy_divider("mmdc_ch0_axi_podf", "periph", base + 0x14, 19, 3, base + 0x48, 4);
+ hws[IMX6QDL_CLK_AXI] = imx_clk_hw_busy_divider("axi", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
+ hws[IMX6QDL_CLK_MMDC_CH0_AXI_PODF] = imx_clk_hw_busy_divider("mmdc_ch0_axi_podf", "periph", base + 0x14, 19, 3, base + 0x48, 4);
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_MMDC_CH1_AXI_CG] = imx_clk_gate("mmdc_ch1_axi_cg", "periph2", base + 0x4, 18);
- clk[IMX6QDL_CLK_MMDC_CH1_AXI_PODF] = imx_clk_busy_divider("mmdc_ch1_axi_podf", "mmdc_ch1_axi_cg", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6QDL_CLK_MMDC_CH1_AXI_CG] = imx_clk_hw_gate("mmdc_ch1_axi_cg", "periph2", base + 0x4, 18);
+ hws[IMX6QDL_CLK_MMDC_CH1_AXI_PODF] = imx_clk_hw_busy_divider("mmdc_ch1_axi_podf", "mmdc_ch1_axi_cg", base + 0x14, 3, 3, base + 0x48, 2);
} else {
- clk[IMX6QDL_CLK_MMDC_CH1_AXI_PODF] = imx_clk_busy_divider("mmdc_ch1_axi_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6QDL_CLK_MMDC_CH1_AXI_PODF] = imx_clk_hw_busy_divider("mmdc_ch1_axi_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
}
- clk[IMX6QDL_CLK_ARM] = imx_clk_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
- clk[IMX6QDL_CLK_AHB] = imx_clk_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
+ hws[IMX6QDL_CLK_ARM] = imx_clk_hw_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
+ hws[IMX6QDL_CLK_AHB] = imx_clk_hw_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
/* name parent_name reg shift */
- clk[IMX6QDL_CLK_APBH_DMA] = imx_clk_gate2("apbh_dma", "usdhc3", base + 0x68, 4);
- clk[IMX6QDL_CLK_ASRC] = imx_clk_gate2_shared("asrc", "asrc_podf", base + 0x68, 6, &share_count_asrc);
- clk[IMX6QDL_CLK_ASRC_IPG] = imx_clk_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
- clk[IMX6QDL_CLK_ASRC_MEM] = imx_clk_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
- clk[IMX6QDL_CLK_CAAM_MEM] = imx_clk_gate2("caam_mem", "ahb", base + 0x68, 8);
- clk[IMX6QDL_CLK_CAAM_ACLK] = imx_clk_gate2("caam_aclk", "ahb", base + 0x68, 10);
- clk[IMX6QDL_CLK_CAAM_IPG] = imx_clk_gate2("caam_ipg", "ipg", base + 0x68, 12);
- clk[IMX6QDL_CLK_CAN1_IPG] = imx_clk_gate2("can1_ipg", "ipg", base + 0x68, 14);
- clk[IMX6QDL_CLK_CAN1_SERIAL] = imx_clk_gate2("can1_serial", "can_root", base + 0x68, 16);
- clk[IMX6QDL_CLK_CAN2_IPG] = imx_clk_gate2("can2_ipg", "ipg", base + 0x68, 18);
- clk[IMX6QDL_CLK_CAN2_SERIAL] = imx_clk_gate2("can2_serial", "can_root", base + 0x68, 20);
- clk[IMX6QDL_CLK_DCIC1] = imx_clk_gate2("dcic1", "ipu1_podf", base + 0x68, 24);
- clk[IMX6QDL_CLK_DCIC2] = imx_clk_gate2("dcic2", "ipu2_podf", base + 0x68, 26);
- clk[IMX6QDL_CLK_ECSPI1] = imx_clk_gate2("ecspi1", "ecspi_root", base + 0x6c, 0);
- clk[IMX6QDL_CLK_ECSPI2] = imx_clk_gate2("ecspi2", "ecspi_root", base + 0x6c, 2);
- clk[IMX6QDL_CLK_ECSPI3] = imx_clk_gate2("ecspi3", "ecspi_root", base + 0x6c, 4);
- clk[IMX6QDL_CLK_ECSPI4] = imx_clk_gate2("ecspi4", "ecspi_root", base + 0x6c, 6);
+ hws[IMX6QDL_CLK_APBH_DMA] = imx_clk_hw_gate2("apbh_dma", "usdhc3", base + 0x68, 4);
+ hws[IMX6QDL_CLK_ASRC] = imx_clk_hw_gate2_shared("asrc", "asrc_podf", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6QDL_CLK_ASRC_IPG] = imx_clk_hw_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6QDL_CLK_ASRC_MEM] = imx_clk_hw_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6QDL_CLK_CAAM_MEM] = imx_clk_hw_gate2("caam_mem", "ahb", base + 0x68, 8);
+ hws[IMX6QDL_CLK_CAAM_ACLK] = imx_clk_hw_gate2("caam_aclk", "ahb", base + 0x68, 10);
+ hws[IMX6QDL_CLK_CAAM_IPG] = imx_clk_hw_gate2("caam_ipg", "ipg", base + 0x68, 12);
+ hws[IMX6QDL_CLK_CAN1_IPG] = imx_clk_hw_gate2("can1_ipg", "ipg", base + 0x68, 14);
+ hws[IMX6QDL_CLK_CAN1_SERIAL] = imx_clk_hw_gate2("can1_serial", "can_root", base + 0x68, 16);
+ hws[IMX6QDL_CLK_CAN2_IPG] = imx_clk_hw_gate2("can2_ipg", "ipg", base + 0x68, 18);
+ hws[IMX6QDL_CLK_CAN2_SERIAL] = imx_clk_hw_gate2("can2_serial", "can_root", base + 0x68, 20);
+ hws[IMX6QDL_CLK_DCIC1] = imx_clk_hw_gate2("dcic1", "ipu1_podf", base + 0x68, 24);
+ hws[IMX6QDL_CLK_DCIC2] = imx_clk_hw_gate2("dcic2", "ipu2_podf", base + 0x68, 26);
+ hws[IMX6QDL_CLK_ECSPI1] = imx_clk_hw_gate2("ecspi1", "ecspi_root", base + 0x6c, 0);
+ hws[IMX6QDL_CLK_ECSPI2] = imx_clk_hw_gate2("ecspi2", "ecspi_root", base + 0x6c, 2);
+ hws[IMX6QDL_CLK_ECSPI3] = imx_clk_hw_gate2("ecspi3", "ecspi_root", base + 0x6c, 4);
+ hws[IMX6QDL_CLK_ECSPI4] = imx_clk_hw_gate2("ecspi4", "ecspi_root", base + 0x6c, 6);
if (clk_on_imx6dl())
- clk[IMX6DL_CLK_I2C4] = imx_clk_gate2("i2c4", "ipg_per", base + 0x6c, 8);
+ hws[IMX6DL_CLK_I2C4] = imx_clk_hw_gate2("i2c4", "ipg_per", base + 0x6c, 8);
else
- clk[IMX6Q_CLK_ECSPI5] = imx_clk_gate2("ecspi5", "ecspi_root", base + 0x6c, 8);
- clk[IMX6QDL_CLK_ENET] = imx_clk_gate2("enet", "ipg", base + 0x6c, 10);
- clk[IMX6QDL_CLK_EPIT1] = imx_clk_gate2("epit1", "ipg", base + 0x6c, 12);
- clk[IMX6QDL_CLK_EPIT2] = imx_clk_gate2("epit2", "ipg", base + 0x6c, 14);
- clk[IMX6QDL_CLK_ESAI_EXTAL] = imx_clk_gate2_shared("esai_extal", "esai_podf", base + 0x6c, 16, &share_count_esai);
- clk[IMX6QDL_CLK_ESAI_IPG] = imx_clk_gate2_shared("esai_ipg", "ahb", base + 0x6c, 16, &share_count_esai);
- clk[IMX6QDL_CLK_ESAI_MEM] = imx_clk_gate2_shared("esai_mem", "ahb", base + 0x6c, 16, &share_count_esai);
- clk[IMX6QDL_CLK_GPT_IPG] = imx_clk_gate2("gpt_ipg", "ipg", base + 0x6c, 20);
- clk[IMX6QDL_CLK_GPT_IPG_PER] = imx_clk_gate2("gpt_ipg_per", "ipg_per", base + 0x6c, 22);
- clk[IMX6QDL_CLK_GPU2D_CORE] = imx_clk_gate2("gpu2d_core", "gpu2d_core_podf", base + 0x6c, 24);
- clk[IMX6QDL_CLK_GPU3D_CORE] = imx_clk_gate2("gpu3d_core", "gpu3d_core_podf", base + 0x6c, 26);
- clk[IMX6QDL_CLK_HDMI_IAHB] = imx_clk_gate2("hdmi_iahb", "ahb", base + 0x70, 0);
- clk[IMX6QDL_CLK_HDMI_ISFR] = imx_clk_gate2("hdmi_isfr", "mipi_core_cfg", base + 0x70, 4);
- clk[IMX6QDL_CLK_I2C1] = imx_clk_gate2("i2c1", "ipg_per", base + 0x70, 6);
- clk[IMX6QDL_CLK_I2C2] = imx_clk_gate2("i2c2", "ipg_per", base + 0x70, 8);
- clk[IMX6QDL_CLK_I2C3] = imx_clk_gate2("i2c3", "ipg_per", base + 0x70, 10);
- clk[IMX6QDL_CLK_IIM] = imx_clk_gate2("iim", "ipg", base + 0x70, 12);
- clk[IMX6QDL_CLK_ENFC] = imx_clk_gate2("enfc", "enfc_podf", base + 0x70, 14);
- clk[IMX6QDL_CLK_VDOA] = imx_clk_gate2("vdoa", "vdo_axi", base + 0x70, 26);
- clk[IMX6QDL_CLK_IPU1] = imx_clk_gate2("ipu1", "ipu1_podf", base + 0x74, 0);
- clk[IMX6QDL_CLK_IPU1_DI0] = imx_clk_gate2("ipu1_di0", "ipu1_di0_sel", base + 0x74, 2);
- clk[IMX6QDL_CLK_IPU1_DI1] = imx_clk_gate2("ipu1_di1", "ipu1_di1_sel", base + 0x74, 4);
- clk[IMX6QDL_CLK_IPU2] = imx_clk_gate2("ipu2", "ipu2_podf", base + 0x74, 6);
- clk[IMX6QDL_CLK_IPU2_DI0] = imx_clk_gate2("ipu2_di0", "ipu2_di0_sel", base + 0x74, 8);
+ hws[IMX6Q_CLK_ECSPI5] = imx_clk_hw_gate2("ecspi5", "ecspi_root", base + 0x6c, 8);
+ hws[IMX6QDL_CLK_ENET] = imx_clk_hw_gate2("enet", "ipg", base + 0x6c, 10);
+ hws[IMX6QDL_CLK_EPIT1] = imx_clk_hw_gate2("epit1", "ipg", base + 0x6c, 12);
+ hws[IMX6QDL_CLK_EPIT2] = imx_clk_hw_gate2("epit2", "ipg", base + 0x6c, 14);
+ hws[IMX6QDL_CLK_ESAI_EXTAL] = imx_clk_hw_gate2_shared("esai_extal", "esai_podf", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6QDL_CLK_ESAI_IPG] = imx_clk_hw_gate2_shared("esai_ipg", "ahb", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6QDL_CLK_ESAI_MEM] = imx_clk_hw_gate2_shared("esai_mem", "ahb", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6QDL_CLK_GPT_IPG] = imx_clk_hw_gate2("gpt_ipg", "ipg", base + 0x6c, 20);
+ hws[IMX6QDL_CLK_GPT_IPG_PER] = imx_clk_hw_gate2("gpt_ipg_per", "ipg_per", base + 0x6c, 22);
+ hws[IMX6QDL_CLK_GPU2D_CORE] = imx_clk_hw_gate2("gpu2d_core", "gpu2d_core_podf", base + 0x6c, 24);
+ hws[IMX6QDL_CLK_GPU3D_CORE] = imx_clk_hw_gate2("gpu3d_core", "gpu3d_core_podf", base + 0x6c, 26);
+ hws[IMX6QDL_CLK_HDMI_IAHB] = imx_clk_hw_gate2("hdmi_iahb", "ahb", base + 0x70, 0);
+ hws[IMX6QDL_CLK_HDMI_ISFR] = imx_clk_hw_gate2("hdmi_isfr", "mipi_core_cfg", base + 0x70, 4);
+ hws[IMX6QDL_CLK_I2C1] = imx_clk_hw_gate2("i2c1", "ipg_per", base + 0x70, 6);
+ hws[IMX6QDL_CLK_I2C2] = imx_clk_hw_gate2("i2c2", "ipg_per", base + 0x70, 8);
+ hws[IMX6QDL_CLK_I2C3] = imx_clk_hw_gate2("i2c3", "ipg_per", base + 0x70, 10);
+ hws[IMX6QDL_CLK_IIM] = imx_clk_hw_gate2("iim", "ipg", base + 0x70, 12);
+ hws[IMX6QDL_CLK_ENFC] = imx_clk_hw_gate2("enfc", "enfc_podf", base + 0x70, 14);
+ hws[IMX6QDL_CLK_VDOA] = imx_clk_hw_gate2("vdoa", "vdo_axi", base + 0x70, 26);
+ hws[IMX6QDL_CLK_IPU1] = imx_clk_hw_gate2("ipu1", "ipu1_podf", base + 0x74, 0);
+ hws[IMX6QDL_CLK_IPU1_DI0] = imx_clk_hw_gate2("ipu1_di0", "ipu1_di0_sel", base + 0x74, 2);
+ hws[IMX6QDL_CLK_IPU1_DI1] = imx_clk_hw_gate2("ipu1_di1", "ipu1_di1_sel", base + 0x74, 4);
+ hws[IMX6QDL_CLK_IPU2] = imx_clk_hw_gate2("ipu2", "ipu2_podf", base + 0x74, 6);
+ hws[IMX6QDL_CLK_IPU2_DI0] = imx_clk_hw_gate2("ipu2_di0", "ipu2_di0_sel", base + 0x74, 8);
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_LDB_DI0] = imx_clk_gate2("ldb_di0", "ldb_di0_sel", base + 0x74, 12);
- clk[IMX6QDL_CLK_LDB_DI1] = imx_clk_gate2("ldb_di1", "ldb_di1_sel", base + 0x74, 14);
+ hws[IMX6QDL_CLK_LDB_DI0] = imx_clk_hw_gate2("ldb_di0", "ldb_di0_sel", base + 0x74, 12);
+ hws[IMX6QDL_CLK_LDB_DI1] = imx_clk_hw_gate2("ldb_di1", "ldb_di1_sel", base + 0x74, 14);
} else {
- clk[IMX6QDL_CLK_LDB_DI0] = imx_clk_gate2("ldb_di0", "ldb_di0_podf", base + 0x74, 12);
- clk[IMX6QDL_CLK_LDB_DI1] = imx_clk_gate2("ldb_di1", "ldb_di1_podf", base + 0x74, 14);
+ hws[IMX6QDL_CLK_LDB_DI0] = imx_clk_hw_gate2("ldb_di0", "ldb_di0_podf", base + 0x74, 12);
+ hws[IMX6QDL_CLK_LDB_DI1] = imx_clk_hw_gate2("ldb_di1", "ldb_di1_podf", base + 0x74, 14);
}
- clk[IMX6QDL_CLK_IPU2_DI1] = imx_clk_gate2("ipu2_di1", "ipu2_di1_sel", base + 0x74, 10);
- clk[IMX6QDL_CLK_HSI_TX] = imx_clk_gate2_shared("hsi_tx", "hsi_tx_podf", base + 0x74, 16, &share_count_mipi_core_cfg);
- clk[IMX6QDL_CLK_MIPI_CORE_CFG] = imx_clk_gate2_shared("mipi_core_cfg", "video_27m", base + 0x74, 16, &share_count_mipi_core_cfg);
- clk[IMX6QDL_CLK_MIPI_IPG] = imx_clk_gate2_shared("mipi_ipg", "ipg", base + 0x74, 16, &share_count_mipi_core_cfg);
+ hws[IMX6QDL_CLK_IPU2_DI1] = imx_clk_hw_gate2("ipu2_di1", "ipu2_di1_sel", base + 0x74, 10);
+ hws[IMX6QDL_CLK_HSI_TX] = imx_clk_hw_gate2_shared("hsi_tx", "hsi_tx_podf", base + 0x74, 16, &share_count_mipi_core_cfg);
+ hws[IMX6QDL_CLK_MIPI_CORE_CFG] = imx_clk_hw_gate2_shared("mipi_core_cfg", "video_27m", base + 0x74, 16, &share_count_mipi_core_cfg);
+ hws[IMX6QDL_CLK_MIPI_IPG] = imx_clk_hw_gate2_shared("mipi_ipg", "ipg", base + 0x74, 16, &share_count_mipi_core_cfg);
+
if (clk_on_imx6dl())
/*
* The multiplexer and divider of the imx6q clock gpu2d get
* redefined/reused as mlb_sys_sel and mlb_sys_clk_podf on imx6dl.
*/
- clk[IMX6QDL_CLK_MLB] = imx_clk_gate2("mlb", "mlb_podf", base + 0x74, 18);
+ hws[IMX6QDL_CLK_MLB] = imx_clk_hw_gate2("mlb", "mlb_podf", base + 0x74, 18);
else
- clk[IMX6QDL_CLK_MLB] = imx_clk_gate2("mlb", "axi", base + 0x74, 18);
- clk[IMX6QDL_CLK_MMDC_CH0_AXI] = imx_clk_gate2_flags("mmdc_ch0_axi", "mmdc_ch0_axi_podf", base + 0x74, 20, CLK_IS_CRITICAL);
- clk[IMX6QDL_CLK_MMDC_CH1_AXI] = imx_clk_gate2("mmdc_ch1_axi", "mmdc_ch1_axi_podf", base + 0x74, 22);
- clk[IMX6QDL_CLK_MMDC_P0_IPG] = imx_clk_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
- clk[IMX6QDL_CLK_OCRAM] = imx_clk_gate2("ocram", "ahb", base + 0x74, 28);
- clk[IMX6QDL_CLK_OPENVG_AXI] = imx_clk_gate2("openvg_axi", "axi", base + 0x74, 30);
- clk[IMX6QDL_CLK_PCIE_AXI] = imx_clk_gate2("pcie_axi", "pcie_axi_sel", base + 0x78, 0);
- clk[IMX6QDL_CLK_PER1_BCH] = imx_clk_gate2("per1_bch", "usdhc3", base + 0x78, 12);
- clk[IMX6QDL_CLK_PWM1] = imx_clk_gate2("pwm1", "ipg_per", base + 0x78, 16);
- clk[IMX6QDL_CLK_PWM2] = imx_clk_gate2("pwm2", "ipg_per", base + 0x78, 18);
- clk[IMX6QDL_CLK_PWM3] = imx_clk_gate2("pwm3", "ipg_per", base + 0x78, 20);
- clk[IMX6QDL_CLK_PWM4] = imx_clk_gate2("pwm4", "ipg_per", base + 0x78, 22);
- clk[IMX6QDL_CLK_GPMI_BCH_APB] = imx_clk_gate2("gpmi_bch_apb", "usdhc3", base + 0x78, 24);
- clk[IMX6QDL_CLK_GPMI_BCH] = imx_clk_gate2("gpmi_bch", "usdhc4", base + 0x78, 26);
- clk[IMX6QDL_CLK_GPMI_IO] = imx_clk_gate2("gpmi_io", "enfc", base + 0x78, 28);
- clk[IMX6QDL_CLK_GPMI_APB] = imx_clk_gate2("gpmi_apb", "usdhc3", base + 0x78, 30);
- clk[IMX6QDL_CLK_ROM] = imx_clk_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
- clk[IMX6QDL_CLK_SATA] = imx_clk_gate2("sata", "ahb", base + 0x7c, 4);
- clk[IMX6QDL_CLK_SDMA] = imx_clk_gate2("sdma", "ahb", base + 0x7c, 6);
- clk[IMX6QDL_CLK_SPBA] = imx_clk_gate2("spba", "ipg", base + 0x7c, 12);
- clk[IMX6QDL_CLK_SPDIF] = imx_clk_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_spdif);
- clk[IMX6QDL_CLK_SPDIF_GCLK] = imx_clk_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_spdif);
- clk[IMX6QDL_CLK_SSI1_IPG] = imx_clk_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
- clk[IMX6QDL_CLK_SSI2_IPG] = imx_clk_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
- clk[IMX6QDL_CLK_SSI3_IPG] = imx_clk_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
- clk[IMX6QDL_CLK_SSI1] = imx_clk_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
- clk[IMX6QDL_CLK_SSI2] = imx_clk_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
- clk[IMX6QDL_CLK_SSI3] = imx_clk_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
- clk[IMX6QDL_CLK_UART_IPG] = imx_clk_gate2("uart_ipg", "ipg", base + 0x7c, 24);
- clk[IMX6QDL_CLK_UART_SERIAL] = imx_clk_gate2("uart_serial", "uart_serial_podf", base + 0x7c, 26);
- clk[IMX6QDL_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
- clk[IMX6QDL_CLK_USDHC1] = imx_clk_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
- clk[IMX6QDL_CLK_USDHC2] = imx_clk_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
- clk[IMX6QDL_CLK_USDHC3] = imx_clk_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
- clk[IMX6QDL_CLK_USDHC4] = imx_clk_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
- clk[IMX6QDL_CLK_EIM_SLOW] = imx_clk_gate2("eim_slow", "eim_slow_podf", base + 0x80, 10);
- clk[IMX6QDL_CLK_VDO_AXI] = imx_clk_gate2("vdo_axi", "vdo_axi_sel", base + 0x80, 12);
- clk[IMX6QDL_CLK_VPU_AXI] = imx_clk_gate2("vpu_axi", "vpu_axi_podf", base + 0x80, 14);
+ hws[IMX6QDL_CLK_MLB] = imx_clk_hw_gate2("mlb", "axi", base + 0x74, 18);
+ hws[IMX6QDL_CLK_MMDC_CH0_AXI] = imx_clk_hw_gate2_flags("mmdc_ch0_axi", "mmdc_ch0_axi_podf", base + 0x74, 20, CLK_IS_CRITICAL);
+ hws[IMX6QDL_CLK_MMDC_CH1_AXI] = imx_clk_hw_gate2("mmdc_ch1_axi", "mmdc_ch1_axi_podf", base + 0x74, 22);
+ hws[IMX6QDL_CLK_MMDC_P0_IPG] = imx_clk_hw_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
+ hws[IMX6QDL_CLK_OCRAM] = imx_clk_hw_gate2("ocram", "ahb", base + 0x74, 28);
+ hws[IMX6QDL_CLK_OPENVG_AXI] = imx_clk_hw_gate2("openvg_axi", "axi", base + 0x74, 30);
+ hws[IMX6QDL_CLK_PCIE_AXI] = imx_clk_hw_gate2("pcie_axi", "pcie_axi_sel", base + 0x78, 0);
+ hws[IMX6QDL_CLK_PER1_BCH] = imx_clk_hw_gate2("per1_bch", "usdhc3", base + 0x78, 12);
+ hws[IMX6QDL_CLK_PWM1] = imx_clk_hw_gate2("pwm1", "ipg_per", base + 0x78, 16);
+ hws[IMX6QDL_CLK_PWM2] = imx_clk_hw_gate2("pwm2", "ipg_per", base + 0x78, 18);
+ hws[IMX6QDL_CLK_PWM3] = imx_clk_hw_gate2("pwm3", "ipg_per", base + 0x78, 20);
+ hws[IMX6QDL_CLK_PWM4] = imx_clk_hw_gate2("pwm4", "ipg_per", base + 0x78, 22);
+ hws[IMX6QDL_CLK_GPMI_BCH_APB] = imx_clk_hw_gate2("gpmi_bch_apb", "usdhc3", base + 0x78, 24);
+ hws[IMX6QDL_CLK_GPMI_BCH] = imx_clk_hw_gate2("gpmi_bch", "usdhc4", base + 0x78, 26);
+ hws[IMX6QDL_CLK_GPMI_IO] = imx_clk_hw_gate2("gpmi_io", "enfc", base + 0x78, 28);
+ hws[IMX6QDL_CLK_GPMI_APB] = imx_clk_hw_gate2("gpmi_apb", "usdhc3", base + 0x78, 30);
+ hws[IMX6QDL_CLK_ROM] = imx_clk_hw_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
+ hws[IMX6QDL_CLK_SATA] = imx_clk_hw_gate2("sata", "ahb", base + 0x7c, 4);
+ hws[IMX6QDL_CLK_SDMA] = imx_clk_hw_gate2("sdma", "ahb", base + 0x7c, 6);
+ hws[IMX6QDL_CLK_SPBA] = imx_clk_hw_gate2("spba", "ipg", base + 0x7c, 12);
+ hws[IMX6QDL_CLK_SPDIF] = imx_clk_hw_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_spdif);
+ hws[IMX6QDL_CLK_SPDIF_GCLK] = imx_clk_hw_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_spdif);
+ hws[IMX6QDL_CLK_SSI1_IPG] = imx_clk_hw_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6QDL_CLK_SSI2_IPG] = imx_clk_hw_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6QDL_CLK_SSI3_IPG] = imx_clk_hw_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6QDL_CLK_SSI1] = imx_clk_hw_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6QDL_CLK_SSI2] = imx_clk_hw_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6QDL_CLK_SSI3] = imx_clk_hw_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6QDL_CLK_UART_IPG] = imx_clk_hw_gate2("uart_ipg", "ipg", base + 0x7c, 24);
+ hws[IMX6QDL_CLK_UART_SERIAL] = imx_clk_hw_gate2("uart_serial", "uart_serial_podf", base + 0x7c, 26);
+ hws[IMX6QDL_CLK_USBOH3] = imx_clk_hw_gate2("usboh3", "ipg", base + 0x80, 0);
+ hws[IMX6QDL_CLK_USDHC1] = imx_clk_hw_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
+ hws[IMX6QDL_CLK_USDHC2] = imx_clk_hw_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
+ hws[IMX6QDL_CLK_USDHC3] = imx_clk_hw_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
+ hws[IMX6QDL_CLK_USDHC4] = imx_clk_hw_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
+ hws[IMX6QDL_CLK_EIM_SLOW] = imx_clk_hw_gate2("eim_slow", "eim_slow_podf", base + 0x80, 10);
+ hws[IMX6QDL_CLK_VDO_AXI] = imx_clk_hw_gate2("vdo_axi", "vdo_axi_sel", base + 0x80, 12);
+ hws[IMX6QDL_CLK_VPU_AXI] = imx_clk_hw_gate2("vpu_axi", "vpu_axi_podf", base + 0x80, 14);
if (clk_on_imx6qp()) {
- clk[IMX6QDL_CLK_PRE0] = imx_clk_gate2("pre0", "pre_axi", base + 0x80, 16);
- clk[IMX6QDL_CLK_PRE1] = imx_clk_gate2("pre1", "pre_axi", base + 0x80, 18);
- clk[IMX6QDL_CLK_PRE2] = imx_clk_gate2("pre2", "pre_axi", base + 0x80, 20);
- clk[IMX6QDL_CLK_PRE3] = imx_clk_gate2("pre3", "pre_axi", base + 0x80, 22);
- clk[IMX6QDL_CLK_PRG0_AXI] = imx_clk_gate2_shared("prg0_axi", "ipu1_podf", base + 0x80, 24, &share_count_prg0);
- clk[IMX6QDL_CLK_PRG1_AXI] = imx_clk_gate2_shared("prg1_axi", "ipu2_podf", base + 0x80, 26, &share_count_prg1);
- clk[IMX6QDL_CLK_PRG0_APB] = imx_clk_gate2_shared("prg0_apb", "ipg", base + 0x80, 24, &share_count_prg0);
- clk[IMX6QDL_CLK_PRG1_APB] = imx_clk_gate2_shared("prg1_apb", "ipg", base + 0x80, 26, &share_count_prg1);
+ hws[IMX6QDL_CLK_PRE0] = imx_clk_hw_gate2("pre0", "pre_axi", base + 0x80, 16);
+ hws[IMX6QDL_CLK_PRE1] = imx_clk_hw_gate2("pre1", "pre_axi", base + 0x80, 18);
+ hws[IMX6QDL_CLK_PRE2] = imx_clk_hw_gate2("pre2", "pre_axi", base + 0x80, 20);
+ hws[IMX6QDL_CLK_PRE3] = imx_clk_hw_gate2("pre3", "pre_axi", base + 0x80, 22);
+ hws[IMX6QDL_CLK_PRG0_AXI] = imx_clk_hw_gate2_shared("prg0_axi", "ipu1_podf", base + 0x80, 24, &share_count_prg0);
+ hws[IMX6QDL_CLK_PRG1_AXI] = imx_clk_hw_gate2_shared("prg1_axi", "ipu2_podf", base + 0x80, 26, &share_count_prg1);
+ hws[IMX6QDL_CLK_PRG0_APB] = imx_clk_hw_gate2_shared("prg0_apb", "ipg", base + 0x80, 24, &share_count_prg0);
+ hws[IMX6QDL_CLK_PRG1_APB] = imx_clk_hw_gate2_shared("prg1_apb", "ipg", base + 0x80, 26, &share_count_prg1);
}
- clk[IMX6QDL_CLK_CKO1] = imx_clk_gate("cko1", "cko1_podf", base + 0x60, 7);
- clk[IMX6QDL_CLK_CKO2] = imx_clk_gate("cko2", "cko2_podf", base + 0x60, 24);
+ hws[IMX6QDL_CLK_CKO1] = imx_clk_hw_gate("cko1", "cko1_podf", base + 0x60, 7);
+ hws[IMX6QDL_CLK_CKO2] = imx_clk_hw_gate("cko2", "cko2_podf", base + 0x60, 24);
/*
* The gpt_3m clock is not available on i.MX6Q TO1.0. Let's point it
* to clock gpt_ipg_per to ease the gpt driver code.
*/
if (clk_on_imx6q() && imx_get_soc_revision() == IMX_CHIP_REVISION_1_0)
- clk[IMX6QDL_CLK_GPT_3M] = clk[IMX6QDL_CLK_GPT_IPG_PER];
+ hws[IMX6QDL_CLK_GPT_3M] = hws[IMX6QDL_CLK_GPT_IPG_PER];
- imx_check_clocks(clk, ARRAY_SIZE(clk));
+ imx_check_clk_hws(hws, IMX6QDL_CLK_END);
- clk_data.clks = clk;
- clk_data.clk_num = ARRAY_SIZE(clk);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
- clk_register_clkdev(clk[IMX6QDL_CLK_ENET_REF], "enet_ref", NULL);
+ clk_hw_register_clkdev(hws[IMX6QDL_CLK_ENET_REF], "enet_ref", NULL);
- clk_set_rate(clk[IMX6QDL_CLK_PLL3_PFD1_540M], 540000000);
+ clk_set_rate(hws[IMX6QDL_CLK_PLL3_PFD1_540M]->clk, 540000000);
if (clk_on_imx6dl())
- clk_set_parent(clk[IMX6QDL_CLK_IPU1_SEL], clk[IMX6QDL_CLK_PLL3_PFD1_540M]);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU1_SEL]->clk, hws[IMX6QDL_CLK_PLL3_PFD1_540M]->clk);
- clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI1_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI0_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI1_PRE_SEL], clk[IMX6QDL_CLK_PLL5_VIDEO_DIV]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI0_SEL], clk[IMX6QDL_CLK_IPU1_DI0_PRE]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU1_DI1_SEL], clk[IMX6QDL_CLK_IPU1_DI1_PRE]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI0_SEL], clk[IMX6QDL_CLK_IPU2_DI0_PRE]);
- clk_set_parent(clk[IMX6QDL_CLK_IPU2_DI1_SEL], clk[IMX6QDL_CLK_IPU2_DI1_PRE]);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU1_DI0_PRE_SEL]->clk, hws[IMX6QDL_CLK_PLL5_VIDEO_DIV]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU1_DI1_PRE_SEL]->clk, hws[IMX6QDL_CLK_PLL5_VIDEO_DIV]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU2_DI0_PRE_SEL]->clk, hws[IMX6QDL_CLK_PLL5_VIDEO_DIV]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU2_DI1_PRE_SEL]->clk, hws[IMX6QDL_CLK_PLL5_VIDEO_DIV]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU1_DI0_SEL]->clk, hws[IMX6QDL_CLK_IPU1_DI0_PRE]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU1_DI1_SEL]->clk, hws[IMX6QDL_CLK_IPU1_DI1_PRE]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU2_DI0_SEL]->clk, hws[IMX6QDL_CLK_IPU2_DI0_PRE]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_IPU2_DI1_SEL]->clk, hws[IMX6QDL_CLK_IPU2_DI1_PRE]->clk);
/*
* The gpmi needs 100MHz frequency in the EDO/Sync mode,
* We can not get the 100MHz from the pll2_pfd0_352m.
* So choose pll2_pfd2_396m as enfc_sel's parent.
*/
- clk_set_parent(clk[IMX6QDL_CLK_ENFC_SEL], clk[IMX6QDL_CLK_PLL2_PFD2_396M]);
+ clk_set_parent(hws[IMX6QDL_CLK_ENFC_SEL]->clk, hws[IMX6QDL_CLK_PLL2_PFD2_396M]->clk);
if (IS_ENABLED(CONFIG_USB_MXS_PHY)) {
- clk_prepare_enable(clk[IMX6QDL_CLK_USBPHY1_GATE]);
- clk_prepare_enable(clk[IMX6QDL_CLK_USBPHY2_GATE]);
+ clk_prepare_enable(hws[IMX6QDL_CLK_USBPHY1_GATE]->clk);
+ clk_prepare_enable(hws[IMX6QDL_CLK_USBPHY2_GATE]->clk);
}
/*
* Let's initially set up CLKO with OSC24M, since this configuration
* is widely used by imx6q board designs to clock audio codec.
*/
- ret = clk_set_parent(clk[IMX6QDL_CLK_CKO2_SEL], clk[IMX6QDL_CLK_OSC]);
+ ret = clk_set_parent(hws[IMX6QDL_CLK_CKO2_SEL]->clk, hws[IMX6QDL_CLK_OSC]->clk);
if (!ret)
- ret = clk_set_parent(clk[IMX6QDL_CLK_CKO], clk[IMX6QDL_CLK_CKO2]);
+ ret = clk_set_parent(hws[IMX6QDL_CLK_CKO]->clk, hws[IMX6QDL_CLK_CKO2]->clk);
if (ret)
pr_warn("failed to set up CLKO: %d\n", ret);
/* Audio-related clocks configuration */
- clk_set_parent(clk[IMX6QDL_CLK_SPDIF_SEL], clk[IMX6QDL_CLK_PLL3_PFD3_454M]);
+ clk_set_parent(hws[IMX6QDL_CLK_SPDIF_SEL]->clk, hws[IMX6QDL_CLK_PLL3_PFD3_454M]->clk);
/* All existing boards with PCIe use LVDS1 */
if (IS_ENABLED(CONFIG_PCI_IMX6))
- clk_set_parent(clk[IMX6QDL_CLK_LVDS1_SEL], clk[IMX6QDL_CLK_SATA_REF_100M]);
+ clk_set_parent(hws[IMX6QDL_CLK_LVDS1_SEL]->clk, hws[IMX6QDL_CLK_SATA_REF_100M]->clk);
/*
* Initialize the GPU clock muxes, so that the maximum specified clock
* rates for the respective SoC are not exceeded.
*/
if (clk_on_imx6dl()) {
- clk_set_parent(clk[IMX6QDL_CLK_GPU3D_CORE_SEL],
- clk[IMX6QDL_CLK_PLL2_PFD1_594M]);
- clk_set_parent(clk[IMX6QDL_CLK_GPU2D_CORE_SEL],
- clk[IMX6QDL_CLK_PLL2_PFD1_594M]);
+ clk_set_parent(hws[IMX6QDL_CLK_GPU3D_CORE_SEL]->clk,
+ hws[IMX6QDL_CLK_PLL2_PFD1_594M]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_GPU2D_CORE_SEL]->clk,
+ hws[IMX6QDL_CLK_PLL2_PFD1_594M]->clk);
} else if (clk_on_imx6q()) {
- clk_set_parent(clk[IMX6QDL_CLK_GPU3D_CORE_SEL],
- clk[IMX6QDL_CLK_MMDC_CH0_AXI]);
- clk_set_parent(clk[IMX6QDL_CLK_GPU3D_SHADER_SEL],
- clk[IMX6QDL_CLK_PLL2_PFD1_594M]);
- clk_set_parent(clk[IMX6QDL_CLK_GPU2D_CORE_SEL],
- clk[IMX6QDL_CLK_PLL3_USB_OTG]);
+ clk_set_parent(hws[IMX6QDL_CLK_GPU3D_CORE_SEL]->clk,
+ hws[IMX6QDL_CLK_MMDC_CH0_AXI]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_GPU3D_SHADER_SEL]->clk,
+ hws[IMX6QDL_CLK_PLL2_PFD1_594M]->clk);
+ clk_set_parent(hws[IMX6QDL_CLK_GPU2D_CORE_SEL]->clk,
+ hws[IMX6QDL_CLK_PLL3_USB_OTG]->clk);
+ }
+
+ for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
+ int index = uart_clk_ids[i];
+
+ uart_clks[i] = &hws[index]->clk;
}
imx_register_uart_clocks(uart_clks);
#include "clk.h"
-#define CCDR 0x4
-#define BM_CCM_CCDR_MMDC_CH0_MASK (1 << 17)
#define CCSR 0xc
#define BM_CCSR_PLL1_SW_CLK_SEL (1 << 2)
#define CACRR 0x10
static unsigned int share_count_ssi3;
static unsigned int share_count_spdif;
-static struct clk *clks[IMX6SL_CLK_END];
-static struct clk_onecell_data clk_data;
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
static void __iomem *ccm_base;
static void __iomem *anatop_base;
imx6sl_enable_pll_arm(false);
}
-static struct clk ** const uart_clks[] __initconst = {
- &clks[IMX6SL_CLK_UART],
- &clks[IMX6SL_CLK_UART_SERIAL],
- NULL
+static const int uart_clk_ids[] __initconst = {
+ IMX6SL_CLK_UART,
+ IMX6SL_CLK_UART_SERIAL,
};
+static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
+
static void __init imx6sl_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
int ret;
+ int i;
+
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX6SL_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
+
+ clk_hw_data->num = IMX6SL_CLK_END;
+ hws = clk_hw_data->hws;
- clks[IMX6SL_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
- clks[IMX6SL_CLK_CKIL] = imx_obtain_fixed_clock("ckil", 0);
- clks[IMX6SL_CLK_OSC] = imx_obtain_fixed_clock("osc", 0);
+ hws[IMX6SL_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
+ hws[IMX6SL_CLK_CKIL] = imx_obtain_fixed_clock_hw("ckil", 0);
+ hws[IMX6SL_CLK_OSC] = imx_obtain_fixed_clock_hw("osc", 0);
/* Clock source from external clock via CLK1 PAD */
- clks[IMX6SL_CLK_ANACLK1] = imx_obtain_fixed_clock("anaclk1", 0);
+ hws[IMX6SL_CLK_ANACLK1] = imx_obtain_fixed_clock_hw("anaclk1", 0);
np = of_find_compatible_node(NULL, NULL, "fsl,imx6sl-anatop");
base = of_iomap(np, 0);
WARN_ON(!base);
anatop_base = base;
- clks[IMX6SL_PLL1_BYPASS_SRC] = imx_clk_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL2_BYPASS_SRC] = imx_clk_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL3_BYPASS_SRC] = imx_clk_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL4_BYPASS_SRC] = imx_clk_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL5_BYPASS_SRC] = imx_clk_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL6_BYPASS_SRC] = imx_clk_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SL_PLL7_BYPASS_SRC] = imx_clk_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL1_BYPASS_SRC] = imx_clk_hw_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL2_BYPASS_SRC] = imx_clk_hw_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL3_BYPASS_SRC] = imx_clk_hw_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL4_BYPASS_SRC] = imx_clk_hw_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL5_BYPASS_SRC] = imx_clk_hw_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL6_BYPASS_SRC] = imx_clk_hw_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SL_PLL7_BYPASS_SRC] = imx_clk_hw_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
/* type name parent_name base div_mask */
- clks[IMX6SL_CLK_PLL1] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
- clks[IMX6SL_CLK_PLL2] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
- clks[IMX6SL_CLK_PLL3] = imx_clk_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
- clks[IMX6SL_CLK_PLL4] = imx_clk_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
- clks[IMX6SL_CLK_PLL5] = imx_clk_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
- clks[IMX6SL_CLK_PLL6] = imx_clk_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
- clks[IMX6SL_CLK_PLL7] = imx_clk_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
-
- clks[IMX6SL_PLL1_BYPASS] = imx_clk_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL2_BYPASS] = imx_clk_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL3_BYPASS] = imx_clk_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL4_BYPASS] = imx_clk_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL5_BYPASS] = imx_clk_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL6_BYPASS] = imx_clk_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SL_PLL7_BYPASS] = imx_clk_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_CLK_PLL1] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
+ hws[IMX6SL_CLK_PLL2] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
+ hws[IMX6SL_CLK_PLL3] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
+ hws[IMX6SL_CLK_PLL4] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
+ hws[IMX6SL_CLK_PLL5] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
+ hws[IMX6SL_CLK_PLL6] = imx_clk_hw_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
+ hws[IMX6SL_CLK_PLL7] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
+
+ hws[IMX6SL_PLL1_BYPASS] = imx_clk_hw_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL2_BYPASS] = imx_clk_hw_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL3_BYPASS] = imx_clk_hw_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL4_BYPASS] = imx_clk_hw_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL5_BYPASS] = imx_clk_hw_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL6_BYPASS] = imx_clk_hw_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SL_PLL7_BYPASS] = imx_clk_hw_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
/* Do not bypass PLLs initially */
- clk_set_parent(clks[IMX6SL_PLL1_BYPASS], clks[IMX6SL_CLK_PLL1]);
- clk_set_parent(clks[IMX6SL_PLL2_BYPASS], clks[IMX6SL_CLK_PLL2]);
- clk_set_parent(clks[IMX6SL_PLL3_BYPASS], clks[IMX6SL_CLK_PLL3]);
- clk_set_parent(clks[IMX6SL_PLL4_BYPASS], clks[IMX6SL_CLK_PLL4]);
- clk_set_parent(clks[IMX6SL_PLL5_BYPASS], clks[IMX6SL_CLK_PLL5]);
- clk_set_parent(clks[IMX6SL_PLL6_BYPASS], clks[IMX6SL_CLK_PLL6]);
- clk_set_parent(clks[IMX6SL_PLL7_BYPASS], clks[IMX6SL_CLK_PLL7]);
-
- clks[IMX6SL_CLK_PLL1_SYS] = imx_clk_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
- clks[IMX6SL_CLK_PLL2_BUS] = imx_clk_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
- clks[IMX6SL_CLK_PLL3_USB_OTG] = imx_clk_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
- clks[IMX6SL_CLK_PLL4_AUDIO] = imx_clk_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
- clks[IMX6SL_CLK_PLL5_VIDEO] = imx_clk_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
- clks[IMX6SL_CLK_PLL6_ENET] = imx_clk_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
- clks[IMX6SL_CLK_PLL7_USB_HOST] = imx_clk_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
-
- clks[IMX6SL_CLK_LVDS1_SEL] = imx_clk_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
- clks[IMX6SL_CLK_LVDS1_OUT] = imx_clk_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x160, 10, BIT(12));
- clks[IMX6SL_CLK_LVDS1_IN] = imx_clk_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
+ clk_set_parent(hws[IMX6SL_PLL1_BYPASS]->clk, hws[IMX6SL_CLK_PLL1]->clk);
+ clk_set_parent(hws[IMX6SL_PLL2_BYPASS]->clk, hws[IMX6SL_CLK_PLL2]->clk);
+ clk_set_parent(hws[IMX6SL_PLL3_BYPASS]->clk, hws[IMX6SL_CLK_PLL3]->clk);
+ clk_set_parent(hws[IMX6SL_PLL4_BYPASS]->clk, hws[IMX6SL_CLK_PLL4]->clk);
+ clk_set_parent(hws[IMX6SL_PLL5_BYPASS]->clk, hws[IMX6SL_CLK_PLL5]->clk);
+ clk_set_parent(hws[IMX6SL_PLL6_BYPASS]->clk, hws[IMX6SL_CLK_PLL6]->clk);
+ clk_set_parent(hws[IMX6SL_PLL7_BYPASS]->clk, hws[IMX6SL_CLK_PLL7]->clk);
+
+ hws[IMX6SL_CLK_PLL1_SYS] = imx_clk_hw_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
+ hws[IMX6SL_CLK_PLL2_BUS] = imx_clk_hw_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
+ hws[IMX6SL_CLK_PLL3_USB_OTG] = imx_clk_hw_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
+ hws[IMX6SL_CLK_PLL4_AUDIO] = imx_clk_hw_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
+ hws[IMX6SL_CLK_PLL5_VIDEO] = imx_clk_hw_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
+ hws[IMX6SL_CLK_PLL6_ENET] = imx_clk_hw_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
+ hws[IMX6SL_CLK_PLL7_USB_HOST] = imx_clk_hw_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
+
+ hws[IMX6SL_CLK_LVDS1_SEL] = imx_clk_hw_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
+ hws[IMX6SL_CLK_LVDS1_OUT] = imx_clk_hw_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x160, 10, BIT(12));
+ hws[IMX6SL_CLK_LVDS1_IN] = imx_clk_hw_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
/*
* usbphy1 and usbphy2 are implemented as dummy gates using reserve
* turned on during boot, and software will not need to control it
* anymore after that.
*/
- clks[IMX6SL_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
- clks[IMX6SL_CLK_USBPHY2] = imx_clk_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
- clks[IMX6SL_CLK_USBPHY1_GATE] = imx_clk_gate("usbphy1_gate", "dummy", base + 0x10, 6);
- clks[IMX6SL_CLK_USBPHY2_GATE] = imx_clk_gate("usbphy2_gate", "dummy", base + 0x20, 6);
+ hws[IMX6SL_CLK_USBPHY1] = imx_clk_hw_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
+ hws[IMX6SL_CLK_USBPHY2] = imx_clk_hw_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
+ hws[IMX6SL_CLK_USBPHY1_GATE] = imx_clk_hw_gate("usbphy1_gate", "dummy", base + 0x10, 6);
+ hws[IMX6SL_CLK_USBPHY2_GATE] = imx_clk_hw_gate("usbphy2_gate", "dummy", base + 0x20, 6);
/* dev name parent_name flags reg shift width div: flags, div_table lock */
- clks[IMX6SL_CLK_PLL4_POST_DIV] = clk_register_divider_table(NULL, "pll4_post_div", "pll4_audio", CLK_SET_RATE_PARENT, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SL_CLK_PLL4_AUDIO_DIV] = clk_register_divider(NULL, "pll4_audio_div", "pll4_post_div", CLK_SET_RATE_PARENT, base + 0x170, 15, 1, 0, &imx_ccm_lock);
- clks[IMX6SL_CLK_PLL5_POST_DIV] = clk_register_divider_table(NULL, "pll5_post_div", "pll5_video", CLK_SET_RATE_PARENT, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SL_CLK_PLL5_VIDEO_DIV] = clk_register_divider_table(NULL, "pll5_video_div", "pll5_post_div", CLK_SET_RATE_PARENT, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
- clks[IMX6SL_CLK_ENET_REF] = clk_register_divider_table(NULL, "enet_ref", "pll6_enet", 0, base + 0xe0, 0, 2, 0, clk_enet_ref_table, &imx_ccm_lock);
+ hws[IMX6SL_CLK_PLL4_POST_DIV] = clk_hw_register_divider_table(NULL, "pll4_post_div", "pll4_audio", CLK_SET_RATE_PARENT, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
+ hws[IMX6SL_CLK_PLL4_AUDIO_DIV] = clk_hw_register_divider(NULL, "pll4_audio_div", "pll4_post_div", CLK_SET_RATE_PARENT, base + 0x170, 15, 1, 0, &imx_ccm_lock);
+ hws[IMX6SL_CLK_PLL5_POST_DIV] = clk_hw_register_divider_table(NULL, "pll5_post_div", "pll5_video", CLK_SET_RATE_PARENT, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
+ hws[IMX6SL_CLK_PLL5_VIDEO_DIV] = clk_hw_register_divider_table(NULL, "pll5_video_div", "pll5_post_div", CLK_SET_RATE_PARENT, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
+ hws[IMX6SL_CLK_ENET_REF] = clk_hw_register_divider_table(NULL, "enet_ref", "pll6_enet", 0, base + 0xe0, 0, 2, 0, clk_enet_ref_table, &imx_ccm_lock);
/* name parent_name reg idx */
- clks[IMX6SL_CLK_PLL2_PFD0] = imx_clk_pfd("pll2_pfd0", "pll2_bus", base + 0x100, 0);
- clks[IMX6SL_CLK_PLL2_PFD1] = imx_clk_pfd("pll2_pfd1", "pll2_bus", base + 0x100, 1);
- clks[IMX6SL_CLK_PLL2_PFD2] = imx_clk_pfd("pll2_pfd2", "pll2_bus", base + 0x100, 2);
- clks[IMX6SL_CLK_PLL3_PFD0] = imx_clk_pfd("pll3_pfd0", "pll3_usb_otg", base + 0xf0, 0);
- clks[IMX6SL_CLK_PLL3_PFD1] = imx_clk_pfd("pll3_pfd1", "pll3_usb_otg", base + 0xf0, 1);
- clks[IMX6SL_CLK_PLL3_PFD2] = imx_clk_pfd("pll3_pfd2", "pll3_usb_otg", base + 0xf0, 2);
- clks[IMX6SL_CLK_PLL3_PFD3] = imx_clk_pfd("pll3_pfd3", "pll3_usb_otg", base + 0xf0, 3);
+ hws[IMX6SL_CLK_PLL2_PFD0] = imx_clk_hw_pfd("pll2_pfd0", "pll2_bus", base + 0x100, 0);
+ hws[IMX6SL_CLK_PLL2_PFD1] = imx_clk_hw_pfd("pll2_pfd1", "pll2_bus", base + 0x100, 1);
+ hws[IMX6SL_CLK_PLL2_PFD2] = imx_clk_hw_pfd("pll2_pfd2", "pll2_bus", base + 0x100, 2);
+ hws[IMX6SL_CLK_PLL3_PFD0] = imx_clk_hw_pfd("pll3_pfd0", "pll3_usb_otg", base + 0xf0, 0);
+ hws[IMX6SL_CLK_PLL3_PFD1] = imx_clk_hw_pfd("pll3_pfd1", "pll3_usb_otg", base + 0xf0, 1);
+ hws[IMX6SL_CLK_PLL3_PFD2] = imx_clk_hw_pfd("pll3_pfd2", "pll3_usb_otg", base + 0xf0, 2);
+ hws[IMX6SL_CLK_PLL3_PFD3] = imx_clk_hw_pfd("pll3_pfd3", "pll3_usb_otg", base + 0xf0, 3);
/* name parent_name mult div */
- clks[IMX6SL_CLK_PLL2_198M] = imx_clk_fixed_factor("pll2_198m", "pll2_pfd2", 1, 2);
- clks[IMX6SL_CLK_PLL3_120M] = imx_clk_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
- clks[IMX6SL_CLK_PLL3_80M] = imx_clk_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
- clks[IMX6SL_CLK_PLL3_60M] = imx_clk_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
+ hws[IMX6SL_CLK_PLL2_198M] = imx_clk_hw_fixed_factor("pll2_198m", "pll2_pfd2", 1, 2);
+ hws[IMX6SL_CLK_PLL3_120M] = imx_clk_hw_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
+ hws[IMX6SL_CLK_PLL3_80M] = imx_clk_hw_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
+ hws[IMX6SL_CLK_PLL3_60M] = imx_clk_hw_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
np = ccm_node;
base = of_iomap(np, 0);
ccm_base = base;
/* name reg shift width parent_names num_parents */
- clks[IMX6SL_CLK_STEP] = imx_clk_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
- clks[IMX6SL_CLK_PLL1_SW] = imx_clk_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
- clks[IMX6SL_CLK_OCRAM_ALT_SEL] = imx_clk_mux("ocram_alt_sel", base + 0x14, 7, 1, ocram_alt_sels, ARRAY_SIZE(ocram_alt_sels));
- clks[IMX6SL_CLK_OCRAM_SEL] = imx_clk_mux("ocram_sel", base + 0x14, 6, 1, ocram_sels, ARRAY_SIZE(ocram_sels));
- clks[IMX6SL_CLK_PRE_PERIPH2_SEL] = imx_clk_mux("pre_periph2_sel", base + 0x18, 21, 2, pre_periph_sels, ARRAY_SIZE(pre_periph_sels));
- clks[IMX6SL_CLK_PRE_PERIPH_SEL] = imx_clk_mux("pre_periph_sel", base + 0x18, 18, 2, pre_periph_sels, ARRAY_SIZE(pre_periph_sels));
- clks[IMX6SL_CLK_PERIPH2_CLK2_SEL] = imx_clk_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
- clks[IMX6SL_CLK_PERIPH_CLK2_SEL] = imx_clk_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
- clks[IMX6SL_CLK_CSI_SEL] = imx_clk_mux("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels));
- clks[IMX6SL_CLK_LCDIF_AXI_SEL] = imx_clk_mux("lcdif_axi_sel", base + 0x3c, 14, 2, lcdif_axi_sels, ARRAY_SIZE(lcdif_axi_sels));
- clks[IMX6SL_CLK_USDHC1_SEL] = imx_clk_fixup_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_USDHC2_SEL] = imx_clk_fixup_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_USDHC3_SEL] = imx_clk_fixup_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_USDHC4_SEL] = imx_clk_fixup_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_SSI1_SEL] = imx_clk_fixup_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_SSI2_SEL] = imx_clk_fixup_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_SSI3_SEL] = imx_clk_fixup_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_PERCLK_SEL] = imx_clk_fixup_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels), imx_cscmr1_fixup);
- clks[IMX6SL_CLK_PXP_AXI_SEL] = imx_clk_mux("pxp_axi_sel", base + 0x34, 6, 3, pxp_axi_sels, ARRAY_SIZE(pxp_axi_sels));
- clks[IMX6SL_CLK_EPDC_AXI_SEL] = imx_clk_mux("epdc_axi_sel", base + 0x34, 15, 3, epdc_axi_sels, ARRAY_SIZE(epdc_axi_sels));
- clks[IMX6SL_CLK_GPU2D_OVG_SEL] = imx_clk_mux("gpu2d_ovg_sel", base + 0x18, 4, 2, gpu2d_ovg_sels, ARRAY_SIZE(gpu2d_ovg_sels));
- clks[IMX6SL_CLK_GPU2D_SEL] = imx_clk_mux("gpu2d_sel", base + 0x18, 8, 2, gpu2d_sels, ARRAY_SIZE(gpu2d_sels));
- clks[IMX6SL_CLK_LCDIF_PIX_SEL] = imx_clk_mux("lcdif_pix_sel", base + 0x38, 6, 3, lcdif_pix_sels, ARRAY_SIZE(lcdif_pix_sels));
- clks[IMX6SL_CLK_EPDC_PIX_SEL] = imx_clk_mux("epdc_pix_sel", base + 0x38, 15, 3, epdc_pix_sels, ARRAY_SIZE(epdc_pix_sels));
- clks[IMX6SL_CLK_SPDIF0_SEL] = imx_clk_mux("spdif0_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SL_CLK_SPDIF1_SEL] = imx_clk_mux("spdif1_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SL_CLK_EXTERN_AUDIO_SEL] = imx_clk_mux("extern_audio_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SL_CLK_ECSPI_SEL] = imx_clk_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
- clks[IMX6SL_CLK_UART_SEL] = imx_clk_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
+ hws[IMX6SL_CLK_STEP] = imx_clk_hw_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
+ hws[IMX6SL_CLK_PLL1_SW] = imx_clk_hw_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
+ hws[IMX6SL_CLK_OCRAM_ALT_SEL] = imx_clk_hw_mux("ocram_alt_sel", base + 0x14, 7, 1, ocram_alt_sels, ARRAY_SIZE(ocram_alt_sels));
+ hws[IMX6SL_CLK_OCRAM_SEL] = imx_clk_hw_mux("ocram_sel", base + 0x14, 6, 1, ocram_sels, ARRAY_SIZE(ocram_sels));
+ hws[IMX6SL_CLK_PRE_PERIPH2_SEL] = imx_clk_hw_mux("pre_periph2_sel", base + 0x18, 21, 2, pre_periph_sels, ARRAY_SIZE(pre_periph_sels));
+ hws[IMX6SL_CLK_PRE_PERIPH_SEL] = imx_clk_hw_mux("pre_periph_sel", base + 0x18, 18, 2, pre_periph_sels, ARRAY_SIZE(pre_periph_sels));
+ hws[IMX6SL_CLK_PERIPH2_CLK2_SEL] = imx_clk_hw_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
+ hws[IMX6SL_CLK_PERIPH_CLK2_SEL] = imx_clk_hw_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
+ hws[IMX6SL_CLK_CSI_SEL] = imx_clk_hw_mux("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels));
+ hws[IMX6SL_CLK_LCDIF_AXI_SEL] = imx_clk_hw_mux("lcdif_axi_sel", base + 0x3c, 14, 2, lcdif_axi_sels, ARRAY_SIZE(lcdif_axi_sels));
+ hws[IMX6SL_CLK_USDHC1_SEL] = imx_clk_hw_fixup_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_USDHC2_SEL] = imx_clk_hw_fixup_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_USDHC3_SEL] = imx_clk_hw_fixup_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_USDHC4_SEL] = imx_clk_hw_fixup_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_SSI1_SEL] = imx_clk_hw_fixup_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_SSI2_SEL] = imx_clk_hw_fixup_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_SSI3_SEL] = imx_clk_hw_fixup_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_PERCLK_SEL] = imx_clk_hw_fixup_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels), imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_PXP_AXI_SEL] = imx_clk_hw_mux("pxp_axi_sel", base + 0x34, 6, 3, pxp_axi_sels, ARRAY_SIZE(pxp_axi_sels));
+ hws[IMX6SL_CLK_EPDC_AXI_SEL] = imx_clk_hw_mux("epdc_axi_sel", base + 0x34, 15, 3, epdc_axi_sels, ARRAY_SIZE(epdc_axi_sels));
+ hws[IMX6SL_CLK_GPU2D_OVG_SEL] = imx_clk_hw_mux("gpu2d_ovg_sel", base + 0x18, 4, 2, gpu2d_ovg_sels, ARRAY_SIZE(gpu2d_ovg_sels));
+ hws[IMX6SL_CLK_GPU2D_SEL] = imx_clk_hw_mux("gpu2d_sel", base + 0x18, 8, 2, gpu2d_sels, ARRAY_SIZE(gpu2d_sels));
+ hws[IMX6SL_CLK_LCDIF_PIX_SEL] = imx_clk_hw_mux("lcdif_pix_sel", base + 0x38, 6, 3, lcdif_pix_sels, ARRAY_SIZE(lcdif_pix_sels));
+ hws[IMX6SL_CLK_EPDC_PIX_SEL] = imx_clk_hw_mux("epdc_pix_sel", base + 0x38, 15, 3, epdc_pix_sels, ARRAY_SIZE(epdc_pix_sels));
+ hws[IMX6SL_CLK_SPDIF0_SEL] = imx_clk_hw_mux("spdif0_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SL_CLK_SPDIF1_SEL] = imx_clk_hw_mux("spdif1_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SL_CLK_EXTERN_AUDIO_SEL] = imx_clk_hw_mux("extern_audio_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SL_CLK_ECSPI_SEL] = imx_clk_hw_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
+ hws[IMX6SL_CLK_UART_SEL] = imx_clk_hw_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
/* name reg shift width busy: reg, shift parent_names num_parents */
- clks[IMX6SL_CLK_PERIPH] = imx_clk_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
- clks[IMX6SL_CLK_PERIPH2] = imx_clk_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
+ hws[IMX6SL_CLK_PERIPH] = imx_clk_hw_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
+ hws[IMX6SL_CLK_PERIPH2] = imx_clk_hw_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
/* name parent_name reg shift width */
- clks[IMX6SL_CLK_OCRAM_PODF] = imx_clk_busy_divider("ocram_podf", "ocram_sel", base + 0x14, 16, 3, base + 0x48, 0);
- clks[IMX6SL_CLK_PERIPH_CLK2_PODF] = imx_clk_divider("periph_clk2_podf", "periph_clk2_sel", base + 0x14, 27, 3);
- clks[IMX6SL_CLK_PERIPH2_CLK2_PODF] = imx_clk_divider("periph2_clk2_podf", "periph2_clk2_sel", base + 0x14, 0, 3);
- clks[IMX6SL_CLK_IPG] = imx_clk_divider("ipg", "ahb", base + 0x14, 8, 2);
- clks[IMX6SL_CLK_CSI_PODF] = imx_clk_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
- clks[IMX6SL_CLK_LCDIF_AXI_PODF] = imx_clk_divider("lcdif_axi_podf", "lcdif_axi_sel", base + 0x3c, 16, 3);
- clks[IMX6SL_CLK_USDHC1_PODF] = imx_clk_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
- clks[IMX6SL_CLK_USDHC2_PODF] = imx_clk_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
- clks[IMX6SL_CLK_USDHC3_PODF] = imx_clk_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
- clks[IMX6SL_CLK_USDHC4_PODF] = imx_clk_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
- clks[IMX6SL_CLK_SSI1_PRED] = imx_clk_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
- clks[IMX6SL_CLK_SSI1_PODF] = imx_clk_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
- clks[IMX6SL_CLK_SSI2_PRED] = imx_clk_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
- clks[IMX6SL_CLK_SSI2_PODF] = imx_clk_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
- clks[IMX6SL_CLK_SSI3_PRED] = imx_clk_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
- clks[IMX6SL_CLK_SSI3_PODF] = imx_clk_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
- clks[IMX6SL_CLK_PERCLK] = imx_clk_fixup_divider("perclk", "perclk_sel", base + 0x1c, 0, 6, imx_cscmr1_fixup);
- clks[IMX6SL_CLK_PXP_AXI_PODF] = imx_clk_divider("pxp_axi_podf", "pxp_axi_sel", base + 0x34, 3, 3);
- clks[IMX6SL_CLK_EPDC_AXI_PODF] = imx_clk_divider("epdc_axi_podf", "epdc_axi_sel", base + 0x34, 12, 3);
- clks[IMX6SL_CLK_GPU2D_OVG_PODF] = imx_clk_divider("gpu2d_ovg_podf", "gpu2d_ovg_sel", base + 0x18, 26, 3);
- clks[IMX6SL_CLK_GPU2D_PODF] = imx_clk_divider("gpu2d_podf", "gpu2d_sel", base + 0x18, 29, 3);
- clks[IMX6SL_CLK_LCDIF_PIX_PRED] = imx_clk_divider("lcdif_pix_pred", "lcdif_pix_sel", base + 0x38, 3, 3);
- clks[IMX6SL_CLK_EPDC_PIX_PRED] = imx_clk_divider("epdc_pix_pred", "epdc_pix_sel", base + 0x38, 12, 3);
- clks[IMX6SL_CLK_LCDIF_PIX_PODF] = imx_clk_fixup_divider("lcdif_pix_podf", "lcdif_pix_pred", base + 0x1c, 20, 3, imx_cscmr1_fixup);
- clks[IMX6SL_CLK_EPDC_PIX_PODF] = imx_clk_divider("epdc_pix_podf", "epdc_pix_pred", base + 0x18, 23, 3);
- clks[IMX6SL_CLK_SPDIF0_PRED] = imx_clk_divider("spdif0_pred", "spdif0_sel", base + 0x30, 25, 3);
- clks[IMX6SL_CLK_SPDIF0_PODF] = imx_clk_divider("spdif0_podf", "spdif0_pred", base + 0x30, 22, 3);
- clks[IMX6SL_CLK_SPDIF1_PRED] = imx_clk_divider("spdif1_pred", "spdif1_sel", base + 0x30, 12, 3);
- clks[IMX6SL_CLK_SPDIF1_PODF] = imx_clk_divider("spdif1_podf", "spdif1_pred", base + 0x30, 9, 3);
- clks[IMX6SL_CLK_EXTERN_AUDIO_PRED] = imx_clk_divider("extern_audio_pred", "extern_audio_sel", base + 0x28, 9, 3);
- clks[IMX6SL_CLK_EXTERN_AUDIO_PODF] = imx_clk_divider("extern_audio_podf", "extern_audio_pred", base + 0x28, 25, 3);
- clks[IMX6SL_CLK_ECSPI_ROOT] = imx_clk_divider("ecspi_root", "ecspi_sel", base + 0x38, 19, 6);
- clks[IMX6SL_CLK_UART_ROOT] = imx_clk_divider("uart_root", "uart_sel", base + 0x24, 0, 6);
+ hws[IMX6SL_CLK_OCRAM_PODF] = imx_clk_hw_busy_divider("ocram_podf", "ocram_sel", base + 0x14, 16, 3, base + 0x48, 0);
+ hws[IMX6SL_CLK_PERIPH_CLK2_PODF] = imx_clk_hw_divider("periph_clk2_podf", "periph_clk2_sel", base + 0x14, 27, 3);
+ hws[IMX6SL_CLK_PERIPH2_CLK2_PODF] = imx_clk_hw_divider("periph2_clk2_podf", "periph2_clk2_sel", base + 0x14, 0, 3);
+ hws[IMX6SL_CLK_IPG] = imx_clk_hw_divider("ipg", "ahb", base + 0x14, 8, 2);
+ hws[IMX6SL_CLK_CSI_PODF] = imx_clk_hw_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
+ hws[IMX6SL_CLK_LCDIF_AXI_PODF] = imx_clk_hw_divider("lcdif_axi_podf", "lcdif_axi_sel", base + 0x3c, 16, 3);
+ hws[IMX6SL_CLK_USDHC1_PODF] = imx_clk_hw_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
+ hws[IMX6SL_CLK_USDHC2_PODF] = imx_clk_hw_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
+ hws[IMX6SL_CLK_USDHC3_PODF] = imx_clk_hw_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
+ hws[IMX6SL_CLK_USDHC4_PODF] = imx_clk_hw_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
+ hws[IMX6SL_CLK_SSI1_PRED] = imx_clk_hw_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
+ hws[IMX6SL_CLK_SSI1_PODF] = imx_clk_hw_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
+ hws[IMX6SL_CLK_SSI2_PRED] = imx_clk_hw_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
+ hws[IMX6SL_CLK_SSI2_PODF] = imx_clk_hw_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
+ hws[IMX6SL_CLK_SSI3_PRED] = imx_clk_hw_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
+ hws[IMX6SL_CLK_SSI3_PODF] = imx_clk_hw_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
+ hws[IMX6SL_CLK_PERCLK] = imx_clk_hw_fixup_divider("perclk", "perclk_sel", base + 0x1c, 0, 6, imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_PXP_AXI_PODF] = imx_clk_hw_divider("pxp_axi_podf", "pxp_axi_sel", base + 0x34, 3, 3);
+ hws[IMX6SL_CLK_EPDC_AXI_PODF] = imx_clk_hw_divider("epdc_axi_podf", "epdc_axi_sel", base + 0x34, 12, 3);
+ hws[IMX6SL_CLK_GPU2D_OVG_PODF] = imx_clk_hw_divider("gpu2d_ovg_podf", "gpu2d_ovg_sel", base + 0x18, 26, 3);
+ hws[IMX6SL_CLK_GPU2D_PODF] = imx_clk_hw_divider("gpu2d_podf", "gpu2d_sel", base + 0x18, 29, 3);
+ hws[IMX6SL_CLK_LCDIF_PIX_PRED] = imx_clk_hw_divider("lcdif_pix_pred", "lcdif_pix_sel", base + 0x38, 3, 3);
+ hws[IMX6SL_CLK_EPDC_PIX_PRED] = imx_clk_hw_divider("epdc_pix_pred", "epdc_pix_sel", base + 0x38, 12, 3);
+ hws[IMX6SL_CLK_LCDIF_PIX_PODF] = imx_clk_hw_fixup_divider("lcdif_pix_podf", "lcdif_pix_pred", base + 0x1c, 20, 3, imx_cscmr1_fixup);
+ hws[IMX6SL_CLK_EPDC_PIX_PODF] = imx_clk_hw_divider("epdc_pix_podf", "epdc_pix_pred", base + 0x18, 23, 3);
+ hws[IMX6SL_CLK_SPDIF0_PRED] = imx_clk_hw_divider("spdif0_pred", "spdif0_sel", base + 0x30, 25, 3);
+ hws[IMX6SL_CLK_SPDIF0_PODF] = imx_clk_hw_divider("spdif0_podf", "spdif0_pred", base + 0x30, 22, 3);
+ hws[IMX6SL_CLK_SPDIF1_PRED] = imx_clk_hw_divider("spdif1_pred", "spdif1_sel", base + 0x30, 12, 3);
+ hws[IMX6SL_CLK_SPDIF1_PODF] = imx_clk_hw_divider("spdif1_podf", "spdif1_pred", base + 0x30, 9, 3);
+ hws[IMX6SL_CLK_EXTERN_AUDIO_PRED] = imx_clk_hw_divider("extern_audio_pred", "extern_audio_sel", base + 0x28, 9, 3);
+ hws[IMX6SL_CLK_EXTERN_AUDIO_PODF] = imx_clk_hw_divider("extern_audio_podf", "extern_audio_pred", base + 0x28, 25, 3);
+ hws[IMX6SL_CLK_ECSPI_ROOT] = imx_clk_hw_divider("ecspi_root", "ecspi_sel", base + 0x38, 19, 6);
+ hws[IMX6SL_CLK_UART_ROOT] = imx_clk_hw_divider("uart_root", "uart_sel", base + 0x24, 0, 6);
/* name parent_name reg shift width busy: reg, shift */
- clks[IMX6SL_CLK_AHB] = imx_clk_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
- clks[IMX6SL_CLK_MMDC_ROOT] = imx_clk_busy_divider("mmdc", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
- clks[IMX6SL_CLK_ARM] = imx_clk_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
+ hws[IMX6SL_CLK_AHB] = imx_clk_hw_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
+ hws[IMX6SL_CLK_MMDC_ROOT] = imx_clk_hw_busy_divider("mmdc", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6SL_CLK_ARM] = imx_clk_hw_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
/* name parent_name reg shift */
- clks[IMX6SL_CLK_ECSPI1] = imx_clk_gate2("ecspi1", "ecspi_root", base + 0x6c, 0);
- clks[IMX6SL_CLK_ECSPI2] = imx_clk_gate2("ecspi2", "ecspi_root", base + 0x6c, 2);
- clks[IMX6SL_CLK_ECSPI3] = imx_clk_gate2("ecspi3", "ecspi_root", base + 0x6c, 4);
- clks[IMX6SL_CLK_ECSPI4] = imx_clk_gate2("ecspi4", "ecspi_root", base + 0x6c, 6);
- clks[IMX6SL_CLK_ENET] = imx_clk_gate2("enet", "ipg", base + 0x6c, 10);
- clks[IMX6SL_CLK_EPIT1] = imx_clk_gate2("epit1", "perclk", base + 0x6c, 12);
- clks[IMX6SL_CLK_EPIT2] = imx_clk_gate2("epit2", "perclk", base + 0x6c, 14);
- clks[IMX6SL_CLK_EXTERN_AUDIO] = imx_clk_gate2("extern_audio", "extern_audio_podf", base + 0x6c, 16);
- clks[IMX6SL_CLK_GPT] = imx_clk_gate2("gpt", "perclk", base + 0x6c, 20);
- clks[IMX6SL_CLK_GPT_SERIAL] = imx_clk_gate2("gpt_serial", "perclk", base + 0x6c, 22);
- clks[IMX6SL_CLK_GPU2D_OVG] = imx_clk_gate2("gpu2d_ovg", "gpu2d_ovg_podf", base + 0x6c, 26);
- clks[IMX6SL_CLK_I2C1] = imx_clk_gate2("i2c1", "perclk", base + 0x70, 6);
- clks[IMX6SL_CLK_I2C2] = imx_clk_gate2("i2c2", "perclk", base + 0x70, 8);
- clks[IMX6SL_CLK_I2C3] = imx_clk_gate2("i2c3", "perclk", base + 0x70, 10);
- clks[IMX6SL_CLK_OCOTP] = imx_clk_gate2("ocotp", "ipg", base + 0x70, 12);
- clks[IMX6SL_CLK_CSI] = imx_clk_gate2("csi", "csi_podf", base + 0x74, 0);
- clks[IMX6SL_CLK_PXP_AXI] = imx_clk_gate2("pxp_axi", "pxp_axi_podf", base + 0x74, 2);
- clks[IMX6SL_CLK_EPDC_AXI] = imx_clk_gate2("epdc_axi", "epdc_axi_podf", base + 0x74, 4);
- clks[IMX6SL_CLK_LCDIF_AXI] = imx_clk_gate2("lcdif_axi", "lcdif_axi_podf", base + 0x74, 6);
- clks[IMX6SL_CLK_LCDIF_PIX] = imx_clk_gate2("lcdif_pix", "lcdif_pix_podf", base + 0x74, 8);
- clks[IMX6SL_CLK_EPDC_PIX] = imx_clk_gate2("epdc_pix", "epdc_pix_podf", base + 0x74, 10);
- clks[IMX6SL_CLK_MMDC_P0_IPG] = imx_clk_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
- clks[IMX6SL_CLK_MMDC_P1_IPG] = imx_clk_gate2("mmdc_p1_ipg", "ipg", base + 0x74, 26);
- clks[IMX6SL_CLK_OCRAM] = imx_clk_gate2("ocram", "ocram_podf", base + 0x74, 28);
- clks[IMX6SL_CLK_PWM1] = imx_clk_gate2("pwm1", "perclk", base + 0x78, 16);
- clks[IMX6SL_CLK_PWM2] = imx_clk_gate2("pwm2", "perclk", base + 0x78, 18);
- clks[IMX6SL_CLK_PWM3] = imx_clk_gate2("pwm3", "perclk", base + 0x78, 20);
- clks[IMX6SL_CLK_PWM4] = imx_clk_gate2("pwm4", "perclk", base + 0x78, 22);
- clks[IMX6SL_CLK_SDMA] = imx_clk_gate2("sdma", "ipg", base + 0x7c, 6);
- clks[IMX6SL_CLK_SPBA] = imx_clk_gate2("spba", "ipg", base + 0x7c, 12);
- clks[IMX6SL_CLK_SPDIF] = imx_clk_gate2_shared("spdif", "spdif0_podf", base + 0x7c, 14, &share_count_spdif);
- clks[IMX6SL_CLK_SPDIF_GCLK] = imx_clk_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_spdif);
- clks[IMX6SL_CLK_SSI1_IPG] = imx_clk_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SL_CLK_SSI2_IPG] = imx_clk_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SL_CLK_SSI3_IPG] = imx_clk_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SL_CLK_SSI1] = imx_clk_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SL_CLK_SSI2] = imx_clk_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SL_CLK_SSI3] = imx_clk_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SL_CLK_UART] = imx_clk_gate2("uart", "ipg", base + 0x7c, 24);
- clks[IMX6SL_CLK_UART_SERIAL] = imx_clk_gate2("uart_serial", "uart_root", base + 0x7c, 26);
- clks[IMX6SL_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
- clks[IMX6SL_CLK_USDHC1] = imx_clk_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
- clks[IMX6SL_CLK_USDHC2] = imx_clk_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
- clks[IMX6SL_CLK_USDHC3] = imx_clk_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
- clks[IMX6SL_CLK_USDHC4] = imx_clk_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
+ hws[IMX6SL_CLK_ECSPI1] = imx_clk_hw_gate2("ecspi1", "ecspi_root", base + 0x6c, 0);
+ hws[IMX6SL_CLK_ECSPI2] = imx_clk_hw_gate2("ecspi2", "ecspi_root", base + 0x6c, 2);
+ hws[IMX6SL_CLK_ECSPI3] = imx_clk_hw_gate2("ecspi3", "ecspi_root", base + 0x6c, 4);
+ hws[IMX6SL_CLK_ECSPI4] = imx_clk_hw_gate2("ecspi4", "ecspi_root", base + 0x6c, 6);
+ hws[IMX6SL_CLK_ENET] = imx_clk_hw_gate2("enet", "ipg", base + 0x6c, 10);
+ hws[IMX6SL_CLK_EPIT1] = imx_clk_hw_gate2("epit1", "perclk", base + 0x6c, 12);
+ hws[IMX6SL_CLK_EPIT2] = imx_clk_hw_gate2("epit2", "perclk", base + 0x6c, 14);
+ hws[IMX6SL_CLK_EXTERN_AUDIO] = imx_clk_hw_gate2("extern_audio", "extern_audio_podf", base + 0x6c, 16);
+ hws[IMX6SL_CLK_GPT] = imx_clk_hw_gate2("gpt", "perclk", base + 0x6c, 20);
+ hws[IMX6SL_CLK_GPT_SERIAL] = imx_clk_hw_gate2("gpt_serial", "perclk", base + 0x6c, 22);
+ hws[IMX6SL_CLK_GPU2D_OVG] = imx_clk_hw_gate2("gpu2d_ovg", "gpu2d_ovg_podf", base + 0x6c, 26);
+ hws[IMX6SL_CLK_I2C1] = imx_clk_hw_gate2("i2c1", "perclk", base + 0x70, 6);
+ hws[IMX6SL_CLK_I2C2] = imx_clk_hw_gate2("i2c2", "perclk", base + 0x70, 8);
+ hws[IMX6SL_CLK_I2C3] = imx_clk_hw_gate2("i2c3", "perclk", base + 0x70, 10);
+ hws[IMX6SL_CLK_OCOTP] = imx_clk_hw_gate2("ocotp", "ipg", base + 0x70, 12);
+ hws[IMX6SL_CLK_CSI] = imx_clk_hw_gate2("csi", "csi_podf", base + 0x74, 0);
+ hws[IMX6SL_CLK_PXP_AXI] = imx_clk_hw_gate2("pxp_axi", "pxp_axi_podf", base + 0x74, 2);
+ hws[IMX6SL_CLK_EPDC_AXI] = imx_clk_hw_gate2("epdc_axi", "epdc_axi_podf", base + 0x74, 4);
+ hws[IMX6SL_CLK_LCDIF_AXI] = imx_clk_hw_gate2("lcdif_axi", "lcdif_axi_podf", base + 0x74, 6);
+ hws[IMX6SL_CLK_LCDIF_PIX] = imx_clk_hw_gate2("lcdif_pix", "lcdif_pix_podf", base + 0x74, 8);
+ hws[IMX6SL_CLK_EPDC_PIX] = imx_clk_hw_gate2("epdc_pix", "epdc_pix_podf", base + 0x74, 10);
+ hws[IMX6SL_CLK_MMDC_P0_IPG] = imx_clk_hw_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
+ hws[IMX6SL_CLK_MMDC_P1_IPG] = imx_clk_hw_gate2("mmdc_p1_ipg", "ipg", base + 0x74, 26);
+ hws[IMX6SL_CLK_OCRAM] = imx_clk_hw_gate2("ocram", "ocram_podf", base + 0x74, 28);
+ hws[IMX6SL_CLK_PWM1] = imx_clk_hw_gate2("pwm1", "perclk", base + 0x78, 16);
+ hws[IMX6SL_CLK_PWM2] = imx_clk_hw_gate2("pwm2", "perclk", base + 0x78, 18);
+ hws[IMX6SL_CLK_PWM3] = imx_clk_hw_gate2("pwm3", "perclk", base + 0x78, 20);
+ hws[IMX6SL_CLK_PWM4] = imx_clk_hw_gate2("pwm4", "perclk", base + 0x78, 22);
+ hws[IMX6SL_CLK_SDMA] = imx_clk_hw_gate2("sdma", "ipg", base + 0x7c, 6);
+ hws[IMX6SL_CLK_SPBA] = imx_clk_hw_gate2("spba", "ipg", base + 0x7c, 12);
+ hws[IMX6SL_CLK_SPDIF] = imx_clk_hw_gate2_shared("spdif", "spdif0_podf", base + 0x7c, 14, &share_count_spdif);
+ hws[IMX6SL_CLK_SPDIF_GCLK] = imx_clk_hw_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_spdif);
+ hws[IMX6SL_CLK_SSI1_IPG] = imx_clk_hw_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SL_CLK_SSI2_IPG] = imx_clk_hw_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SL_CLK_SSI3_IPG] = imx_clk_hw_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SL_CLK_SSI1] = imx_clk_hw_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SL_CLK_SSI2] = imx_clk_hw_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SL_CLK_SSI3] = imx_clk_hw_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SL_CLK_UART] = imx_clk_hw_gate2("uart", "ipg", base + 0x7c, 24);
+ hws[IMX6SL_CLK_UART_SERIAL] = imx_clk_hw_gate2("uart_serial", "uart_root", base + 0x7c, 26);
+ hws[IMX6SL_CLK_USBOH3] = imx_clk_hw_gate2("usboh3", "ipg", base + 0x80, 0);
+ hws[IMX6SL_CLK_USDHC1] = imx_clk_hw_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
+ hws[IMX6SL_CLK_USDHC2] = imx_clk_hw_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
+ hws[IMX6SL_CLK_USDHC3] = imx_clk_hw_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
+ hws[IMX6SL_CLK_USDHC4] = imx_clk_hw_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
/* Ensure the MMDC CH0 handshake is bypassed */
- writel_relaxed(readl_relaxed(base + CCDR) |
- BM_CCM_CCDR_MMDC_CH0_MASK, base + CCDR);
+ imx_mmdc_mask_handshake(base, 0);
- imx_check_clocks(clks, ARRAY_SIZE(clks));
+ imx_check_clk_hws(hws, IMX6SL_CLK_END);
- clk_data.clks = clks;
- clk_data.clk_num = ARRAY_SIZE(clks);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
/* Ensure the AHB clk is at 132MHz. */
- ret = clk_set_rate(clks[IMX6SL_CLK_AHB], 132000000);
+ ret = clk_set_rate(hws[IMX6SL_CLK_AHB]->clk, 132000000);
if (ret)
pr_warn("%s: failed to set AHB clock rate %d!\n",
__func__, ret);
if (IS_ENABLED(CONFIG_USB_MXS_PHY)) {
- clk_prepare_enable(clks[IMX6SL_CLK_USBPHY1_GATE]);
- clk_prepare_enable(clks[IMX6SL_CLK_USBPHY2_GATE]);
+ clk_prepare_enable(hws[IMX6SL_CLK_USBPHY1_GATE]->clk);
+ clk_prepare_enable(hws[IMX6SL_CLK_USBPHY2_GATE]->clk);
}
/* Audio-related clocks configuration */
- clk_set_parent(clks[IMX6SL_CLK_SPDIF0_SEL], clks[IMX6SL_CLK_PLL3_PFD3]);
+ clk_set_parent(hws[IMX6SL_CLK_SPDIF0_SEL]->clk, hws[IMX6SL_CLK_PLL3_PFD3]->clk);
/* set PLL5 video as lcdif pix parent clock */
- clk_set_parent(clks[IMX6SL_CLK_LCDIF_PIX_SEL],
- clks[IMX6SL_CLK_PLL5_VIDEO_DIV]);
+ clk_set_parent(hws[IMX6SL_CLK_LCDIF_PIX_SEL]->clk,
+ hws[IMX6SL_CLK_PLL5_VIDEO_DIV]->clk);
- clk_set_parent(clks[IMX6SL_CLK_LCDIF_AXI_SEL],
- clks[IMX6SL_CLK_PLL2_PFD2]);
+ clk_set_parent(hws[IMX6SL_CLK_LCDIF_AXI_SEL]->clk,
+ hws[IMX6SL_CLK_PLL2_PFD2]->clk);
+
+ for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
+ int index = uart_clk_ids[i];
+
+ uart_clks[i] = &hws[index]->clk;
+ }
imx_register_uart_clocks(uart_clks);
}
#include <dt-bindings/clock/imx6sll-clock.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/io.h>
#include "clk.h"
#define CCM_ANALOG_PLL_BYPASS (0x1 << 16)
-#define BM_CCM_CCDR_MMDC_CH0_MASK (0x2 << 16)
#define xPLL_CLR(offset) (offset + 0x8)
static const char *pll_bypass_src_sels[] = { "osc", "dummy", };
static const char *epdc_pre_sels[] = { "pll2_bus", "pll3_usb_otg", "pll5_video_div", "pll2_pfd0_352m", "pll2_pfd2_396m", "pll3_pfd2_508m", };
static const char *epdc_sels[] = { "epdc_podf", "ipp_di0", "ipp_di1", "ldb_di0", "ldb_di1", };
-static struct clk *clks[IMX6SLL_CLK_END];
-static struct clk_onecell_data clk_data;
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
static const struct clk_div_table post_div_table[] = {
{ .val = 2, .div = 1, },
static u32 share_count_ssi2;
static u32 share_count_ssi3;
-static struct clk ** const uart_clks[] __initconst = {
- &clks[IMX6SLL_CLK_UART1_IPG],
- &clks[IMX6SLL_CLK_UART1_SERIAL],
- &clks[IMX6SLL_CLK_UART2_IPG],
- &clks[IMX6SLL_CLK_UART2_SERIAL],
- &clks[IMX6SLL_CLK_UART3_IPG],
- &clks[IMX6SLL_CLK_UART3_SERIAL],
- &clks[IMX6SLL_CLK_UART4_IPG],
- &clks[IMX6SLL_CLK_UART4_SERIAL],
- &clks[IMX6SLL_CLK_UART5_IPG],
- &clks[IMX6SLL_CLK_UART5_SERIAL],
- NULL
+static const int uart_clk_ids[] __initconst = {
+ IMX6SLL_CLK_UART1_IPG,
+ IMX6SLL_CLK_UART1_SERIAL,
+ IMX6SLL_CLK_UART2_IPG,
+ IMX6SLL_CLK_UART2_SERIAL,
+ IMX6SLL_CLK_UART3_IPG,
+ IMX6SLL_CLK_UART3_SERIAL,
+ IMX6SLL_CLK_UART4_IPG,
+ IMX6SLL_CLK_UART4_SERIAL,
+ IMX6SLL_CLK_UART5_IPG,
+ IMX6SLL_CLK_UART5_SERIAL,
};
+static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
+
static void __init imx6sll_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
+ int i;
+
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX6SLL_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
+
+ clk_hw_data->num = IMX6SLL_CLK_END;
+ hws = clk_hw_data->hws;
- clks[IMX6SLL_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
+ hws[IMX6SLL_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
- clks[IMX6SLL_CLK_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
- clks[IMX6SLL_CLK_OSC] = of_clk_get_by_name(ccm_node, "osc");
+ hws[IMX6SLL_CLK_CKIL] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ckil"));
+ hws[IMX6SLL_CLK_OSC] = __clk_get_hw(of_clk_get_by_name(ccm_node, "osc"));
/* ipp_di clock is external input */
- clks[IMX6SLL_CLK_IPP_DI0] = of_clk_get_by_name(ccm_node, "ipp_di0");
- clks[IMX6SLL_CLK_IPP_DI1] = of_clk_get_by_name(ccm_node, "ipp_di1");
+ hws[IMX6SLL_CLK_IPP_DI0] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di0"));
+ hws[IMX6SLL_CLK_IPP_DI1] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di1"));
np = of_find_compatible_node(NULL, NULL, "fsl,imx6sll-anatop");
base = of_iomap(np, 0);
writel_relaxed(CCM_ANALOG_PLL_BYPASS, base + xPLL_CLR(0xa0));
writel_relaxed(CCM_ANALOG_PLL_BYPASS, base + xPLL_CLR(0xe0));
- clks[IMX6SLL_PLL1_BYPASS_SRC] = imx_clk_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL2_BYPASS_SRC] = imx_clk_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL3_BYPASS_SRC] = imx_clk_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL4_BYPASS_SRC] = imx_clk_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL5_BYPASS_SRC] = imx_clk_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL6_BYPASS_SRC] = imx_clk_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SLL_PLL7_BYPASS_SRC] = imx_clk_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
-
- clks[IMX6SLL_CLK_PLL1] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll1", "pll1_bypass_src", base + 0x00, 0x7f);
- clks[IMX6SLL_CLK_PLL2] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll2", "pll2_bypass_src", base + 0x30, 0x1);
- clks[IMX6SLL_CLK_PLL3] = imx_clk_pllv3(IMX_PLLV3_USB, "pll3", "pll3_bypass_src", base + 0x10, 0x3);
- clks[IMX6SLL_CLK_PLL4] = imx_clk_pllv3(IMX_PLLV3_AV, "pll4", "pll4_bypass_src", base + 0x70, 0x7f);
- clks[IMX6SLL_CLK_PLL5] = imx_clk_pllv3(IMX_PLLV3_AV, "pll5", "pll5_bypass_src", base + 0xa0, 0x7f);
- clks[IMX6SLL_CLK_PLL6] = imx_clk_pllv3(IMX_PLLV3_ENET, "pll6", "pll6_bypass_src", base + 0xe0, 0x3);
- clks[IMX6SLL_CLK_PLL7] = imx_clk_pllv3(IMX_PLLV3_USB, "pll7", "pll7_bypass_src", base + 0x20, 0x3);
-
- clks[IMX6SLL_PLL1_BYPASS] = imx_clk_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL2_BYPASS] = imx_clk_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL3_BYPASS] = imx_clk_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL4_BYPASS] = imx_clk_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL5_BYPASS] = imx_clk_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL6_BYPASS] = imx_clk_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SLL_PLL7_BYPASS] = imx_clk_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
-
- clks[IMX6SLL_CLK_PLL1_SYS] = imx_clk_fixed_factor("pll1_sys", "pll1_bypass", 1, 1);
- clks[IMX6SLL_CLK_PLL2_BUS] = imx_clk_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
- clks[IMX6SLL_CLK_PLL3_USB_OTG] = imx_clk_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
- clks[IMX6SLL_CLK_PLL4_AUDIO] = imx_clk_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
- clks[IMX6SLL_CLK_PLL5_VIDEO] = imx_clk_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
- clks[IMX6SLL_CLK_PLL6_ENET] = imx_clk_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
- clks[IMX6SLL_CLK_PLL7_USB_HOST] = imx_clk_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
+ hws[IMX6SLL_PLL1_BYPASS_SRC] = imx_clk_hw_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL2_BYPASS_SRC] = imx_clk_hw_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL3_BYPASS_SRC] = imx_clk_hw_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL4_BYPASS_SRC] = imx_clk_hw_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL5_BYPASS_SRC] = imx_clk_hw_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL6_BYPASS_SRC] = imx_clk_hw_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SLL_PLL7_BYPASS_SRC] = imx_clk_hw_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+
+ hws[IMX6SLL_CLK_PLL1] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll1", "pll1_bypass_src", base + 0x00, 0x7f);
+ hws[IMX6SLL_CLK_PLL2] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll2", "pll2_bypass_src", base + 0x30, 0x1);
+ hws[IMX6SLL_CLK_PLL3] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll3", "pll3_bypass_src", base + 0x10, 0x3);
+ hws[IMX6SLL_CLK_PLL4] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll4", "pll4_bypass_src", base + 0x70, 0x7f);
+ hws[IMX6SLL_CLK_PLL5] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll5", "pll5_bypass_src", base + 0xa0, 0x7f);
+ hws[IMX6SLL_CLK_PLL6] = imx_clk_hw_pllv3(IMX_PLLV3_ENET, "pll6", "pll6_bypass_src", base + 0xe0, 0x3);
+ hws[IMX6SLL_CLK_PLL7] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll7", "pll7_bypass_src", base + 0x20, 0x3);
+
+ hws[IMX6SLL_PLL1_BYPASS] = imx_clk_hw_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL2_BYPASS] = imx_clk_hw_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL3_BYPASS] = imx_clk_hw_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL4_BYPASS] = imx_clk_hw_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL5_BYPASS] = imx_clk_hw_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL6_BYPASS] = imx_clk_hw_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SLL_PLL7_BYPASS] = imx_clk_hw_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
+
+ hws[IMX6SLL_CLK_PLL1_SYS] = imx_clk_hw_fixed_factor("pll1_sys", "pll1_bypass", 1, 1);
+ hws[IMX6SLL_CLK_PLL2_BUS] = imx_clk_hw_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
+ hws[IMX6SLL_CLK_PLL3_USB_OTG] = imx_clk_hw_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
+ hws[IMX6SLL_CLK_PLL4_AUDIO] = imx_clk_hw_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
+ hws[IMX6SLL_CLK_PLL5_VIDEO] = imx_clk_hw_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
+ hws[IMX6SLL_CLK_PLL6_ENET] = imx_clk_hw_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
+ hws[IMX6SLL_CLK_PLL7_USB_HOST] = imx_clk_hw_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
/*
* Bit 20 is the reserved and read-only bit, we do this only for:
* - Keep refcount when do usbphy clk_enable/disable, in that case,
* the clk framework many need to enable/disable usbphy's parent
*/
- clks[IMX6SLL_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
- clks[IMX6SLL_CLK_USBPHY2] = imx_clk_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
+ hws[IMX6SLL_CLK_USBPHY1] = imx_clk_hw_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
+ hws[IMX6SLL_CLK_USBPHY2] = imx_clk_hw_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
/*
* usbphy*_gate needs to be on after system boots up, and software
* never needs to control it anymore.
*/
if (IS_ENABLED(CONFIG_USB_MXS_PHY)) {
- clks[IMX6SLL_CLK_USBPHY1_GATE] = imx_clk_gate_flags("usbphy1_gate", "dummy", base + 0x10, 6, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_USBPHY2_GATE] = imx_clk_gate_flags("usbphy2_gate", "dummy", base + 0x20, 6, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_USBPHY1_GATE] = imx_clk_hw_gate_flags("usbphy1_gate", "dummy", base + 0x10, 6, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_USBPHY2_GATE] = imx_clk_hw_gate_flags("usbphy2_gate", "dummy", base + 0x20, 6, CLK_IS_CRITICAL);
}
/* name parent_name reg idx */
- clks[IMX6SLL_CLK_PLL2_PFD0] = imx_clk_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
- clks[IMX6SLL_CLK_PLL2_PFD1] = imx_clk_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
- clks[IMX6SLL_CLK_PLL2_PFD2] = imx_clk_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
- clks[IMX6SLL_CLK_PLL2_PFD3] = imx_clk_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
- clks[IMX6SLL_CLK_PLL3_PFD0] = imx_clk_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
- clks[IMX6SLL_CLK_PLL3_PFD1] = imx_clk_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
- clks[IMX6SLL_CLK_PLL3_PFD2] = imx_clk_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
- clks[IMX6SLL_CLK_PLL3_PFD3] = imx_clk_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
-
- clks[IMX6SLL_CLK_PLL4_POST_DIV] = clk_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
+ hws[IMX6SLL_CLK_PLL2_PFD0] = imx_clk_hw_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
+ hws[IMX6SLL_CLK_PLL2_PFD1] = imx_clk_hw_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
+ hws[IMX6SLL_CLK_PLL2_PFD2] = imx_clk_hw_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
+ hws[IMX6SLL_CLK_PLL2_PFD3] = imx_clk_hw_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
+ hws[IMX6SLL_CLK_PLL3_PFD0] = imx_clk_hw_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
+ hws[IMX6SLL_CLK_PLL3_PFD1] = imx_clk_hw_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
+ hws[IMX6SLL_CLK_PLL3_PFD2] = imx_clk_hw_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
+ hws[IMX6SLL_CLK_PLL3_PFD3] = imx_clk_hw_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
+
+ hws[IMX6SLL_CLK_PLL4_POST_DIV] = clk_hw_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SLL_CLK_PLL4_AUDIO_DIV] = clk_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
+ hws[IMX6SLL_CLK_PLL4_AUDIO_DIV] = clk_hw_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x170, 15, 1, 0, &imx_ccm_lock);
- clks[IMX6SLL_CLK_PLL5_POST_DIV] = clk_register_divider_table(NULL, "pll5_post_div", "pll5_video",
+ hws[IMX6SLL_CLK_PLL5_POST_DIV] = clk_hw_register_divider_table(NULL, "pll5_post_div", "pll5_video",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SLL_CLK_PLL5_VIDEO_DIV] = clk_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
+ hws[IMX6SLL_CLK_PLL5_VIDEO_DIV] = clk_hw_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
/* name parent_name mult div */
- clks[IMX6SLL_CLK_PLL2_198M] = imx_clk_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
- clks[IMX6SLL_CLK_PLL3_120M] = imx_clk_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
- clks[IMX6SLL_CLK_PLL3_80M] = imx_clk_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
- clks[IMX6SLL_CLK_PLL3_60M] = imx_clk_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
+ hws[IMX6SLL_CLK_PLL2_198M] = imx_clk_hw_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
+ hws[IMX6SLL_CLK_PLL3_120M] = imx_clk_hw_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
+ hws[IMX6SLL_CLK_PLL3_80M] = imx_clk_hw_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
+ hws[IMX6SLL_CLK_PLL3_60M] = imx_clk_hw_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
np = ccm_node;
base = of_iomap(np, 0);
WARN_ON(!base);
- clks[IMX6SLL_CLK_STEP] = imx_clk_mux("step", base + 0x0c, 8, 1, step_sels, ARRAY_SIZE(step_sels));
- clks[IMX6SLL_CLK_PLL1_SW] = imx_clk_mux_flags("pll1_sw", base + 0x0c, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels), 0);
- clks[IMX6SLL_CLK_AXI_ALT_SEL] = imx_clk_mux("axi_alt_sel", base + 0x14, 7, 1, axi_alt_sels, ARRAY_SIZE(axi_alt_sels));
- clks[IMX6SLL_CLK_AXI_SEL] = imx_clk_mux_flags("axi_sel", base + 0x14, 6, 1, axi_sels, ARRAY_SIZE(axi_sels), 0);
- clks[IMX6SLL_CLK_PERIPH_PRE] = imx_clk_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
- clks[IMX6SLL_CLK_PERIPH2_PRE] = imx_clk_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
- clks[IMX6SLL_CLK_PERIPH_CLK2_SEL] = imx_clk_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
- clks[IMX6SLL_CLK_PERIPH2_CLK2_SEL] = imx_clk_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
- clks[IMX6SLL_CLK_USDHC1_SEL] = imx_clk_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SLL_CLK_USDHC2_SEL] = imx_clk_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SLL_CLK_USDHC3_SEL] = imx_clk_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SLL_CLK_SSI1_SEL] = imx_clk_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SLL_CLK_SSI2_SEL] = imx_clk_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SLL_CLK_SSI3_SEL] = imx_clk_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SLL_CLK_PERCLK_SEL] = imx_clk_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
- clks[IMX6SLL_CLK_UART_SEL] = imx_clk_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
- clks[IMX6SLL_CLK_SPDIF_SEL] = imx_clk_mux("spdif_sel", base + 0x30, 20, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
- clks[IMX6SLL_CLK_EXTERN_AUDIO_SEL] = imx_clk_mux("extern_audio_sel", base + 0x30, 7, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
- clks[IMX6SLL_CLK_EPDC_PRE_SEL] = imx_clk_mux("epdc_pre_sel", base + 0x34, 15, 3, epdc_pre_sels, ARRAY_SIZE(epdc_pre_sels));
- clks[IMX6SLL_CLK_EPDC_SEL] = imx_clk_mux("epdc_sel", base + 0x34, 9, 3, epdc_sels, ARRAY_SIZE(epdc_sels));
- clks[IMX6SLL_CLK_ECSPI_SEL] = imx_clk_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
- clks[IMX6SLL_CLK_LCDIF_PRE_SEL] = imx_clk_mux("lcdif_pre_sel", base + 0x38, 15, 3, lcdif_pre_sels, ARRAY_SIZE(lcdif_pre_sels));
- clks[IMX6SLL_CLK_LCDIF_SEL] = imx_clk_mux("lcdif_sel", base + 0x38, 9, 3, lcdif_sels, ARRAY_SIZE(lcdif_sels));
-
- clks[IMX6SLL_CLK_PERIPH] = imx_clk_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
- clks[IMX6SLL_CLK_PERIPH2] = imx_clk_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
-
- clks[IMX6SLL_CLK_PERIPH_CLK2] = imx_clk_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
- clks[IMX6SLL_CLK_PERIPH2_CLK2] = imx_clk_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
- clks[IMX6SLL_CLK_IPG] = imx_clk_divider("ipg", "ahb", base + 0x14, 8, 2);
- clks[IMX6SLL_CLK_LCDIF_PODF] = imx_clk_divider("lcdif_podf", "lcdif_pred", base + 0x18, 23, 3);
- clks[IMX6SLL_CLK_PERCLK] = imx_clk_divider("perclk", "perclk_sel", base + 0x1c, 0, 6);
- clks[IMX6SLL_CLK_USDHC3_PODF] = imx_clk_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
- clks[IMX6SLL_CLK_USDHC2_PODF] = imx_clk_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
- clks[IMX6SLL_CLK_USDHC1_PODF] = imx_clk_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
- clks[IMX6SLL_CLK_UART_PODF] = imx_clk_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
- clks[IMX6SLL_CLK_SSI3_PRED] = imx_clk_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
- clks[IMX6SLL_CLK_SSI3_PODF] = imx_clk_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
- clks[IMX6SLL_CLK_SSI1_PRED] = imx_clk_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
- clks[IMX6SLL_CLK_SSI1_PODF] = imx_clk_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
- clks[IMX6SLL_CLK_SSI2_PRED] = imx_clk_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
- clks[IMX6SLL_CLK_SSI2_PODF] = imx_clk_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
- clks[IMX6SLL_CLK_SPDIF_PRED] = imx_clk_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
- clks[IMX6SLL_CLK_SPDIF_PODF] = imx_clk_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
- clks[IMX6SLL_CLK_EXTERN_AUDIO_PRED] = imx_clk_divider("extern_audio_pred", "extern_audio_sel", base + 0x30, 12, 3);
- clks[IMX6SLL_CLK_EXTERN_AUDIO_PODF] = imx_clk_divider("extern_audio_podf", "extern_audio_pred", base + 0x30, 9, 3);
- clks[IMX6SLL_CLK_EPDC_PODF] = imx_clk_divider("epdc_podf", "epdc_pre_sel", base + 0x34, 12, 3);
- clks[IMX6SLL_CLK_ECSPI_PODF] = imx_clk_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
- clks[IMX6SLL_CLK_LCDIF_PRED] = imx_clk_divider("lcdif_pred", "lcdif_pre_sel", base + 0x38, 12, 3);
-
- clks[IMX6SLL_CLK_ARM] = imx_clk_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
- clks[IMX6SLL_CLK_MMDC_PODF] = imx_clk_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
- clks[IMX6SLL_CLK_AXI_PODF] = imx_clk_busy_divider("axi", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
- clks[IMX6SLL_CLK_AHB] = imx_clk_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
-
- clks[IMX6SLL_CLK_LDB_DI0_DIV_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
- clks[IMX6SLL_CLK_LDB_DI0_DIV_7] = imx_clk_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
- clks[IMX6SLL_CLK_LDB_DI1_DIV_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
- clks[IMX6SLL_CLK_LDB_DI1_DIV_7] = imx_clk_fixed_factor("ldb_di1_div_7", "ldb_di1_sel", 1, 7);
-
- clks[IMX6SLL_CLK_LDB_DI0_SEL] = imx_clk_mux("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels));
- clks[IMX6SLL_CLK_LDB_DI1_SEL] = imx_clk_mux("ldb_di1_sel", base + 0x1c, 7, 3, ldb_di1_sels, ARRAY_SIZE(ldb_di1_sels));
- clks[IMX6SLL_CLK_LDB_DI0_DIV_SEL] = imx_clk_mux("ldb_di0_div_sel", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels));
- clks[IMX6SLL_CLK_LDB_DI1_DIV_SEL] = imx_clk_mux("ldb_di1_div_sel", base + 0x20, 10, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels));
+ hws[IMX6SLL_CLK_STEP] = imx_clk_hw_mux("step", base + 0x0c, 8, 1, step_sels, ARRAY_SIZE(step_sels));
+ hws[IMX6SLL_CLK_PLL1_SW] = imx_clk_hw_mux_flags("pll1_sw", base + 0x0c, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels), 0);
+ hws[IMX6SLL_CLK_AXI_ALT_SEL] = imx_clk_hw_mux("axi_alt_sel", base + 0x14, 7, 1, axi_alt_sels, ARRAY_SIZE(axi_alt_sels));
+ hws[IMX6SLL_CLK_AXI_SEL] = imx_clk_hw_mux_flags("axi_sel", base + 0x14, 6, 1, axi_sels, ARRAY_SIZE(axi_sels), 0);
+ hws[IMX6SLL_CLK_PERIPH_PRE] = imx_clk_hw_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
+ hws[IMX6SLL_CLK_PERIPH2_PRE] = imx_clk_hw_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
+ hws[IMX6SLL_CLK_PERIPH_CLK2_SEL] = imx_clk_hw_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
+ hws[IMX6SLL_CLK_PERIPH2_CLK2_SEL] = imx_clk_hw_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
+ hws[IMX6SLL_CLK_USDHC1_SEL] = imx_clk_hw_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SLL_CLK_USDHC2_SEL] = imx_clk_hw_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SLL_CLK_USDHC3_SEL] = imx_clk_hw_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SLL_CLK_SSI1_SEL] = imx_clk_hw_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SLL_CLK_SSI2_SEL] = imx_clk_hw_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SLL_CLK_SSI3_SEL] = imx_clk_hw_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SLL_CLK_PERCLK_SEL] = imx_clk_hw_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
+ hws[IMX6SLL_CLK_UART_SEL] = imx_clk_hw_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
+ hws[IMX6SLL_CLK_SPDIF_SEL] = imx_clk_hw_mux("spdif_sel", base + 0x30, 20, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
+ hws[IMX6SLL_CLK_EXTERN_AUDIO_SEL] = imx_clk_hw_mux("extern_audio_sel", base + 0x30, 7, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
+ hws[IMX6SLL_CLK_EPDC_PRE_SEL] = imx_clk_hw_mux("epdc_pre_sel", base + 0x34, 15, 3, epdc_pre_sels, ARRAY_SIZE(epdc_pre_sels));
+ hws[IMX6SLL_CLK_EPDC_SEL] = imx_clk_hw_mux("epdc_sel", base + 0x34, 9, 3, epdc_sels, ARRAY_SIZE(epdc_sels));
+ hws[IMX6SLL_CLK_ECSPI_SEL] = imx_clk_hw_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
+ hws[IMX6SLL_CLK_LCDIF_PRE_SEL] = imx_clk_hw_mux("lcdif_pre_sel", base + 0x38, 15, 3, lcdif_pre_sels, ARRAY_SIZE(lcdif_pre_sels));
+ hws[IMX6SLL_CLK_LCDIF_SEL] = imx_clk_hw_mux("lcdif_sel", base + 0x38, 9, 3, lcdif_sels, ARRAY_SIZE(lcdif_sels));
+
+ hws[IMX6SLL_CLK_PERIPH] = imx_clk_hw_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
+ hws[IMX6SLL_CLK_PERIPH2] = imx_clk_hw_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
+
+ hws[IMX6SLL_CLK_PERIPH_CLK2] = imx_clk_hw_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
+ hws[IMX6SLL_CLK_PERIPH2_CLK2] = imx_clk_hw_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
+ hws[IMX6SLL_CLK_IPG] = imx_clk_hw_divider("ipg", "ahb", base + 0x14, 8, 2);
+ hws[IMX6SLL_CLK_LCDIF_PODF] = imx_clk_hw_divider("lcdif_podf", "lcdif_pred", base + 0x18, 23, 3);
+ hws[IMX6SLL_CLK_PERCLK] = imx_clk_hw_divider("perclk", "perclk_sel", base + 0x1c, 0, 6);
+ hws[IMX6SLL_CLK_USDHC3_PODF] = imx_clk_hw_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
+ hws[IMX6SLL_CLK_USDHC2_PODF] = imx_clk_hw_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
+ hws[IMX6SLL_CLK_USDHC1_PODF] = imx_clk_hw_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
+ hws[IMX6SLL_CLK_UART_PODF] = imx_clk_hw_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
+ hws[IMX6SLL_CLK_SSI3_PRED] = imx_clk_hw_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
+ hws[IMX6SLL_CLK_SSI3_PODF] = imx_clk_hw_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
+ hws[IMX6SLL_CLK_SSI1_PRED] = imx_clk_hw_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
+ hws[IMX6SLL_CLK_SSI1_PODF] = imx_clk_hw_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
+ hws[IMX6SLL_CLK_SSI2_PRED] = imx_clk_hw_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
+ hws[IMX6SLL_CLK_SSI2_PODF] = imx_clk_hw_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
+ hws[IMX6SLL_CLK_SPDIF_PRED] = imx_clk_hw_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
+ hws[IMX6SLL_CLK_SPDIF_PODF] = imx_clk_hw_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
+ hws[IMX6SLL_CLK_EXTERN_AUDIO_PRED] = imx_clk_hw_divider("extern_audio_pred", "extern_audio_sel", base + 0x30, 12, 3);
+ hws[IMX6SLL_CLK_EXTERN_AUDIO_PODF] = imx_clk_hw_divider("extern_audio_podf", "extern_audio_pred", base + 0x30, 9, 3);
+ hws[IMX6SLL_CLK_EPDC_PODF] = imx_clk_hw_divider("epdc_podf", "epdc_pre_sel", base + 0x34, 12, 3);
+ hws[IMX6SLL_CLK_ECSPI_PODF] = imx_clk_hw_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
+ hws[IMX6SLL_CLK_LCDIF_PRED] = imx_clk_hw_divider("lcdif_pred", "lcdif_pre_sel", base + 0x38, 12, 3);
+
+ hws[IMX6SLL_CLK_ARM] = imx_clk_hw_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
+ hws[IMX6SLL_CLK_MMDC_PODF] = imx_clk_hw_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6SLL_CLK_AXI_PODF] = imx_clk_hw_busy_divider("axi", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
+ hws[IMX6SLL_CLK_AHB] = imx_clk_hw_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
+
+ hws[IMX6SLL_CLK_LDB_DI0_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
+ hws[IMX6SLL_CLK_LDB_DI0_DIV_7] = imx_clk_hw_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
+ hws[IMX6SLL_CLK_LDB_DI1_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
+ hws[IMX6SLL_CLK_LDB_DI1_DIV_7] = imx_clk_hw_fixed_factor("ldb_di1_div_7", "ldb_di1_sel", 1, 7);
+
+ hws[IMX6SLL_CLK_LDB_DI0_SEL] = imx_clk_hw_mux("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels));
+ hws[IMX6SLL_CLK_LDB_DI1_SEL] = imx_clk_hw_mux("ldb_di1_sel", base + 0x1c, 7, 3, ldb_di1_sels, ARRAY_SIZE(ldb_di1_sels));
+ hws[IMX6SLL_CLK_LDB_DI0_DIV_SEL] = imx_clk_hw_mux("ldb_di0_div_sel", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels));
+ hws[IMX6SLL_CLK_LDB_DI1_DIV_SEL] = imx_clk_hw_mux("ldb_di1_div_sel", base + 0x20, 10, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels));
/* CCGR0 */
- clks[IMX6SLL_CLK_AIPSTZ1] = imx_clk_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_AIPSTZ2] = imx_clk_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_DCP] = imx_clk_gate2("dcp", "ahb", base + 0x68, 10);
- clks[IMX6SLL_CLK_UART2_IPG] = imx_clk_gate2("uart2_ipg", "ipg", base + 0x68, 28);
- clks[IMX6SLL_CLK_UART2_SERIAL] = imx_clk_gate2("uart2_serial", "uart_podf", base + 0x68, 28);
- clks[IMX6SLL_CLK_GPIO2] = imx_clk_gate2("gpio2", "ipg", base + 0x68, 30);
+ hws[IMX6SLL_CLK_AIPSTZ1] = imx_clk_hw_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_AIPSTZ2] = imx_clk_hw_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_DCP] = imx_clk_hw_gate2("dcp", "ahb", base + 0x68, 10);
+ hws[IMX6SLL_CLK_UART2_IPG] = imx_clk_hw_gate2("uart2_ipg", "ipg", base + 0x68, 28);
+ hws[IMX6SLL_CLK_UART2_SERIAL] = imx_clk_hw_gate2("uart2_serial", "uart_podf", base + 0x68, 28);
+ hws[IMX6SLL_CLK_GPIO2] = imx_clk_hw_gate2("gpio2", "ipg", base + 0x68, 30);
/* CCGR1 */
- clks[IMX6SLL_CLK_ECSPI1] = imx_clk_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
- clks[IMX6SLL_CLK_ECSPI2] = imx_clk_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
- clks[IMX6SLL_CLK_ECSPI3] = imx_clk_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
- clks[IMX6SLL_CLK_ECSPI4] = imx_clk_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
- clks[IMX6SLL_CLK_UART3_IPG] = imx_clk_gate2("uart3_ipg", "ipg", base + 0x6c, 10);
- clks[IMX6SLL_CLK_UART3_SERIAL] = imx_clk_gate2("uart3_serial", "uart_podf", base + 0x6c, 10);
- clks[IMX6SLL_CLK_EPIT1] = imx_clk_gate2("epit1", "perclk", base + 0x6c, 12);
- clks[IMX6SLL_CLK_EPIT2] = imx_clk_gate2("epit2", "perclk", base + 0x6c, 14);
- clks[IMX6SLL_CLK_GPT_BUS] = imx_clk_gate2("gpt1_bus", "perclk", base + 0x6c, 20);
- clks[IMX6SLL_CLK_GPT_SERIAL] = imx_clk_gate2("gpt1_serial", "perclk", base + 0x6c, 22);
- clks[IMX6SLL_CLK_UART4_IPG] = imx_clk_gate2("uart4_ipg", "ipg", base + 0x6c, 24);
- clks[IMX6SLL_CLK_UART4_SERIAL] = imx_clk_gate2("uart4_serial", "uart_podf", base + 0x6c, 24);
- clks[IMX6SLL_CLK_GPIO1] = imx_clk_gate2("gpio1", "ipg", base + 0x6c, 26);
- clks[IMX6SLL_CLK_GPIO5] = imx_clk_gate2("gpio5", "ipg", base + 0x6c, 30);
+ hws[IMX6SLL_CLK_ECSPI1] = imx_clk_hw_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
+ hws[IMX6SLL_CLK_ECSPI2] = imx_clk_hw_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
+ hws[IMX6SLL_CLK_ECSPI3] = imx_clk_hw_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
+ hws[IMX6SLL_CLK_ECSPI4] = imx_clk_hw_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
+ hws[IMX6SLL_CLK_UART3_IPG] = imx_clk_hw_gate2("uart3_ipg", "ipg", base + 0x6c, 10);
+ hws[IMX6SLL_CLK_UART3_SERIAL] = imx_clk_hw_gate2("uart3_serial", "uart_podf", base + 0x6c, 10);
+ hws[IMX6SLL_CLK_EPIT1] = imx_clk_hw_gate2("epit1", "perclk", base + 0x6c, 12);
+ hws[IMX6SLL_CLK_EPIT2] = imx_clk_hw_gate2("epit2", "perclk", base + 0x6c, 14);
+ hws[IMX6SLL_CLK_GPT_BUS] = imx_clk_hw_gate2("gpt1_bus", "perclk", base + 0x6c, 20);
+ hws[IMX6SLL_CLK_GPT_SERIAL] = imx_clk_hw_gate2("gpt1_serial", "perclk", base + 0x6c, 22);
+ hws[IMX6SLL_CLK_UART4_IPG] = imx_clk_hw_gate2("uart4_ipg", "ipg", base + 0x6c, 24);
+ hws[IMX6SLL_CLK_UART4_SERIAL] = imx_clk_hw_gate2("uart4_serial", "uart_podf", base + 0x6c, 24);
+ hws[IMX6SLL_CLK_GPIO1] = imx_clk_hw_gate2("gpio1", "ipg", base + 0x6c, 26);
+ hws[IMX6SLL_CLK_GPIO5] = imx_clk_hw_gate2("gpio5", "ipg", base + 0x6c, 30);
/* CCGR2 */
- clks[IMX6SLL_CLK_GPIO6] = imx_clk_gate2("gpio6", "ipg", base + 0x70, 0);
- clks[IMX6SLL_CLK_CSI] = imx_clk_gate2("csi", "axi", base + 0x70, 2);
- clks[IMX6SLL_CLK_I2C1] = imx_clk_gate2("i2c1", "perclk", base + 0x70, 6);
- clks[IMX6SLL_CLK_I2C2] = imx_clk_gate2("i2c2", "perclk", base + 0x70, 8);
- clks[IMX6SLL_CLK_I2C3] = imx_clk_gate2("i2c3", "perclk", base + 0x70, 10);
- clks[IMX6SLL_CLK_OCOTP] = imx_clk_gate2("ocotp", "ipg", base + 0x70, 12);
- clks[IMX6SLL_CLK_GPIO3] = imx_clk_gate2("gpio3", "ipg", base + 0x70, 26);
- clks[IMX6SLL_CLK_LCDIF_APB] = imx_clk_gate2("lcdif_apb", "axi", base + 0x70, 28);
- clks[IMX6SLL_CLK_PXP] = imx_clk_gate2("pxp", "axi", base + 0x70, 30);
+ hws[IMX6SLL_CLK_GPIO6] = imx_clk_hw_gate2("gpio6", "ipg", base + 0x70, 0);
+ hws[IMX6SLL_CLK_CSI] = imx_clk_hw_gate2("csi", "axi", base + 0x70, 2);
+ hws[IMX6SLL_CLK_I2C1] = imx_clk_hw_gate2("i2c1", "perclk", base + 0x70, 6);
+ hws[IMX6SLL_CLK_I2C2] = imx_clk_hw_gate2("i2c2", "perclk", base + 0x70, 8);
+ hws[IMX6SLL_CLK_I2C3] = imx_clk_hw_gate2("i2c3", "perclk", base + 0x70, 10);
+ hws[IMX6SLL_CLK_OCOTP] = imx_clk_hw_gate2("ocotp", "ipg", base + 0x70, 12);
+ hws[IMX6SLL_CLK_GPIO3] = imx_clk_hw_gate2("gpio3", "ipg", base + 0x70, 26);
+ hws[IMX6SLL_CLK_LCDIF_APB] = imx_clk_hw_gate2("lcdif_apb", "axi", base + 0x70, 28);
+ hws[IMX6SLL_CLK_PXP] = imx_clk_hw_gate2("pxp", "axi", base + 0x70, 30);
/* CCGR3 */
- clks[IMX6SLL_CLK_UART5_IPG] = imx_clk_gate2("uart5_ipg", "ipg", base + 0x74, 2);
- clks[IMX6SLL_CLK_UART5_SERIAL] = imx_clk_gate2("uart5_serial", "uart_podf", base + 0x74, 2);
- clks[IMX6SLL_CLK_EPDC_AXI] = imx_clk_gate2("epdc_aclk", "axi", base + 0x74, 4);
- clks[IMX6SLL_CLK_EPDC_PIX] = imx_clk_gate2("epdc_pix", "epdc_podf", base + 0x74, 4);
- clks[IMX6SLL_CLK_LCDIF_PIX] = imx_clk_gate2("lcdif_pix", "lcdif_podf", base + 0x74, 10);
- clks[IMX6SLL_CLK_GPIO4] = imx_clk_gate2("gpio4", "ipg", base + 0x74, 12);
- clks[IMX6SLL_CLK_WDOG1] = imx_clk_gate2("wdog1", "ipg", base + 0x74, 16);
- clks[IMX6SLL_CLK_MMDC_P0_FAST] = imx_clk_gate_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_MMDC_P0_IPG] = imx_clk_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_MMDC_P1_IPG] = imx_clk_gate2("mmdc_p1_ipg", "ipg", base + 0x74, 26);
- clks[IMX6SLL_CLK_OCRAM] = imx_clk_gate_flags("ocram","ahb", base + 0x74, 28, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_UART5_IPG] = imx_clk_hw_gate2("uart5_ipg", "ipg", base + 0x74, 2);
+ hws[IMX6SLL_CLK_UART5_SERIAL] = imx_clk_hw_gate2("uart5_serial", "uart_podf", base + 0x74, 2);
+ hws[IMX6SLL_CLK_EPDC_AXI] = imx_clk_hw_gate2("epdc_aclk", "axi", base + 0x74, 4);
+ hws[IMX6SLL_CLK_EPDC_PIX] = imx_clk_hw_gate2("epdc_pix", "epdc_podf", base + 0x74, 4);
+ hws[IMX6SLL_CLK_LCDIF_PIX] = imx_clk_hw_gate2("lcdif_pix", "lcdif_podf", base + 0x74, 10);
+ hws[IMX6SLL_CLK_GPIO4] = imx_clk_hw_gate2("gpio4", "ipg", base + 0x74, 12);
+ hws[IMX6SLL_CLK_WDOG1] = imx_clk_hw_gate2("wdog1", "ipg", base + 0x74, 16);
+ hws[IMX6SLL_CLK_MMDC_P0_FAST] = imx_clk_hw_gate_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_MMDC_P0_IPG] = imx_clk_hw_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_MMDC_P1_IPG] = imx_clk_hw_gate2_flags("mmdc_p1_ipg", "ipg", base + 0x74, 26, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_OCRAM] = imx_clk_hw_gate_flags("ocram", "ahb", base + 0x74, 28, CLK_IS_CRITICAL);
/* CCGR4 */
- clks[IMX6SLL_CLK_PWM1] = imx_clk_gate2("pwm1", "perclk", base + 0x78, 16);
- clks[IMX6SLL_CLK_PWM2] = imx_clk_gate2("pwm2", "perclk", base + 0x78, 18);
- clks[IMX6SLL_CLK_PWM3] = imx_clk_gate2("pwm3", "perclk", base + 0x78, 20);
- clks[IMX6SLL_CLK_PWM4] = imx_clk_gate2("pwm4", "perclk", base + 0x78, 22);
+ hws[IMX6SLL_CLK_PWM1] = imx_clk_hw_gate2("pwm1", "perclk", base + 0x78, 16);
+ hws[IMX6SLL_CLK_PWM2] = imx_clk_hw_gate2("pwm2", "perclk", base + 0x78, 18);
+ hws[IMX6SLL_CLK_PWM3] = imx_clk_hw_gate2("pwm3", "perclk", base + 0x78, 20);
+ hws[IMX6SLL_CLK_PWM4] = imx_clk_hw_gate2("pwm4", "perclk", base + 0x78, 22);
/* CCGR5 */
- clks[IMX6SLL_CLK_ROM] = imx_clk_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
- clks[IMX6SLL_CLK_SDMA] = imx_clk_gate2("sdma", "ahb", base + 0x7c, 6);
- clks[IMX6SLL_CLK_WDOG2] = imx_clk_gate2("wdog2", "ipg", base + 0x7c, 10);
- clks[IMX6SLL_CLK_SPBA] = imx_clk_gate2("spba", "ipg", base + 0x7c, 12);
- clks[IMX6SLL_CLK_EXTERN_AUDIO] = imx_clk_gate2_shared("extern_audio", "extern_audio_podf", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SLL_CLK_SPDIF] = imx_clk_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SLL_CLK_SPDIF_GCLK] = imx_clk_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SLL_CLK_SSI1] = imx_clk_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SLL_CLK_SSI1_IPG] = imx_clk_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SLL_CLK_SSI2] = imx_clk_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SLL_CLK_SSI2_IPG] = imx_clk_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SLL_CLK_SSI3] = imx_clk_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SLL_CLK_SSI3_IPG] = imx_clk_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SLL_CLK_UART1_IPG] = imx_clk_gate2("uart1_ipg", "ipg", base + 0x7c, 24);
- clks[IMX6SLL_CLK_UART1_SERIAL] = imx_clk_gate2("uart1_serial", "uart_podf", base + 0x7c, 24);
+ hws[IMX6SLL_CLK_ROM] = imx_clk_hw_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
+ hws[IMX6SLL_CLK_SDMA] = imx_clk_hw_gate2("sdma", "ahb", base + 0x7c, 6);
+ hws[IMX6SLL_CLK_WDOG2] = imx_clk_hw_gate2("wdog2", "ipg", base + 0x7c, 10);
+ hws[IMX6SLL_CLK_SPBA] = imx_clk_hw_gate2("spba", "ipg", base + 0x7c, 12);
+ hws[IMX6SLL_CLK_EXTERN_AUDIO] = imx_clk_hw_gate2_shared("extern_audio", "extern_audio_podf", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SLL_CLK_SPDIF] = imx_clk_hw_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SLL_CLK_SPDIF_GCLK] = imx_clk_hw_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SLL_CLK_SSI1] = imx_clk_hw_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SLL_CLK_SSI1_IPG] = imx_clk_hw_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SLL_CLK_SSI2] = imx_clk_hw_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SLL_CLK_SSI2_IPG] = imx_clk_hw_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SLL_CLK_SSI3] = imx_clk_hw_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SLL_CLK_SSI3_IPG] = imx_clk_hw_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SLL_CLK_UART1_IPG] = imx_clk_hw_gate2("uart1_ipg", "ipg", base + 0x7c, 24);
+ hws[IMX6SLL_CLK_UART1_SERIAL] = imx_clk_hw_gate2("uart1_serial", "uart_podf", base + 0x7c, 24);
/* CCGR6 */
- clks[IMX6SLL_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
- clks[IMX6SLL_CLK_USDHC1] = imx_clk_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
- clks[IMX6SLL_CLK_USDHC2] = imx_clk_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
- clks[IMX6SLL_CLK_USDHC3] = imx_clk_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
+ hws[IMX6SLL_CLK_USBOH3] = imx_clk_hw_gate2("usboh3", "ipg", base + 0x80, 0);
+ hws[IMX6SLL_CLK_USDHC1] = imx_clk_hw_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
+ hws[IMX6SLL_CLK_USDHC2] = imx_clk_hw_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
+ hws[IMX6SLL_CLK_USDHC3] = imx_clk_hw_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
/* mask handshake of mmdc */
- writel_relaxed(BM_CCM_CCDR_MMDC_CH0_MASK, base + 0x4);
+ imx_mmdc_mask_handshake(base, 0);
- imx_check_clocks(clks, ARRAY_SIZE(clks));
+ imx_check_clk_hws(hws, IMX6SLL_CLK_END);
- clk_data.clks = clks;
- clk_data.clk_num = ARRAY_SIZE(clks);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
+
+ for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
+ int index = uart_clk_ids[i];
+
+ uart_clks[i] = &hws[index]->clk;
+ }
imx_register_uart_clocks(uart_clks);
/* Lower the AHB clock rate before changing the clock source. */
- clk_set_rate(clks[IMX6SLL_CLK_AHB], 99000000);
+ clk_set_rate(hws[IMX6SLL_CLK_AHB]->clk, 99000000);
/* Change periph_pre clock to pll2_bus to adjust AXI rate to 264MHz */
- clk_set_parent(clks[IMX6SLL_CLK_PERIPH_CLK2_SEL], clks[IMX6SLL_CLK_PLL3_USB_OTG]);
- clk_set_parent(clks[IMX6SLL_CLK_PERIPH], clks[IMX6SLL_CLK_PERIPH_CLK2]);
- clk_set_parent(clks[IMX6SLL_CLK_PERIPH_PRE], clks[IMX6SLL_CLK_PLL2_BUS]);
- clk_set_parent(clks[IMX6SLL_CLK_PERIPH], clks[IMX6SLL_CLK_PERIPH_PRE]);
+ clk_set_parent(hws[IMX6SLL_CLK_PERIPH_CLK2_SEL]->clk, hws[IMX6SLL_CLK_PLL3_USB_OTG]->clk);
+ clk_set_parent(hws[IMX6SLL_CLK_PERIPH]->clk, hws[IMX6SLL_CLK_PERIPH_CLK2]->clk);
+ clk_set_parent(hws[IMX6SLL_CLK_PERIPH_PRE]->clk, hws[IMX6SLL_CLK_PLL2_BUS]->clk);
+ clk_set_parent(hws[IMX6SLL_CLK_PERIPH]->clk, hws[IMX6SLL_CLK_PERIPH_PRE]->clk);
- clk_set_rate(clks[IMX6SLL_CLK_AHB], 132000000);
+ clk_set_rate(hws[IMX6SLL_CLK_AHB]->clk, 132000000);
}
CLK_OF_DECLARE_DRIVER(imx6sll, "fsl,imx6sll-ccm", imx6sll_clocks_init);
#include <dt-bindings/clock/imx6sx-clock.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/io.h>
#include "clk.h"
-#define CCDR 0x4
-#define BM_CCM_CCDR_MMDC_CH0_MASK (0x2 << 16)
-
static const char *step_sels[] = { "osc", "pll2_pfd2_396m", };
static const char *pll1_sw_sels[] = { "pll1_sys", "step", };
static const char *periph_pre_sels[] = { "pll2_bus", "pll2_pfd2_396m", "pll2_pfd0_352m", "pll2_198m", };
static const char *pll6_bypass_sels[] = { "pll6", "pll6_bypass_src", };
static const char *pll7_bypass_sels[] = { "pll7", "pll7_bypass_src", };
-static struct clk *clks[IMX6SX_CLK_CLK_END];
-static struct clk_onecell_data clk_data;
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
static const struct clk_div_table clk_enet_ref_table[] = {
{ .val = 0, .div = 20, },
static u32 share_count_sai1;
static u32 share_count_sai2;
-static struct clk ** const uart_clks[] __initconst = {
- &clks[IMX6SX_CLK_UART_IPG],
- &clks[IMX6SX_CLK_UART_SERIAL],
- NULL
+static const int uart_clk_ids[] __initconst = {
+ IMX6SX_CLK_UART_IPG,
+ IMX6SX_CLK_UART_SERIAL,
};
+static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
+
static void __init imx6sx_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
+ int i;
+
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX6SX_CLK_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
+
+ clk_hw_data->num = IMX6SX_CLK_CLK_END;
+ hws = clk_hw_data->hws;
- clks[IMX6SX_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
+ hws[IMX6SX_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
- clks[IMX6SX_CLK_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
- clks[IMX6SX_CLK_OSC] = of_clk_get_by_name(ccm_node, "osc");
+ hws[IMX6SX_CLK_CKIL] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ckil"));
+ hws[IMX6SX_CLK_OSC] = __clk_get_hw(of_clk_get_by_name(ccm_node, "osc"));
/* ipp_di clock is external input */
- clks[IMX6SX_CLK_IPP_DI0] = of_clk_get_by_name(ccm_node, "ipp_di0");
- clks[IMX6SX_CLK_IPP_DI1] = of_clk_get_by_name(ccm_node, "ipp_di1");
+ hws[IMX6SX_CLK_IPP_DI0] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di0"));
+ hws[IMX6SX_CLK_IPP_DI1] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di1"));
/* Clock source from external clock via CLK1/2 PAD */
- clks[IMX6SX_CLK_ANACLK1] = of_clk_get_by_name(ccm_node, "anaclk1");
- clks[IMX6SX_CLK_ANACLK2] = of_clk_get_by_name(ccm_node, "anaclk2");
+ hws[IMX6SX_CLK_ANACLK1] = __clk_get_hw(of_clk_get_by_name(ccm_node, "anaclk1"));
+ hws[IMX6SX_CLK_ANACLK2] = __clk_get_hw(of_clk_get_by_name(ccm_node, "anaclk2"));
np = of_find_compatible_node(NULL, NULL, "fsl,imx6sx-anatop");
base = of_iomap(np, 0);
WARN_ON(!base);
of_node_put(np);
- clks[IMX6SX_PLL1_BYPASS_SRC] = imx_clk_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL2_BYPASS_SRC] = imx_clk_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL3_BYPASS_SRC] = imx_clk_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL4_BYPASS_SRC] = imx_clk_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL5_BYPASS_SRC] = imx_clk_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL6_BYPASS_SRC] = imx_clk_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6SX_PLL7_BYPASS_SRC] = imx_clk_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL1_BYPASS_SRC] = imx_clk_hw_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL2_BYPASS_SRC] = imx_clk_hw_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL3_BYPASS_SRC] = imx_clk_hw_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL4_BYPASS_SRC] = imx_clk_hw_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL5_BYPASS_SRC] = imx_clk_hw_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL6_BYPASS_SRC] = imx_clk_hw_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6SX_PLL7_BYPASS_SRC] = imx_clk_hw_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
/* type name parent_name base div_mask */
- clks[IMX6SX_CLK_PLL1] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
- clks[IMX6SX_CLK_PLL2] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
- clks[IMX6SX_CLK_PLL3] = imx_clk_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
- clks[IMX6SX_CLK_PLL4] = imx_clk_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
- clks[IMX6SX_CLK_PLL5] = imx_clk_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
- clks[IMX6SX_CLK_PLL6] = imx_clk_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
- clks[IMX6SX_CLK_PLL7] = imx_clk_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
-
- clks[IMX6SX_PLL1_BYPASS] = imx_clk_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL2_BYPASS] = imx_clk_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL3_BYPASS] = imx_clk_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL4_BYPASS] = imx_clk_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL5_BYPASS] = imx_clk_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL6_BYPASS] = imx_clk_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_PLL7_BYPASS] = imx_clk_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_PLL1] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
+ hws[IMX6SX_CLK_PLL2] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
+ hws[IMX6SX_CLK_PLL3] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
+ hws[IMX6SX_CLK_PLL4] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
+ hws[IMX6SX_CLK_PLL5] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
+ hws[IMX6SX_CLK_PLL6] = imx_clk_hw_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
+ hws[IMX6SX_CLK_PLL7] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
+
+ hws[IMX6SX_PLL1_BYPASS] = imx_clk_hw_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL2_BYPASS] = imx_clk_hw_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL3_BYPASS] = imx_clk_hw_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL4_BYPASS] = imx_clk_hw_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL5_BYPASS] = imx_clk_hw_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL6_BYPASS] = imx_clk_hw_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_PLL7_BYPASS] = imx_clk_hw_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
/* Do not bypass PLLs initially */
- clk_set_parent(clks[IMX6SX_PLL1_BYPASS], clks[IMX6SX_CLK_PLL1]);
- clk_set_parent(clks[IMX6SX_PLL2_BYPASS], clks[IMX6SX_CLK_PLL2]);
- clk_set_parent(clks[IMX6SX_PLL3_BYPASS], clks[IMX6SX_CLK_PLL3]);
- clk_set_parent(clks[IMX6SX_PLL4_BYPASS], clks[IMX6SX_CLK_PLL4]);
- clk_set_parent(clks[IMX6SX_PLL5_BYPASS], clks[IMX6SX_CLK_PLL5]);
- clk_set_parent(clks[IMX6SX_PLL6_BYPASS], clks[IMX6SX_CLK_PLL6]);
- clk_set_parent(clks[IMX6SX_PLL7_BYPASS], clks[IMX6SX_CLK_PLL7]);
-
- clks[IMX6SX_CLK_PLL1_SYS] = imx_clk_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
- clks[IMX6SX_CLK_PLL2_BUS] = imx_clk_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
- clks[IMX6SX_CLK_PLL3_USB_OTG] = imx_clk_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
- clks[IMX6SX_CLK_PLL4_AUDIO] = imx_clk_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
- clks[IMX6SX_CLK_PLL5_VIDEO] = imx_clk_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
- clks[IMX6SX_CLK_PLL6_ENET] = imx_clk_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
- clks[IMX6SX_CLK_PLL7_USB_HOST] = imx_clk_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
+ clk_set_parent(hws[IMX6SX_PLL1_BYPASS]->clk, hws[IMX6SX_CLK_PLL1]->clk);
+ clk_set_parent(hws[IMX6SX_PLL2_BYPASS]->clk, hws[IMX6SX_CLK_PLL2]->clk);
+ clk_set_parent(hws[IMX6SX_PLL3_BYPASS]->clk, hws[IMX6SX_CLK_PLL3]->clk);
+ clk_set_parent(hws[IMX6SX_PLL4_BYPASS]->clk, hws[IMX6SX_CLK_PLL4]->clk);
+ clk_set_parent(hws[IMX6SX_PLL5_BYPASS]->clk, hws[IMX6SX_CLK_PLL5]->clk);
+ clk_set_parent(hws[IMX6SX_PLL6_BYPASS]->clk, hws[IMX6SX_CLK_PLL6]->clk);
+ clk_set_parent(hws[IMX6SX_PLL7_BYPASS]->clk, hws[IMX6SX_CLK_PLL7]->clk);
+
+ hws[IMX6SX_CLK_PLL1_SYS] = imx_clk_hw_gate("pll1_sys", "pll1_bypass", base + 0x00, 13);
+ hws[IMX6SX_CLK_PLL2_BUS] = imx_clk_hw_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
+ hws[IMX6SX_CLK_PLL3_USB_OTG] = imx_clk_hw_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
+ hws[IMX6SX_CLK_PLL4_AUDIO] = imx_clk_hw_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
+ hws[IMX6SX_CLK_PLL5_VIDEO] = imx_clk_hw_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
+ hws[IMX6SX_CLK_PLL6_ENET] = imx_clk_hw_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
+ hws[IMX6SX_CLK_PLL7_USB_HOST] = imx_clk_hw_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
/*
* Bit 20 is the reserved and read-only bit, we do this only for:
* - Keep refcount when do usbphy clk_enable/disable, in that case,
* the clk framework may need to enable/disable usbphy's parent
*/
- clks[IMX6SX_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
- clks[IMX6SX_CLK_USBPHY2] = imx_clk_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
+ hws[IMX6SX_CLK_USBPHY1] = imx_clk_hw_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
+ hws[IMX6SX_CLK_USBPHY2] = imx_clk_hw_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
/*
* usbphy*_gate needs to be on after system boots up, and software
* never needs to control it anymore.
*/
- clks[IMX6SX_CLK_USBPHY1_GATE] = imx_clk_gate("usbphy1_gate", "dummy", base + 0x10, 6);
- clks[IMX6SX_CLK_USBPHY2_GATE] = imx_clk_gate("usbphy2_gate", "dummy", base + 0x20, 6);
+ hws[IMX6SX_CLK_USBPHY1_GATE] = imx_clk_hw_gate("usbphy1_gate", "dummy", base + 0x10, 6);
+ hws[IMX6SX_CLK_USBPHY2_GATE] = imx_clk_hw_gate("usbphy2_gate", "dummy", base + 0x20, 6);
/* FIXME 100MHz is used for pcie ref for all imx6 pcie, excepted imx6q */
- clks[IMX6SX_CLK_PCIE_REF] = imx_clk_fixed_factor("pcie_ref", "pll6_enet", 1, 5);
- clks[IMX6SX_CLK_PCIE_REF_125M] = imx_clk_gate("pcie_ref_125m", "pcie_ref", base + 0xe0, 19);
+ hws[IMX6SX_CLK_PCIE_REF] = imx_clk_hw_fixed_factor("pcie_ref", "pll6_enet", 1, 5);
+ hws[IMX6SX_CLK_PCIE_REF_125M] = imx_clk_hw_gate("pcie_ref_125m", "pcie_ref", base + 0xe0, 19);
- clks[IMX6SX_CLK_LVDS1_OUT] = imx_clk_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x160, 10, BIT(12));
- clks[IMX6SX_CLK_LVDS2_OUT] = imx_clk_gate_exclusive("lvds2_out", "lvds2_sel", base + 0x160, 11, BIT(13));
- clks[IMX6SX_CLK_LVDS1_IN] = imx_clk_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
- clks[IMX6SX_CLK_LVDS2_IN] = imx_clk_gate_exclusive("lvds2_in", "anaclk2", base + 0x160, 13, BIT(11));
+ hws[IMX6SX_CLK_LVDS1_OUT] = imx_clk_hw_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x160, 10, BIT(12));
+ hws[IMX6SX_CLK_LVDS2_OUT] = imx_clk_hw_gate_exclusive("lvds2_out", "lvds2_sel", base + 0x160, 11, BIT(13));
+ hws[IMX6SX_CLK_LVDS1_IN] = imx_clk_hw_gate_exclusive("lvds1_in", "anaclk1", base + 0x160, 12, BIT(10));
+ hws[IMX6SX_CLK_LVDS2_IN] = imx_clk_hw_gate_exclusive("lvds2_in", "anaclk2", base + 0x160, 13, BIT(11));
- clks[IMX6SX_CLK_ENET_REF] = clk_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
+ hws[IMX6SX_CLK_ENET_REF] = clk_hw_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
base + 0xe0, 0, 2, 0, clk_enet_ref_table,
&imx_ccm_lock);
- clks[IMX6SX_CLK_ENET2_REF] = clk_register_divider_table(NULL, "enet2_ref", "pll6_enet", 0,
+ hws[IMX6SX_CLK_ENET2_REF] = clk_hw_register_divider_table(NULL, "enet2_ref", "pll6_enet", 0,
base + 0xe0, 2, 2, 0, clk_enet_ref_table,
&imx_ccm_lock);
- clks[IMX6SX_CLK_ENET2_REF_125M] = imx_clk_gate("enet2_ref_125m", "enet2_ref", base + 0xe0, 20);
+ hws[IMX6SX_CLK_ENET2_REF_125M] = imx_clk_hw_gate("enet2_ref_125m", "enet2_ref", base + 0xe0, 20);
- clks[IMX6SX_CLK_ENET_PTP_REF] = imx_clk_fixed_factor("enet_ptp_ref", "pll6_enet", 1, 20);
- clks[IMX6SX_CLK_ENET_PTP] = imx_clk_gate("enet_ptp_25m", "enet_ptp_ref", base + 0xe0, 21);
+ hws[IMX6SX_CLK_ENET_PTP_REF] = imx_clk_hw_fixed_factor("enet_ptp_ref", "pll6_enet", 1, 20);
+ hws[IMX6SX_CLK_ENET_PTP] = imx_clk_hw_gate("enet_ptp_25m", "enet_ptp_ref", base + 0xe0, 21);
/* name parent_name reg idx */
- clks[IMX6SX_CLK_PLL2_PFD0] = imx_clk_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
- clks[IMX6SX_CLK_PLL2_PFD1] = imx_clk_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
- clks[IMX6SX_CLK_PLL2_PFD2] = imx_clk_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
- clks[IMX6SX_CLK_PLL2_PFD3] = imx_clk_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
- clks[IMX6SX_CLK_PLL3_PFD0] = imx_clk_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
- clks[IMX6SX_CLK_PLL3_PFD1] = imx_clk_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
- clks[IMX6SX_CLK_PLL3_PFD2] = imx_clk_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
- clks[IMX6SX_CLK_PLL3_PFD3] = imx_clk_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
+ hws[IMX6SX_CLK_PLL2_PFD0] = imx_clk_hw_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
+ hws[IMX6SX_CLK_PLL2_PFD1] = imx_clk_hw_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
+ hws[IMX6SX_CLK_PLL2_PFD2] = imx_clk_hw_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
+ hws[IMX6SX_CLK_PLL2_PFD3] = imx_clk_hw_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
+ hws[IMX6SX_CLK_PLL3_PFD0] = imx_clk_hw_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
+ hws[IMX6SX_CLK_PLL3_PFD1] = imx_clk_hw_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
+ hws[IMX6SX_CLK_PLL3_PFD2] = imx_clk_hw_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
+ hws[IMX6SX_CLK_PLL3_PFD3] = imx_clk_hw_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
/* name parent_name mult div */
- clks[IMX6SX_CLK_PLL2_198M] = imx_clk_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
- clks[IMX6SX_CLK_PLL3_120M] = imx_clk_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
- clks[IMX6SX_CLK_PLL3_80M] = imx_clk_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
- clks[IMX6SX_CLK_PLL3_60M] = imx_clk_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
- clks[IMX6SX_CLK_TWD] = imx_clk_fixed_factor("twd", "arm", 1, 2);
- clks[IMX6SX_CLK_GPT_3M] = imx_clk_fixed_factor("gpt_3m", "osc", 1, 8);
-
- clks[IMX6SX_CLK_PLL4_POST_DIV] = clk_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
+ hws[IMX6SX_CLK_PLL2_198M] = imx_clk_hw_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
+ hws[IMX6SX_CLK_PLL3_120M] = imx_clk_hw_fixed_factor("pll3_120m", "pll3_usb_otg", 1, 4);
+ hws[IMX6SX_CLK_PLL3_80M] = imx_clk_hw_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
+ hws[IMX6SX_CLK_PLL3_60M] = imx_clk_hw_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
+ hws[IMX6SX_CLK_TWD] = imx_clk_hw_fixed_factor("twd", "arm", 1, 2);
+ hws[IMX6SX_CLK_GPT_3M] = imx_clk_hw_fixed_factor("gpt_3m", "osc", 1, 8);
+
+ hws[IMX6SX_CLK_PLL4_POST_DIV] = clk_hw_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
CLK_SET_RATE_PARENT, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SX_CLK_PLL4_AUDIO_DIV] = clk_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
+ hws[IMX6SX_CLK_PLL4_AUDIO_DIV] = clk_hw_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
CLK_SET_RATE_PARENT, base + 0x170, 15, 1, 0, &imx_ccm_lock);
- clks[IMX6SX_CLK_PLL5_POST_DIV] = clk_register_divider_table(NULL, "pll5_post_div", "pll5_video",
+ hws[IMX6SX_CLK_PLL5_POST_DIV] = clk_hw_register_divider_table(NULL, "pll5_post_div", "pll5_video",
CLK_SET_RATE_PARENT, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6SX_CLK_PLL5_VIDEO_DIV] = clk_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
+ hws[IMX6SX_CLK_PLL5_VIDEO_DIV] = clk_hw_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
CLK_SET_RATE_PARENT, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
/* name reg shift width parent_names num_parents */
- clks[IMX6SX_CLK_LVDS1_SEL] = imx_clk_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
- clks[IMX6SX_CLK_LVDS2_SEL] = imx_clk_mux("lvds2_sel", base + 0x160, 5, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
+ hws[IMX6SX_CLK_LVDS1_SEL] = imx_clk_hw_mux("lvds1_sel", base + 0x160, 0, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
+ hws[IMX6SX_CLK_LVDS2_SEL] = imx_clk_hw_mux("lvds2_sel", base + 0x160, 5, 5, lvds_sels, ARRAY_SIZE(lvds_sels));
np = ccm_node;
base = of_iomap(np, 0);
WARN_ON(!base);
/* name reg shift width parent_names num_parents */
- clks[IMX6SX_CLK_STEP] = imx_clk_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
- clks[IMX6SX_CLK_PLL1_SW] = imx_clk_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
- clks[IMX6SX_CLK_OCRAM_SEL] = imx_clk_mux("ocram_sel", base + 0x14, 6, 2, ocram_sels, ARRAY_SIZE(ocram_sels));
- clks[IMX6SX_CLK_PERIPH_PRE] = imx_clk_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
- clks[IMX6SX_CLK_PERIPH2_PRE] = imx_clk_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
- clks[IMX6SX_CLK_PERIPH_CLK2_SEL] = imx_clk_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
- clks[IMX6SX_CLK_PERIPH2_CLK2_SEL] = imx_clk_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
- clks[IMX6SX_CLK_PCIE_AXI_SEL] = imx_clk_mux("pcie_axi_sel", base + 0x18, 10, 1, pcie_axi_sels, ARRAY_SIZE(pcie_axi_sels));
- clks[IMX6SX_CLK_GPU_AXI_SEL] = imx_clk_mux("gpu_axi_sel", base + 0x18, 8, 2, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
- clks[IMX6SX_CLK_GPU_CORE_SEL] = imx_clk_mux("gpu_core_sel", base + 0x18, 4, 2, gpu_core_sels, ARRAY_SIZE(gpu_core_sels));
- clks[IMX6SX_CLK_EIM_SLOW_SEL] = imx_clk_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
- clks[IMX6SX_CLK_USDHC1_SEL] = imx_clk_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SX_CLK_USDHC2_SEL] = imx_clk_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SX_CLK_USDHC3_SEL] = imx_clk_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SX_CLK_USDHC4_SEL] = imx_clk_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6SX_CLK_SSI3_SEL] = imx_clk_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SX_CLK_SSI2_SEL] = imx_clk_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SX_CLK_SSI1_SEL] = imx_clk_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
- clks[IMX6SX_CLK_QSPI1_SEL] = imx_clk_mux_flags("qspi1_sel", base + 0x1c, 7, 3, qspi1_sels, ARRAY_SIZE(qspi1_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_PERCLK_SEL] = imx_clk_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
- clks[IMX6SX_CLK_VID_SEL] = imx_clk_mux("vid_sel", base + 0x20, 21, 3, vid_sels, ARRAY_SIZE(vid_sels));
- clks[IMX6SX_CLK_ESAI_SEL] = imx_clk_mux("esai_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SX_CLK_CAN_SEL] = imx_clk_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
- clks[IMX6SX_CLK_UART_SEL] = imx_clk_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
- clks[IMX6SX_CLK_QSPI2_SEL] = imx_clk_mux_flags("qspi2_sel", base + 0x2c, 15, 3, qspi2_sels, ARRAY_SIZE(qspi2_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_SPDIF_SEL] = imx_clk_mux("spdif_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SX_CLK_AUDIO_SEL] = imx_clk_mux("audio_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
- clks[IMX6SX_CLK_ENET_PRE_SEL] = imx_clk_mux("enet_pre_sel", base + 0x34, 15, 3, enet_pre_sels, ARRAY_SIZE(enet_pre_sels));
- clks[IMX6SX_CLK_ENET_SEL] = imx_clk_mux("enet_sel", base + 0x34, 9, 3, enet_sels, ARRAY_SIZE(enet_sels));
- clks[IMX6SX_CLK_M4_PRE_SEL] = imx_clk_mux("m4_pre_sel", base + 0x34, 6, 3, m4_pre_sels, ARRAY_SIZE(m4_pre_sels));
- clks[IMX6SX_CLK_M4_SEL] = imx_clk_mux("m4_sel", base + 0x34, 0, 3, m4_sels, ARRAY_SIZE(m4_sels));
- clks[IMX6SX_CLK_ECSPI_SEL] = imx_clk_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
- clks[IMX6SX_CLK_LCDIF2_PRE_SEL] = imx_clk_mux("lcdif2_pre_sel", base + 0x38, 6, 3, lcdif2_pre_sels, ARRAY_SIZE(lcdif2_pre_sels));
- clks[IMX6SX_CLK_LCDIF2_SEL] = imx_clk_mux("lcdif2_sel", base + 0x38, 0, 3, lcdif2_sels, ARRAY_SIZE(lcdif2_sels));
- clks[IMX6SX_CLK_DISPLAY_SEL] = imx_clk_mux("display_sel", base + 0x3c, 14, 2, display_sels, ARRAY_SIZE(display_sels));
- clks[IMX6SX_CLK_CSI_SEL] = imx_clk_mux("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels));
- clks[IMX6SX_CLK_CKO1_SEL] = imx_clk_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
- clks[IMX6SX_CLK_CKO2_SEL] = imx_clk_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
- clks[IMX6SX_CLK_CKO] = imx_clk_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
-
- clks[IMX6SX_CLK_LDB_DI1_DIV_SEL] = imx_clk_mux_flags("ldb_di1_div_sel", base + 0x20, 11, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_LDB_DI0_DIV_SEL] = imx_clk_mux_flags("ldb_di0_div_sel", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_LDB_DI1_SEL] = imx_clk_mux_flags("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di1_sels, ARRAY_SIZE(ldb_di1_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_LDB_DI0_SEL] = imx_clk_mux_flags("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_LCDIF1_PRE_SEL] = imx_clk_mux_flags("lcdif1_pre_sel", base + 0x38, 15, 3, lcdif1_pre_sels, ARRAY_SIZE(lcdif1_pre_sels), CLK_SET_RATE_PARENT);
- clks[IMX6SX_CLK_LCDIF1_SEL] = imx_clk_mux_flags("lcdif1_sel", base + 0x38, 9, 3, lcdif1_sels, ARRAY_SIZE(lcdif1_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_STEP] = imx_clk_hw_mux("step", base + 0xc, 8, 1, step_sels, ARRAY_SIZE(step_sels));
+ hws[IMX6SX_CLK_PLL1_SW] = imx_clk_hw_mux("pll1_sw", base + 0xc, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels));
+ hws[IMX6SX_CLK_OCRAM_SEL] = imx_clk_hw_mux("ocram_sel", base + 0x14, 6, 2, ocram_sels, ARRAY_SIZE(ocram_sels));
+ hws[IMX6SX_CLK_PERIPH_PRE] = imx_clk_hw_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
+ hws[IMX6SX_CLK_PERIPH2_PRE] = imx_clk_hw_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
+ hws[IMX6SX_CLK_PERIPH_CLK2_SEL] = imx_clk_hw_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
+ hws[IMX6SX_CLK_PERIPH2_CLK2_SEL] = imx_clk_hw_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
+ hws[IMX6SX_CLK_PCIE_AXI_SEL] = imx_clk_hw_mux("pcie_axi_sel", base + 0x18, 10, 1, pcie_axi_sels, ARRAY_SIZE(pcie_axi_sels));
+ hws[IMX6SX_CLK_GPU_AXI_SEL] = imx_clk_hw_mux("gpu_axi_sel", base + 0x18, 8, 2, gpu_axi_sels, ARRAY_SIZE(gpu_axi_sels));
+ hws[IMX6SX_CLK_GPU_CORE_SEL] = imx_clk_hw_mux("gpu_core_sel", base + 0x18, 4, 2, gpu_core_sels, ARRAY_SIZE(gpu_core_sels));
+ hws[IMX6SX_CLK_EIM_SLOW_SEL] = imx_clk_hw_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
+ hws[IMX6SX_CLK_USDHC1_SEL] = imx_clk_hw_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SX_CLK_USDHC2_SEL] = imx_clk_hw_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SX_CLK_USDHC3_SEL] = imx_clk_hw_mux("usdhc3_sel", base + 0x1c, 18, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SX_CLK_USDHC4_SEL] = imx_clk_hw_mux("usdhc4_sel", base + 0x1c, 19, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6SX_CLK_SSI3_SEL] = imx_clk_hw_mux("ssi3_sel", base + 0x1c, 14, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SX_CLK_SSI2_SEL] = imx_clk_hw_mux("ssi2_sel", base + 0x1c, 12, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SX_CLK_SSI1_SEL] = imx_clk_hw_mux("ssi1_sel", base + 0x1c, 10, 2, ssi_sels, ARRAY_SIZE(ssi_sels));
+ hws[IMX6SX_CLK_QSPI1_SEL] = imx_clk_hw_mux_flags("qspi1_sel", base + 0x1c, 7, 3, qspi1_sels, ARRAY_SIZE(qspi1_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_PERCLK_SEL] = imx_clk_hw_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
+ hws[IMX6SX_CLK_VID_SEL] = imx_clk_hw_mux("vid_sel", base + 0x20, 21, 3, vid_sels, ARRAY_SIZE(vid_sels));
+ hws[IMX6SX_CLK_ESAI_SEL] = imx_clk_hw_mux("esai_sel", base + 0x20, 19, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SX_CLK_CAN_SEL] = imx_clk_hw_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
+ hws[IMX6SX_CLK_UART_SEL] = imx_clk_hw_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
+ hws[IMX6SX_CLK_QSPI2_SEL] = imx_clk_hw_mux_flags("qspi2_sel", base + 0x2c, 15, 3, qspi2_sels, ARRAY_SIZE(qspi2_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_SPDIF_SEL] = imx_clk_hw_mux("spdif_sel", base + 0x30, 20, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SX_CLK_AUDIO_SEL] = imx_clk_hw_mux("audio_sel", base + 0x30, 7, 2, audio_sels, ARRAY_SIZE(audio_sels));
+ hws[IMX6SX_CLK_ENET_PRE_SEL] = imx_clk_hw_mux("enet_pre_sel", base + 0x34, 15, 3, enet_pre_sels, ARRAY_SIZE(enet_pre_sels));
+ hws[IMX6SX_CLK_ENET_SEL] = imx_clk_hw_mux("enet_sel", base + 0x34, 9, 3, enet_sels, ARRAY_SIZE(enet_sels));
+ hws[IMX6SX_CLK_M4_PRE_SEL] = imx_clk_hw_mux("m4_pre_sel", base + 0x34, 6, 3, m4_pre_sels, ARRAY_SIZE(m4_pre_sels));
+ hws[IMX6SX_CLK_M4_SEL] = imx_clk_hw_mux("m4_sel", base + 0x34, 0, 3, m4_sels, ARRAY_SIZE(m4_sels));
+ hws[IMX6SX_CLK_ECSPI_SEL] = imx_clk_hw_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
+ hws[IMX6SX_CLK_LCDIF2_PRE_SEL] = imx_clk_hw_mux("lcdif2_pre_sel", base + 0x38, 6, 3, lcdif2_pre_sels, ARRAY_SIZE(lcdif2_pre_sels));
+ hws[IMX6SX_CLK_LCDIF2_SEL] = imx_clk_hw_mux("lcdif2_sel", base + 0x38, 0, 3, lcdif2_sels, ARRAY_SIZE(lcdif2_sels));
+ hws[IMX6SX_CLK_DISPLAY_SEL] = imx_clk_hw_mux("display_sel", base + 0x3c, 14, 2, display_sels, ARRAY_SIZE(display_sels));
+ hws[IMX6SX_CLK_CSI_SEL] = imx_clk_hw_mux("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels));
+ hws[IMX6SX_CLK_CKO1_SEL] = imx_clk_hw_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
+ hws[IMX6SX_CLK_CKO2_SEL] = imx_clk_hw_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
+ hws[IMX6SX_CLK_CKO] = imx_clk_hw_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
+
+ hws[IMX6SX_CLK_LDB_DI1_DIV_SEL] = imx_clk_hw_mux_flags("ldb_di1_div_sel", base + 0x20, 11, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_LDB_DI0_DIV_SEL] = imx_clk_hw_mux_flags("ldb_di0_div_sel", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_LDB_DI1_SEL] = imx_clk_hw_mux_flags("ldb_di1_sel", base + 0x2c, 12, 3, ldb_di1_sels, ARRAY_SIZE(ldb_di1_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_LDB_DI0_SEL] = imx_clk_hw_mux_flags("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_LCDIF1_PRE_SEL] = imx_clk_hw_mux_flags("lcdif1_pre_sel", base + 0x38, 15, 3, lcdif1_pre_sels, ARRAY_SIZE(lcdif1_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6SX_CLK_LCDIF1_SEL] = imx_clk_hw_mux_flags("lcdif1_sel", base + 0x38, 9, 3, lcdif1_sels, ARRAY_SIZE(lcdif1_sels), CLK_SET_RATE_PARENT);
/* name parent_name reg shift width */
- clks[IMX6SX_CLK_PERIPH_CLK2] = imx_clk_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
- clks[IMX6SX_CLK_PERIPH2_CLK2] = imx_clk_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
- clks[IMX6SX_CLK_IPG] = imx_clk_divider("ipg", "ahb", base + 0x14, 8, 2);
- clks[IMX6SX_CLK_GPU_CORE_PODF] = imx_clk_divider("gpu_core_podf", "gpu_core_sel", base + 0x18, 29, 3);
- clks[IMX6SX_CLK_GPU_AXI_PODF] = imx_clk_divider("gpu_axi_podf", "gpu_axi_sel", base + 0x18, 26, 3);
- clks[IMX6SX_CLK_LCDIF1_PODF] = imx_clk_divider("lcdif1_podf", "lcdif1_pred", base + 0x18, 23, 3);
- clks[IMX6SX_CLK_QSPI1_PODF] = imx_clk_divider("qspi1_podf", "qspi1_sel", base + 0x1c, 26, 3);
- clks[IMX6SX_CLK_EIM_SLOW_PODF] = imx_clk_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
- clks[IMX6SX_CLK_LCDIF2_PODF] = imx_clk_divider("lcdif2_podf", "lcdif2_pred", base + 0x1c, 20, 3);
- clks[IMX6SX_CLK_PERCLK] = imx_clk_divider_flags("perclk", "perclk_sel", base + 0x1c, 0, 6, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_VID_PODF] = imx_clk_divider("vid_podf", "vid_sel", base + 0x20, 24, 2);
- clks[IMX6SX_CLK_CAN_PODF] = imx_clk_divider("can_podf", "can_sel", base + 0x20, 2, 6);
- clks[IMX6SX_CLK_USDHC4_PODF] = imx_clk_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
- clks[IMX6SX_CLK_USDHC3_PODF] = imx_clk_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
- clks[IMX6SX_CLK_USDHC2_PODF] = imx_clk_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
- clks[IMX6SX_CLK_USDHC1_PODF] = imx_clk_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
- clks[IMX6SX_CLK_UART_PODF] = imx_clk_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
- clks[IMX6SX_CLK_ESAI_PRED] = imx_clk_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
- clks[IMX6SX_CLK_ESAI_PODF] = imx_clk_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
- clks[IMX6SX_CLK_SSI3_PRED] = imx_clk_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
- clks[IMX6SX_CLK_SSI3_PODF] = imx_clk_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
- clks[IMX6SX_CLK_SSI1_PRED] = imx_clk_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
- clks[IMX6SX_CLK_SSI1_PODF] = imx_clk_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
- clks[IMX6SX_CLK_QSPI2_PRED] = imx_clk_divider("qspi2_pred", "qspi2_sel", base + 0x2c, 18, 3);
- clks[IMX6SX_CLK_QSPI2_PODF] = imx_clk_divider("qspi2_podf", "qspi2_pred", base + 0x2c, 21, 6);
- clks[IMX6SX_CLK_SSI2_PRED] = imx_clk_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
- clks[IMX6SX_CLK_SSI2_PODF] = imx_clk_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
- clks[IMX6SX_CLK_SPDIF_PRED] = imx_clk_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
- clks[IMX6SX_CLK_SPDIF_PODF] = imx_clk_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
- clks[IMX6SX_CLK_AUDIO_PRED] = imx_clk_divider("audio_pred", "audio_sel", base + 0x30, 12, 3);
- clks[IMX6SX_CLK_AUDIO_PODF] = imx_clk_divider("audio_podf", "audio_pred", base + 0x30, 9, 3);
- clks[IMX6SX_CLK_ENET_PODF] = imx_clk_divider("enet_podf", "enet_pre_sel", base + 0x34, 12, 3);
- clks[IMX6SX_CLK_M4_PODF] = imx_clk_divider("m4_podf", "m4_sel", base + 0x34, 3, 3);
- clks[IMX6SX_CLK_ECSPI_PODF] = imx_clk_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
- clks[IMX6SX_CLK_LCDIF1_PRED] = imx_clk_divider("lcdif1_pred", "lcdif1_pre_sel", base + 0x38, 12, 3);
- clks[IMX6SX_CLK_LCDIF2_PRED] = imx_clk_divider("lcdif2_pred", "lcdif2_pre_sel", base + 0x38, 3, 3);
- clks[IMX6SX_CLK_DISPLAY_PODF] = imx_clk_divider("display_podf", "display_sel", base + 0x3c, 16, 3);
- clks[IMX6SX_CLK_CSI_PODF] = imx_clk_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
- clks[IMX6SX_CLK_CKO1_PODF] = imx_clk_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
- clks[IMX6SX_CLK_CKO2_PODF] = imx_clk_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
-
- clks[IMX6SX_CLK_LDB_DI0_DIV_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
- clks[IMX6SX_CLK_LDB_DI0_DIV_7] = imx_clk_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
- clks[IMX6SX_CLK_LDB_DI1_DIV_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
- clks[IMX6SX_CLK_LDB_DI1_DIV_7] = imx_clk_fixed_factor("ldb_di1_div_7", "ldb_di1_sel", 1, 7);
+ hws[IMX6SX_CLK_PERIPH_CLK2] = imx_clk_hw_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
+ hws[IMX6SX_CLK_PERIPH2_CLK2] = imx_clk_hw_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
+ hws[IMX6SX_CLK_IPG] = imx_clk_hw_divider("ipg", "ahb", base + 0x14, 8, 2);
+ hws[IMX6SX_CLK_GPU_CORE_PODF] = imx_clk_hw_divider("gpu_core_podf", "gpu_core_sel", base + 0x18, 29, 3);
+ hws[IMX6SX_CLK_GPU_AXI_PODF] = imx_clk_hw_divider("gpu_axi_podf", "gpu_axi_sel", base + 0x18, 26, 3);
+ hws[IMX6SX_CLK_LCDIF1_PODF] = imx_clk_hw_divider("lcdif1_podf", "lcdif1_pred", base + 0x18, 23, 3);
+ hws[IMX6SX_CLK_QSPI1_PODF] = imx_clk_hw_divider("qspi1_podf", "qspi1_sel", base + 0x1c, 26, 3);
+ hws[IMX6SX_CLK_EIM_SLOW_PODF] = imx_clk_hw_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
+ hws[IMX6SX_CLK_LCDIF2_PODF] = imx_clk_hw_divider("lcdif2_podf", "lcdif2_pred", base + 0x1c, 20, 3);
+ hws[IMX6SX_CLK_PERCLK] = imx_clk_hw_divider_flags("perclk", "perclk_sel", base + 0x1c, 0, 6, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_VID_PODF] = imx_clk_hw_divider("vid_podf", "vid_sel", base + 0x20, 24, 2);
+ hws[IMX6SX_CLK_CAN_PODF] = imx_clk_hw_divider("can_podf", "can_sel", base + 0x20, 2, 6);
+ hws[IMX6SX_CLK_USDHC4_PODF] = imx_clk_hw_divider("usdhc4_podf", "usdhc4_sel", base + 0x24, 22, 3);
+ hws[IMX6SX_CLK_USDHC3_PODF] = imx_clk_hw_divider("usdhc3_podf", "usdhc3_sel", base + 0x24, 19, 3);
+ hws[IMX6SX_CLK_USDHC2_PODF] = imx_clk_hw_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
+ hws[IMX6SX_CLK_USDHC1_PODF] = imx_clk_hw_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
+ hws[IMX6SX_CLK_UART_PODF] = imx_clk_hw_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
+ hws[IMX6SX_CLK_ESAI_PRED] = imx_clk_hw_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
+ hws[IMX6SX_CLK_ESAI_PODF] = imx_clk_hw_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
+ hws[IMX6SX_CLK_SSI3_PRED] = imx_clk_hw_divider("ssi3_pred", "ssi3_sel", base + 0x28, 22, 3);
+ hws[IMX6SX_CLK_SSI3_PODF] = imx_clk_hw_divider("ssi3_podf", "ssi3_pred", base + 0x28, 16, 6);
+ hws[IMX6SX_CLK_SSI1_PRED] = imx_clk_hw_divider("ssi1_pred", "ssi1_sel", base + 0x28, 6, 3);
+ hws[IMX6SX_CLK_SSI1_PODF] = imx_clk_hw_divider("ssi1_podf", "ssi1_pred", base + 0x28, 0, 6);
+ hws[IMX6SX_CLK_QSPI2_PRED] = imx_clk_hw_divider("qspi2_pred", "qspi2_sel", base + 0x2c, 18, 3);
+ hws[IMX6SX_CLK_QSPI2_PODF] = imx_clk_hw_divider("qspi2_podf", "qspi2_pred", base + 0x2c, 21, 6);
+ hws[IMX6SX_CLK_SSI2_PRED] = imx_clk_hw_divider("ssi2_pred", "ssi2_sel", base + 0x2c, 6, 3);
+ hws[IMX6SX_CLK_SSI2_PODF] = imx_clk_hw_divider("ssi2_podf", "ssi2_pred", base + 0x2c, 0, 6);
+ hws[IMX6SX_CLK_SPDIF_PRED] = imx_clk_hw_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
+ hws[IMX6SX_CLK_SPDIF_PODF] = imx_clk_hw_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
+ hws[IMX6SX_CLK_AUDIO_PRED] = imx_clk_hw_divider("audio_pred", "audio_sel", base + 0x30, 12, 3);
+ hws[IMX6SX_CLK_AUDIO_PODF] = imx_clk_hw_divider("audio_podf", "audio_pred", base + 0x30, 9, 3);
+ hws[IMX6SX_CLK_ENET_PODF] = imx_clk_hw_divider("enet_podf", "enet_pre_sel", base + 0x34, 12, 3);
+ hws[IMX6SX_CLK_M4_PODF] = imx_clk_hw_divider("m4_podf", "m4_sel", base + 0x34, 3, 3);
+ hws[IMX6SX_CLK_ECSPI_PODF] = imx_clk_hw_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
+ hws[IMX6SX_CLK_LCDIF1_PRED] = imx_clk_hw_divider("lcdif1_pred", "lcdif1_pre_sel", base + 0x38, 12, 3);
+ hws[IMX6SX_CLK_LCDIF2_PRED] = imx_clk_hw_divider("lcdif2_pred", "lcdif2_pre_sel", base + 0x38, 3, 3);
+ hws[IMX6SX_CLK_DISPLAY_PODF] = imx_clk_hw_divider("display_podf", "display_sel", base + 0x3c, 16, 3);
+ hws[IMX6SX_CLK_CSI_PODF] = imx_clk_hw_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
+ hws[IMX6SX_CLK_CKO1_PODF] = imx_clk_hw_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
+ hws[IMX6SX_CLK_CKO2_PODF] = imx_clk_hw_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
+
+ hws[IMX6SX_CLK_LDB_DI0_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
+ hws[IMX6SX_CLK_LDB_DI0_DIV_7] = imx_clk_hw_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
+ hws[IMX6SX_CLK_LDB_DI1_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di1_div_3_5", "ldb_di1_sel", 2, 7);
+ hws[IMX6SX_CLK_LDB_DI1_DIV_7] = imx_clk_hw_fixed_factor("ldb_di1_div_7", "ldb_di1_sel", 1, 7);
/* name reg shift width busy: reg, shift parent_names num_parents */
- clks[IMX6SX_CLK_PERIPH] = imx_clk_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
- clks[IMX6SX_CLK_PERIPH2] = imx_clk_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
+ hws[IMX6SX_CLK_PERIPH] = imx_clk_hw_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
+ hws[IMX6SX_CLK_PERIPH2] = imx_clk_hw_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
/* name parent_name reg shift width busy: reg, shift */
- clks[IMX6SX_CLK_OCRAM_PODF] = imx_clk_busy_divider("ocram_podf", "ocram_sel", base + 0x14, 16, 3, base + 0x48, 0);
- clks[IMX6SX_CLK_AHB] = imx_clk_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
- clks[IMX6SX_CLK_MMDC_PODF] = imx_clk_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
- clks[IMX6SX_CLK_ARM] = imx_clk_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
+ hws[IMX6SX_CLK_OCRAM_PODF] = imx_clk_hw_busy_divider("ocram_podf", "ocram_sel", base + 0x14, 16, 3, base + 0x48, 0);
+ hws[IMX6SX_CLK_AHB] = imx_clk_hw_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
+ hws[IMX6SX_CLK_MMDC_PODF] = imx_clk_hw_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6SX_CLK_ARM] = imx_clk_hw_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
/* name parent_name reg shift */
/* CCGR0 */
- clks[IMX6SX_CLK_AIPS_TZ1] = imx_clk_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_AIPS_TZ2] = imx_clk_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_APBH_DMA] = imx_clk_gate2("apbh_dma", "usdhc3", base + 0x68, 4);
- clks[IMX6SX_CLK_ASRC_MEM] = imx_clk_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
- clks[IMX6SX_CLK_ASRC_IPG] = imx_clk_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
- clks[IMX6SX_CLK_CAAM_MEM] = imx_clk_gate2("caam_mem", "ahb", base + 0x68, 8);
- clks[IMX6SX_CLK_CAAM_ACLK] = imx_clk_gate2("caam_aclk", "ahb", base + 0x68, 10);
- clks[IMX6SX_CLK_CAAM_IPG] = imx_clk_gate2("caam_ipg", "ipg", base + 0x68, 12);
- clks[IMX6SX_CLK_CAN1_IPG] = imx_clk_gate2("can1_ipg", "ipg", base + 0x68, 14);
- clks[IMX6SX_CLK_CAN1_SERIAL] = imx_clk_gate2("can1_serial", "can_podf", base + 0x68, 16);
- clks[IMX6SX_CLK_CAN2_IPG] = imx_clk_gate2("can2_ipg", "ipg", base + 0x68, 18);
- clks[IMX6SX_CLK_CAN2_SERIAL] = imx_clk_gate2("can2_serial", "can_podf", base + 0x68, 20);
- clks[IMX6SX_CLK_DCIC1] = imx_clk_gate2("dcic1", "display_podf", base + 0x68, 24);
- clks[IMX6SX_CLK_DCIC2] = imx_clk_gate2("dcic2", "display_podf", base + 0x68, 26);
- clks[IMX6SX_CLK_AIPS_TZ3] = imx_clk_gate2_flags("aips_tz3", "ahb", base + 0x68, 30, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_AIPS_TZ1] = imx_clk_hw_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_AIPS_TZ2] = imx_clk_hw_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_APBH_DMA] = imx_clk_hw_gate2("apbh_dma", "usdhc3", base + 0x68, 4);
+ hws[IMX6SX_CLK_ASRC_MEM] = imx_clk_hw_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6SX_CLK_ASRC_IPG] = imx_clk_hw_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6SX_CLK_CAAM_MEM] = imx_clk_hw_gate2("caam_mem", "ahb", base + 0x68, 8);
+ hws[IMX6SX_CLK_CAAM_ACLK] = imx_clk_hw_gate2("caam_aclk", "ahb", base + 0x68, 10);
+ hws[IMX6SX_CLK_CAAM_IPG] = imx_clk_hw_gate2("caam_ipg", "ipg", base + 0x68, 12);
+ hws[IMX6SX_CLK_CAN1_IPG] = imx_clk_hw_gate2("can1_ipg", "ipg", base + 0x68, 14);
+ hws[IMX6SX_CLK_CAN1_SERIAL] = imx_clk_hw_gate2("can1_serial", "can_podf", base + 0x68, 16);
+ hws[IMX6SX_CLK_CAN2_IPG] = imx_clk_hw_gate2("can2_ipg", "ipg", base + 0x68, 18);
+ hws[IMX6SX_CLK_CAN2_SERIAL] = imx_clk_hw_gate2("can2_serial", "can_podf", base + 0x68, 20);
+ hws[IMX6SX_CLK_DCIC1] = imx_clk_hw_gate2("dcic1", "display_podf", base + 0x68, 24);
+ hws[IMX6SX_CLK_DCIC2] = imx_clk_hw_gate2("dcic2", "display_podf", base + 0x68, 26);
+ hws[IMX6SX_CLK_AIPS_TZ3] = imx_clk_hw_gate2_flags("aips_tz3", "ahb", base + 0x68, 30, CLK_IS_CRITICAL);
/* CCGR1 */
- clks[IMX6SX_CLK_ECSPI1] = imx_clk_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
- clks[IMX6SX_CLK_ECSPI2] = imx_clk_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
- clks[IMX6SX_CLK_ECSPI3] = imx_clk_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
- clks[IMX6SX_CLK_ECSPI4] = imx_clk_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
- clks[IMX6SX_CLK_ECSPI5] = imx_clk_gate2("ecspi5", "ecspi_podf", base + 0x6c, 8);
- clks[IMX6SX_CLK_EPIT1] = imx_clk_gate2("epit1", "perclk", base + 0x6c, 12);
- clks[IMX6SX_CLK_EPIT2] = imx_clk_gate2("epit2", "perclk", base + 0x6c, 14);
- clks[IMX6SX_CLK_ESAI_EXTAL] = imx_clk_gate2_shared("esai_extal", "esai_podf", base + 0x6c, 16, &share_count_esai);
- clks[IMX6SX_CLK_ESAI_IPG] = imx_clk_gate2_shared("esai_ipg", "ahb", base + 0x6c, 16, &share_count_esai);
- clks[IMX6SX_CLK_ESAI_MEM] = imx_clk_gate2_shared("esai_mem", "ahb", base + 0x6c, 16, &share_count_esai);
- clks[IMX6SX_CLK_WAKEUP] = imx_clk_gate2_flags("wakeup", "ipg", base + 0x6c, 18, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_GPT_BUS] = imx_clk_gate2("gpt_bus", "perclk", base + 0x6c, 20);
- clks[IMX6SX_CLK_GPT_SERIAL] = imx_clk_gate2("gpt_serial", "perclk", base + 0x6c, 22);
- clks[IMX6SX_CLK_GPU] = imx_clk_gate2("gpu", "gpu_core_podf", base + 0x6c, 26);
- clks[IMX6SX_CLK_OCRAM_S] = imx_clk_gate2("ocram_s", "ahb", base + 0x6c, 28);
- clks[IMX6SX_CLK_CANFD] = imx_clk_gate2("canfd", "can_podf", base + 0x6c, 30);
+ hws[IMX6SX_CLK_ECSPI1] = imx_clk_hw_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
+ hws[IMX6SX_CLK_ECSPI2] = imx_clk_hw_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
+ hws[IMX6SX_CLK_ECSPI3] = imx_clk_hw_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
+ hws[IMX6SX_CLK_ECSPI4] = imx_clk_hw_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
+ hws[IMX6SX_CLK_ECSPI5] = imx_clk_hw_gate2("ecspi5", "ecspi_podf", base + 0x6c, 8);
+ hws[IMX6SX_CLK_EPIT1] = imx_clk_hw_gate2("epit1", "perclk", base + 0x6c, 12);
+ hws[IMX6SX_CLK_EPIT2] = imx_clk_hw_gate2("epit2", "perclk", base + 0x6c, 14);
+ hws[IMX6SX_CLK_ESAI_EXTAL] = imx_clk_hw_gate2_shared("esai_extal", "esai_podf", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6SX_CLK_ESAI_IPG] = imx_clk_hw_gate2_shared("esai_ipg", "ahb", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6SX_CLK_ESAI_MEM] = imx_clk_hw_gate2_shared("esai_mem", "ahb", base + 0x6c, 16, &share_count_esai);
+ hws[IMX6SX_CLK_WAKEUP] = imx_clk_hw_gate2_flags("wakeup", "ipg", base + 0x6c, 18, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_GPT_BUS] = imx_clk_hw_gate2("gpt_bus", "perclk", base + 0x6c, 20);
+ hws[IMX6SX_CLK_GPT_SERIAL] = imx_clk_hw_gate2("gpt_serial", "perclk", base + 0x6c, 22);
+ hws[IMX6SX_CLK_GPU] = imx_clk_hw_gate2("gpu", "gpu_core_podf", base + 0x6c, 26);
+ hws[IMX6SX_CLK_OCRAM_S] = imx_clk_hw_gate2("ocram_s", "ahb", base + 0x6c, 28);
+ hws[IMX6SX_CLK_CANFD] = imx_clk_hw_gate2("canfd", "can_podf", base + 0x6c, 30);
/* CCGR2 */
- clks[IMX6SX_CLK_CSI] = imx_clk_gate2("csi", "csi_podf", base + 0x70, 2);
- clks[IMX6SX_CLK_I2C1] = imx_clk_gate2("i2c1", "perclk", base + 0x70, 6);
- clks[IMX6SX_CLK_I2C2] = imx_clk_gate2("i2c2", "perclk", base + 0x70, 8);
- clks[IMX6SX_CLK_I2C3] = imx_clk_gate2("i2c3", "perclk", base + 0x70, 10);
- clks[IMX6SX_CLK_OCOTP] = imx_clk_gate2("ocotp", "ipg", base + 0x70, 12);
- clks[IMX6SX_CLK_IOMUXC] = imx_clk_gate2("iomuxc", "lcdif1_podf", base + 0x70, 14);
- clks[IMX6SX_CLK_IPMUX1] = imx_clk_gate2_flags("ipmux1", "ahb", base + 0x70, 16, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_IPMUX2] = imx_clk_gate2_flags("ipmux2", "ahb", base + 0x70, 18, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_IPMUX3] = imx_clk_gate2_flags("ipmux3", "ahb", base + 0x70, 20, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_TZASC1] = imx_clk_gate2_flags("tzasc1", "mmdc_podf", base + 0x70, 22, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_LCDIF_APB] = imx_clk_gate2("lcdif_apb", "display_podf", base + 0x70, 28);
- clks[IMX6SX_CLK_PXP_AXI] = imx_clk_gate2("pxp_axi", "display_podf", base + 0x70, 30);
+ hws[IMX6SX_CLK_CSI] = imx_clk_hw_gate2("csi", "csi_podf", base + 0x70, 2);
+ hws[IMX6SX_CLK_I2C1] = imx_clk_hw_gate2("i2c1", "perclk", base + 0x70, 6);
+ hws[IMX6SX_CLK_I2C2] = imx_clk_hw_gate2("i2c2", "perclk", base + 0x70, 8);
+ hws[IMX6SX_CLK_I2C3] = imx_clk_hw_gate2("i2c3", "perclk", base + 0x70, 10);
+ hws[IMX6SX_CLK_OCOTP] = imx_clk_hw_gate2("ocotp", "ipg", base + 0x70, 12);
+ hws[IMX6SX_CLK_IOMUXC] = imx_clk_hw_gate2("iomuxc", "lcdif1_podf", base + 0x70, 14);
+ hws[IMX6SX_CLK_IPMUX1] = imx_clk_hw_gate2_flags("ipmux1", "ahb", base + 0x70, 16, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_IPMUX2] = imx_clk_hw_gate2_flags("ipmux2", "ahb", base + 0x70, 18, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_IPMUX3] = imx_clk_hw_gate2_flags("ipmux3", "ahb", base + 0x70, 20, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_TZASC1] = imx_clk_hw_gate2_flags("tzasc1", "mmdc_podf", base + 0x70, 22, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_LCDIF_APB] = imx_clk_hw_gate2("lcdif_apb", "display_podf", base + 0x70, 28);
+ hws[IMX6SX_CLK_PXP_AXI] = imx_clk_hw_gate2("pxp_axi", "display_podf", base + 0x70, 30);
/* CCGR3 */
- clks[IMX6SX_CLK_M4] = imx_clk_gate2("m4", "m4_podf", base + 0x74, 2);
- clks[IMX6SX_CLK_ENET] = imx_clk_gate2("enet", "ipg", base + 0x74, 4);
- clks[IMX6SX_CLK_ENET_AHB] = imx_clk_gate2("enet_ahb", "enet_sel", base + 0x74, 4);
- clks[IMX6SX_CLK_DISPLAY_AXI] = imx_clk_gate2("display_axi", "display_podf", base + 0x74, 6);
- clks[IMX6SX_CLK_LCDIF2_PIX] = imx_clk_gate2("lcdif2_pix", "lcdif2_sel", base + 0x74, 8);
- clks[IMX6SX_CLK_LCDIF1_PIX] = imx_clk_gate2("lcdif1_pix", "lcdif1_sel", base + 0x74, 10);
- clks[IMX6SX_CLK_LDB_DI0] = imx_clk_gate2("ldb_di0", "ldb_di0_div_sel", base + 0x74, 12);
- clks[IMX6SX_CLK_QSPI1] = imx_clk_gate2("qspi1", "qspi1_podf", base + 0x74, 14);
- clks[IMX6SX_CLK_MLB] = imx_clk_gate2("mlb", "ahb", base + 0x74, 18);
- clks[IMX6SX_CLK_MMDC_P0_FAST] = imx_clk_gate2_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_MMDC_P0_IPG] = imx_clk_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_MMDC_P1_IPG] = imx_clk_gate2("mmdc_p1_ipg", "ipg", base + 0x74, 26);
- clks[IMX6SX_CLK_OCRAM] = imx_clk_gate2_flags("ocram", "ocram_podf", base + 0x74, 28, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_M4] = imx_clk_hw_gate2("m4", "m4_podf", base + 0x74, 2);
+ hws[IMX6SX_CLK_ENET] = imx_clk_hw_gate2("enet", "ipg", base + 0x74, 4);
+ hws[IMX6SX_CLK_ENET_AHB] = imx_clk_hw_gate2("enet_ahb", "enet_sel", base + 0x74, 4);
+ hws[IMX6SX_CLK_DISPLAY_AXI] = imx_clk_hw_gate2("display_axi", "display_podf", base + 0x74, 6);
+ hws[IMX6SX_CLK_LCDIF2_PIX] = imx_clk_hw_gate2("lcdif2_pix", "lcdif2_sel", base + 0x74, 8);
+ hws[IMX6SX_CLK_LCDIF1_PIX] = imx_clk_hw_gate2("lcdif1_pix", "lcdif1_sel", base + 0x74, 10);
+ hws[IMX6SX_CLK_LDB_DI0] = imx_clk_hw_gate2("ldb_di0", "ldb_di0_div_sel", base + 0x74, 12);
+ hws[IMX6SX_CLK_QSPI1] = imx_clk_hw_gate2("qspi1", "qspi1_podf", base + 0x74, 14);
+ hws[IMX6SX_CLK_MLB] = imx_clk_hw_gate2("mlb", "ahb", base + 0x74, 18);
+ hws[IMX6SX_CLK_MMDC_P0_FAST] = imx_clk_hw_gate2_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_MMDC_P0_IPG] = imx_clk_hw_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_MMDC_P1_IPG] = imx_clk_hw_gate2_flags("mmdc_p1_ipg", "ipg", base + 0x74, 26, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_OCRAM] = imx_clk_hw_gate2_flags("ocram", "ocram_podf", base + 0x74, 28, CLK_IS_CRITICAL);
/* CCGR4 */
- clks[IMX6SX_CLK_PCIE_AXI] = imx_clk_gate2("pcie_axi", "display_podf", base + 0x78, 0);
- clks[IMX6SX_CLK_QSPI2] = imx_clk_gate2("qspi2", "qspi2_podf", base + 0x78, 10);
- clks[IMX6SX_CLK_PER1_BCH] = imx_clk_gate2("per1_bch", "usdhc3", base + 0x78, 12);
- clks[IMX6SX_CLK_PER2_MAIN] = imx_clk_gate2_flags("per2_main", "ahb", base + 0x78, 14, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_PWM1] = imx_clk_gate2("pwm1", "perclk", base + 0x78, 16);
- clks[IMX6SX_CLK_PWM2] = imx_clk_gate2("pwm2", "perclk", base + 0x78, 18);
- clks[IMX6SX_CLK_PWM3] = imx_clk_gate2("pwm3", "perclk", base + 0x78, 20);
- clks[IMX6SX_CLK_PWM4] = imx_clk_gate2("pwm4", "perclk", base + 0x78, 22);
- clks[IMX6SX_CLK_GPMI_BCH_APB] = imx_clk_gate2("gpmi_bch_apb", "usdhc3", base + 0x78, 24);
- clks[IMX6SX_CLK_GPMI_BCH] = imx_clk_gate2("gpmi_bch", "usdhc4", base + 0x78, 26);
- clks[IMX6SX_CLK_GPMI_IO] = imx_clk_gate2("gpmi_io", "qspi2_podf", base + 0x78, 28);
- clks[IMX6SX_CLK_GPMI_APB] = imx_clk_gate2("gpmi_apb", "usdhc3", base + 0x78, 30);
+ hws[IMX6SX_CLK_PCIE_AXI] = imx_clk_hw_gate2("pcie_axi", "display_podf", base + 0x78, 0);
+ hws[IMX6SX_CLK_QSPI2] = imx_clk_hw_gate2("qspi2", "qspi2_podf", base + 0x78, 10);
+ hws[IMX6SX_CLK_PER1_BCH] = imx_clk_hw_gate2("per1_bch", "usdhc3", base + 0x78, 12);
+ hws[IMX6SX_CLK_PER2_MAIN] = imx_clk_hw_gate2_flags("per2_main", "ahb", base + 0x78, 14, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_PWM1] = imx_clk_hw_gate2("pwm1", "perclk", base + 0x78, 16);
+ hws[IMX6SX_CLK_PWM2] = imx_clk_hw_gate2("pwm2", "perclk", base + 0x78, 18);
+ hws[IMX6SX_CLK_PWM3] = imx_clk_hw_gate2("pwm3", "perclk", base + 0x78, 20);
+ hws[IMX6SX_CLK_PWM4] = imx_clk_hw_gate2("pwm4", "perclk", base + 0x78, 22);
+ hws[IMX6SX_CLK_GPMI_BCH_APB] = imx_clk_hw_gate2("gpmi_bch_apb", "usdhc3", base + 0x78, 24);
+ hws[IMX6SX_CLK_GPMI_BCH] = imx_clk_hw_gate2("gpmi_bch", "usdhc4", base + 0x78, 26);
+ hws[IMX6SX_CLK_GPMI_IO] = imx_clk_hw_gate2("gpmi_io", "qspi2_podf", base + 0x78, 28);
+ hws[IMX6SX_CLK_GPMI_APB] = imx_clk_hw_gate2("gpmi_apb", "usdhc3", base + 0x78, 30);
/* CCGR5 */
- clks[IMX6SX_CLK_ROM] = imx_clk_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
- clks[IMX6SX_CLK_SDMA] = imx_clk_gate2("sdma", "ahb", base + 0x7c, 6);
- clks[IMX6SX_CLK_SPBA] = imx_clk_gate2("spba", "ipg", base + 0x7c, 12);
- clks[IMX6SX_CLK_AUDIO] = imx_clk_gate2_shared("audio", "audio_podf", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SX_CLK_SPDIF] = imx_clk_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SX_CLK_SPDIF_GCLK] = imx_clk_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
- clks[IMX6SX_CLK_SSI1_IPG] = imx_clk_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SX_CLK_SSI2_IPG] = imx_clk_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SX_CLK_SSI3_IPG] = imx_clk_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SX_CLK_SSI1] = imx_clk_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
- clks[IMX6SX_CLK_SSI2] = imx_clk_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
- clks[IMX6SX_CLK_SSI3] = imx_clk_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
- clks[IMX6SX_CLK_UART_IPG] = imx_clk_gate2("uart_ipg", "ipg", base + 0x7c, 24);
- clks[IMX6SX_CLK_UART_SERIAL] = imx_clk_gate2("uart_serial", "uart_podf", base + 0x7c, 26);
- clks[IMX6SX_CLK_SAI1_IPG] = imx_clk_gate2_shared("sai1_ipg", "ipg", base + 0x7c, 28, &share_count_sai1);
- clks[IMX6SX_CLK_SAI2_IPG] = imx_clk_gate2_shared("sai2_ipg", "ipg", base + 0x7c, 30, &share_count_sai2);
- clks[IMX6SX_CLK_SAI1] = imx_clk_gate2_shared("sai1", "ssi1_podf", base + 0x7c, 28, &share_count_sai1);
- clks[IMX6SX_CLK_SAI2] = imx_clk_gate2_shared("sai2", "ssi2_podf", base + 0x7c, 30, &share_count_sai2);
+ hws[IMX6SX_CLK_ROM] = imx_clk_hw_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
+ hws[IMX6SX_CLK_SDMA] = imx_clk_hw_gate2("sdma", "ahb", base + 0x7c, 6);
+ hws[IMX6SX_CLK_SPBA] = imx_clk_hw_gate2("spba", "ipg", base + 0x7c, 12);
+ hws[IMX6SX_CLK_AUDIO] = imx_clk_hw_gate2_shared("audio", "audio_podf", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SX_CLK_SPDIF] = imx_clk_hw_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SX_CLK_SPDIF_GCLK] = imx_clk_hw_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6SX_CLK_SSI1_IPG] = imx_clk_hw_gate2_shared("ssi1_ipg", "ipg", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SX_CLK_SSI2_IPG] = imx_clk_hw_gate2_shared("ssi2_ipg", "ipg", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SX_CLK_SSI3_IPG] = imx_clk_hw_gate2_shared("ssi3_ipg", "ipg", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SX_CLK_SSI1] = imx_clk_hw_gate2_shared("ssi1", "ssi1_podf", base + 0x7c, 18, &share_count_ssi1);
+ hws[IMX6SX_CLK_SSI2] = imx_clk_hw_gate2_shared("ssi2", "ssi2_podf", base + 0x7c, 20, &share_count_ssi2);
+ hws[IMX6SX_CLK_SSI3] = imx_clk_hw_gate2_shared("ssi3", "ssi3_podf", base + 0x7c, 22, &share_count_ssi3);
+ hws[IMX6SX_CLK_UART_IPG] = imx_clk_hw_gate2("uart_ipg", "ipg", base + 0x7c, 24);
+ hws[IMX6SX_CLK_UART_SERIAL] = imx_clk_hw_gate2("uart_serial", "uart_podf", base + 0x7c, 26);
+ hws[IMX6SX_CLK_SAI1_IPG] = imx_clk_hw_gate2_shared("sai1_ipg", "ipg", base + 0x7c, 28, &share_count_sai1);
+ hws[IMX6SX_CLK_SAI2_IPG] = imx_clk_hw_gate2_shared("sai2_ipg", "ipg", base + 0x7c, 30, &share_count_sai2);
+ hws[IMX6SX_CLK_SAI1] = imx_clk_hw_gate2_shared("sai1", "ssi1_podf", base + 0x7c, 28, &share_count_sai1);
+ hws[IMX6SX_CLK_SAI2] = imx_clk_hw_gate2_shared("sai2", "ssi2_podf", base + 0x7c, 30, &share_count_sai2);
/* CCGR6 */
- clks[IMX6SX_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
- clks[IMX6SX_CLK_USDHC1] = imx_clk_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
- clks[IMX6SX_CLK_USDHC2] = imx_clk_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
- clks[IMX6SX_CLK_USDHC3] = imx_clk_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
- clks[IMX6SX_CLK_USDHC4] = imx_clk_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
- clks[IMX6SX_CLK_EIM_SLOW] = imx_clk_gate2("eim_slow", "eim_slow_podf", base + 0x80, 10);
- clks[IMX6SX_CLK_PWM8] = imx_clk_gate2("pwm8", "perclk", base + 0x80, 16);
- clks[IMX6SX_CLK_VADC] = imx_clk_gate2("vadc", "vid_podf", base + 0x80, 20);
- clks[IMX6SX_CLK_GIS] = imx_clk_gate2("gis", "display_podf", base + 0x80, 22);
- clks[IMX6SX_CLK_I2C4] = imx_clk_gate2("i2c4", "perclk", base + 0x80, 24);
- clks[IMX6SX_CLK_PWM5] = imx_clk_gate2("pwm5", "perclk", base + 0x80, 26);
- clks[IMX6SX_CLK_PWM6] = imx_clk_gate2("pwm6", "perclk", base + 0x80, 28);
- clks[IMX6SX_CLK_PWM7] = imx_clk_gate2("pwm7", "perclk", base + 0x80, 30);
-
- clks[IMX6SX_CLK_CKO1] = imx_clk_gate("cko1", "cko1_podf", base + 0x60, 7);
- clks[IMX6SX_CLK_CKO2] = imx_clk_gate("cko2", "cko2_podf", base + 0x60, 24);
+ hws[IMX6SX_CLK_USBOH3] = imx_clk_hw_gate2("usboh3", "ipg", base + 0x80, 0);
+ hws[IMX6SX_CLK_USDHC1] = imx_clk_hw_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
+ hws[IMX6SX_CLK_USDHC2] = imx_clk_hw_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
+ hws[IMX6SX_CLK_USDHC3] = imx_clk_hw_gate2("usdhc3", "usdhc3_podf", base + 0x80, 6);
+ hws[IMX6SX_CLK_USDHC4] = imx_clk_hw_gate2("usdhc4", "usdhc4_podf", base + 0x80, 8);
+ hws[IMX6SX_CLK_EIM_SLOW] = imx_clk_hw_gate2("eim_slow", "eim_slow_podf", base + 0x80, 10);
+ hws[IMX6SX_CLK_PWM8] = imx_clk_hw_gate2("pwm8", "perclk", base + 0x80, 16);
+ hws[IMX6SX_CLK_VADC] = imx_clk_hw_gate2("vadc", "vid_podf", base + 0x80, 20);
+ hws[IMX6SX_CLK_GIS] = imx_clk_hw_gate2("gis", "display_podf", base + 0x80, 22);
+ hws[IMX6SX_CLK_I2C4] = imx_clk_hw_gate2("i2c4", "perclk", base + 0x80, 24);
+ hws[IMX6SX_CLK_PWM5] = imx_clk_hw_gate2("pwm5", "perclk", base + 0x80, 26);
+ hws[IMX6SX_CLK_PWM6] = imx_clk_hw_gate2("pwm6", "perclk", base + 0x80, 28);
+ hws[IMX6SX_CLK_PWM7] = imx_clk_hw_gate2("pwm7", "perclk", base + 0x80, 30);
+
+ hws[IMX6SX_CLK_CKO1] = imx_clk_hw_gate("cko1", "cko1_podf", base + 0x60, 7);
+ hws[IMX6SX_CLK_CKO2] = imx_clk_hw_gate("cko2", "cko2_podf", base + 0x60, 24);
/* mask handshake of mmdc */
- writel_relaxed(BM_CCM_CCDR_MMDC_CH0_MASK, base + CCDR);
+ imx_mmdc_mask_handshake(base, 0);
- imx_check_clocks(clks, ARRAY_SIZE(clks));
+ imx_check_clk_hws(hws, IMX6SX_CLK_CLK_END);
- clk_data.clks = clks;
- clk_data.clk_num = ARRAY_SIZE(clks);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
if (IS_ENABLED(CONFIG_USB_MXS_PHY)) {
- clk_prepare_enable(clks[IMX6SX_CLK_USBPHY1_GATE]);
- clk_prepare_enable(clks[IMX6SX_CLK_USBPHY2_GATE]);
+ clk_prepare_enable(hws[IMX6SX_CLK_USBPHY1_GATE]->clk);
+ clk_prepare_enable(hws[IMX6SX_CLK_USBPHY2_GATE]->clk);
}
/* Set the default 132MHz for EIM module */
- clk_set_parent(clks[IMX6SX_CLK_EIM_SLOW_SEL], clks[IMX6SX_CLK_PLL2_PFD2]);
- clk_set_rate(clks[IMX6SX_CLK_EIM_SLOW], 132000000);
+ clk_set_parent(hws[IMX6SX_CLK_EIM_SLOW_SEL]->clk, hws[IMX6SX_CLK_PLL2_PFD2]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_EIM_SLOW]->clk, 132000000);
/* set parent clock for LCDIF1 pixel clock */
- clk_set_parent(clks[IMX6SX_CLK_LCDIF1_PRE_SEL], clks[IMX6SX_CLK_PLL5_VIDEO_DIV]);
- clk_set_parent(clks[IMX6SX_CLK_LCDIF1_SEL], clks[IMX6SX_CLK_LCDIF1_PODF]);
+ clk_set_parent(hws[IMX6SX_CLK_LCDIF1_PRE_SEL]->clk, hws[IMX6SX_CLK_PLL5_VIDEO_DIV]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_LCDIF1_SEL]->clk, hws[IMX6SX_CLK_LCDIF1_PODF]->clk);
/* Set the parent clks of PCIe lvds1 and pcie_axi to be pcie ref, axi */
- if (clk_set_parent(clks[IMX6SX_CLK_LVDS1_SEL], clks[IMX6SX_CLK_PCIE_REF_125M]))
+ if (clk_set_parent(hws[IMX6SX_CLK_LVDS1_SEL]->clk, hws[IMX6SX_CLK_PCIE_REF_125M]->clk))
pr_err("Failed to set pcie bus parent clk.\n");
- if (clk_set_parent(clks[IMX6SX_CLK_PCIE_AXI_SEL], clks[IMX6SX_CLK_AXI]))
- pr_err("Failed to set pcie parent clk.\n");
/*
* Init enet system AHB clock, set to 200MHz
* pll2_pfd2_396m-> ENET_PODF-> ENET_AHB
*/
- clk_set_parent(clks[IMX6SX_CLK_ENET_PRE_SEL], clks[IMX6SX_CLK_PLL2_PFD2]);
- clk_set_parent(clks[IMX6SX_CLK_ENET_SEL], clks[IMX6SX_CLK_ENET_PODF]);
- clk_set_rate(clks[IMX6SX_CLK_ENET_PODF], 200000000);
- clk_set_rate(clks[IMX6SX_CLK_ENET_REF], 125000000);
- clk_set_rate(clks[IMX6SX_CLK_ENET2_REF], 125000000);
+ clk_set_parent(hws[IMX6SX_CLK_ENET_PRE_SEL]->clk, hws[IMX6SX_CLK_PLL2_PFD2]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_ENET_SEL]->clk, hws[IMX6SX_CLK_ENET_PODF]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_ENET_PODF]->clk, 200000000);
+ clk_set_rate(hws[IMX6SX_CLK_ENET_REF]->clk, 125000000);
+ clk_set_rate(hws[IMX6SX_CLK_ENET2_REF]->clk, 125000000);
/* Audio clocks */
- clk_set_rate(clks[IMX6SX_CLK_PLL4_AUDIO_DIV], 393216000);
+ clk_set_rate(hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk, 393216000);
- clk_set_parent(clks[IMX6SX_CLK_SPDIF_SEL], clks[IMX6SX_CLK_PLL4_AUDIO_DIV]);
- clk_set_rate(clks[IMX6SX_CLK_SPDIF_PODF], 98304000);
+ clk_set_parent(hws[IMX6SX_CLK_SPDIF_SEL]->clk, hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_SPDIF_PODF]->clk, 98304000);
- clk_set_parent(clks[IMX6SX_CLK_AUDIO_SEL], clks[IMX6SX_CLK_PLL3_USB_OTG]);
- clk_set_rate(clks[IMX6SX_CLK_AUDIO_PODF], 24000000);
+ clk_set_parent(hws[IMX6SX_CLK_AUDIO_SEL]->clk, hws[IMX6SX_CLK_PLL3_USB_OTG]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_AUDIO_PODF]->clk, 24000000);
- clk_set_parent(clks[IMX6SX_CLK_SSI1_SEL], clks[IMX6SX_CLK_PLL4_AUDIO_DIV]);
- clk_set_parent(clks[IMX6SX_CLK_SSI2_SEL], clks[IMX6SX_CLK_PLL4_AUDIO_DIV]);
- clk_set_parent(clks[IMX6SX_CLK_SSI3_SEL], clks[IMX6SX_CLK_PLL4_AUDIO_DIV]);
- clk_set_rate(clks[IMX6SX_CLK_SSI1_PODF], 24576000);
- clk_set_rate(clks[IMX6SX_CLK_SSI2_PODF], 24576000);
- clk_set_rate(clks[IMX6SX_CLK_SSI3_PODF], 24576000);
+ clk_set_parent(hws[IMX6SX_CLK_SSI1_SEL]->clk, hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_SSI2_SEL]->clk, hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_SSI3_SEL]->clk, hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_SSI1_PODF]->clk, 24576000);
+ clk_set_rate(hws[IMX6SX_CLK_SSI2_PODF]->clk, 24576000);
+ clk_set_rate(hws[IMX6SX_CLK_SSI3_PODF]->clk, 24576000);
- clk_set_parent(clks[IMX6SX_CLK_ESAI_SEL], clks[IMX6SX_CLK_PLL4_AUDIO_DIV]);
- clk_set_rate(clks[IMX6SX_CLK_ESAI_PODF], 24576000);
+ clk_set_parent(hws[IMX6SX_CLK_ESAI_SEL]->clk, hws[IMX6SX_CLK_PLL4_AUDIO_DIV]->clk);
+ clk_set_rate(hws[IMX6SX_CLK_ESAI_PODF]->clk, 24576000);
/* Set parent clock for vadc */
- clk_set_parent(clks[IMX6SX_CLK_VID_SEL], clks[IMX6SX_CLK_PLL3_USB_OTG]);
+ clk_set_parent(hws[IMX6SX_CLK_VID_SEL]->clk, hws[IMX6SX_CLK_PLL3_USB_OTG]->clk);
/* default parent of can_sel clock is invalid, manually set it here */
- clk_set_parent(clks[IMX6SX_CLK_CAN_SEL], clks[IMX6SX_CLK_PLL3_60M]);
+ clk_set_parent(hws[IMX6SX_CLK_CAN_SEL]->clk, hws[IMX6SX_CLK_PLL3_60M]->clk);
/* Update gpu clock from default 528M to 720M */
- clk_set_parent(clks[IMX6SX_CLK_GPU_CORE_SEL], clks[IMX6SX_CLK_PLL3_PFD0]);
- clk_set_parent(clks[IMX6SX_CLK_GPU_AXI_SEL], clks[IMX6SX_CLK_PLL3_PFD0]);
+ clk_set_parent(hws[IMX6SX_CLK_GPU_CORE_SEL]->clk, hws[IMX6SX_CLK_PLL3_PFD0]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_GPU_AXI_SEL]->clk, hws[IMX6SX_CLK_PLL3_PFD0]->clk);
- clk_set_parent(clks[IMX6SX_CLK_QSPI1_SEL], clks[IMX6SX_CLK_PLL2_BUS]);
- clk_set_parent(clks[IMX6SX_CLK_QSPI2_SEL], clks[IMX6SX_CLK_PLL2_BUS]);
+ clk_set_parent(hws[IMX6SX_CLK_QSPI1_SEL]->clk, hws[IMX6SX_CLK_PLL2_BUS]->clk);
+ clk_set_parent(hws[IMX6SX_CLK_QSPI2_SEL]->clk, hws[IMX6SX_CLK_PLL2_BUS]->clk);
+
+ for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
+ int index = uart_clk_ids[i];
+
+ uart_clks[i] = &hws[index]->clk;
+ }
imx_register_uart_clocks(uart_clks);
}
#include <dt-bindings/clock/imx6ul-clock.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/io.h>
#include "clk.h"
-#define BM_CCM_CCDR_MMDC_CH0_MASK (0x2 << 16)
-#define CCDR 0x4
-
static const char *pll_bypass_src_sels[] = { "osc", "dummy", };
static const char *pll1_bypass_sels[] = { "pll1", "pll1_bypass_src", };
static const char *pll2_bypass_sels[] = { "pll2", "pll2_bypass_src", };
"dummy", "dummy", "dummy", "dummy", "uart_serial", "spdif", "dummy", "dummy", };
static const char *cko_sels[] = { "cko1", "cko2", };
-static struct clk *clks[IMX6UL_CLK_END];
-static struct clk_onecell_data clk_data;
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
static const struct clk_div_table clk_enet_ref_table[] = {
{ .val = 0, .div = 20, },
struct device_node *np;
void __iomem *base;
- clks[IMX6UL_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX6UL_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
+
+ clk_hw_data->num = IMX6UL_CLK_END;
+ hws = clk_hw_data->hws;
+
+ hws[IMX6UL_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
- clks[IMX6UL_CLK_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
- clks[IMX6UL_CLK_OSC] = of_clk_get_by_name(ccm_node, "osc");
+ hws[IMX6UL_CLK_CKIL] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ckil"));
+ hws[IMX6UL_CLK_OSC] = __clk_get_hw(of_clk_get_by_name(ccm_node, "osc"));
/* ipp_di clock is external input */
- clks[IMX6UL_CLK_IPP_DI0] = of_clk_get_by_name(ccm_node, "ipp_di0");
- clks[IMX6UL_CLK_IPP_DI1] = of_clk_get_by_name(ccm_node, "ipp_di1");
+ hws[IMX6UL_CLK_IPP_DI0] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di0"));
+ hws[IMX6UL_CLK_IPP_DI1] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ipp_di1"));
np = of_find_compatible_node(NULL, NULL, "fsl,imx6ul-anatop");
base = of_iomap(np, 0);
of_node_put(np);
WARN_ON(!base);
- clks[IMX6UL_PLL1_BYPASS_SRC] = imx_clk_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL2_BYPASS_SRC] = imx_clk_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL3_BYPASS_SRC] = imx_clk_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL4_BYPASS_SRC] = imx_clk_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL5_BYPASS_SRC] = imx_clk_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL6_BYPASS_SRC] = imx_clk_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
- clks[IMX6UL_PLL7_BYPASS_SRC] = imx_clk_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
-
- clks[IMX6UL_CLK_PLL1] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
- clks[IMX6UL_CLK_PLL2] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
- clks[IMX6UL_CLK_PLL3] = imx_clk_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
- clks[IMX6UL_CLK_PLL4] = imx_clk_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
- clks[IMX6UL_CLK_PLL5] = imx_clk_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
- clks[IMX6UL_CLK_PLL6] = imx_clk_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
- clks[IMX6UL_CLK_PLL7] = imx_clk_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
-
- clks[IMX6UL_PLL1_BYPASS] = imx_clk_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL2_BYPASS] = imx_clk_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL3_BYPASS] = imx_clk_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL4_BYPASS] = imx_clk_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL5_BYPASS] = imx_clk_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL6_BYPASS] = imx_clk_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_PLL7_BYPASS] = imx_clk_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_CLK_CSI_SEL] = imx_clk_mux_flags("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL1_BYPASS_SRC] = imx_clk_hw_mux("pll1_bypass_src", base + 0x00, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL2_BYPASS_SRC] = imx_clk_hw_mux("pll2_bypass_src", base + 0x30, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL3_BYPASS_SRC] = imx_clk_hw_mux("pll3_bypass_src", base + 0x10, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL4_BYPASS_SRC] = imx_clk_hw_mux("pll4_bypass_src", base + 0x70, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL5_BYPASS_SRC] = imx_clk_hw_mux("pll5_bypass_src", base + 0xa0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL6_BYPASS_SRC] = imx_clk_hw_mux("pll6_bypass_src", base + 0xe0, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+ hws[IMX6UL_PLL7_BYPASS_SRC] = imx_clk_hw_mux("pll7_bypass_src", base + 0x20, 14, 1, pll_bypass_src_sels, ARRAY_SIZE(pll_bypass_src_sels));
+
+ hws[IMX6UL_CLK_PLL1] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll1", "osc", base + 0x00, 0x7f);
+ hws[IMX6UL_CLK_PLL2] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll2", "osc", base + 0x30, 0x1);
+ hws[IMX6UL_CLK_PLL3] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll3", "osc", base + 0x10, 0x3);
+ hws[IMX6UL_CLK_PLL4] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll4", "osc", base + 0x70, 0x7f);
+ hws[IMX6UL_CLK_PLL5] = imx_clk_hw_pllv3(IMX_PLLV3_AV, "pll5", "osc", base + 0xa0, 0x7f);
+ hws[IMX6UL_CLK_PLL6] = imx_clk_hw_pllv3(IMX_PLLV3_ENET, "pll6", "osc", base + 0xe0, 0x3);
+ hws[IMX6UL_CLK_PLL7] = imx_clk_hw_pllv3(IMX_PLLV3_USB, "pll7", "osc", base + 0x20, 0x3);
+
+ hws[IMX6UL_PLL1_BYPASS] = imx_clk_hw_mux_flags("pll1_bypass", base + 0x00, 16, 1, pll1_bypass_sels, ARRAY_SIZE(pll1_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL2_BYPASS] = imx_clk_hw_mux_flags("pll2_bypass", base + 0x30, 16, 1, pll2_bypass_sels, ARRAY_SIZE(pll2_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL3_BYPASS] = imx_clk_hw_mux_flags("pll3_bypass", base + 0x10, 16, 1, pll3_bypass_sels, ARRAY_SIZE(pll3_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL4_BYPASS] = imx_clk_hw_mux_flags("pll4_bypass", base + 0x70, 16, 1, pll4_bypass_sels, ARRAY_SIZE(pll4_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL5_BYPASS] = imx_clk_hw_mux_flags("pll5_bypass", base + 0xa0, 16, 1, pll5_bypass_sels, ARRAY_SIZE(pll5_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL6_BYPASS] = imx_clk_hw_mux_flags("pll6_bypass", base + 0xe0, 16, 1, pll6_bypass_sels, ARRAY_SIZE(pll6_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_PLL7_BYPASS] = imx_clk_hw_mux_flags("pll7_bypass", base + 0x20, 16, 1, pll7_bypass_sels, ARRAY_SIZE(pll7_bypass_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_CLK_CSI_SEL] = imx_clk_hw_mux_flags("csi_sel", base + 0x3c, 9, 2, csi_sels, ARRAY_SIZE(csi_sels), CLK_SET_RATE_PARENT);
/* Do not bypass PLLs initially */
- clk_set_parent(clks[IMX6UL_PLL1_BYPASS], clks[IMX6UL_CLK_PLL1]);
- clk_set_parent(clks[IMX6UL_PLL2_BYPASS], clks[IMX6UL_CLK_PLL2]);
- clk_set_parent(clks[IMX6UL_PLL3_BYPASS], clks[IMX6UL_CLK_PLL3]);
- clk_set_parent(clks[IMX6UL_PLL4_BYPASS], clks[IMX6UL_CLK_PLL4]);
- clk_set_parent(clks[IMX6UL_PLL5_BYPASS], clks[IMX6UL_CLK_PLL5]);
- clk_set_parent(clks[IMX6UL_PLL6_BYPASS], clks[IMX6UL_CLK_PLL6]);
- clk_set_parent(clks[IMX6UL_PLL7_BYPASS], clks[IMX6UL_CLK_PLL7]);
-
- clks[IMX6UL_CLK_PLL1_SYS] = imx_clk_fixed_factor("pll1_sys", "pll1_bypass", 1, 1);
- clks[IMX6UL_CLK_PLL2_BUS] = imx_clk_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
- clks[IMX6UL_CLK_PLL3_USB_OTG] = imx_clk_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
- clks[IMX6UL_CLK_PLL4_AUDIO] = imx_clk_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
- clks[IMX6UL_CLK_PLL5_VIDEO] = imx_clk_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
- clks[IMX6UL_CLK_PLL6_ENET] = imx_clk_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
- clks[IMX6UL_CLK_PLL7_USB_HOST] = imx_clk_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
+ clk_set_parent(hws[IMX6UL_PLL1_BYPASS]->clk, hws[IMX6UL_CLK_PLL1]->clk);
+ clk_set_parent(hws[IMX6UL_PLL2_BYPASS]->clk, hws[IMX6UL_CLK_PLL2]->clk);
+ clk_set_parent(hws[IMX6UL_PLL3_BYPASS]->clk, hws[IMX6UL_CLK_PLL3]->clk);
+ clk_set_parent(hws[IMX6UL_PLL4_BYPASS]->clk, hws[IMX6UL_CLK_PLL4]->clk);
+ clk_set_parent(hws[IMX6UL_PLL5_BYPASS]->clk, hws[IMX6UL_CLK_PLL5]->clk);
+ clk_set_parent(hws[IMX6UL_PLL6_BYPASS]->clk, hws[IMX6UL_CLK_PLL6]->clk);
+ clk_set_parent(hws[IMX6UL_PLL7_BYPASS]->clk, hws[IMX6UL_CLK_PLL7]->clk);
+
+ hws[IMX6UL_CLK_PLL1_SYS] = imx_clk_hw_fixed_factor("pll1_sys", "pll1_bypass", 1, 1);
+ hws[IMX6UL_CLK_PLL2_BUS] = imx_clk_hw_gate("pll2_bus", "pll2_bypass", base + 0x30, 13);
+ hws[IMX6UL_CLK_PLL3_USB_OTG] = imx_clk_hw_gate("pll3_usb_otg", "pll3_bypass", base + 0x10, 13);
+ hws[IMX6UL_CLK_PLL4_AUDIO] = imx_clk_hw_gate("pll4_audio", "pll4_bypass", base + 0x70, 13);
+ hws[IMX6UL_CLK_PLL5_VIDEO] = imx_clk_hw_gate("pll5_video", "pll5_bypass", base + 0xa0, 13);
+ hws[IMX6UL_CLK_PLL6_ENET] = imx_clk_hw_gate("pll6_enet", "pll6_bypass", base + 0xe0, 13);
+ hws[IMX6UL_CLK_PLL7_USB_HOST] = imx_clk_hw_gate("pll7_usb_host", "pll7_bypass", base + 0x20, 13);
/*
* Bit 20 is the reserved and read-only bit, we do this only for:
* - Keep refcount when do usbphy clk_enable/disable, in that case,
* the clk framework many need to enable/disable usbphy's parent
*/
- clks[IMX6UL_CLK_USBPHY1] = imx_clk_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
- clks[IMX6UL_CLK_USBPHY2] = imx_clk_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
+ hws[IMX6UL_CLK_USBPHY1] = imx_clk_hw_gate("usbphy1", "pll3_usb_otg", base + 0x10, 20);
+ hws[IMX6UL_CLK_USBPHY2] = imx_clk_hw_gate("usbphy2", "pll7_usb_host", base + 0x20, 20);
/*
* usbphy*_gate needs to be on after system boots up, and software
* never needs to control it anymore.
*/
- clks[IMX6UL_CLK_USBPHY1_GATE] = imx_clk_gate("usbphy1_gate", "dummy", base + 0x10, 6);
- clks[IMX6UL_CLK_USBPHY2_GATE] = imx_clk_gate("usbphy2_gate", "dummy", base + 0x20, 6);
+ hws[IMX6UL_CLK_USBPHY1_GATE] = imx_clk_hw_gate("usbphy1_gate", "dummy", base + 0x10, 6);
+ hws[IMX6UL_CLK_USBPHY2_GATE] = imx_clk_hw_gate("usbphy2_gate", "dummy", base + 0x20, 6);
/* name parent_name reg idx */
- clks[IMX6UL_CLK_PLL2_PFD0] = imx_clk_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
- clks[IMX6UL_CLK_PLL2_PFD1] = imx_clk_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
- clks[IMX6UL_CLK_PLL2_PFD2] = imx_clk_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
- clks[IMX6UL_CLK_PLL2_PFD3] = imx_clk_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
- clks[IMX6UL_CLK_PLL3_PFD0] = imx_clk_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
- clks[IMX6UL_CLK_PLL3_PFD1] = imx_clk_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
- clks[IMX6UL_CLK_PLL3_PFD2] = imx_clk_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
- clks[IMX6UL_CLK_PLL3_PFD3] = imx_clk_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
-
- clks[IMX6UL_CLK_ENET_REF] = clk_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
+ hws[IMX6UL_CLK_PLL2_PFD0] = imx_clk_hw_pfd("pll2_pfd0_352m", "pll2_bus", base + 0x100, 0);
+ hws[IMX6UL_CLK_PLL2_PFD1] = imx_clk_hw_pfd("pll2_pfd1_594m", "pll2_bus", base + 0x100, 1);
+ hws[IMX6UL_CLK_PLL2_PFD2] = imx_clk_hw_pfd("pll2_pfd2_396m", "pll2_bus", base + 0x100, 2);
+ hws[IMX6UL_CLK_PLL2_PFD3] = imx_clk_hw_pfd("pll2_pfd3_594m", "pll2_bus", base + 0x100, 3);
+ hws[IMX6UL_CLK_PLL3_PFD0] = imx_clk_hw_pfd("pll3_pfd0_720m", "pll3_usb_otg", base + 0xf0, 0);
+ hws[IMX6UL_CLK_PLL3_PFD1] = imx_clk_hw_pfd("pll3_pfd1_540m", "pll3_usb_otg", base + 0xf0, 1);
+ hws[IMX6UL_CLK_PLL3_PFD2] = imx_clk_hw_pfd("pll3_pfd2_508m", "pll3_usb_otg", base + 0xf0, 2);
+ hws[IMX6UL_CLK_PLL3_PFD3] = imx_clk_hw_pfd("pll3_pfd3_454m", "pll3_usb_otg", base + 0xf0, 3);
+
+ hws[IMX6UL_CLK_ENET_REF] = clk_hw_register_divider_table(NULL, "enet_ref", "pll6_enet", 0,
base + 0xe0, 0, 2, 0, clk_enet_ref_table, &imx_ccm_lock);
- clks[IMX6UL_CLK_ENET2_REF] = clk_register_divider_table(NULL, "enet2_ref", "pll6_enet", 0,
+ hws[IMX6UL_CLK_ENET2_REF] = clk_hw_register_divider_table(NULL, "enet2_ref", "pll6_enet", 0,
base + 0xe0, 2, 2, 0, clk_enet_ref_table, &imx_ccm_lock);
- clks[IMX6UL_CLK_ENET2_REF_125M] = imx_clk_gate("enet_ref_125m", "enet2_ref", base + 0xe0, 20);
- clks[IMX6UL_CLK_ENET_PTP_REF] = imx_clk_fixed_factor("enet_ptp_ref", "pll6_enet", 1, 20);
- clks[IMX6UL_CLK_ENET_PTP] = imx_clk_gate("enet_ptp", "enet_ptp_ref", base + 0xe0, 21);
+ hws[IMX6UL_CLK_ENET2_REF_125M] = imx_clk_hw_gate("enet_ref_125m", "enet2_ref", base + 0xe0, 20);
+ hws[IMX6UL_CLK_ENET_PTP_REF] = imx_clk_hw_fixed_factor("enet_ptp_ref", "pll6_enet", 1, 20);
+ hws[IMX6UL_CLK_ENET_PTP] = imx_clk_hw_gate("enet_ptp", "enet_ptp_ref", base + 0xe0, 21);
- clks[IMX6UL_CLK_PLL4_POST_DIV] = clk_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
+ hws[IMX6UL_CLK_PLL4_POST_DIV] = clk_hw_register_divider_table(NULL, "pll4_post_div", "pll4_audio",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x70, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6UL_CLK_PLL4_AUDIO_DIV] = clk_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
+ hws[IMX6UL_CLK_PLL4_AUDIO_DIV] = clk_hw_register_divider(NULL, "pll4_audio_div", "pll4_post_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x170, 15, 1, 0, &imx_ccm_lock);
- clks[IMX6UL_CLK_PLL5_POST_DIV] = clk_register_divider_table(NULL, "pll5_post_div", "pll5_video",
+ hws[IMX6UL_CLK_PLL5_POST_DIV] = clk_hw_register_divider_table(NULL, "pll5_post_div", "pll5_video",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0xa0, 19, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX6UL_CLK_PLL5_VIDEO_DIV] = clk_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
+ hws[IMX6UL_CLK_PLL5_VIDEO_DIV] = clk_hw_register_divider_table(NULL, "pll5_video_div", "pll5_post_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x170, 30, 2, 0, video_div_table, &imx_ccm_lock);
/* name parent_name mult div */
- clks[IMX6UL_CLK_PLL2_198M] = imx_clk_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
- clks[IMX6UL_CLK_PLL3_80M] = imx_clk_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
- clks[IMX6UL_CLK_PLL3_60M] = imx_clk_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
- clks[IMX6UL_CLK_GPT_3M] = imx_clk_fixed_factor("gpt_3m", "osc", 1, 8);
+ hws[IMX6UL_CLK_PLL2_198M] = imx_clk_hw_fixed_factor("pll2_198m", "pll2_pfd2_396m", 1, 2);
+ hws[IMX6UL_CLK_PLL3_80M] = imx_clk_hw_fixed_factor("pll3_80m", "pll3_usb_otg", 1, 6);
+ hws[IMX6UL_CLK_PLL3_60M] = imx_clk_hw_fixed_factor("pll3_60m", "pll3_usb_otg", 1, 8);
+ hws[IMX6UL_CLK_GPT_3M] = imx_clk_hw_fixed_factor("gpt_3m", "osc", 1, 8);
np = ccm_node;
base = of_iomap(np, 0);
WARN_ON(!base);
- clks[IMX6UL_CA7_SECONDARY_SEL] = imx_clk_mux("ca7_secondary_sel", base + 0xc, 3, 1, ca7_secondary_sels, ARRAY_SIZE(ca7_secondary_sels));
- clks[IMX6UL_CLK_STEP] = imx_clk_mux("step", base + 0x0c, 8, 1, step_sels, ARRAY_SIZE(step_sels));
- clks[IMX6UL_CLK_PLL1_SW] = imx_clk_mux_flags("pll1_sw", base + 0x0c, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels), 0);
- clks[IMX6UL_CLK_AXI_ALT_SEL] = imx_clk_mux("axi_alt_sel", base + 0x14, 7, 1, axi_alt_sels, ARRAY_SIZE(axi_alt_sels));
- clks[IMX6UL_CLK_AXI_SEL] = imx_clk_mux_flags("axi_sel", base + 0x14, 6, 1, axi_sels, ARRAY_SIZE(axi_sels), 0);
- clks[IMX6UL_CLK_PERIPH_PRE] = imx_clk_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
- clks[IMX6UL_CLK_PERIPH2_PRE] = imx_clk_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
- clks[IMX6UL_CLK_PERIPH_CLK2_SEL] = imx_clk_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
- clks[IMX6UL_CLK_PERIPH2_CLK2_SEL] = imx_clk_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
- clks[IMX6UL_CLK_EIM_SLOW_SEL] = imx_clk_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
- clks[IMX6UL_CLK_GPMI_SEL] = imx_clk_mux("gpmi_sel", base + 0x1c, 19, 1, gpmi_sels, ARRAY_SIZE(gpmi_sels));
- clks[IMX6UL_CLK_BCH_SEL] = imx_clk_mux("bch_sel", base + 0x1c, 18, 1, bch_sels, ARRAY_SIZE(bch_sels));
- clks[IMX6UL_CLK_USDHC2_SEL] = imx_clk_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6UL_CLK_USDHC1_SEL] = imx_clk_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
- clks[IMX6UL_CLK_SAI3_SEL] = imx_clk_mux("sai3_sel", base + 0x1c, 14, 2, sai_sels, ARRAY_SIZE(sai_sels));
- clks[IMX6UL_CLK_SAI2_SEL] = imx_clk_mux("sai2_sel", base + 0x1c, 12, 2, sai_sels, ARRAY_SIZE(sai_sels));
- clks[IMX6UL_CLK_SAI1_SEL] = imx_clk_mux("sai1_sel", base + 0x1c, 10, 2, sai_sels, ARRAY_SIZE(sai_sels));
- clks[IMX6UL_CLK_QSPI1_SEL] = imx_clk_mux("qspi1_sel", base + 0x1c, 7, 3, qspi1_sels, ARRAY_SIZE(qspi1_sels));
- clks[IMX6UL_CLK_PERCLK_SEL] = imx_clk_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
- clks[IMX6UL_CLK_CAN_SEL] = imx_clk_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
+ hws[IMX6UL_CA7_SECONDARY_SEL] = imx_clk_hw_mux("ca7_secondary_sel", base + 0xc, 3, 1, ca7_secondary_sels, ARRAY_SIZE(ca7_secondary_sels));
+ hws[IMX6UL_CLK_STEP] = imx_clk_hw_mux("step", base + 0x0c, 8, 1, step_sels, ARRAY_SIZE(step_sels));
+ hws[IMX6UL_CLK_PLL1_SW] = imx_clk_hw_mux_flags("pll1_sw", base + 0x0c, 2, 1, pll1_sw_sels, ARRAY_SIZE(pll1_sw_sels), 0);
+ hws[IMX6UL_CLK_AXI_ALT_SEL] = imx_clk_hw_mux("axi_alt_sel", base + 0x14, 7, 1, axi_alt_sels, ARRAY_SIZE(axi_alt_sels));
+ hws[IMX6UL_CLK_AXI_SEL] = imx_clk_hw_mux_flags("axi_sel", base + 0x14, 6, 1, axi_sels, ARRAY_SIZE(axi_sels), 0);
+ hws[IMX6UL_CLK_PERIPH_PRE] = imx_clk_hw_mux("periph_pre", base + 0x18, 18, 2, periph_pre_sels, ARRAY_SIZE(periph_pre_sels));
+ hws[IMX6UL_CLK_PERIPH2_PRE] = imx_clk_hw_mux("periph2_pre", base + 0x18, 21, 2, periph2_pre_sels, ARRAY_SIZE(periph2_pre_sels));
+ hws[IMX6UL_CLK_PERIPH_CLK2_SEL] = imx_clk_hw_mux("periph_clk2_sel", base + 0x18, 12, 2, periph_clk2_sels, ARRAY_SIZE(periph_clk2_sels));
+ hws[IMX6UL_CLK_PERIPH2_CLK2_SEL] = imx_clk_hw_mux("periph2_clk2_sel", base + 0x18, 20, 1, periph2_clk2_sels, ARRAY_SIZE(periph2_clk2_sels));
+ hws[IMX6UL_CLK_EIM_SLOW_SEL] = imx_clk_hw_mux("eim_slow_sel", base + 0x1c, 29, 2, eim_slow_sels, ARRAY_SIZE(eim_slow_sels));
+ hws[IMX6UL_CLK_GPMI_SEL] = imx_clk_hw_mux("gpmi_sel", base + 0x1c, 19, 1, gpmi_sels, ARRAY_SIZE(gpmi_sels));
+ hws[IMX6UL_CLK_BCH_SEL] = imx_clk_hw_mux("bch_sel", base + 0x1c, 18, 1, bch_sels, ARRAY_SIZE(bch_sels));
+ hws[IMX6UL_CLK_USDHC2_SEL] = imx_clk_hw_mux("usdhc2_sel", base + 0x1c, 17, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6UL_CLK_USDHC1_SEL] = imx_clk_hw_mux("usdhc1_sel", base + 0x1c, 16, 1, usdhc_sels, ARRAY_SIZE(usdhc_sels));
+ hws[IMX6UL_CLK_SAI3_SEL] = imx_clk_hw_mux("sai3_sel", base + 0x1c, 14, 2, sai_sels, ARRAY_SIZE(sai_sels));
+ hws[IMX6UL_CLK_SAI2_SEL] = imx_clk_hw_mux("sai2_sel", base + 0x1c, 12, 2, sai_sels, ARRAY_SIZE(sai_sels));
+ hws[IMX6UL_CLK_SAI1_SEL] = imx_clk_hw_mux("sai1_sel", base + 0x1c, 10, 2, sai_sels, ARRAY_SIZE(sai_sels));
+ hws[IMX6UL_CLK_QSPI1_SEL] = imx_clk_hw_mux("qspi1_sel", base + 0x1c, 7, 3, qspi1_sels, ARRAY_SIZE(qspi1_sels));
+ hws[IMX6UL_CLK_PERCLK_SEL] = imx_clk_hw_mux("perclk_sel", base + 0x1c, 6, 1, perclk_sels, ARRAY_SIZE(perclk_sels));
+ hws[IMX6UL_CLK_CAN_SEL] = imx_clk_hw_mux("can_sel", base + 0x20, 8, 2, can_sels, ARRAY_SIZE(can_sels));
if (clk_on_imx6ull())
- clks[IMX6ULL_CLK_ESAI_SEL] = imx_clk_mux("esai_sel", base + 0x20, 19, 2, esai_sels, ARRAY_SIZE(esai_sels));
- clks[IMX6UL_CLK_UART_SEL] = imx_clk_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
- clks[IMX6UL_CLK_ENFC_SEL] = imx_clk_mux("enfc_sel", base + 0x2c, 15, 3, enfc_sels, ARRAY_SIZE(enfc_sels));
- clks[IMX6UL_CLK_LDB_DI0_SEL] = imx_clk_mux("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels));
- clks[IMX6UL_CLK_SPDIF_SEL] = imx_clk_mux("spdif_sel", base + 0x30, 20, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
+ hws[IMX6ULL_CLK_ESAI_SEL] = imx_clk_hw_mux("esai_sel", base + 0x20, 19, 2, esai_sels, ARRAY_SIZE(esai_sels));
+ hws[IMX6UL_CLK_UART_SEL] = imx_clk_hw_mux("uart_sel", base + 0x24, 6, 1, uart_sels, ARRAY_SIZE(uart_sels));
+ hws[IMX6UL_CLK_ENFC_SEL] = imx_clk_hw_mux("enfc_sel", base + 0x2c, 15, 3, enfc_sels, ARRAY_SIZE(enfc_sels));
+ hws[IMX6UL_CLK_LDB_DI0_SEL] = imx_clk_hw_mux("ldb_di0_sel", base + 0x2c, 9, 3, ldb_di0_sels, ARRAY_SIZE(ldb_di0_sels));
+ hws[IMX6UL_CLK_SPDIF_SEL] = imx_clk_hw_mux("spdif_sel", base + 0x30, 20, 2, spdif_sels, ARRAY_SIZE(spdif_sels));
if (clk_on_imx6ul()) {
- clks[IMX6UL_CLK_SIM_PRE_SEL] = imx_clk_mux("sim_pre_sel", base + 0x34, 15, 3, sim_pre_sels, ARRAY_SIZE(sim_pre_sels));
- clks[IMX6UL_CLK_SIM_SEL] = imx_clk_mux("sim_sel", base + 0x34, 9, 3, sim_sels, ARRAY_SIZE(sim_sels));
+ hws[IMX6UL_CLK_SIM_PRE_SEL] = imx_clk_hw_mux("sim_pre_sel", base + 0x34, 15, 3, sim_pre_sels, ARRAY_SIZE(sim_pre_sels));
+ hws[IMX6UL_CLK_SIM_SEL] = imx_clk_hw_mux("sim_sel", base + 0x34, 9, 3, sim_sels, ARRAY_SIZE(sim_sels));
} else if (clk_on_imx6ull()) {
- clks[IMX6ULL_CLK_EPDC_PRE_SEL] = imx_clk_mux("epdc_pre_sel", base + 0x34, 15, 3, epdc_pre_sels, ARRAY_SIZE(epdc_pre_sels));
- clks[IMX6ULL_CLK_EPDC_SEL] = imx_clk_mux("epdc_sel", base + 0x34, 9, 3, epdc_sels, ARRAY_SIZE(epdc_sels));
+ hws[IMX6ULL_CLK_EPDC_PRE_SEL] = imx_clk_hw_mux("epdc_pre_sel", base + 0x34, 15, 3, epdc_pre_sels, ARRAY_SIZE(epdc_pre_sels));
+ hws[IMX6ULL_CLK_EPDC_SEL] = imx_clk_hw_mux("epdc_sel", base + 0x34, 9, 3, epdc_sels, ARRAY_SIZE(epdc_sels));
}
- clks[IMX6UL_CLK_ECSPI_SEL] = imx_clk_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
- clks[IMX6UL_CLK_LCDIF_PRE_SEL] = imx_clk_mux_flags("lcdif_pre_sel", base + 0x38, 15, 3, lcdif_pre_sels, ARRAY_SIZE(lcdif_pre_sels), CLK_SET_RATE_PARENT);
- clks[IMX6UL_CLK_LCDIF_SEL] = imx_clk_mux("lcdif_sel", base + 0x38, 9, 3, lcdif_sels, ARRAY_SIZE(lcdif_sels));
-
- clks[IMX6UL_CLK_LDB_DI0_DIV_SEL] = imx_clk_mux("ldb_di0", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels));
- clks[IMX6UL_CLK_LDB_DI1_DIV_SEL] = imx_clk_mux("ldb_di1", base + 0x20, 11, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels));
-
- clks[IMX6UL_CLK_CKO1_SEL] = imx_clk_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
- clks[IMX6UL_CLK_CKO2_SEL] = imx_clk_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
- clks[IMX6UL_CLK_CKO] = imx_clk_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
-
- clks[IMX6UL_CLK_LDB_DI0_DIV_3_5] = imx_clk_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
- clks[IMX6UL_CLK_LDB_DI0_DIV_7] = imx_clk_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
- clks[IMX6UL_CLK_LDB_DI1_DIV_3_5] = imx_clk_fixed_factor("ldb_di1_div_3_5", "qspi1_sel", 2, 7);
- clks[IMX6UL_CLK_LDB_DI1_DIV_7] = imx_clk_fixed_factor("ldb_di1_div_7", "qspi1_sel", 1, 7);
-
- clks[IMX6UL_CLK_PERIPH] = imx_clk_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
- clks[IMX6UL_CLK_PERIPH2] = imx_clk_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
-
- clks[IMX6UL_CLK_PERIPH_CLK2] = imx_clk_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
- clks[IMX6UL_CLK_PERIPH2_CLK2] = imx_clk_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
- clks[IMX6UL_CLK_IPG] = imx_clk_divider("ipg", "ahb", base + 0x14, 8, 2);
- clks[IMX6UL_CLK_LCDIF_PODF] = imx_clk_divider("lcdif_podf", "lcdif_pred", base + 0x18, 23, 3);
- clks[IMX6UL_CLK_QSPI1_PDOF] = imx_clk_divider("qspi1_podf", "qspi1_sel", base + 0x1c, 26, 3);
- clks[IMX6UL_CLK_EIM_SLOW_PODF] = imx_clk_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
- clks[IMX6UL_CLK_PERCLK] = imx_clk_divider("perclk", "perclk_sel", base + 0x1c, 0, 6);
- clks[IMX6UL_CLK_CAN_PODF] = imx_clk_divider("can_podf", "can_sel", base + 0x20, 2, 6);
- clks[IMX6UL_CLK_GPMI_PODF] = imx_clk_divider("gpmi_podf", "gpmi_sel", base + 0x24, 22, 3);
- clks[IMX6UL_CLK_BCH_PODF] = imx_clk_divider("bch_podf", "bch_sel", base + 0x24, 19, 3);
- clks[IMX6UL_CLK_USDHC2_PODF] = imx_clk_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
- clks[IMX6UL_CLK_USDHC1_PODF] = imx_clk_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
- clks[IMX6UL_CLK_UART_PODF] = imx_clk_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
- clks[IMX6UL_CLK_SAI3_PRED] = imx_clk_divider("sai3_pred", "sai3_sel", base + 0x28, 22, 3);
- clks[IMX6UL_CLK_SAI3_PODF] = imx_clk_divider("sai3_podf", "sai3_pred", base + 0x28, 16, 6);
- clks[IMX6UL_CLK_SAI1_PRED] = imx_clk_divider("sai1_pred", "sai1_sel", base + 0x28, 6, 3);
- clks[IMX6UL_CLK_SAI1_PODF] = imx_clk_divider("sai1_podf", "sai1_pred", base + 0x28, 0, 6);
+ hws[IMX6UL_CLK_ECSPI_SEL] = imx_clk_hw_mux("ecspi_sel", base + 0x38, 18, 1, ecspi_sels, ARRAY_SIZE(ecspi_sels));
+ hws[IMX6UL_CLK_LCDIF_PRE_SEL] = imx_clk_hw_mux_flags("lcdif_pre_sel", base + 0x38, 15, 3, lcdif_pre_sels, ARRAY_SIZE(lcdif_pre_sels), CLK_SET_RATE_PARENT);
+ hws[IMX6UL_CLK_LCDIF_SEL] = imx_clk_hw_mux("lcdif_sel", base + 0x38, 9, 3, lcdif_sels, ARRAY_SIZE(lcdif_sels));
+
+ hws[IMX6UL_CLK_LDB_DI0_DIV_SEL] = imx_clk_hw_mux("ldb_di0", base + 0x20, 10, 1, ldb_di0_div_sels, ARRAY_SIZE(ldb_di0_div_sels));
+ hws[IMX6UL_CLK_LDB_DI1_DIV_SEL] = imx_clk_hw_mux("ldb_di1", base + 0x20, 11, 1, ldb_di1_div_sels, ARRAY_SIZE(ldb_di1_div_sels));
+
+ hws[IMX6UL_CLK_CKO1_SEL] = imx_clk_hw_mux("cko1_sel", base + 0x60, 0, 4, cko1_sels, ARRAY_SIZE(cko1_sels));
+ hws[IMX6UL_CLK_CKO2_SEL] = imx_clk_hw_mux("cko2_sel", base + 0x60, 16, 5, cko2_sels, ARRAY_SIZE(cko2_sels));
+ hws[IMX6UL_CLK_CKO] = imx_clk_hw_mux("cko", base + 0x60, 8, 1, cko_sels, ARRAY_SIZE(cko_sels));
+
+ hws[IMX6UL_CLK_LDB_DI0_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di0_div_3_5", "ldb_di0_sel", 2, 7);
+ hws[IMX6UL_CLK_LDB_DI0_DIV_7] = imx_clk_hw_fixed_factor("ldb_di0_div_7", "ldb_di0_sel", 1, 7);
+ hws[IMX6UL_CLK_LDB_DI1_DIV_3_5] = imx_clk_hw_fixed_factor("ldb_di1_div_3_5", "qspi1_sel", 2, 7);
+ hws[IMX6UL_CLK_LDB_DI1_DIV_7] = imx_clk_hw_fixed_factor("ldb_di1_div_7", "qspi1_sel", 1, 7);
+
+ hws[IMX6UL_CLK_PERIPH] = imx_clk_hw_busy_mux("periph", base + 0x14, 25, 1, base + 0x48, 5, periph_sels, ARRAY_SIZE(periph_sels));
+ hws[IMX6UL_CLK_PERIPH2] = imx_clk_hw_busy_mux("periph2", base + 0x14, 26, 1, base + 0x48, 3, periph2_sels, ARRAY_SIZE(periph2_sels));
+
+ hws[IMX6UL_CLK_PERIPH_CLK2] = imx_clk_hw_divider("periph_clk2", "periph_clk2_sel", base + 0x14, 27, 3);
+ hws[IMX6UL_CLK_PERIPH2_CLK2] = imx_clk_hw_divider("periph2_clk2", "periph2_clk2_sel", base + 0x14, 0, 3);
+ hws[IMX6UL_CLK_IPG] = imx_clk_hw_divider("ipg", "ahb", base + 0x14, 8, 2);
+ hws[IMX6UL_CLK_LCDIF_PODF] = imx_clk_hw_divider("lcdif_podf", "lcdif_pred", base + 0x18, 23, 3);
+ hws[IMX6UL_CLK_QSPI1_PDOF] = imx_clk_hw_divider("qspi1_podf", "qspi1_sel", base + 0x1c, 26, 3);
+ hws[IMX6UL_CLK_EIM_SLOW_PODF] = imx_clk_hw_divider("eim_slow_podf", "eim_slow_sel", base + 0x1c, 23, 3);
+ hws[IMX6UL_CLK_PERCLK] = imx_clk_hw_divider("perclk", "perclk_sel", base + 0x1c, 0, 6);
+ hws[IMX6UL_CLK_CAN_PODF] = imx_clk_hw_divider("can_podf", "can_sel", base + 0x20, 2, 6);
+ hws[IMX6UL_CLK_GPMI_PODF] = imx_clk_hw_divider("gpmi_podf", "gpmi_sel", base + 0x24, 22, 3);
+ hws[IMX6UL_CLK_BCH_PODF] = imx_clk_hw_divider("bch_podf", "bch_sel", base + 0x24, 19, 3);
+ hws[IMX6UL_CLK_USDHC2_PODF] = imx_clk_hw_divider("usdhc2_podf", "usdhc2_sel", base + 0x24, 16, 3);
+ hws[IMX6UL_CLK_USDHC1_PODF] = imx_clk_hw_divider("usdhc1_podf", "usdhc1_sel", base + 0x24, 11, 3);
+ hws[IMX6UL_CLK_UART_PODF] = imx_clk_hw_divider("uart_podf", "uart_sel", base + 0x24, 0, 6);
+ hws[IMX6UL_CLK_SAI3_PRED] = imx_clk_hw_divider("sai3_pred", "sai3_sel", base + 0x28, 22, 3);
+ hws[IMX6UL_CLK_SAI3_PODF] = imx_clk_hw_divider("sai3_podf", "sai3_pred", base + 0x28, 16, 6);
+ hws[IMX6UL_CLK_SAI1_PRED] = imx_clk_hw_divider("sai1_pred", "sai1_sel", base + 0x28, 6, 3);
+ hws[IMX6UL_CLK_SAI1_PODF] = imx_clk_hw_divider("sai1_podf", "sai1_pred", base + 0x28, 0, 6);
if (clk_on_imx6ull()) {
- clks[IMX6ULL_CLK_ESAI_PRED] = imx_clk_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
- clks[IMX6ULL_CLK_ESAI_PODF] = imx_clk_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
+ hws[IMX6ULL_CLK_ESAI_PRED] = imx_clk_hw_divider("esai_pred", "esai_sel", base + 0x28, 9, 3);
+ hws[IMX6ULL_CLK_ESAI_PODF] = imx_clk_hw_divider("esai_podf", "esai_pred", base + 0x28, 25, 3);
}
- clks[IMX6UL_CLK_ENFC_PRED] = imx_clk_divider("enfc_pred", "enfc_sel", base + 0x2c, 18, 3);
- clks[IMX6UL_CLK_ENFC_PODF] = imx_clk_divider("enfc_podf", "enfc_pred", base + 0x2c, 21, 6);
- clks[IMX6UL_CLK_SAI2_PRED] = imx_clk_divider("sai2_pred", "sai2_sel", base + 0x2c, 6, 3);
- clks[IMX6UL_CLK_SAI2_PODF] = imx_clk_divider("sai2_podf", "sai2_pred", base + 0x2c, 0, 6);
- clks[IMX6UL_CLK_SPDIF_PRED] = imx_clk_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
- clks[IMX6UL_CLK_SPDIF_PODF] = imx_clk_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
+ hws[IMX6UL_CLK_ENFC_PRED] = imx_clk_hw_divider("enfc_pred", "enfc_sel", base + 0x2c, 18, 3);
+ hws[IMX6UL_CLK_ENFC_PODF] = imx_clk_hw_divider("enfc_podf", "enfc_pred", base + 0x2c, 21, 6);
+ hws[IMX6UL_CLK_SAI2_PRED] = imx_clk_hw_divider("sai2_pred", "sai2_sel", base + 0x2c, 6, 3);
+ hws[IMX6UL_CLK_SAI2_PODF] = imx_clk_hw_divider("sai2_podf", "sai2_pred", base + 0x2c, 0, 6);
+ hws[IMX6UL_CLK_SPDIF_PRED] = imx_clk_hw_divider("spdif_pred", "spdif_sel", base + 0x30, 25, 3);
+ hws[IMX6UL_CLK_SPDIF_PODF] = imx_clk_hw_divider("spdif_podf", "spdif_pred", base + 0x30, 22, 3);
if (clk_on_imx6ul())
- clks[IMX6UL_CLK_SIM_PODF] = imx_clk_divider("sim_podf", "sim_pre_sel", base + 0x34, 12, 3);
+ hws[IMX6UL_CLK_SIM_PODF] = imx_clk_hw_divider("sim_podf", "sim_pre_sel", base + 0x34, 12, 3);
else if (clk_on_imx6ull())
- clks[IMX6ULL_CLK_EPDC_PODF] = imx_clk_divider("epdc_podf", "epdc_pre_sel", base + 0x34, 12, 3);
- clks[IMX6UL_CLK_ECSPI_PODF] = imx_clk_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
- clks[IMX6UL_CLK_LCDIF_PRED] = imx_clk_divider("lcdif_pred", "lcdif_pre_sel", base + 0x38, 12, 3);
- clks[IMX6UL_CLK_CSI_PODF] = imx_clk_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
+ hws[IMX6ULL_CLK_EPDC_PODF] = imx_clk_hw_divider("epdc_podf", "epdc_pre_sel", base + 0x34, 12, 3);
+ hws[IMX6UL_CLK_ECSPI_PODF] = imx_clk_hw_divider("ecspi_podf", "ecspi_sel", base + 0x38, 19, 6);
+ hws[IMX6UL_CLK_LCDIF_PRED] = imx_clk_hw_divider("lcdif_pred", "lcdif_pre_sel", base + 0x38, 12, 3);
+ hws[IMX6UL_CLK_CSI_PODF] = imx_clk_hw_divider("csi_podf", "csi_sel", base + 0x3c, 11, 3);
- clks[IMX6UL_CLK_CKO1_PODF] = imx_clk_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
- clks[IMX6UL_CLK_CKO2_PODF] = imx_clk_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
+ hws[IMX6UL_CLK_CKO1_PODF] = imx_clk_hw_divider("cko1_podf", "cko1_sel", base + 0x60, 4, 3);
+ hws[IMX6UL_CLK_CKO2_PODF] = imx_clk_hw_divider("cko2_podf", "cko2_sel", base + 0x60, 21, 3);
- clks[IMX6UL_CLK_ARM] = imx_clk_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
- clks[IMX6UL_CLK_MMDC_PODF] = imx_clk_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
- clks[IMX6UL_CLK_AXI_PODF] = imx_clk_busy_divider("axi_podf", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
- clks[IMX6UL_CLK_AHB] = imx_clk_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
+ hws[IMX6UL_CLK_ARM] = imx_clk_hw_busy_divider("arm", "pll1_sw", base + 0x10, 0, 3, base + 0x48, 16);
+ hws[IMX6UL_CLK_MMDC_PODF] = imx_clk_hw_busy_divider("mmdc_podf", "periph2", base + 0x14, 3, 3, base + 0x48, 2);
+ hws[IMX6UL_CLK_AXI_PODF] = imx_clk_hw_busy_divider("axi_podf", "axi_sel", base + 0x14, 16, 3, base + 0x48, 0);
+ hws[IMX6UL_CLK_AHB] = imx_clk_hw_busy_divider("ahb", "periph", base + 0x14, 10, 3, base + 0x48, 1);
/* CCGR0 */
- clks[IMX6UL_CLK_AIPSTZ1] = imx_clk_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
- clks[IMX6UL_CLK_AIPSTZ2] = imx_clk_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
- clks[IMX6UL_CLK_APBHDMA] = imx_clk_gate2("apbh_dma", "bch_podf", base + 0x68, 4);
- clks[IMX6UL_CLK_ASRC_IPG] = imx_clk_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
- clks[IMX6UL_CLK_ASRC_MEM] = imx_clk_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6UL_CLK_AIPSTZ1] = imx_clk_hw_gate2_flags("aips_tz1", "ahb", base + 0x68, 0, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_AIPSTZ2] = imx_clk_hw_gate2_flags("aips_tz2", "ahb", base + 0x68, 2, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_APBHDMA] = imx_clk_hw_gate2("apbh_dma", "bch_podf", base + 0x68, 4);
+ hws[IMX6UL_CLK_ASRC_IPG] = imx_clk_hw_gate2_shared("asrc_ipg", "ahb", base + 0x68, 6, &share_count_asrc);
+ hws[IMX6UL_CLK_ASRC_MEM] = imx_clk_hw_gate2_shared("asrc_mem", "ahb", base + 0x68, 6, &share_count_asrc);
if (clk_on_imx6ul()) {
- clks[IMX6UL_CLK_CAAM_MEM] = imx_clk_gate2("caam_mem", "ahb", base + 0x68, 8);
- clks[IMX6UL_CLK_CAAM_ACLK] = imx_clk_gate2("caam_aclk", "ahb", base + 0x68, 10);
- clks[IMX6UL_CLK_CAAM_IPG] = imx_clk_gate2("caam_ipg", "ipg", base + 0x68, 12);
+ hws[IMX6UL_CLK_CAAM_MEM] = imx_clk_hw_gate2("caam_mem", "ahb", base + 0x68, 8);
+ hws[IMX6UL_CLK_CAAM_ACLK] = imx_clk_hw_gate2("caam_aclk", "ahb", base + 0x68, 10);
+ hws[IMX6UL_CLK_CAAM_IPG] = imx_clk_hw_gate2("caam_ipg", "ipg", base + 0x68, 12);
} else if (clk_on_imx6ull()) {
- clks[IMX6ULL_CLK_DCP_CLK] = imx_clk_gate2("dcp", "ahb", base + 0x68, 10);
- clks[IMX6UL_CLK_ENET] = imx_clk_gate2("enet", "ipg", base + 0x68, 12);
- clks[IMX6UL_CLK_ENET_AHB] = imx_clk_gate2("enet_ahb", "ahb", base + 0x68, 12);
+ hws[IMX6ULL_CLK_DCP_CLK] = imx_clk_hw_gate2("dcp", "ahb", base + 0x68, 10);
+ hws[IMX6UL_CLK_ENET] = imx_clk_hw_gate2("enet", "ipg", base + 0x68, 12);
+ hws[IMX6UL_CLK_ENET_AHB] = imx_clk_hw_gate2("enet_ahb", "ahb", base + 0x68, 12);
}
- clks[IMX6UL_CLK_CAN1_IPG] = imx_clk_gate2("can1_ipg", "ipg", base + 0x68, 14);
- clks[IMX6UL_CLK_CAN1_SERIAL] = imx_clk_gate2("can1_serial", "can_podf", base + 0x68, 16);
- clks[IMX6UL_CLK_CAN2_IPG] = imx_clk_gate2("can2_ipg", "ipg", base + 0x68, 18);
- clks[IMX6UL_CLK_CAN2_SERIAL] = imx_clk_gate2("can2_serial", "can_podf", base + 0x68, 20);
- clks[IMX6UL_CLK_GPT2_BUS] = imx_clk_gate2("gpt2_bus", "perclk", base + 0x68, 24);
- clks[IMX6UL_CLK_GPT2_SERIAL] = imx_clk_gate2("gpt2_serial", "perclk", base + 0x68, 26);
- clks[IMX6UL_CLK_UART2_IPG] = imx_clk_gate2("uart2_ipg", "ipg", base + 0x68, 28);
- clks[IMX6UL_CLK_UART2_SERIAL] = imx_clk_gate2("uart2_serial", "uart_podf", base + 0x68, 28);
+ hws[IMX6UL_CLK_CAN1_IPG] = imx_clk_hw_gate2("can1_ipg", "ipg", base + 0x68, 14);
+ hws[IMX6UL_CLK_CAN1_SERIAL] = imx_clk_hw_gate2("can1_serial", "can_podf", base + 0x68, 16);
+ hws[IMX6UL_CLK_CAN2_IPG] = imx_clk_hw_gate2("can2_ipg", "ipg", base + 0x68, 18);
+ hws[IMX6UL_CLK_CAN2_SERIAL] = imx_clk_hw_gate2("can2_serial", "can_podf", base + 0x68, 20);
+ hws[IMX6UL_CLK_GPT2_BUS] = imx_clk_hw_gate2("gpt2_bus", "perclk", base + 0x68, 24);
+ hws[IMX6UL_CLK_GPT2_SERIAL] = imx_clk_hw_gate2("gpt2_serial", "perclk", base + 0x68, 26);
+ hws[IMX6UL_CLK_UART2_IPG] = imx_clk_hw_gate2("uart2_ipg", "ipg", base + 0x68, 28);
+ hws[IMX6UL_CLK_UART2_SERIAL] = imx_clk_hw_gate2("uart2_serial", "uart_podf", base + 0x68, 28);
if (clk_on_imx6ull())
- clks[IMX6UL_CLK_AIPSTZ3] = imx_clk_gate2("aips_tz3", "ahb", base + 0x80, 18);
- clks[IMX6UL_CLK_GPIO2] = imx_clk_gate2("gpio2", "ipg", base + 0x68, 30);
+ hws[IMX6UL_CLK_AIPSTZ3] = imx_clk_hw_gate2("aips_tz3", "ahb", base + 0x80, 18);
+ hws[IMX6UL_CLK_GPIO2] = imx_clk_hw_gate2("gpio2", "ipg", base + 0x68, 30);
/* CCGR1 */
- clks[IMX6UL_CLK_ECSPI1] = imx_clk_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
- clks[IMX6UL_CLK_ECSPI2] = imx_clk_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
- clks[IMX6UL_CLK_ECSPI3] = imx_clk_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
- clks[IMX6UL_CLK_ECSPI4] = imx_clk_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
- clks[IMX6UL_CLK_ADC2] = imx_clk_gate2("adc2", "ipg", base + 0x6c, 8);
- clks[IMX6UL_CLK_UART3_IPG] = imx_clk_gate2("uart3_ipg", "ipg", base + 0x6c, 10);
- clks[IMX6UL_CLK_UART3_SERIAL] = imx_clk_gate2("uart3_serial", "uart_podf", base + 0x6c, 10);
- clks[IMX6UL_CLK_EPIT1] = imx_clk_gate2("epit1", "perclk", base + 0x6c, 12);
- clks[IMX6UL_CLK_EPIT2] = imx_clk_gate2("epit2", "perclk", base + 0x6c, 14);
- clks[IMX6UL_CLK_ADC1] = imx_clk_gate2("adc1", "ipg", base + 0x6c, 16);
- clks[IMX6UL_CLK_GPT1_BUS] = imx_clk_gate2("gpt1_bus", "perclk", base + 0x6c, 20);
- clks[IMX6UL_CLK_GPT1_SERIAL] = imx_clk_gate2("gpt1_serial", "perclk", base + 0x6c, 22);
- clks[IMX6UL_CLK_UART4_IPG] = imx_clk_gate2("uart4_ipg", "ipg", base + 0x6c, 24);
- clks[IMX6UL_CLK_UART4_SERIAL] = imx_clk_gate2("uart4_serial", "uart_podf", base + 0x6c, 24);
- clks[IMX6UL_CLK_GPIO1] = imx_clk_gate2("gpio1", "ipg", base + 0x6c, 26);
- clks[IMX6UL_CLK_GPIO5] = imx_clk_gate2("gpio5", "ipg", base + 0x6c, 30);
+ hws[IMX6UL_CLK_ECSPI1] = imx_clk_hw_gate2("ecspi1", "ecspi_podf", base + 0x6c, 0);
+ hws[IMX6UL_CLK_ECSPI2] = imx_clk_hw_gate2("ecspi2", "ecspi_podf", base + 0x6c, 2);
+ hws[IMX6UL_CLK_ECSPI3] = imx_clk_hw_gate2("ecspi3", "ecspi_podf", base + 0x6c, 4);
+ hws[IMX6UL_CLK_ECSPI4] = imx_clk_hw_gate2("ecspi4", "ecspi_podf", base + 0x6c, 6);
+ hws[IMX6UL_CLK_ADC2] = imx_clk_hw_gate2("adc2", "ipg", base + 0x6c, 8);
+ hws[IMX6UL_CLK_UART3_IPG] = imx_clk_hw_gate2("uart3_ipg", "ipg", base + 0x6c, 10);
+ hws[IMX6UL_CLK_UART3_SERIAL] = imx_clk_hw_gate2("uart3_serial", "uart_podf", base + 0x6c, 10);
+ hws[IMX6UL_CLK_EPIT1] = imx_clk_hw_gate2("epit1", "perclk", base + 0x6c, 12);
+ hws[IMX6UL_CLK_EPIT2] = imx_clk_hw_gate2("epit2", "perclk", base + 0x6c, 14);
+ hws[IMX6UL_CLK_ADC1] = imx_clk_hw_gate2("adc1", "ipg", base + 0x6c, 16);
+ hws[IMX6UL_CLK_GPT1_BUS] = imx_clk_hw_gate2("gpt1_bus", "perclk", base + 0x6c, 20);
+ hws[IMX6UL_CLK_GPT1_SERIAL] = imx_clk_hw_gate2("gpt1_serial", "perclk", base + 0x6c, 22);
+ hws[IMX6UL_CLK_UART4_IPG] = imx_clk_hw_gate2("uart4_ipg", "ipg", base + 0x6c, 24);
+ hws[IMX6UL_CLK_UART4_SERIAL] = imx_clk_hw_gate2("uart4_serial", "uart_podf", base + 0x6c, 24);
+ hws[IMX6UL_CLK_GPIO1] = imx_clk_hw_gate2("gpio1", "ipg", base + 0x6c, 26);
+ hws[IMX6UL_CLK_GPIO5] = imx_clk_hw_gate2("gpio5", "ipg", base + 0x6c, 30);
/* CCGR2 */
if (clk_on_imx6ull()) {
- clks[IMX6ULL_CLK_ESAI_EXTAL] = imx_clk_gate2_shared("esai_extal", "esai_podf", base + 0x70, 0, &share_count_esai);
- clks[IMX6ULL_CLK_ESAI_IPG] = imx_clk_gate2_shared("esai_ipg", "ahb", base + 0x70, 0, &share_count_esai);
- clks[IMX6ULL_CLK_ESAI_MEM] = imx_clk_gate2_shared("esai_mem", "ahb", base + 0x70, 0, &share_count_esai);
+ hws[IMX6ULL_CLK_ESAI_EXTAL] = imx_clk_hw_gate2_shared("esai_extal", "esai_podf", base + 0x70, 0, &share_count_esai);
+ hws[IMX6ULL_CLK_ESAI_IPG] = imx_clk_hw_gate2_shared("esai_ipg", "ahb", base + 0x70, 0, &share_count_esai);
+ hws[IMX6ULL_CLK_ESAI_MEM] = imx_clk_hw_gate2_shared("esai_mem", "ahb", base + 0x70, 0, &share_count_esai);
}
- clks[IMX6UL_CLK_CSI] = imx_clk_gate2("csi", "csi_podf", base + 0x70, 2);
- clks[IMX6UL_CLK_I2C1] = imx_clk_gate2("i2c1", "perclk", base + 0x70, 6);
- clks[IMX6UL_CLK_I2C2] = imx_clk_gate2("i2c2", "perclk", base + 0x70, 8);
- clks[IMX6UL_CLK_I2C3] = imx_clk_gate2("i2c3", "perclk", base + 0x70, 10);
- clks[IMX6UL_CLK_OCOTP] = imx_clk_gate2("ocotp", "ipg", base + 0x70, 12);
- clks[IMX6UL_CLK_IOMUXC] = imx_clk_gate2("iomuxc", "lcdif_podf", base + 0x70, 14);
- clks[IMX6UL_CLK_GPIO3] = imx_clk_gate2("gpio3", "ipg", base + 0x70, 26);
- clks[IMX6UL_CLK_LCDIF_APB] = imx_clk_gate2("lcdif_apb", "axi", base + 0x70, 28);
- clks[IMX6UL_CLK_PXP] = imx_clk_gate2("pxp", "axi", base + 0x70, 30);
+ hws[IMX6UL_CLK_CSI] = imx_clk_hw_gate2("csi", "csi_podf", base + 0x70, 2);
+ hws[IMX6UL_CLK_I2C1] = imx_clk_hw_gate2("i2c1", "perclk", base + 0x70, 6);
+ hws[IMX6UL_CLK_I2C2] = imx_clk_hw_gate2("i2c2", "perclk", base + 0x70, 8);
+ hws[IMX6UL_CLK_I2C3] = imx_clk_hw_gate2("i2c3", "perclk", base + 0x70, 10);
+ hws[IMX6UL_CLK_OCOTP] = imx_clk_hw_gate2("ocotp", "ipg", base + 0x70, 12);
+ hws[IMX6UL_CLK_IOMUXC] = imx_clk_hw_gate2("iomuxc", "lcdif_podf", base + 0x70, 14);
+ hws[IMX6UL_CLK_GPIO3] = imx_clk_hw_gate2("gpio3", "ipg", base + 0x70, 26);
+ hws[IMX6UL_CLK_LCDIF_APB] = imx_clk_hw_gate2("lcdif_apb", "axi", base + 0x70, 28);
+ hws[IMX6UL_CLK_PXP] = imx_clk_hw_gate2("pxp", "axi", base + 0x70, 30);
/* CCGR3 */
- clks[IMX6UL_CLK_UART5_IPG] = imx_clk_gate2("uart5_ipg", "ipg", base + 0x74, 2);
- clks[IMX6UL_CLK_UART5_SERIAL] = imx_clk_gate2("uart5_serial", "uart_podf", base + 0x74, 2);
+ hws[IMX6UL_CLK_UART5_IPG] = imx_clk_hw_gate2("uart5_ipg", "ipg", base + 0x74, 2);
+ hws[IMX6UL_CLK_UART5_SERIAL] = imx_clk_hw_gate2("uart5_serial", "uart_podf", base + 0x74, 2);
if (clk_on_imx6ul()) {
- clks[IMX6UL_CLK_ENET] = imx_clk_gate2("enet", "ipg", base + 0x74, 4);
- clks[IMX6UL_CLK_ENET_AHB] = imx_clk_gate2("enet_ahb", "ahb", base + 0x74, 4);
+ hws[IMX6UL_CLK_ENET] = imx_clk_hw_gate2("enet", "ipg", base + 0x74, 4);
+ hws[IMX6UL_CLK_ENET_AHB] = imx_clk_hw_gate2("enet_ahb", "ahb", base + 0x74, 4);
} else if (clk_on_imx6ull()) {
- clks[IMX6ULL_CLK_EPDC_ACLK] = imx_clk_gate2("epdc_aclk", "axi", base + 0x74, 4);
- clks[IMX6ULL_CLK_EPDC_PIX] = imx_clk_gate2("epdc_pix", "epdc_podf", base + 0x74, 4);
+ hws[IMX6ULL_CLK_EPDC_ACLK] = imx_clk_hw_gate2("epdc_aclk", "axi", base + 0x74, 4);
+ hws[IMX6ULL_CLK_EPDC_PIX] = imx_clk_hw_gate2("epdc_pix", "epdc_podf", base + 0x74, 4);
}
- clks[IMX6UL_CLK_UART6_IPG] = imx_clk_gate2("uart6_ipg", "ipg", base + 0x74, 6);
- clks[IMX6UL_CLK_UART6_SERIAL] = imx_clk_gate2("uart6_serial", "uart_podf", base + 0x74, 6);
- clks[IMX6UL_CLK_LCDIF_PIX] = imx_clk_gate2("lcdif_pix", "lcdif_podf", base + 0x74, 10);
- clks[IMX6UL_CLK_GPIO4] = imx_clk_gate2("gpio4", "ipg", base + 0x74, 12);
- clks[IMX6UL_CLK_QSPI] = imx_clk_gate2("qspi1", "qspi1_podf", base + 0x74, 14);
- clks[IMX6UL_CLK_WDOG1] = imx_clk_gate2("wdog1", "ipg", base + 0x74, 16);
- clks[IMX6UL_CLK_MMDC_P0_FAST] = imx_clk_gate_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
- clks[IMX6UL_CLK_MMDC_P0_IPG] = imx_clk_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
- clks[IMX6UL_CLK_MMDC_P1_IPG] = imx_clk_gate2("mmdc_p1_ipg", "ipg", base + 0x74, 26);
- clks[IMX6UL_CLK_AXI] = imx_clk_gate_flags("axi", "axi_podf", base + 0x74, 28, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_UART6_IPG] = imx_clk_hw_gate2("uart6_ipg", "ipg", base + 0x74, 6);
+ hws[IMX6UL_CLK_UART6_SERIAL] = imx_clk_hw_gate2("uart6_serial", "uart_podf", base + 0x74, 6);
+ hws[IMX6UL_CLK_LCDIF_PIX] = imx_clk_hw_gate2("lcdif_pix", "lcdif_podf", base + 0x74, 10);
+ hws[IMX6UL_CLK_GPIO4] = imx_clk_hw_gate2("gpio4", "ipg", base + 0x74, 12);
+ hws[IMX6UL_CLK_QSPI] = imx_clk_hw_gate2("qspi1", "qspi1_podf", base + 0x74, 14);
+ hws[IMX6UL_CLK_WDOG1] = imx_clk_hw_gate2("wdog1", "ipg", base + 0x74, 16);
+ hws[IMX6UL_CLK_MMDC_P0_FAST] = imx_clk_hw_gate_flags("mmdc_p0_fast", "mmdc_podf", base + 0x74, 20, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_MMDC_P0_IPG] = imx_clk_hw_gate2_flags("mmdc_p0_ipg", "ipg", base + 0x74, 24, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_MMDC_P1_IPG] = imx_clk_hw_gate2_flags("mmdc_p1_ipg", "ipg", base + 0x74, 26, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_AXI] = imx_clk_hw_gate_flags("axi", "axi_podf", base + 0x74, 28, CLK_IS_CRITICAL);
/* CCGR4 */
- clks[IMX6UL_CLK_PER_BCH] = imx_clk_gate2("per_bch", "bch_podf", base + 0x78, 12);
- clks[IMX6UL_CLK_PWM1] = imx_clk_gate2("pwm1", "perclk", base + 0x78, 16);
- clks[IMX6UL_CLK_PWM2] = imx_clk_gate2("pwm2", "perclk", base + 0x78, 18);
- clks[IMX6UL_CLK_PWM3] = imx_clk_gate2("pwm3", "perclk", base + 0x78, 20);
- clks[IMX6UL_CLK_PWM4] = imx_clk_gate2("pwm4", "perclk", base + 0x78, 22);
- clks[IMX6UL_CLK_GPMI_BCH_APB] = imx_clk_gate2("gpmi_bch_apb", "bch_podf", base + 0x78, 24);
- clks[IMX6UL_CLK_GPMI_BCH] = imx_clk_gate2("gpmi_bch", "gpmi_podf", base + 0x78, 26);
- clks[IMX6UL_CLK_GPMI_IO] = imx_clk_gate2("gpmi_io", "enfc_podf", base + 0x78, 28);
- clks[IMX6UL_CLK_GPMI_APB] = imx_clk_gate2("gpmi_apb", "bch_podf", base + 0x78, 30);
+ hws[IMX6UL_CLK_PER_BCH] = imx_clk_hw_gate2("per_bch", "bch_podf", base + 0x78, 12);
+ hws[IMX6UL_CLK_PWM1] = imx_clk_hw_gate2("pwm1", "perclk", base + 0x78, 16);
+ hws[IMX6UL_CLK_PWM2] = imx_clk_hw_gate2("pwm2", "perclk", base + 0x78, 18);
+ hws[IMX6UL_CLK_PWM3] = imx_clk_hw_gate2("pwm3", "perclk", base + 0x78, 20);
+ hws[IMX6UL_CLK_PWM4] = imx_clk_hw_gate2("pwm4", "perclk", base + 0x78, 22);
+ hws[IMX6UL_CLK_GPMI_BCH_APB] = imx_clk_hw_gate2("gpmi_bch_apb", "bch_podf", base + 0x78, 24);
+ hws[IMX6UL_CLK_GPMI_BCH] = imx_clk_hw_gate2("gpmi_bch", "gpmi_podf", base + 0x78, 26);
+ hws[IMX6UL_CLK_GPMI_IO] = imx_clk_hw_gate2("gpmi_io", "enfc_podf", base + 0x78, 28);
+ hws[IMX6UL_CLK_GPMI_APB] = imx_clk_hw_gate2("gpmi_apb", "bch_podf", base + 0x78, 30);
/* CCGR5 */
- clks[IMX6UL_CLK_ROM] = imx_clk_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
- clks[IMX6UL_CLK_SDMA] = imx_clk_gate2("sdma", "ahb", base + 0x7c, 6);
- clks[IMX6UL_CLK_KPP] = imx_clk_gate2("kpp", "ipg", base + 0x7c, 8);
- clks[IMX6UL_CLK_WDOG2] = imx_clk_gate2("wdog2", "ipg", base + 0x7c, 10);
- clks[IMX6UL_CLK_SPBA] = imx_clk_gate2("spba", "ipg", base + 0x7c, 12);
- clks[IMX6UL_CLK_SPDIF] = imx_clk_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
- clks[IMX6UL_CLK_SPDIF_GCLK] = imx_clk_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
- clks[IMX6UL_CLK_SAI3] = imx_clk_gate2_shared("sai3", "sai3_podf", base + 0x7c, 22, &share_count_sai3);
- clks[IMX6UL_CLK_SAI3_IPG] = imx_clk_gate2_shared("sai3_ipg", "ipg", base + 0x7c, 22, &share_count_sai3);
- clks[IMX6UL_CLK_UART1_IPG] = imx_clk_gate2("uart1_ipg", "ipg", base + 0x7c, 24);
- clks[IMX6UL_CLK_UART1_SERIAL] = imx_clk_gate2("uart1_serial", "uart_podf", base + 0x7c, 24);
- clks[IMX6UL_CLK_UART7_IPG] = imx_clk_gate2("uart7_ipg", "ipg", base + 0x7c, 26);
- clks[IMX6UL_CLK_UART7_SERIAL] = imx_clk_gate2("uart7_serial", "uart_podf", base + 0x7c, 26);
- clks[IMX6UL_CLK_SAI1] = imx_clk_gate2_shared("sai1", "sai1_podf", base + 0x7c, 28, &share_count_sai1);
- clks[IMX6UL_CLK_SAI1_IPG] = imx_clk_gate2_shared("sai1_ipg", "ipg", base + 0x7c, 28, &share_count_sai1);
- clks[IMX6UL_CLK_SAI2] = imx_clk_gate2_shared("sai2", "sai2_podf", base + 0x7c, 30, &share_count_sai2);
- clks[IMX6UL_CLK_SAI2_IPG] = imx_clk_gate2_shared("sai2_ipg", "ipg", base + 0x7c, 30, &share_count_sai2);
+ hws[IMX6UL_CLK_ROM] = imx_clk_hw_gate2_flags("rom", "ahb", base + 0x7c, 0, CLK_IS_CRITICAL);
+ hws[IMX6UL_CLK_SDMA] = imx_clk_hw_gate2("sdma", "ahb", base + 0x7c, 6);
+ hws[IMX6UL_CLK_KPP] = imx_clk_hw_gate2("kpp", "ipg", base + 0x7c, 8);
+ hws[IMX6UL_CLK_WDOG2] = imx_clk_hw_gate2("wdog2", "ipg", base + 0x7c, 10);
+ hws[IMX6UL_CLK_SPBA] = imx_clk_hw_gate2("spba", "ipg", base + 0x7c, 12);
+ hws[IMX6UL_CLK_SPDIF] = imx_clk_hw_gate2_shared("spdif", "spdif_podf", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6UL_CLK_SPDIF_GCLK] = imx_clk_hw_gate2_shared("spdif_gclk", "ipg", base + 0x7c, 14, &share_count_audio);
+ hws[IMX6UL_CLK_SAI3] = imx_clk_hw_gate2_shared("sai3", "sai3_podf", base + 0x7c, 22, &share_count_sai3);
+ hws[IMX6UL_CLK_SAI3_IPG] = imx_clk_hw_gate2_shared("sai3_ipg", "ipg", base + 0x7c, 22, &share_count_sai3);
+ hws[IMX6UL_CLK_UART1_IPG] = imx_clk_hw_gate2("uart1_ipg", "ipg", base + 0x7c, 24);
+ hws[IMX6UL_CLK_UART1_SERIAL] = imx_clk_hw_gate2("uart1_serial", "uart_podf", base + 0x7c, 24);
+ hws[IMX6UL_CLK_UART7_IPG] = imx_clk_hw_gate2("uart7_ipg", "ipg", base + 0x7c, 26);
+ hws[IMX6UL_CLK_UART7_SERIAL] = imx_clk_hw_gate2("uart7_serial", "uart_podf", base + 0x7c, 26);
+ hws[IMX6UL_CLK_SAI1] = imx_clk_hw_gate2_shared("sai1", "sai1_podf", base + 0x7c, 28, &share_count_sai1);
+ hws[IMX6UL_CLK_SAI1_IPG] = imx_clk_hw_gate2_shared("sai1_ipg", "ipg", base + 0x7c, 28, &share_count_sai1);
+ hws[IMX6UL_CLK_SAI2] = imx_clk_hw_gate2_shared("sai2", "sai2_podf", base + 0x7c, 30, &share_count_sai2);
+ hws[IMX6UL_CLK_SAI2_IPG] = imx_clk_hw_gate2_shared("sai2_ipg", "ipg", base + 0x7c, 30, &share_count_sai2);
/* CCGR6 */
- clks[IMX6UL_CLK_USBOH3] = imx_clk_gate2("usboh3", "ipg", base + 0x80, 0);
- clks[IMX6UL_CLK_USDHC1] = imx_clk_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
- clks[IMX6UL_CLK_USDHC2] = imx_clk_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
+ hws[IMX6UL_CLK_USBOH3] = imx_clk_hw_gate2("usboh3", "ipg", base + 0x80, 0);
+ hws[IMX6UL_CLK_USDHC1] = imx_clk_hw_gate2("usdhc1", "usdhc1_podf", base + 0x80, 2);
+ hws[IMX6UL_CLK_USDHC2] = imx_clk_hw_gate2("usdhc2", "usdhc2_podf", base + 0x80, 4);
if (clk_on_imx6ul()) {
- clks[IMX6UL_CLK_SIM1] = imx_clk_gate2("sim1", "sim_sel", base + 0x80, 6);
- clks[IMX6UL_CLK_SIM2] = imx_clk_gate2("sim2", "sim_sel", base + 0x80, 8);
+ hws[IMX6UL_CLK_SIM1] = imx_clk_hw_gate2("sim1", "sim_sel", base + 0x80, 6);
+ hws[IMX6UL_CLK_SIM2] = imx_clk_hw_gate2("sim2", "sim_sel", base + 0x80, 8);
}
- clks[IMX6UL_CLK_EIM] = imx_clk_gate2("eim", "eim_slow_podf", base + 0x80, 10);
- clks[IMX6UL_CLK_PWM8] = imx_clk_gate2("pwm8", "perclk", base + 0x80, 16);
- clks[IMX6UL_CLK_UART8_IPG] = imx_clk_gate2("uart8_ipg", "ipg", base + 0x80, 14);
- clks[IMX6UL_CLK_UART8_SERIAL] = imx_clk_gate2("uart8_serial", "uart_podf", base + 0x80, 14);
- clks[IMX6UL_CLK_WDOG3] = imx_clk_gate2("wdog3", "ipg", base + 0x80, 20);
- clks[IMX6UL_CLK_I2C4] = imx_clk_gate2("i2c4", "perclk", base + 0x80, 24);
- clks[IMX6UL_CLK_PWM5] = imx_clk_gate2("pwm5", "perclk", base + 0x80, 26);
- clks[IMX6UL_CLK_PWM6] = imx_clk_gate2("pwm6", "perclk", base + 0x80, 28);
- clks[IMX6UL_CLK_PWM7] = imx_clk_gate2("pwm7", "perclk", base + 0x80, 30);
+ hws[IMX6UL_CLK_EIM] = imx_clk_hw_gate2("eim", "eim_slow_podf", base + 0x80, 10);
+ hws[IMX6UL_CLK_PWM8] = imx_clk_hw_gate2("pwm8", "perclk", base + 0x80, 16);
+ hws[IMX6UL_CLK_UART8_IPG] = imx_clk_hw_gate2("uart8_ipg", "ipg", base + 0x80, 14);
+ hws[IMX6UL_CLK_UART8_SERIAL] = imx_clk_hw_gate2("uart8_serial", "uart_podf", base + 0x80, 14);
+ hws[IMX6UL_CLK_WDOG3] = imx_clk_hw_gate2("wdog3", "ipg", base + 0x80, 20);
+ hws[IMX6UL_CLK_I2C4] = imx_clk_hw_gate2("i2c4", "perclk", base + 0x80, 24);
+ hws[IMX6UL_CLK_PWM5] = imx_clk_hw_gate2("pwm5", "perclk", base + 0x80, 26);
+ hws[IMX6UL_CLK_PWM6] = imx_clk_hw_gate2("pwm6", "perclk", base + 0x80, 28);
+ hws[IMX6UL_CLK_PWM7] = imx_clk_hw_gate2("pwm7", "perclk", base + 0x80, 30);
/* CCOSR */
- clks[IMX6UL_CLK_CKO1] = imx_clk_gate("cko1", "cko1_podf", base + 0x60, 7);
- clks[IMX6UL_CLK_CKO2] = imx_clk_gate("cko2", "cko2_podf", base + 0x60, 24);
+ hws[IMX6UL_CLK_CKO1] = imx_clk_hw_gate("cko1", "cko1_podf", base + 0x60, 7);
+ hws[IMX6UL_CLK_CKO2] = imx_clk_hw_gate("cko2", "cko2_podf", base + 0x60, 24);
/* mask handshake of mmdc */
- writel_relaxed(BM_CCM_CCDR_MMDC_CH0_MASK, base + CCDR);
+ imx_mmdc_mask_handshake(base, 0);
- imx_check_clocks(clks, ARRAY_SIZE(clks));
+ imx_check_clk_hws(hws, IMX6UL_CLK_END);
- clk_data.clks = clks;
- clk_data.clk_num = ARRAY_SIZE(clks);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
/*
* Lower the AHB clock rate before changing the parent clock source,
* AXI clock rate, so we need to lower AHB rate first to make sure at
* any time, AHB rate is <= 133MHz.
*/
- clk_set_rate(clks[IMX6UL_CLK_AHB], 99000000);
+ clk_set_rate(hws[IMX6UL_CLK_AHB]->clk, 99000000);
/* Change periph_pre clock to pll2_bus to adjust AXI rate to 264MHz */
- clk_set_parent(clks[IMX6UL_CLK_PERIPH_CLK2_SEL], clks[IMX6UL_CLK_OSC]);
- clk_set_parent(clks[IMX6UL_CLK_PERIPH], clks[IMX6UL_CLK_PERIPH_CLK2]);
- clk_set_parent(clks[IMX6UL_CLK_PERIPH_PRE], clks[IMX6UL_CLK_PLL2_BUS]);
- clk_set_parent(clks[IMX6UL_CLK_PERIPH], clks[IMX6UL_CLK_PERIPH_PRE]);
+ clk_set_parent(hws[IMX6UL_CLK_PERIPH_CLK2_SEL]->clk, hws[IMX6UL_CLK_OSC]->clk);
+ clk_set_parent(hws[IMX6UL_CLK_PERIPH]->clk, hws[IMX6UL_CLK_PERIPH_CLK2]->clk);
+ clk_set_parent(hws[IMX6UL_CLK_PERIPH_PRE]->clk, hws[IMX6UL_CLK_PLL2_BUS]->clk);
+ clk_set_parent(hws[IMX6UL_CLK_PERIPH]->clk, hws[IMX6UL_CLK_PERIPH_PRE]->clk);
/* Make sure AHB rate is 132MHz */
- clk_set_rate(clks[IMX6UL_CLK_AHB], 132000000);
+ clk_set_rate(hws[IMX6UL_CLK_AHB]->clk, 132000000);
/* set perclk to from OSC */
- clk_set_parent(clks[IMX6UL_CLK_PERCLK_SEL], clks[IMX6UL_CLK_OSC]);
+ clk_set_parent(hws[IMX6UL_CLK_PERCLK_SEL]->clk, hws[IMX6UL_CLK_OSC]->clk);
- clk_set_rate(clks[IMX6UL_CLK_ENET_REF], 50000000);
- clk_set_rate(clks[IMX6UL_CLK_ENET2_REF], 50000000);
- clk_set_rate(clks[IMX6UL_CLK_CSI], 24000000);
+ clk_set_rate(hws[IMX6UL_CLK_ENET_REF]->clk, 50000000);
+ clk_set_rate(hws[IMX6UL_CLK_ENET2_REF]->clk, 50000000);
+ clk_set_rate(hws[IMX6UL_CLK_CSI]->clk, 24000000);
if (clk_on_imx6ull())
- clk_prepare_enable(clks[IMX6UL_CLK_AIPSTZ3]);
+ clk_prepare_enable(hws[IMX6UL_CLK_AIPSTZ3]->clk);
if (IS_ENABLED(CONFIG_USB_MXS_PHY)) {
- clk_prepare_enable(clks[IMX6UL_CLK_USBPHY1_GATE]);
- clk_prepare_enable(clks[IMX6UL_CLK_USBPHY2_GATE]);
+ clk_prepare_enable(hws[IMX6UL_CLK_USBPHY1_GATE]->clk);
+ clk_prepare_enable(hws[IMX6UL_CLK_USBPHY2_GATE]->clk);
}
- clk_set_parent(clks[IMX6UL_CLK_CAN_SEL], clks[IMX6UL_CLK_PLL3_60M]);
+ clk_set_parent(hws[IMX6UL_CLK_CAN_SEL]->clk, hws[IMX6UL_CLK_PLL3_60M]->clk);
if (clk_on_imx6ul())
- clk_set_parent(clks[IMX6UL_CLK_SIM_PRE_SEL], clks[IMX6UL_CLK_PLL3_USB_OTG]);
+ clk_set_parent(hws[IMX6UL_CLK_SIM_PRE_SEL]->clk, hws[IMX6UL_CLK_PLL3_USB_OTG]->clk);
else if (clk_on_imx6ull())
- clk_set_parent(clks[IMX6ULL_CLK_EPDC_PRE_SEL], clks[IMX6UL_CLK_PLL3_PFD2]);
+ clk_set_parent(hws[IMX6ULL_CLK_EPDC_PRE_SEL]->clk, hws[IMX6UL_CLK_PLL3_PFD2]->clk);
- clk_set_parent(clks[IMX6UL_CLK_ENFC_SEL], clks[IMX6UL_CLK_PLL2_PFD2]);
+ clk_set_parent(hws[IMX6UL_CLK_ENFC_SEL]->clk, hws[IMX6UL_CLK_PLL2_PFD2]->clk);
}
CLK_OF_DECLARE(imx6ul, "fsl,imx6ul-ccm", imx6ul_clocks_init);
#include <dt-bindings/clock/imx7d-clock.h>
#include <linux/clk.h>
#include <linux/clkdev.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/io.h>
{ }
};
-static struct clk *clks[IMX7D_CLK_END];
static const char *arm_a7_sel[] = { "osc", "pll_arm_main_clk",
"pll_enet_500m_clk", "pll_dram_main_clk",
"pll_sys_main_clk", "pll_sys_pfd0_392m_clk", "pll_audio_post_div",
static const char *pll_audio_bypass_sel[] = { "pll_audio_main", "pll_audio_main_src", };
static const char *pll_video_bypass_sel[] = { "pll_video_main", "pll_video_main_src", };
-static struct clk_onecell_data clk_data;
-
-static struct clk ** const uart_clks[] __initconst = {
- &clks[IMX7D_UART1_ROOT_CLK],
- &clks[IMX7D_UART2_ROOT_CLK],
- &clks[IMX7D_UART3_ROOT_CLK],
- &clks[IMX7D_UART4_ROOT_CLK],
- &clks[IMX7D_UART5_ROOT_CLK],
- &clks[IMX7D_UART6_ROOT_CLK],
- &clks[IMX7D_UART7_ROOT_CLK],
- NULL
+static struct clk_hw **hws;
+static struct clk_hw_onecell_data *clk_hw_data;
+
+static const int uart_clk_ids[] __initconst = {
+ IMX7D_UART1_ROOT_CLK,
+ IMX7D_UART2_ROOT_CLK,
+ IMX7D_UART3_ROOT_CLK,
+ IMX7D_UART4_ROOT_CLK,
+ IMX7D_UART5_ROOT_CLK,
+ IMX7D_UART6_ROOT_CLK,
+ IMX7D_UART7_ROOT_CLK,
};
+static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
+
static void __init imx7d_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
+ int i;
+
+ clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
+ IMX7D_CLK_END), GFP_KERNEL);
+ if (WARN_ON(!clk_hw_data))
+ return;
+
+ clk_hw_data->num = IMX7D_CLK_END;
+ hws = clk_hw_data->hws;
- clks[IMX7D_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
- clks[IMX7D_OSC_24M_CLK] = of_clk_get_by_name(ccm_node, "osc");
- clks[IMX7D_CKIL] = of_clk_get_by_name(ccm_node, "ckil");
+ hws[IMX7D_CLK_DUMMY] = imx_clk_hw_fixed("dummy", 0);
+ hws[IMX7D_OSC_24M_CLK] = __clk_get_hw(of_clk_get_by_name(ccm_node, "osc"));
+ hws[IMX7D_CKIL] = __clk_get_hw(of_clk_get_by_name(ccm_node, "ckil"));
np = of_find_compatible_node(NULL, NULL, "fsl,imx7d-anatop");
base = of_iomap(np, 0);
WARN_ON(!base);
of_node_put(np);
- clks[IMX7D_PLL_ARM_MAIN_SRC] = imx_clk_mux("pll_arm_main_src", base + 0x60, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
- clks[IMX7D_PLL_DRAM_MAIN_SRC] = imx_clk_mux("pll_dram_main_src", base + 0x70, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
- clks[IMX7D_PLL_SYS_MAIN_SRC] = imx_clk_mux("pll_sys_main_src", base + 0xb0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
- clks[IMX7D_PLL_ENET_MAIN_SRC] = imx_clk_mux("pll_enet_main_src", base + 0xe0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
- clks[IMX7D_PLL_AUDIO_MAIN_SRC] = imx_clk_mux("pll_audio_main_src", base + 0xf0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
- clks[IMX7D_PLL_VIDEO_MAIN_SRC] = imx_clk_mux("pll_video_main_src", base + 0x130, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
-
- clks[IMX7D_PLL_ARM_MAIN] = imx_clk_pllv3(IMX_PLLV3_SYS, "pll_arm_main", "osc", base + 0x60, 0x7f);
- clks[IMX7D_PLL_DRAM_MAIN] = imx_clk_pllv3(IMX_PLLV3_DDR_IMX7, "pll_dram_main", "osc", base + 0x70, 0x7f);
- clks[IMX7D_PLL_SYS_MAIN] = imx_clk_pllv3(IMX_PLLV3_GENERIC, "pll_sys_main", "osc", base + 0xb0, 0x1);
- clks[IMX7D_PLL_ENET_MAIN] = imx_clk_pllv3(IMX_PLLV3_ENET_IMX7, "pll_enet_main", "osc", base + 0xe0, 0x0);
- clks[IMX7D_PLL_AUDIO_MAIN] = imx_clk_pllv3(IMX_PLLV3_AV_IMX7, "pll_audio_main", "osc", base + 0xf0, 0x7f);
- clks[IMX7D_PLL_VIDEO_MAIN] = imx_clk_pllv3(IMX_PLLV3_AV_IMX7, "pll_video_main", "osc", base + 0x130, 0x7f);
-
- clks[IMX7D_PLL_ARM_MAIN_BYPASS] = imx_clk_mux_flags("pll_arm_main_bypass", base + 0x60, 16, 1, pll_arm_bypass_sel, ARRAY_SIZE(pll_arm_bypass_sel), CLK_SET_RATE_PARENT);
- clks[IMX7D_PLL_DRAM_MAIN_BYPASS] = imx_clk_mux_flags("pll_dram_main_bypass", base + 0x70, 16, 1, pll_dram_bypass_sel, ARRAY_SIZE(pll_dram_bypass_sel), CLK_SET_RATE_PARENT);
- clks[IMX7D_PLL_SYS_MAIN_BYPASS] = imx_clk_mux_flags("pll_sys_main_bypass", base + 0xb0, 16, 1, pll_sys_bypass_sel, ARRAY_SIZE(pll_sys_bypass_sel), CLK_SET_RATE_PARENT);
- clks[IMX7D_PLL_ENET_MAIN_BYPASS] = imx_clk_mux_flags("pll_enet_main_bypass", base + 0xe0, 16, 1, pll_enet_bypass_sel, ARRAY_SIZE(pll_enet_bypass_sel), CLK_SET_RATE_PARENT);
- clks[IMX7D_PLL_AUDIO_MAIN_BYPASS] = imx_clk_mux_flags("pll_audio_main_bypass", base + 0xf0, 16, 1, pll_audio_bypass_sel, ARRAY_SIZE(pll_audio_bypass_sel), CLK_SET_RATE_PARENT);
- clks[IMX7D_PLL_VIDEO_MAIN_BYPASS] = imx_clk_mux_flags("pll_video_main_bypass", base + 0x130, 16, 1, pll_video_bypass_sel, ARRAY_SIZE(pll_video_bypass_sel), CLK_SET_RATE_PARENT);
-
- clks[IMX7D_PLL_ARM_MAIN_CLK] = imx_clk_gate("pll_arm_main_clk", "pll_arm_main_bypass", base + 0x60, 13);
- clks[IMX7D_PLL_DRAM_MAIN_CLK] = imx_clk_gate("pll_dram_main_clk", "pll_dram_test_div", base + 0x70, 13);
- clks[IMX7D_PLL_SYS_MAIN_CLK] = imx_clk_gate("pll_sys_main_clk", "pll_sys_main_bypass", base + 0xb0, 13);
- clks[IMX7D_PLL_AUDIO_MAIN_CLK] = imx_clk_gate("pll_audio_main_clk", "pll_audio_main_bypass", base + 0xf0, 13);
- clks[IMX7D_PLL_VIDEO_MAIN_CLK] = imx_clk_gate("pll_video_main_clk", "pll_video_main_bypass", base + 0x130, 13);
-
- clks[IMX7D_PLL_DRAM_TEST_DIV] = clk_register_divider_table(NULL, "pll_dram_test_div", "pll_dram_main_bypass",
+ hws[IMX7D_PLL_ARM_MAIN_SRC] = imx_clk_hw_mux("pll_arm_main_src", base + 0x60, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+ hws[IMX7D_PLL_DRAM_MAIN_SRC] = imx_clk_hw_mux("pll_dram_main_src", base + 0x70, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+ hws[IMX7D_PLL_SYS_MAIN_SRC] = imx_clk_hw_mux("pll_sys_main_src", base + 0xb0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+ hws[IMX7D_PLL_ENET_MAIN_SRC] = imx_clk_hw_mux("pll_enet_main_src", base + 0xe0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+ hws[IMX7D_PLL_AUDIO_MAIN_SRC] = imx_clk_hw_mux("pll_audio_main_src", base + 0xf0, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+ hws[IMX7D_PLL_VIDEO_MAIN_SRC] = imx_clk_hw_mux("pll_video_main_src", base + 0x130, 14, 2, pll_bypass_src_sel, ARRAY_SIZE(pll_bypass_src_sel));
+
+ hws[IMX7D_PLL_ARM_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_SYS, "pll_arm_main", "osc", base + 0x60, 0x7f);
+ hws[IMX7D_PLL_DRAM_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_DDR_IMX7, "pll_dram_main", "osc", base + 0x70, 0x7f);
+ hws[IMX7D_PLL_SYS_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_GENERIC, "pll_sys_main", "osc", base + 0xb0, 0x1);
+ hws[IMX7D_PLL_ENET_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_ENET_IMX7, "pll_enet_main", "osc", base + 0xe0, 0x0);
+ hws[IMX7D_PLL_AUDIO_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_AV_IMX7, "pll_audio_main", "osc", base + 0xf0, 0x7f);
+ hws[IMX7D_PLL_VIDEO_MAIN] = imx_clk_hw_pllv3(IMX_PLLV3_AV_IMX7, "pll_video_main", "osc", base + 0x130, 0x7f);
+
+ hws[IMX7D_PLL_ARM_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_arm_main_bypass", base + 0x60, 16, 1, pll_arm_bypass_sel, ARRAY_SIZE(pll_arm_bypass_sel), CLK_SET_RATE_PARENT);
+ hws[IMX7D_PLL_DRAM_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_dram_main_bypass", base + 0x70, 16, 1, pll_dram_bypass_sel, ARRAY_SIZE(pll_dram_bypass_sel), CLK_SET_RATE_PARENT);
+ hws[IMX7D_PLL_SYS_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_sys_main_bypass", base + 0xb0, 16, 1, pll_sys_bypass_sel, ARRAY_SIZE(pll_sys_bypass_sel), CLK_SET_RATE_PARENT);
+ hws[IMX7D_PLL_ENET_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_enet_main_bypass", base + 0xe0, 16, 1, pll_enet_bypass_sel, ARRAY_SIZE(pll_enet_bypass_sel), CLK_SET_RATE_PARENT);
+ hws[IMX7D_PLL_AUDIO_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_audio_main_bypass", base + 0xf0, 16, 1, pll_audio_bypass_sel, ARRAY_SIZE(pll_audio_bypass_sel), CLK_SET_RATE_PARENT);
+ hws[IMX7D_PLL_VIDEO_MAIN_BYPASS] = imx_clk_hw_mux_flags("pll_video_main_bypass", base + 0x130, 16, 1, pll_video_bypass_sel, ARRAY_SIZE(pll_video_bypass_sel), CLK_SET_RATE_PARENT);
+
+ hws[IMX7D_PLL_ARM_MAIN_CLK] = imx_clk_hw_gate("pll_arm_main_clk", "pll_arm_main_bypass", base + 0x60, 13);
+ hws[IMX7D_PLL_DRAM_MAIN_CLK] = imx_clk_hw_gate("pll_dram_main_clk", "pll_dram_test_div", base + 0x70, 13);
+ hws[IMX7D_PLL_SYS_MAIN_CLK] = imx_clk_hw_gate("pll_sys_main_clk", "pll_sys_main_bypass", base + 0xb0, 13);
+ hws[IMX7D_PLL_AUDIO_MAIN_CLK] = imx_clk_hw_gate("pll_audio_main_clk", "pll_audio_main_bypass", base + 0xf0, 13);
+ hws[IMX7D_PLL_VIDEO_MAIN_CLK] = imx_clk_hw_gate("pll_video_main_clk", "pll_video_main_bypass", base + 0x130, 13);
+
+ hws[IMX7D_PLL_DRAM_TEST_DIV] = clk_hw_register_divider_table(NULL, "pll_dram_test_div", "pll_dram_main_bypass",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x70, 21, 2, 0, test_div_table, &imx_ccm_lock);
- clks[IMX7D_PLL_AUDIO_TEST_DIV] = clk_register_divider_table(NULL, "pll_audio_test_div", "pll_audio_main_clk",
+ hws[IMX7D_PLL_AUDIO_TEST_DIV] = clk_hw_register_divider_table(NULL, "pll_audio_test_div", "pll_audio_main_clk",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0xf0, 19, 2, 0, test_div_table, &imx_ccm_lock);
- clks[IMX7D_PLL_AUDIO_POST_DIV] = clk_register_divider_table(NULL, "pll_audio_post_div", "pll_audio_test_div",
+ hws[IMX7D_PLL_AUDIO_POST_DIV] = clk_hw_register_divider_table(NULL, "pll_audio_post_div", "pll_audio_test_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0xf0, 22, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX7D_PLL_VIDEO_TEST_DIV] = clk_register_divider_table(NULL, "pll_video_test_div", "pll_video_main_clk",
+ hws[IMX7D_PLL_VIDEO_TEST_DIV] = clk_hw_register_divider_table(NULL, "pll_video_test_div", "pll_video_main_clk",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x130, 19, 2, 0, test_div_table, &imx_ccm_lock);
- clks[IMX7D_PLL_VIDEO_POST_DIV] = clk_register_divider_table(NULL, "pll_video_post_div", "pll_video_test_div",
+ hws[IMX7D_PLL_VIDEO_POST_DIV] = clk_hw_register_divider_table(NULL, "pll_video_post_div", "pll_video_test_div",
CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, base + 0x130, 22, 2, 0, post_div_table, &imx_ccm_lock);
- clks[IMX7D_PLL_SYS_PFD0_392M_CLK] = imx_clk_pfd("pll_sys_pfd0_392m_clk", "pll_sys_main_clk", base + 0xc0, 0);
- clks[IMX7D_PLL_SYS_PFD1_332M_CLK] = imx_clk_pfd("pll_sys_pfd1_332m_clk", "pll_sys_main_clk", base + 0xc0, 1);
- clks[IMX7D_PLL_SYS_PFD2_270M_CLK] = imx_clk_pfd("pll_sys_pfd2_270m_clk", "pll_sys_main_clk", base + 0xc0, 2);
-
- clks[IMX7D_PLL_SYS_PFD3_CLK] = imx_clk_pfd("pll_sys_pfd3_clk", "pll_sys_main_clk", base + 0xc0, 3);
- clks[IMX7D_PLL_SYS_PFD4_CLK] = imx_clk_pfd("pll_sys_pfd4_clk", "pll_sys_main_clk", base + 0xd0, 0);
- clks[IMX7D_PLL_SYS_PFD5_CLK] = imx_clk_pfd("pll_sys_pfd5_clk", "pll_sys_main_clk", base + 0xd0, 1);
- clks[IMX7D_PLL_SYS_PFD6_CLK] = imx_clk_pfd("pll_sys_pfd6_clk", "pll_sys_main_clk", base + 0xd0, 2);
- clks[IMX7D_PLL_SYS_PFD7_CLK] = imx_clk_pfd("pll_sys_pfd7_clk", "pll_sys_main_clk", base + 0xd0, 3);
-
- clks[IMX7D_PLL_SYS_MAIN_480M] = imx_clk_fixed_factor("pll_sys_main_480m", "pll_sys_main_clk", 1, 1);
- clks[IMX7D_PLL_SYS_MAIN_240M] = imx_clk_fixed_factor("pll_sys_main_240m", "pll_sys_main_clk", 1, 2);
- clks[IMX7D_PLL_SYS_MAIN_120M] = imx_clk_fixed_factor("pll_sys_main_120m", "pll_sys_main_clk", 1, 4);
- clks[IMX7D_PLL_DRAM_MAIN_533M] = imx_clk_fixed_factor("pll_dram_533m", "pll_dram_main_clk", 1, 2);
-
- clks[IMX7D_PLL_SYS_MAIN_480M_CLK] = imx_clk_gate_dis_flags("pll_sys_main_480m_clk", "pll_sys_main_480m", base + 0xb0, 4, CLK_IS_CRITICAL);
- clks[IMX7D_PLL_SYS_MAIN_240M_CLK] = imx_clk_gate_dis("pll_sys_main_240m_clk", "pll_sys_main_240m", base + 0xb0, 5);
- clks[IMX7D_PLL_SYS_MAIN_120M_CLK] = imx_clk_gate_dis("pll_sys_main_120m_clk", "pll_sys_main_120m", base + 0xb0, 6);
- clks[IMX7D_PLL_DRAM_MAIN_533M_CLK] = imx_clk_gate("pll_dram_533m_clk", "pll_dram_533m", base + 0x70, 12);
-
- clks[IMX7D_PLL_SYS_PFD0_196M] = imx_clk_fixed_factor("pll_sys_pfd0_196m", "pll_sys_pfd0_392m_clk", 1, 2);
- clks[IMX7D_PLL_SYS_PFD1_166M] = imx_clk_fixed_factor("pll_sys_pfd1_166m", "pll_sys_pfd1_332m_clk", 1, 2);
- clks[IMX7D_PLL_SYS_PFD2_135M] = imx_clk_fixed_factor("pll_sys_pfd2_135m", "pll_sys_pfd2_270m_clk", 1, 2);
-
- clks[IMX7D_PLL_SYS_PFD0_196M_CLK] = imx_clk_gate_dis("pll_sys_pfd0_196m_clk", "pll_sys_pfd0_196m", base + 0xb0, 26);
- clks[IMX7D_PLL_SYS_PFD1_166M_CLK] = imx_clk_gate_dis("pll_sys_pfd1_166m_clk", "pll_sys_pfd1_166m", base + 0xb0, 27);
- clks[IMX7D_PLL_SYS_PFD2_135M_CLK] = imx_clk_gate_dis("pll_sys_pfd2_135m_clk", "pll_sys_pfd2_135m", base + 0xb0, 28);
-
- clks[IMX7D_PLL_ENET_MAIN_CLK] = imx_clk_fixed_factor("pll_enet_main_clk", "pll_enet_main_bypass", 1, 1);
- clks[IMX7D_PLL_ENET_MAIN_500M] = imx_clk_fixed_factor("pll_enet_500m", "pll_enet_main_clk", 1, 2);
- clks[IMX7D_PLL_ENET_MAIN_250M] = imx_clk_fixed_factor("pll_enet_250m", "pll_enet_main_clk", 1, 4);
- clks[IMX7D_PLL_ENET_MAIN_125M] = imx_clk_fixed_factor("pll_enet_125m", "pll_enet_main_clk", 1, 8);
- clks[IMX7D_PLL_ENET_MAIN_100M] = imx_clk_fixed_factor("pll_enet_100m", "pll_enet_main_clk", 1, 10);
- clks[IMX7D_PLL_ENET_MAIN_50M] = imx_clk_fixed_factor("pll_enet_50m", "pll_enet_main_clk", 1, 20);
- clks[IMX7D_PLL_ENET_MAIN_40M] = imx_clk_fixed_factor("pll_enet_40m", "pll_enet_main_clk", 1, 25);
- clks[IMX7D_PLL_ENET_MAIN_25M] = imx_clk_fixed_factor("pll_enet_25m", "pll_enet_main_clk", 1, 40);
-
- clks[IMX7D_PLL_ENET_MAIN_500M_CLK] = imx_clk_gate("pll_enet_500m_clk", "pll_enet_500m", base + 0xe0, 12);
- clks[IMX7D_PLL_ENET_MAIN_250M_CLK] = imx_clk_gate("pll_enet_250m_clk", "pll_enet_250m", base + 0xe0, 11);
- clks[IMX7D_PLL_ENET_MAIN_125M_CLK] = imx_clk_gate("pll_enet_125m_clk", "pll_enet_125m", base + 0xe0, 10);
- clks[IMX7D_PLL_ENET_MAIN_100M_CLK] = imx_clk_gate("pll_enet_100m_clk", "pll_enet_100m", base + 0xe0, 9);
- clks[IMX7D_PLL_ENET_MAIN_50M_CLK] = imx_clk_gate("pll_enet_50m_clk", "pll_enet_50m", base + 0xe0, 8);
- clks[IMX7D_PLL_ENET_MAIN_40M_CLK] = imx_clk_gate("pll_enet_40m_clk", "pll_enet_40m", base + 0xe0, 7);
- clks[IMX7D_PLL_ENET_MAIN_25M_CLK] = imx_clk_gate("pll_enet_25m_clk", "pll_enet_25m", base + 0xe0, 6);
-
- clks[IMX7D_LVDS1_OUT_SEL] = imx_clk_mux("lvds1_sel", base + 0x170, 0, 5, lvds1_sel, ARRAY_SIZE(lvds1_sel));
- clks[IMX7D_LVDS1_OUT_CLK] = imx_clk_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x170, 5, BIT(6));
+ hws[IMX7D_PLL_SYS_PFD0_392M_CLK] = imx_clk_hw_pfd("pll_sys_pfd0_392m_clk", "pll_sys_main_clk", base + 0xc0, 0);
+ hws[IMX7D_PLL_SYS_PFD1_332M_CLK] = imx_clk_hw_pfd("pll_sys_pfd1_332m_clk", "pll_sys_main_clk", base + 0xc0, 1);
+ hws[IMX7D_PLL_SYS_PFD2_270M_CLK] = imx_clk_hw_pfd("pll_sys_pfd2_270m_clk", "pll_sys_main_clk", base + 0xc0, 2);
+
+ hws[IMX7D_PLL_SYS_PFD3_CLK] = imx_clk_hw_pfd("pll_sys_pfd3_clk", "pll_sys_main_clk", base + 0xc0, 3);
+ hws[IMX7D_PLL_SYS_PFD4_CLK] = imx_clk_hw_pfd("pll_sys_pfd4_clk", "pll_sys_main_clk", base + 0xd0, 0);
+ hws[IMX7D_PLL_SYS_PFD5_CLK] = imx_clk_hw_pfd("pll_sys_pfd5_clk", "pll_sys_main_clk", base + 0xd0, 1);
+ hws[IMX7D_PLL_SYS_PFD6_CLK] = imx_clk_hw_pfd("pll_sys_pfd6_clk", "pll_sys_main_clk", base + 0xd0, 2);
+ hws[IMX7D_PLL_SYS_PFD7_CLK] = imx_clk_hw_pfd("pll_sys_pfd7_clk", "pll_sys_main_clk", base + 0xd0, 3);
+
+ hws[IMX7D_PLL_SYS_MAIN_480M] = imx_clk_hw_fixed_factor("pll_sys_main_480m", "pll_sys_main_clk", 1, 1);
+ hws[IMX7D_PLL_SYS_MAIN_240M] = imx_clk_hw_fixed_factor("pll_sys_main_240m", "pll_sys_main_clk", 1, 2);
+ hws[IMX7D_PLL_SYS_MAIN_120M] = imx_clk_hw_fixed_factor("pll_sys_main_120m", "pll_sys_main_clk", 1, 4);
+ hws[IMX7D_PLL_DRAM_MAIN_533M] = imx_clk_hw_fixed_factor("pll_dram_533m", "pll_dram_main_clk", 1, 2);
+
+ hws[IMX7D_PLL_SYS_MAIN_480M_CLK] = imx_clk_hw_gate_dis_flags("pll_sys_main_480m_clk", "pll_sys_main_480m", base + 0xb0, 4, CLK_IS_CRITICAL);
+ hws[IMX7D_PLL_SYS_MAIN_240M_CLK] = imx_clk_hw_gate_dis("pll_sys_main_240m_clk", "pll_sys_main_240m", base + 0xb0, 5);
+ hws[IMX7D_PLL_SYS_MAIN_120M_CLK] = imx_clk_hw_gate_dis("pll_sys_main_120m_clk", "pll_sys_main_120m", base + 0xb0, 6);
+ hws[IMX7D_PLL_DRAM_MAIN_533M_CLK] = imx_clk_hw_gate("pll_dram_533m_clk", "pll_dram_533m", base + 0x70, 12);
+
+ hws[IMX7D_PLL_SYS_PFD0_196M] = imx_clk_hw_fixed_factor("pll_sys_pfd0_196m", "pll_sys_pfd0_392m_clk", 1, 2);
+ hws[IMX7D_PLL_SYS_PFD1_166M] = imx_clk_hw_fixed_factor("pll_sys_pfd1_166m", "pll_sys_pfd1_332m_clk", 1, 2);
+ hws[IMX7D_PLL_SYS_PFD2_135M] = imx_clk_hw_fixed_factor("pll_sys_pfd2_135m", "pll_sys_pfd2_270m_clk", 1, 2);
+
+ hws[IMX7D_PLL_SYS_PFD0_196M_CLK] = imx_clk_hw_gate_dis("pll_sys_pfd0_196m_clk", "pll_sys_pfd0_196m", base + 0xb0, 26);
+ hws[IMX7D_PLL_SYS_PFD1_166M_CLK] = imx_clk_hw_gate_dis("pll_sys_pfd1_166m_clk", "pll_sys_pfd1_166m", base + 0xb0, 27);
+ hws[IMX7D_PLL_SYS_PFD2_135M_CLK] = imx_clk_hw_gate_dis("pll_sys_pfd2_135m_clk", "pll_sys_pfd2_135m", base + 0xb0, 28);
+
+ hws[IMX7D_PLL_ENET_MAIN_CLK] = imx_clk_hw_fixed_factor("pll_enet_main_clk", "pll_enet_main_bypass", 1, 1);
+ hws[IMX7D_PLL_ENET_MAIN_500M] = imx_clk_hw_fixed_factor("pll_enet_500m", "pll_enet_main_clk", 1, 2);
+ hws[IMX7D_PLL_ENET_MAIN_250M] = imx_clk_hw_fixed_factor("pll_enet_250m", "pll_enet_main_clk", 1, 4);
+ hws[IMX7D_PLL_ENET_MAIN_125M] = imx_clk_hw_fixed_factor("pll_enet_125m", "pll_enet_main_clk", 1, 8);
+ hws[IMX7D_PLL_ENET_MAIN_100M] = imx_clk_hw_fixed_factor("pll_enet_100m", "pll_enet_main_clk", 1, 10);
+ hws[IMX7D_PLL_ENET_MAIN_50M] = imx_clk_hw_fixed_factor("pll_enet_50m", "pll_enet_main_clk", 1, 20);
+ hws[IMX7D_PLL_ENET_MAIN_40M] = imx_clk_hw_fixed_factor("pll_enet_40m", "pll_enet_main_clk", 1, 25);
+ hws[IMX7D_PLL_ENET_MAIN_25M] = imx_clk_hw_fixed_factor("pll_enet_25m", "pll_enet_main_clk", 1, 40);
+
+ hws[IMX7D_PLL_ENET_MAIN_500M_CLK] = imx_clk_hw_gate("pll_enet_500m_clk", "pll_enet_500m", base + 0xe0, 12);
+ hws[IMX7D_PLL_ENET_MAIN_250M_CLK] = imx_clk_hw_gate("pll_enet_250m_clk", "pll_enet_250m", base + 0xe0, 11);
+ hws[IMX7D_PLL_ENET_MAIN_125M_CLK] = imx_clk_hw_gate("pll_enet_125m_clk", "pll_enet_125m", base + 0xe0, 10);
+ hws[IMX7D_PLL_ENET_MAIN_100M_CLK] = imx_clk_hw_gate("pll_enet_100m_clk", "pll_enet_100m", base + 0xe0, 9);
+ hws[IMX7D_PLL_ENET_MAIN_50M_CLK] = imx_clk_hw_gate("pll_enet_50m_clk", "pll_enet_50m", base + 0xe0, 8);
+ hws[IMX7D_PLL_ENET_MAIN_40M_CLK] = imx_clk_hw_gate("pll_enet_40m_clk", "pll_enet_40m", base + 0xe0, 7);
+ hws[IMX7D_PLL_ENET_MAIN_25M_CLK] = imx_clk_hw_gate("pll_enet_25m_clk", "pll_enet_25m", base + 0xe0, 6);
+
+ hws[IMX7D_LVDS1_OUT_SEL] = imx_clk_hw_mux("lvds1_sel", base + 0x170, 0, 5, lvds1_sel, ARRAY_SIZE(lvds1_sel));
+ hws[IMX7D_LVDS1_OUT_CLK] = imx_clk_hw_gate_exclusive("lvds1_out", "lvds1_sel", base + 0x170, 5, BIT(6));
np = ccm_node;
base = of_iomap(np, 0);
WARN_ON(!base);
- clks[IMX7D_ARM_A7_ROOT_SRC] = imx_clk_mux2("arm_a7_src", base + 0x8000, 24, 3, arm_a7_sel, ARRAY_SIZE(arm_a7_sel));
- clks[IMX7D_ARM_M4_ROOT_SRC] = imx_clk_mux2("arm_m4_src", base + 0x8080, 24, 3, arm_m4_sel, ARRAY_SIZE(arm_m4_sel));
- clks[IMX7D_MAIN_AXI_ROOT_SRC] = imx_clk_mux2("axi_src", base + 0x8800, 24, 3, axi_sel, ARRAY_SIZE(axi_sel));
- clks[IMX7D_DISP_AXI_ROOT_SRC] = imx_clk_mux2("disp_axi_src", base + 0x8880, 24, 3, disp_axi_sel, ARRAY_SIZE(disp_axi_sel));
- clks[IMX7D_ENET_AXI_ROOT_SRC] = imx_clk_mux2("enet_axi_src", base + 0x8900, 24, 3, enet_axi_sel, ARRAY_SIZE(enet_axi_sel));
- clks[IMX7D_NAND_USDHC_BUS_ROOT_SRC] = imx_clk_mux2("nand_usdhc_src", base + 0x8980, 24, 3, nand_usdhc_bus_sel, ARRAY_SIZE(nand_usdhc_bus_sel));
- clks[IMX7D_AHB_CHANNEL_ROOT_SRC] = imx_clk_mux2("ahb_src", base + 0x9000, 24, 3, ahb_channel_sel, ARRAY_SIZE(ahb_channel_sel));
- clks[IMX7D_DRAM_PHYM_ROOT_SRC] = imx_clk_mux2("dram_phym_src", base + 0x9800, 24, 1, dram_phym_sel, ARRAY_SIZE(dram_phym_sel));
- clks[IMX7D_DRAM_ROOT_SRC] = imx_clk_mux2("dram_src", base + 0x9880, 24, 1, dram_sel, ARRAY_SIZE(dram_sel));
- clks[IMX7D_DRAM_PHYM_ALT_ROOT_SRC] = imx_clk_mux2("dram_phym_alt_src", base + 0xa000, 24, 3, dram_phym_alt_sel, ARRAY_SIZE(dram_phym_alt_sel));
- clks[IMX7D_DRAM_ALT_ROOT_SRC] = imx_clk_mux2("dram_alt_src", base + 0xa080, 24, 3, dram_alt_sel, ARRAY_SIZE(dram_alt_sel));
- clks[IMX7D_USB_HSIC_ROOT_SRC] = imx_clk_mux2("usb_hsic_src", base + 0xa100, 24, 3, usb_hsic_sel, ARRAY_SIZE(usb_hsic_sel));
- clks[IMX7D_PCIE_CTRL_ROOT_SRC] = imx_clk_mux2("pcie_ctrl_src", base + 0xa180, 24, 3, pcie_ctrl_sel, ARRAY_SIZE(pcie_ctrl_sel));
- clks[IMX7D_PCIE_PHY_ROOT_SRC] = imx_clk_mux2("pcie_phy_src", base + 0xa200, 24, 3, pcie_phy_sel, ARRAY_SIZE(pcie_phy_sel));
- clks[IMX7D_EPDC_PIXEL_ROOT_SRC] = imx_clk_mux2("epdc_pixel_src", base + 0xa280, 24, 3, epdc_pixel_sel, ARRAY_SIZE(epdc_pixel_sel));
- clks[IMX7D_LCDIF_PIXEL_ROOT_SRC] = imx_clk_mux2("lcdif_pixel_src", base + 0xa300, 24, 3, lcdif_pixel_sel, ARRAY_SIZE(lcdif_pixel_sel));
- clks[IMX7D_MIPI_DSI_ROOT_SRC] = imx_clk_mux2("mipi_dsi_src", base + 0xa380, 24, 3, mipi_dsi_sel, ARRAY_SIZE(mipi_dsi_sel));
- clks[IMX7D_MIPI_CSI_ROOT_SRC] = imx_clk_mux2("mipi_csi_src", base + 0xa400, 24, 3, mipi_csi_sel, ARRAY_SIZE(mipi_csi_sel));
- clks[IMX7D_MIPI_DPHY_ROOT_SRC] = imx_clk_mux2("mipi_dphy_src", base + 0xa480, 24, 3, mipi_dphy_sel, ARRAY_SIZE(mipi_dphy_sel));
- clks[IMX7D_SAI1_ROOT_SRC] = imx_clk_mux2("sai1_src", base + 0xa500, 24, 3, sai1_sel, ARRAY_SIZE(sai1_sel));
- clks[IMX7D_SAI2_ROOT_SRC] = imx_clk_mux2("sai2_src", base + 0xa580, 24, 3, sai2_sel, ARRAY_SIZE(sai2_sel));
- clks[IMX7D_SAI3_ROOT_SRC] = imx_clk_mux2("sai3_src", base + 0xa600, 24, 3, sai3_sel, ARRAY_SIZE(sai3_sel));
- clks[IMX7D_SPDIF_ROOT_SRC] = imx_clk_mux2("spdif_src", base + 0xa680, 24, 3, spdif_sel, ARRAY_SIZE(spdif_sel));
- clks[IMX7D_ENET1_REF_ROOT_SRC] = imx_clk_mux2("enet1_ref_src", base + 0xa700, 24, 3, enet1_ref_sel, ARRAY_SIZE(enet1_ref_sel));
- clks[IMX7D_ENET1_TIME_ROOT_SRC] = imx_clk_mux2("enet1_time_src", base + 0xa780, 24, 3, enet1_time_sel, ARRAY_SIZE(enet1_time_sel));
- clks[IMX7D_ENET2_REF_ROOT_SRC] = imx_clk_mux2("enet2_ref_src", base + 0xa800, 24, 3, enet2_ref_sel, ARRAY_SIZE(enet2_ref_sel));
- clks[IMX7D_ENET2_TIME_ROOT_SRC] = imx_clk_mux2("enet2_time_src", base + 0xa880, 24, 3, enet2_time_sel, ARRAY_SIZE(enet2_time_sel));
- clks[IMX7D_ENET_PHY_REF_ROOT_SRC] = imx_clk_mux2("enet_phy_ref_src", base + 0xa900, 24, 3, enet_phy_ref_sel, ARRAY_SIZE(enet_phy_ref_sel));
- clks[IMX7D_EIM_ROOT_SRC] = imx_clk_mux2("eim_src", base + 0xa980, 24, 3, eim_sel, ARRAY_SIZE(eim_sel));
- clks[IMX7D_NAND_ROOT_SRC] = imx_clk_mux2("nand_src", base + 0xaa00, 24, 3, nand_sel, ARRAY_SIZE(nand_sel));
- clks[IMX7D_QSPI_ROOT_SRC] = imx_clk_mux2("qspi_src", base + 0xaa80, 24, 3, qspi_sel, ARRAY_SIZE(qspi_sel));
- clks[IMX7D_USDHC1_ROOT_SRC] = imx_clk_mux2("usdhc1_src", base + 0xab00, 24, 3, usdhc1_sel, ARRAY_SIZE(usdhc1_sel));
- clks[IMX7D_USDHC2_ROOT_SRC] = imx_clk_mux2("usdhc2_src", base + 0xab80, 24, 3, usdhc2_sel, ARRAY_SIZE(usdhc2_sel));
- clks[IMX7D_USDHC3_ROOT_SRC] = imx_clk_mux2("usdhc3_src", base + 0xac00, 24, 3, usdhc3_sel, ARRAY_SIZE(usdhc3_sel));
- clks[IMX7D_CAN1_ROOT_SRC] = imx_clk_mux2("can1_src", base + 0xac80, 24, 3, can1_sel, ARRAY_SIZE(can1_sel));
- clks[IMX7D_CAN2_ROOT_SRC] = imx_clk_mux2("can2_src", base + 0xad00, 24, 3, can2_sel, ARRAY_SIZE(can2_sel));
- clks[IMX7D_I2C1_ROOT_SRC] = imx_clk_mux2("i2c1_src", base + 0xad80, 24, 3, i2c1_sel, ARRAY_SIZE(i2c1_sel));
- clks[IMX7D_I2C2_ROOT_SRC] = imx_clk_mux2("i2c2_src", base + 0xae00, 24, 3, i2c2_sel, ARRAY_SIZE(i2c2_sel));
- clks[IMX7D_I2C3_ROOT_SRC] = imx_clk_mux2("i2c3_src", base + 0xae80, 24, 3, i2c3_sel, ARRAY_SIZE(i2c3_sel));
- clks[IMX7D_I2C4_ROOT_SRC] = imx_clk_mux2("i2c4_src", base + 0xaf00, 24, 3, i2c4_sel, ARRAY_SIZE(i2c4_sel));
- clks[IMX7D_UART1_ROOT_SRC] = imx_clk_mux2("uart1_src", base + 0xaf80, 24, 3, uart1_sel, ARRAY_SIZE(uart1_sel));
- clks[IMX7D_UART2_ROOT_SRC] = imx_clk_mux2("uart2_src", base + 0xb000, 24, 3, uart2_sel, ARRAY_SIZE(uart2_sel));
- clks[IMX7D_UART3_ROOT_SRC] = imx_clk_mux2("uart3_src", base + 0xb080, 24, 3, uart3_sel, ARRAY_SIZE(uart3_sel));
- clks[IMX7D_UART4_ROOT_SRC] = imx_clk_mux2("uart4_src", base + 0xb100, 24, 3, uart4_sel, ARRAY_SIZE(uart4_sel));
- clks[IMX7D_UART5_ROOT_SRC] = imx_clk_mux2("uart5_src", base + 0xb180, 24, 3, uart5_sel, ARRAY_SIZE(uart5_sel));
- clks[IMX7D_UART6_ROOT_SRC] = imx_clk_mux2("uart6_src", base + 0xb200, 24, 3, uart6_sel, ARRAY_SIZE(uart6_sel));
- clks[IMX7D_UART7_ROOT_SRC] = imx_clk_mux2("uart7_src", base + 0xb280, 24, 3, uart7_sel, ARRAY_SIZE(uart7_sel));
- clks[IMX7D_ECSPI1_ROOT_SRC] = imx_clk_mux2("ecspi1_src", base + 0xb300, 24, 3, ecspi1_sel, ARRAY_SIZE(ecspi1_sel));
- clks[IMX7D_ECSPI2_ROOT_SRC] = imx_clk_mux2("ecspi2_src", base + 0xb380, 24, 3, ecspi2_sel, ARRAY_SIZE(ecspi2_sel));
- clks[IMX7D_ECSPI3_ROOT_SRC] = imx_clk_mux2("ecspi3_src", base + 0xb400, 24, 3, ecspi3_sel, ARRAY_SIZE(ecspi3_sel));
- clks[IMX7D_ECSPI4_ROOT_SRC] = imx_clk_mux2("ecspi4_src", base + 0xb480, 24, 3, ecspi4_sel, ARRAY_SIZE(ecspi4_sel));
- clks[IMX7D_PWM1_ROOT_SRC] = imx_clk_mux2("pwm1_src", base + 0xb500, 24, 3, pwm1_sel, ARRAY_SIZE(pwm1_sel));
- clks[IMX7D_PWM2_ROOT_SRC] = imx_clk_mux2("pwm2_src", base + 0xb580, 24, 3, pwm2_sel, ARRAY_SIZE(pwm2_sel));
- clks[IMX7D_PWM3_ROOT_SRC] = imx_clk_mux2("pwm3_src", base + 0xb600, 24, 3, pwm3_sel, ARRAY_SIZE(pwm3_sel));
- clks[IMX7D_PWM4_ROOT_SRC] = imx_clk_mux2("pwm4_src", base + 0xb680, 24, 3, pwm4_sel, ARRAY_SIZE(pwm4_sel));
- clks[IMX7D_FLEXTIMER1_ROOT_SRC] = imx_clk_mux2("flextimer1_src", base + 0xb700, 24, 3, flextimer1_sel, ARRAY_SIZE(flextimer1_sel));
- clks[IMX7D_FLEXTIMER2_ROOT_SRC] = imx_clk_mux2("flextimer2_src", base + 0xb780, 24, 3, flextimer2_sel, ARRAY_SIZE(flextimer2_sel));
- clks[IMX7D_SIM1_ROOT_SRC] = imx_clk_mux2("sim1_src", base + 0xb800, 24, 3, sim1_sel, ARRAY_SIZE(sim1_sel));
- clks[IMX7D_SIM2_ROOT_SRC] = imx_clk_mux2("sim2_src", base + 0xb880, 24, 3, sim2_sel, ARRAY_SIZE(sim2_sel));
- clks[IMX7D_GPT1_ROOT_SRC] = imx_clk_mux2("gpt1_src", base + 0xb900, 24, 3, gpt1_sel, ARRAY_SIZE(gpt1_sel));
- clks[IMX7D_GPT2_ROOT_SRC] = imx_clk_mux2("gpt2_src", base + 0xb980, 24, 3, gpt2_sel, ARRAY_SIZE(gpt2_sel));
- clks[IMX7D_GPT3_ROOT_SRC] = imx_clk_mux2("gpt3_src", base + 0xba00, 24, 3, gpt3_sel, ARRAY_SIZE(gpt3_sel));
- clks[IMX7D_GPT4_ROOT_SRC] = imx_clk_mux2("gpt4_src", base + 0xba80, 24, 3, gpt4_sel, ARRAY_SIZE(gpt4_sel));
- clks[IMX7D_TRACE_ROOT_SRC] = imx_clk_mux2("trace_src", base + 0xbb00, 24, 3, trace_sel, ARRAY_SIZE(trace_sel));
- clks[IMX7D_WDOG_ROOT_SRC] = imx_clk_mux2("wdog_src", base + 0xbb80, 24, 3, wdog_sel, ARRAY_SIZE(wdog_sel));
- clks[IMX7D_CSI_MCLK_ROOT_SRC] = imx_clk_mux2("csi_mclk_src", base + 0xbc00, 24, 3, csi_mclk_sel, ARRAY_SIZE(csi_mclk_sel));
- clks[IMX7D_AUDIO_MCLK_ROOT_SRC] = imx_clk_mux2("audio_mclk_src", base + 0xbc80, 24, 3, audio_mclk_sel, ARRAY_SIZE(audio_mclk_sel));
- clks[IMX7D_WRCLK_ROOT_SRC] = imx_clk_mux2("wrclk_src", base + 0xbd00, 24, 3, wrclk_sel, ARRAY_SIZE(wrclk_sel));
- clks[IMX7D_CLKO1_ROOT_SRC] = imx_clk_mux2("clko1_src", base + 0xbd80, 24, 3, clko1_sel, ARRAY_SIZE(clko1_sel));
- clks[IMX7D_CLKO2_ROOT_SRC] = imx_clk_mux2("clko2_src", base + 0xbe00, 24, 3, clko2_sel, ARRAY_SIZE(clko2_sel));
-
- clks[IMX7D_ARM_A7_ROOT_CG] = imx_clk_gate3("arm_a7_cg", "arm_a7_src", base + 0x8000, 28);
- clks[IMX7D_ARM_M4_ROOT_CG] = imx_clk_gate3("arm_m4_cg", "arm_m4_src", base + 0x8080, 28);
- clks[IMX7D_MAIN_AXI_ROOT_CG] = imx_clk_gate3("axi_cg", "axi_src", base + 0x8800, 28);
- clks[IMX7D_DISP_AXI_ROOT_CG] = imx_clk_gate3("disp_axi_cg", "disp_axi_src", base + 0x8880, 28);
- clks[IMX7D_ENET_AXI_ROOT_CG] = imx_clk_gate3("enet_axi_cg", "enet_axi_src", base + 0x8900, 28);
- clks[IMX7D_NAND_USDHC_BUS_ROOT_CG] = imx_clk_gate3("nand_usdhc_cg", "nand_usdhc_src", base + 0x8980, 28);
- clks[IMX7D_AHB_CHANNEL_ROOT_CG] = imx_clk_gate3("ahb_cg", "ahb_src", base + 0x9000, 28);
- clks[IMX7D_DRAM_PHYM_ROOT_CG] = imx_clk_gate3("dram_phym_cg", "dram_phym_src", base + 0x9800, 28);
- clks[IMX7D_DRAM_ROOT_CG] = imx_clk_gate3("dram_cg", "dram_src", base + 0x9880, 28);
- clks[IMX7D_DRAM_PHYM_ALT_ROOT_CG] = imx_clk_gate3("dram_phym_alt_cg", "dram_phym_alt_src", base + 0xa000, 28);
- clks[IMX7D_DRAM_ALT_ROOT_CG] = imx_clk_gate3("dram_alt_cg", "dram_alt_src", base + 0xa080, 28);
- clks[IMX7D_USB_HSIC_ROOT_CG] = imx_clk_gate3("usb_hsic_cg", "usb_hsic_src", base + 0xa100, 28);
- clks[IMX7D_PCIE_CTRL_ROOT_CG] = imx_clk_gate3("pcie_ctrl_cg", "pcie_ctrl_src", base + 0xa180, 28);
- clks[IMX7D_PCIE_PHY_ROOT_CG] = imx_clk_gate3("pcie_phy_cg", "pcie_phy_src", base + 0xa200, 28);
- clks[IMX7D_EPDC_PIXEL_ROOT_CG] = imx_clk_gate3("epdc_pixel_cg", "epdc_pixel_src", base + 0xa280, 28);
- clks[IMX7D_LCDIF_PIXEL_ROOT_CG] = imx_clk_gate3("lcdif_pixel_cg", "lcdif_pixel_src", base + 0xa300, 28);
- clks[IMX7D_MIPI_DSI_ROOT_CG] = imx_clk_gate3("mipi_dsi_cg", "mipi_dsi_src", base + 0xa380, 28);
- clks[IMX7D_MIPI_CSI_ROOT_CG] = imx_clk_gate3("mipi_csi_cg", "mipi_csi_src", base + 0xa400, 28);
- clks[IMX7D_MIPI_DPHY_ROOT_CG] = imx_clk_gate3("mipi_dphy_cg", "mipi_dphy_src", base + 0xa480, 28);
- clks[IMX7D_SAI1_ROOT_CG] = imx_clk_gate3("sai1_cg", "sai1_src", base + 0xa500, 28);
- clks[IMX7D_SAI2_ROOT_CG] = imx_clk_gate3("sai2_cg", "sai2_src", base + 0xa580, 28);
- clks[IMX7D_SAI3_ROOT_CG] = imx_clk_gate3("sai3_cg", "sai3_src", base + 0xa600, 28);
- clks[IMX7D_SPDIF_ROOT_CG] = imx_clk_gate3("spdif_cg", "spdif_src", base + 0xa680, 28);
- clks[IMX7D_ENET1_REF_ROOT_CG] = imx_clk_gate3("enet1_ref_cg", "enet1_ref_src", base + 0xa700, 28);
- clks[IMX7D_ENET1_TIME_ROOT_CG] = imx_clk_gate3("enet1_time_cg", "enet1_time_src", base + 0xa780, 28);
- clks[IMX7D_ENET2_REF_ROOT_CG] = imx_clk_gate3("enet2_ref_cg", "enet2_ref_src", base + 0xa800, 28);
- clks[IMX7D_ENET2_TIME_ROOT_CG] = imx_clk_gate3("enet2_time_cg", "enet2_time_src", base + 0xa880, 28);
- clks[IMX7D_ENET_PHY_REF_ROOT_CG] = imx_clk_gate3("enet_phy_ref_cg", "enet_phy_ref_src", base + 0xa900, 28);
- clks[IMX7D_EIM_ROOT_CG] = imx_clk_gate3("eim_cg", "eim_src", base + 0xa980, 28);
- clks[IMX7D_NAND_ROOT_CG] = imx_clk_gate3("nand_cg", "nand_src", base + 0xaa00, 28);
- clks[IMX7D_QSPI_ROOT_CG] = imx_clk_gate3("qspi_cg", "qspi_src", base + 0xaa80, 28);
- clks[IMX7D_USDHC1_ROOT_CG] = imx_clk_gate3("usdhc1_cg", "usdhc1_src", base + 0xab00, 28);
- clks[IMX7D_USDHC2_ROOT_CG] = imx_clk_gate3("usdhc2_cg", "usdhc2_src", base + 0xab80, 28);
- clks[IMX7D_USDHC3_ROOT_CG] = imx_clk_gate3("usdhc3_cg", "usdhc3_src", base + 0xac00, 28);
- clks[IMX7D_CAN1_ROOT_CG] = imx_clk_gate3("can1_cg", "can1_src", base + 0xac80, 28);
- clks[IMX7D_CAN2_ROOT_CG] = imx_clk_gate3("can2_cg", "can2_src", base + 0xad00, 28);
- clks[IMX7D_I2C1_ROOT_CG] = imx_clk_gate3("i2c1_cg", "i2c1_src", base + 0xad80, 28);
- clks[IMX7D_I2C2_ROOT_CG] = imx_clk_gate3("i2c2_cg", "i2c2_src", base + 0xae00, 28);
- clks[IMX7D_I2C3_ROOT_CG] = imx_clk_gate3("i2c3_cg", "i2c3_src", base + 0xae80, 28);
- clks[IMX7D_I2C4_ROOT_CG] = imx_clk_gate3("i2c4_cg", "i2c4_src", base + 0xaf00, 28);
- clks[IMX7D_UART1_ROOT_CG] = imx_clk_gate3("uart1_cg", "uart1_src", base + 0xaf80, 28);
- clks[IMX7D_UART2_ROOT_CG] = imx_clk_gate3("uart2_cg", "uart2_src", base + 0xb000, 28);
- clks[IMX7D_UART3_ROOT_CG] = imx_clk_gate3("uart3_cg", "uart3_src", base + 0xb080, 28);
- clks[IMX7D_UART4_ROOT_CG] = imx_clk_gate3("uart4_cg", "uart4_src", base + 0xb100, 28);
- clks[IMX7D_UART5_ROOT_CG] = imx_clk_gate3("uart5_cg", "uart5_src", base + 0xb180, 28);
- clks[IMX7D_UART6_ROOT_CG] = imx_clk_gate3("uart6_cg", "uart6_src", base + 0xb200, 28);
- clks[IMX7D_UART7_ROOT_CG] = imx_clk_gate3("uart7_cg", "uart7_src", base + 0xb280, 28);
- clks[IMX7D_ECSPI1_ROOT_CG] = imx_clk_gate3("ecspi1_cg", "ecspi1_src", base + 0xb300, 28);
- clks[IMX7D_ECSPI2_ROOT_CG] = imx_clk_gate3("ecspi2_cg", "ecspi2_src", base + 0xb380, 28);
- clks[IMX7D_ECSPI3_ROOT_CG] = imx_clk_gate3("ecspi3_cg", "ecspi3_src", base + 0xb400, 28);
- clks[IMX7D_ECSPI4_ROOT_CG] = imx_clk_gate3("ecspi4_cg", "ecspi4_src", base + 0xb480, 28);
- clks[IMX7D_PWM1_ROOT_CG] = imx_clk_gate3("pwm1_cg", "pwm1_src", base + 0xb500, 28);
- clks[IMX7D_PWM2_ROOT_CG] = imx_clk_gate3("pwm2_cg", "pwm2_src", base + 0xb580, 28);
- clks[IMX7D_PWM3_ROOT_CG] = imx_clk_gate3("pwm3_cg", "pwm3_src", base + 0xb600, 28);
- clks[IMX7D_PWM4_ROOT_CG] = imx_clk_gate3("pwm4_cg", "pwm4_src", base + 0xb680, 28);
- clks[IMX7D_FLEXTIMER1_ROOT_CG] = imx_clk_gate3("flextimer1_cg", "flextimer1_src", base + 0xb700, 28);
- clks[IMX7D_FLEXTIMER2_ROOT_CG] = imx_clk_gate3("flextimer2_cg", "flextimer2_src", base + 0xb780, 28);
- clks[IMX7D_SIM1_ROOT_CG] = imx_clk_gate3("sim1_cg", "sim1_src", base + 0xb800, 28);
- clks[IMX7D_SIM2_ROOT_CG] = imx_clk_gate3("sim2_cg", "sim2_src", base + 0xb880, 28);
- clks[IMX7D_GPT1_ROOT_CG] = imx_clk_gate3("gpt1_cg", "gpt1_src", base + 0xb900, 28);
- clks[IMX7D_GPT2_ROOT_CG] = imx_clk_gate3("gpt2_cg", "gpt2_src", base + 0xb980, 28);
- clks[IMX7D_GPT3_ROOT_CG] = imx_clk_gate3("gpt3_cg", "gpt3_src", base + 0xbA00, 28);
- clks[IMX7D_GPT4_ROOT_CG] = imx_clk_gate3("gpt4_cg", "gpt4_src", base + 0xbA80, 28);
- clks[IMX7D_TRACE_ROOT_CG] = imx_clk_gate3("trace_cg", "trace_src", base + 0xbb00, 28);
- clks[IMX7D_WDOG_ROOT_CG] = imx_clk_gate3("wdog_cg", "wdog_src", base + 0xbb80, 28);
- clks[IMX7D_CSI_MCLK_ROOT_CG] = imx_clk_gate3("csi_mclk_cg", "csi_mclk_src", base + 0xbc00, 28);
- clks[IMX7D_AUDIO_MCLK_ROOT_CG] = imx_clk_gate3("audio_mclk_cg", "audio_mclk_src", base + 0xbc80, 28);
- clks[IMX7D_WRCLK_ROOT_CG] = imx_clk_gate3("wrclk_cg", "wrclk_src", base + 0xbd00, 28);
- clks[IMX7D_CLKO1_ROOT_CG] = imx_clk_gate3("clko1_cg", "clko1_src", base + 0xbd80, 28);
- clks[IMX7D_CLKO2_ROOT_CG] = imx_clk_gate3("clko2_cg", "clko2_src", base + 0xbe00, 28);
-
- clks[IMX7D_MAIN_AXI_ROOT_PRE_DIV] = imx_clk_divider2("axi_pre_div", "axi_cg", base + 0x8800, 16, 3);
- clks[IMX7D_DISP_AXI_ROOT_PRE_DIV] = imx_clk_divider2("disp_axi_pre_div", "disp_axi_cg", base + 0x8880, 16, 3);
- clks[IMX7D_ENET_AXI_ROOT_PRE_DIV] = imx_clk_divider2("enet_axi_pre_div", "enet_axi_cg", base + 0x8900, 16, 3);
- clks[IMX7D_NAND_USDHC_BUS_ROOT_PRE_DIV] = imx_clk_divider2("nand_usdhc_pre_div", "nand_usdhc_cg", base + 0x8980, 16, 3);
- clks[IMX7D_AHB_CHANNEL_ROOT_PRE_DIV] = imx_clk_divider2("ahb_pre_div", "ahb_cg", base + 0x9000, 16, 3);
- clks[IMX7D_DRAM_PHYM_ALT_ROOT_PRE_DIV] = imx_clk_divider2("dram_phym_alt_pre_div", "dram_phym_alt_cg", base + 0xa000, 16, 3);
- clks[IMX7D_DRAM_ALT_ROOT_PRE_DIV] = imx_clk_divider2("dram_alt_pre_div", "dram_alt_cg", base + 0xa080, 16, 3);
- clks[IMX7D_USB_HSIC_ROOT_PRE_DIV] = imx_clk_divider2("usb_hsic_pre_div", "usb_hsic_cg", base + 0xa100, 16, 3);
- clks[IMX7D_PCIE_CTRL_ROOT_PRE_DIV] = imx_clk_divider2("pcie_ctrl_pre_div", "pcie_ctrl_cg", base + 0xa180, 16, 3);
- clks[IMX7D_PCIE_PHY_ROOT_PRE_DIV] = imx_clk_divider2("pcie_phy_pre_div", "pcie_phy_cg", base + 0xa200, 16, 3);
- clks[IMX7D_EPDC_PIXEL_ROOT_PRE_DIV] = imx_clk_divider2("epdc_pixel_pre_div", "epdc_pixel_cg", base + 0xa280, 16, 3);
- clks[IMX7D_LCDIF_PIXEL_ROOT_PRE_DIV] = imx_clk_divider2("lcdif_pixel_pre_div", "lcdif_pixel_cg", base + 0xa300, 16, 3);
- clks[IMX7D_MIPI_DSI_ROOT_PRE_DIV] = imx_clk_divider2("mipi_dsi_pre_div", "mipi_dsi_cg", base + 0xa380, 16, 3);
- clks[IMX7D_MIPI_CSI_ROOT_PRE_DIV] = imx_clk_divider2("mipi_csi_pre_div", "mipi_csi_cg", base + 0xa400, 16, 3);
- clks[IMX7D_MIPI_DPHY_ROOT_PRE_DIV] = imx_clk_divider2("mipi_dphy_pre_div", "mipi_dphy_cg", base + 0xa480, 16, 3);
- clks[IMX7D_SAI1_ROOT_PRE_DIV] = imx_clk_divider2("sai1_pre_div", "sai1_cg", base + 0xa500, 16, 3);
- clks[IMX7D_SAI2_ROOT_PRE_DIV] = imx_clk_divider2("sai2_pre_div", "sai2_cg", base + 0xa580, 16, 3);
- clks[IMX7D_SAI3_ROOT_PRE_DIV] = imx_clk_divider2("sai3_pre_div", "sai3_cg", base + 0xa600, 16, 3);
- clks[IMX7D_SPDIF_ROOT_PRE_DIV] = imx_clk_divider2("spdif_pre_div", "spdif_cg", base + 0xa680, 16, 3);
- clks[IMX7D_ENET1_REF_ROOT_PRE_DIV] = imx_clk_divider2("enet1_ref_pre_div", "enet1_ref_cg", base + 0xa700, 16, 3);
- clks[IMX7D_ENET1_TIME_ROOT_PRE_DIV] = imx_clk_divider2("enet1_time_pre_div", "enet1_time_cg", base + 0xa780, 16, 3);
- clks[IMX7D_ENET2_REF_ROOT_PRE_DIV] = imx_clk_divider2("enet2_ref_pre_div", "enet2_ref_cg", base + 0xa800, 16, 3);
- clks[IMX7D_ENET2_TIME_ROOT_PRE_DIV] = imx_clk_divider2("enet2_time_pre_div", "enet2_time_cg", base + 0xa880, 16, 3);
- clks[IMX7D_ENET_PHY_REF_ROOT_PRE_DIV] = imx_clk_divider2("enet_phy_ref_pre_div", "enet_phy_ref_cg", base + 0xa900, 16, 3);
- clks[IMX7D_EIM_ROOT_PRE_DIV] = imx_clk_divider2("eim_pre_div", "eim_cg", base + 0xa980, 16, 3);
- clks[IMX7D_NAND_ROOT_PRE_DIV] = imx_clk_divider2("nand_pre_div", "nand_cg", base + 0xaa00, 16, 3);
- clks[IMX7D_QSPI_ROOT_PRE_DIV] = imx_clk_divider2("qspi_pre_div", "qspi_cg", base + 0xaa80, 16, 3);
- clks[IMX7D_USDHC1_ROOT_PRE_DIV] = imx_clk_divider2("usdhc1_pre_div", "usdhc1_cg", base + 0xab00, 16, 3);
- clks[IMX7D_USDHC2_ROOT_PRE_DIV] = imx_clk_divider2("usdhc2_pre_div", "usdhc2_cg", base + 0xab80, 16, 3);
- clks[IMX7D_USDHC3_ROOT_PRE_DIV] = imx_clk_divider2("usdhc3_pre_div", "usdhc3_cg", base + 0xac00, 16, 3);
- clks[IMX7D_CAN1_ROOT_PRE_DIV] = imx_clk_divider2("can1_pre_div", "can1_cg", base + 0xac80, 16, 3);
- clks[IMX7D_CAN2_ROOT_PRE_DIV] = imx_clk_divider2("can2_pre_div", "can2_cg", base + 0xad00, 16, 3);
- clks[IMX7D_I2C1_ROOT_PRE_DIV] = imx_clk_divider2("i2c1_pre_div", "i2c1_cg", base + 0xad80, 16, 3);
- clks[IMX7D_I2C2_ROOT_PRE_DIV] = imx_clk_divider2("i2c2_pre_div", "i2c2_cg", base + 0xae00, 16, 3);
- clks[IMX7D_I2C3_ROOT_PRE_DIV] = imx_clk_divider2("i2c3_pre_div", "i2c3_cg", base + 0xae80, 16, 3);
- clks[IMX7D_I2C4_ROOT_PRE_DIV] = imx_clk_divider2("i2c4_pre_div", "i2c4_cg", base + 0xaf00, 16, 3);
- clks[IMX7D_UART1_ROOT_PRE_DIV] = imx_clk_divider2("uart1_pre_div", "uart1_cg", base + 0xaf80, 16, 3);
- clks[IMX7D_UART2_ROOT_PRE_DIV] = imx_clk_divider2("uart2_pre_div", "uart2_cg", base + 0xb000, 16, 3);
- clks[IMX7D_UART3_ROOT_PRE_DIV] = imx_clk_divider2("uart3_pre_div", "uart3_cg", base + 0xb080, 16, 3);
- clks[IMX7D_UART4_ROOT_PRE_DIV] = imx_clk_divider2("uart4_pre_div", "uart4_cg", base + 0xb100, 16, 3);
- clks[IMX7D_UART5_ROOT_PRE_DIV] = imx_clk_divider2("uart5_pre_div", "uart5_cg", base + 0xb180, 16, 3);
- clks[IMX7D_UART6_ROOT_PRE_DIV] = imx_clk_divider2("uart6_pre_div", "uart6_cg", base + 0xb200, 16, 3);
- clks[IMX7D_UART7_ROOT_PRE_DIV] = imx_clk_divider2("uart7_pre_div", "uart7_cg", base + 0xb280, 16, 3);
- clks[IMX7D_ECSPI1_ROOT_PRE_DIV] = imx_clk_divider2("ecspi1_pre_div", "ecspi1_cg", base + 0xb300, 16, 3);
- clks[IMX7D_ECSPI2_ROOT_PRE_DIV] = imx_clk_divider2("ecspi2_pre_div", "ecspi2_cg", base + 0xb380, 16, 3);
- clks[IMX7D_ECSPI3_ROOT_PRE_DIV] = imx_clk_divider2("ecspi3_pre_div", "ecspi3_cg", base + 0xb400, 16, 3);
- clks[IMX7D_ECSPI4_ROOT_PRE_DIV] = imx_clk_divider2("ecspi4_pre_div", "ecspi4_cg", base + 0xb480, 16, 3);
- clks[IMX7D_PWM1_ROOT_PRE_DIV] = imx_clk_divider2("pwm1_pre_div", "pwm1_cg", base + 0xb500, 16, 3);
- clks[IMX7D_PWM2_ROOT_PRE_DIV] = imx_clk_divider2("pwm2_pre_div", "pwm2_cg", base + 0xb580, 16, 3);
- clks[IMX7D_PWM3_ROOT_PRE_DIV] = imx_clk_divider2("pwm3_pre_div", "pwm3_cg", base + 0xb600, 16, 3);
- clks[IMX7D_PWM4_ROOT_PRE_DIV] = imx_clk_divider2("pwm4_pre_div", "pwm4_cg", base + 0xb680, 16, 3);
- clks[IMX7D_FLEXTIMER1_ROOT_PRE_DIV] = imx_clk_divider2("flextimer1_pre_div", "flextimer1_cg", base + 0xb700, 16, 3);
- clks[IMX7D_FLEXTIMER2_ROOT_PRE_DIV] = imx_clk_divider2("flextimer2_pre_div", "flextimer2_cg", base + 0xb780, 16, 3);
- clks[IMX7D_SIM1_ROOT_PRE_DIV] = imx_clk_divider2("sim1_pre_div", "sim1_cg", base + 0xb800, 16, 3);
- clks[IMX7D_SIM2_ROOT_PRE_DIV] = imx_clk_divider2("sim2_pre_div", "sim2_cg", base + 0xb880, 16, 3);
- clks[IMX7D_GPT1_ROOT_PRE_DIV] = imx_clk_divider2("gpt1_pre_div", "gpt1_cg", base + 0xb900, 16, 3);
- clks[IMX7D_GPT2_ROOT_PRE_DIV] = imx_clk_divider2("gpt2_pre_div", "gpt2_cg", base + 0xb980, 16, 3);
- clks[IMX7D_GPT3_ROOT_PRE_DIV] = imx_clk_divider2("gpt3_pre_div", "gpt3_cg", base + 0xba00, 16, 3);
- clks[IMX7D_GPT4_ROOT_PRE_DIV] = imx_clk_divider2("gpt4_pre_div", "gpt4_cg", base + 0xba80, 16, 3);
- clks[IMX7D_TRACE_ROOT_PRE_DIV] = imx_clk_divider2("trace_pre_div", "trace_cg", base + 0xbb00, 16, 3);
- clks[IMX7D_WDOG_ROOT_PRE_DIV] = imx_clk_divider2("wdog_pre_div", "wdog_cg", base + 0xbb80, 16, 3);
- clks[IMX7D_CSI_MCLK_ROOT_PRE_DIV] = imx_clk_divider2("csi_mclk_pre_div", "csi_mclk_cg", base + 0xbc00, 16, 3);
- clks[IMX7D_AUDIO_MCLK_ROOT_PRE_DIV] = imx_clk_divider2("audio_mclk_pre_div", "audio_mclk_cg", base + 0xbc80, 16, 3);
- clks[IMX7D_WRCLK_ROOT_PRE_DIV] = imx_clk_divider2("wrclk_pre_div", "wrclk_cg", base + 0xbd00, 16, 3);
- clks[IMX7D_CLKO1_ROOT_PRE_DIV] = imx_clk_divider2("clko1_pre_div", "clko1_cg", base + 0xbd80, 16, 3);
- clks[IMX7D_CLKO2_ROOT_PRE_DIV] = imx_clk_divider2("clko2_pre_div", "clko2_cg", base + 0xbe00, 16, 3);
-
- clks[IMX7D_ARM_A7_ROOT_DIV] = imx_clk_divider2("arm_a7_div", "arm_a7_cg", base + 0x8000, 0, 3);
- clks[IMX7D_ARM_M4_ROOT_DIV] = imx_clk_divider2("arm_m4_div", "arm_m4_cg", base + 0x8080, 0, 3);
- clks[IMX7D_MAIN_AXI_ROOT_DIV] = imx_clk_divider2("axi_post_div", "axi_pre_div", base + 0x8800, 0, 6);
- clks[IMX7D_DISP_AXI_ROOT_DIV] = imx_clk_divider2("disp_axi_post_div", "disp_axi_pre_div", base + 0x8880, 0, 6);
- clks[IMX7D_ENET_AXI_ROOT_DIV] = imx_clk_divider2("enet_axi_post_div", "enet_axi_pre_div", base + 0x8900, 0, 6);
- clks[IMX7D_NAND_USDHC_BUS_ROOT_CLK] = imx_clk_divider2("nand_usdhc_root_clk", "nand_usdhc_pre_div", base + 0x8980, 0, 6);
- clks[IMX7D_AHB_CHANNEL_ROOT_DIV] = imx_clk_divider2("ahb_root_clk", "ahb_pre_div", base + 0x9000, 0, 6);
- clks[IMX7D_IPG_ROOT_CLK] = imx_clk_divider_flags("ipg_root_clk", "ahb_root_clk", base + 0x9080, 0, 2, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE | CLK_SET_RATE_PARENT);
- clks[IMX7D_DRAM_ROOT_DIV] = imx_clk_divider2("dram_post_div", "dram_cg", base + 0x9880, 0, 3);
- clks[IMX7D_DRAM_PHYM_ALT_ROOT_DIV] = imx_clk_divider2("dram_phym_alt_post_div", "dram_phym_alt_pre_div", base + 0xa000, 0, 3);
- clks[IMX7D_DRAM_ALT_ROOT_DIV] = imx_clk_divider2("dram_alt_post_div", "dram_alt_pre_div", base + 0xa080, 0, 3);
- clks[IMX7D_USB_HSIC_ROOT_DIV] = imx_clk_divider2("usb_hsic_post_div", "usb_hsic_pre_div", base + 0xa100, 0, 6);
- clks[IMX7D_PCIE_CTRL_ROOT_DIV] = imx_clk_divider2("pcie_ctrl_post_div", "pcie_ctrl_pre_div", base + 0xa180, 0, 6);
- clks[IMX7D_PCIE_PHY_ROOT_DIV] = imx_clk_divider2("pcie_phy_post_div", "pcie_phy_pre_div", base + 0xa200, 0, 6);
- clks[IMX7D_EPDC_PIXEL_ROOT_DIV] = imx_clk_divider2("epdc_pixel_post_div", "epdc_pixel_pre_div", base + 0xa280, 0, 6);
- clks[IMX7D_LCDIF_PIXEL_ROOT_DIV] = imx_clk_divider2("lcdif_pixel_post_div", "lcdif_pixel_pre_div", base + 0xa300, 0, 6);
- clks[IMX7D_MIPI_DSI_ROOT_DIV] = imx_clk_divider2("mipi_dsi_post_div", "mipi_dsi_pre_div", base + 0xa380, 0, 6);
- clks[IMX7D_MIPI_CSI_ROOT_DIV] = imx_clk_divider2("mipi_csi_post_div", "mipi_csi_pre_div", base + 0xa400, 0, 6);
- clks[IMX7D_MIPI_DPHY_ROOT_DIV] = imx_clk_divider2("mipi_dphy_post_div", "mipi_dphy_pre_div", base + 0xa480, 0, 6);
- clks[IMX7D_SAI1_ROOT_DIV] = imx_clk_divider2("sai1_post_div", "sai1_pre_div", base + 0xa500, 0, 6);
- clks[IMX7D_SAI2_ROOT_DIV] = imx_clk_divider2("sai2_post_div", "sai2_pre_div", base + 0xa580, 0, 6);
- clks[IMX7D_SAI3_ROOT_DIV] = imx_clk_divider2("sai3_post_div", "sai3_pre_div", base + 0xa600, 0, 6);
- clks[IMX7D_SPDIF_ROOT_DIV] = imx_clk_divider2("spdif_post_div", "spdif_pre_div", base + 0xa680, 0, 6);
- clks[IMX7D_ENET1_REF_ROOT_DIV] = imx_clk_divider2("enet1_ref_post_div", "enet1_ref_pre_div", base + 0xa700, 0, 6);
- clks[IMX7D_ENET1_TIME_ROOT_DIV] = imx_clk_divider2("enet1_time_post_div", "enet1_time_pre_div", base + 0xa780, 0, 6);
- clks[IMX7D_ENET2_REF_ROOT_DIV] = imx_clk_divider2("enet2_ref_post_div", "enet2_ref_pre_div", base + 0xa800, 0, 6);
- clks[IMX7D_ENET2_TIME_ROOT_DIV] = imx_clk_divider2("enet2_time_post_div", "enet2_time_pre_div", base + 0xa880, 0, 6);
- clks[IMX7D_ENET_PHY_REF_ROOT_CLK] = imx_clk_divider2("enet_phy_ref_root_clk", "enet_phy_ref_pre_div", base + 0xa900, 0, 6);
- clks[IMX7D_EIM_ROOT_DIV] = imx_clk_divider2("eim_post_div", "eim_pre_div", base + 0xa980, 0, 6);
- clks[IMX7D_NAND_ROOT_CLK] = imx_clk_divider2("nand_root_clk", "nand_pre_div", base + 0xaa00, 0, 6);
- clks[IMX7D_QSPI_ROOT_DIV] = imx_clk_divider2("qspi_post_div", "qspi_pre_div", base + 0xaa80, 0, 6);
- clks[IMX7D_USDHC1_ROOT_DIV] = imx_clk_divider2("usdhc1_post_div", "usdhc1_pre_div", base + 0xab00, 0, 6);
- clks[IMX7D_USDHC2_ROOT_DIV] = imx_clk_divider2("usdhc2_post_div", "usdhc2_pre_div", base + 0xab80, 0, 6);
- clks[IMX7D_USDHC3_ROOT_DIV] = imx_clk_divider2("usdhc3_post_div", "usdhc3_pre_div", base + 0xac00, 0, 6);
- clks[IMX7D_CAN1_ROOT_DIV] = imx_clk_divider2("can1_post_div", "can1_pre_div", base + 0xac80, 0, 6);
- clks[IMX7D_CAN2_ROOT_DIV] = imx_clk_divider2("can2_post_div", "can2_pre_div", base + 0xad00, 0, 6);
- clks[IMX7D_I2C1_ROOT_DIV] = imx_clk_divider2("i2c1_post_div", "i2c1_pre_div", base + 0xad80, 0, 6);
- clks[IMX7D_I2C2_ROOT_DIV] = imx_clk_divider2("i2c2_post_div", "i2c2_pre_div", base + 0xae00, 0, 6);
- clks[IMX7D_I2C3_ROOT_DIV] = imx_clk_divider2("i2c3_post_div", "i2c3_pre_div", base + 0xae80, 0, 6);
- clks[IMX7D_I2C4_ROOT_DIV] = imx_clk_divider2("i2c4_post_div", "i2c4_pre_div", base + 0xaf00, 0, 6);
- clks[IMX7D_UART1_ROOT_DIV] = imx_clk_divider2("uart1_post_div", "uart1_pre_div", base + 0xaf80, 0, 6);
- clks[IMX7D_UART2_ROOT_DIV] = imx_clk_divider2("uart2_post_div", "uart2_pre_div", base + 0xb000, 0, 6);
- clks[IMX7D_UART3_ROOT_DIV] = imx_clk_divider2("uart3_post_div", "uart3_pre_div", base + 0xb080, 0, 6);
- clks[IMX7D_UART4_ROOT_DIV] = imx_clk_divider2("uart4_post_div", "uart4_pre_div", base + 0xb100, 0, 6);
- clks[IMX7D_UART5_ROOT_DIV] = imx_clk_divider2("uart5_post_div", "uart5_pre_div", base + 0xb180, 0, 6);
- clks[IMX7D_UART6_ROOT_DIV] = imx_clk_divider2("uart6_post_div", "uart6_pre_div", base + 0xb200, 0, 6);
- clks[IMX7D_UART7_ROOT_DIV] = imx_clk_divider2("uart7_post_div", "uart7_pre_div", base + 0xb280, 0, 6);
- clks[IMX7D_ECSPI1_ROOT_DIV] = imx_clk_divider2("ecspi1_post_div", "ecspi1_pre_div", base + 0xb300, 0, 6);
- clks[IMX7D_ECSPI2_ROOT_DIV] = imx_clk_divider2("ecspi2_post_div", "ecspi2_pre_div", base + 0xb380, 0, 6);
- clks[IMX7D_ECSPI3_ROOT_DIV] = imx_clk_divider2("ecspi3_post_div", "ecspi3_pre_div", base + 0xb400, 0, 6);
- clks[IMX7D_ECSPI4_ROOT_DIV] = imx_clk_divider2("ecspi4_post_div", "ecspi4_pre_div", base + 0xb480, 0, 6);
- clks[IMX7D_PWM1_ROOT_DIV] = imx_clk_divider2("pwm1_post_div", "pwm1_pre_div", base + 0xb500, 0, 6);
- clks[IMX7D_PWM2_ROOT_DIV] = imx_clk_divider2("pwm2_post_div", "pwm2_pre_div", base + 0xb580, 0, 6);
- clks[IMX7D_PWM3_ROOT_DIV] = imx_clk_divider2("pwm3_post_div", "pwm3_pre_div", base + 0xb600, 0, 6);
- clks[IMX7D_PWM4_ROOT_DIV] = imx_clk_divider2("pwm4_post_div", "pwm4_pre_div", base + 0xb680, 0, 6);
- clks[IMX7D_FLEXTIMER1_ROOT_DIV] = imx_clk_divider2("flextimer1_post_div", "flextimer1_pre_div", base + 0xb700, 0, 6);
- clks[IMX7D_FLEXTIMER2_ROOT_DIV] = imx_clk_divider2("flextimer2_post_div", "flextimer2_pre_div", base + 0xb780, 0, 6);
- clks[IMX7D_SIM1_ROOT_DIV] = imx_clk_divider2("sim1_post_div", "sim1_pre_div", base + 0xb800, 0, 6);
- clks[IMX7D_SIM2_ROOT_DIV] = imx_clk_divider2("sim2_post_div", "sim2_pre_div", base + 0xb880, 0, 6);
- clks[IMX7D_GPT1_ROOT_DIV] = imx_clk_divider2("gpt1_post_div", "gpt1_pre_div", base + 0xb900, 0, 6);
- clks[IMX7D_GPT2_ROOT_DIV] = imx_clk_divider2("gpt2_post_div", "gpt2_pre_div", base + 0xb980, 0, 6);
- clks[IMX7D_GPT3_ROOT_DIV] = imx_clk_divider2("gpt3_post_div", "gpt3_pre_div", base + 0xba00, 0, 6);
- clks[IMX7D_GPT4_ROOT_DIV] = imx_clk_divider2("gpt4_post_div", "gpt4_pre_div", base + 0xba80, 0, 6);
- clks[IMX7D_TRACE_ROOT_DIV] = imx_clk_divider2("trace_post_div", "trace_pre_div", base + 0xbb00, 0, 6);
- clks[IMX7D_WDOG_ROOT_DIV] = imx_clk_divider2("wdog_post_div", "wdog_pre_div", base + 0xbb80, 0, 6);
- clks[IMX7D_CSI_MCLK_ROOT_DIV] = imx_clk_divider2("csi_mclk_post_div", "csi_mclk_pre_div", base + 0xbc00, 0, 6);
- clks[IMX7D_AUDIO_MCLK_ROOT_DIV] = imx_clk_divider2("audio_mclk_post_div", "audio_mclk_pre_div", base + 0xbc80, 0, 6);
- clks[IMX7D_WRCLK_ROOT_DIV] = imx_clk_divider2("wrclk_post_div", "wrclk_pre_div", base + 0xbd00, 0, 6);
- clks[IMX7D_CLKO1_ROOT_DIV] = imx_clk_divider2("clko1_post_div", "clko1_pre_div", base + 0xbd80, 0, 6);
- clks[IMX7D_CLKO2_ROOT_DIV] = imx_clk_divider2("clko2_post_div", "clko2_pre_div", base + 0xbe00, 0, 6);
-
- clks[IMX7D_ARM_A7_ROOT_CLK] = imx_clk_gate2_flags("arm_a7_root_clk", "arm_a7_div", base + 0x4000, 0, CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_ARM_M4_ROOT_CLK] = imx_clk_gate4("arm_m4_root_clk", "arm_m4_div", base + 0x4010, 0);
- clks[IMX7D_MAIN_AXI_ROOT_CLK] = imx_clk_gate2_flags("main_axi_root_clk", "axi_post_div", base + 0x4040, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_DISP_AXI_ROOT_CLK] = imx_clk_gate4("disp_axi_root_clk", "disp_axi_post_div", base + 0x4050, 0);
- clks[IMX7D_ENET_AXI_ROOT_CLK] = imx_clk_gate4("enet_axi_root_clk", "enet_axi_post_div", base + 0x4060, 0);
- clks[IMX7D_OCRAM_CLK] = imx_clk_gate4("ocram_clk", "main_axi_root_clk", base + 0x4110, 0);
- clks[IMX7D_OCRAM_S_CLK] = imx_clk_gate4("ocram_s_clk", "ahb_root_clk", base + 0x4120, 0);
- clks[IMX7D_DRAM_ROOT_CLK] = imx_clk_gate2_flags("dram_root_clk", "dram_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_DRAM_PHYM_ROOT_CLK] = imx_clk_gate2_flags("dram_phym_root_clk", "dram_phym_cg", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_DRAM_PHYM_ALT_ROOT_CLK] = imx_clk_gate2_flags("dram_phym_alt_root_clk", "dram_phym_alt_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_DRAM_ALT_ROOT_CLK] = imx_clk_gate2_flags("dram_alt_root_clk", "dram_alt_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
- clks[IMX7D_OCOTP_CLK] = imx_clk_gate4("ocotp_clk", "ipg_root_clk", base + 0x4230, 0);
- clks[IMX7D_SNVS_CLK] = imx_clk_gate4("snvs_clk", "ipg_root_clk", base + 0x4250, 0);
- clks[IMX7D_MU_ROOT_CLK] = imx_clk_gate4("mu_root_clk", "ipg_root_clk", base + 0x4270, 0);
- clks[IMX7D_CAAM_CLK] = imx_clk_gate4("caam_clk", "ipg_root_clk", base + 0x4240, 0);
- clks[IMX7D_USB_HSIC_ROOT_CLK] = imx_clk_gate4("usb_hsic_root_clk", "usb_hsic_post_div", base + 0x4690, 0);
- clks[IMX7D_SDMA_CORE_CLK] = imx_clk_gate4("sdma_root_clk", "ahb_root_clk", base + 0x4480, 0);
- clks[IMX7D_PCIE_CTRL_ROOT_CLK] = imx_clk_gate4("pcie_ctrl_root_clk", "pcie_ctrl_post_div", base + 0x4600, 0);
- clks[IMX7D_PCIE_PHY_ROOT_CLK] = imx_clk_gate4("pcie_phy_root_clk", "pcie_phy_post_div", base + 0x4600, 0);
- clks[IMX7D_EPDC_PIXEL_ROOT_CLK] = imx_clk_gate4("epdc_pixel_root_clk", "epdc_pixel_post_div", base + 0x44a0, 0);
- clks[IMX7D_LCDIF_PIXEL_ROOT_CLK] = imx_clk_gate4("lcdif_pixel_root_clk", "lcdif_pixel_post_div", base + 0x44b0, 0);
- clks[IMX7D_MIPI_DSI_ROOT_CLK] = imx_clk_gate4("mipi_dsi_root_clk", "mipi_dsi_post_div", base + 0x4650, 0);
- clks[IMX7D_MIPI_CSI_ROOT_CLK] = imx_clk_gate4("mipi_csi_root_clk", "mipi_csi_post_div", base + 0x4640, 0);
- clks[IMX7D_MIPI_DPHY_ROOT_CLK] = imx_clk_gate4("mipi_dphy_root_clk", "mipi_dphy_post_div", base + 0x4660, 0);
- clks[IMX7D_ENET1_IPG_ROOT_CLK] = imx_clk_gate2_shared2("enet1_ipg_root_clk", "enet_axi_post_div", base + 0x4700, 0, &share_count_enet1);
- clks[IMX7D_ENET1_TIME_ROOT_CLK] = imx_clk_gate2_shared2("enet1_time_root_clk", "enet1_time_post_div", base + 0x4700, 0, &share_count_enet1);
- clks[IMX7D_ENET2_IPG_ROOT_CLK] = imx_clk_gate2_shared2("enet2_ipg_root_clk", "enet_axi_post_div", base + 0x4710, 0, &share_count_enet2);
- clks[IMX7D_ENET2_TIME_ROOT_CLK] = imx_clk_gate2_shared2("enet2_time_root_clk", "enet2_time_post_div", base + 0x4710, 0, &share_count_enet2);
- clks[IMX7D_SAI1_ROOT_CLK] = imx_clk_gate2_shared2("sai1_root_clk", "sai1_post_div", base + 0x48c0, 0, &share_count_sai1);
- clks[IMX7D_SAI1_IPG_CLK] = imx_clk_gate2_shared2("sai1_ipg_clk", "ipg_root_clk", base + 0x48c0, 0, &share_count_sai1);
- clks[IMX7D_SAI2_ROOT_CLK] = imx_clk_gate2_shared2("sai2_root_clk", "sai2_post_div", base + 0x48d0, 0, &share_count_sai2);
- clks[IMX7D_SAI2_IPG_CLK] = imx_clk_gate2_shared2("sai2_ipg_clk", "ipg_root_clk", base + 0x48d0, 0, &share_count_sai2);
- clks[IMX7D_SAI3_ROOT_CLK] = imx_clk_gate2_shared2("sai3_root_clk", "sai3_post_div", base + 0x48e0, 0, &share_count_sai3);
- clks[IMX7D_SAI3_IPG_CLK] = imx_clk_gate2_shared2("sai3_ipg_clk", "ipg_root_clk", base + 0x48e0, 0, &share_count_sai3);
- clks[IMX7D_SPDIF_ROOT_CLK] = imx_clk_gate4("spdif_root_clk", "spdif_post_div", base + 0x44d0, 0);
- clks[IMX7D_EIM_ROOT_CLK] = imx_clk_gate4("eim_root_clk", "eim_post_div", base + 0x4160, 0);
- clks[IMX7D_NAND_RAWNAND_CLK] = imx_clk_gate2_shared2("nand_rawnand_clk", "nand_root_clk", base + 0x4140, 0, &share_count_nand);
- clks[IMX7D_NAND_USDHC_BUS_RAWNAND_CLK] = imx_clk_gate2_shared2("nand_usdhc_rawnand_clk", "nand_usdhc_root_clk", base + 0x4140, 0, &share_count_nand);
- clks[IMX7D_QSPI_ROOT_CLK] = imx_clk_gate4("qspi_root_clk", "qspi_post_div", base + 0x4150, 0);
- clks[IMX7D_USDHC1_ROOT_CLK] = imx_clk_gate4("usdhc1_root_clk", "usdhc1_post_div", base + 0x46c0, 0);
- clks[IMX7D_USDHC2_ROOT_CLK] = imx_clk_gate4("usdhc2_root_clk", "usdhc2_post_div", base + 0x46d0, 0);
- clks[IMX7D_USDHC3_ROOT_CLK] = imx_clk_gate4("usdhc3_root_clk", "usdhc3_post_div", base + 0x46e0, 0);
- clks[IMX7D_CAN1_ROOT_CLK] = imx_clk_gate4("can1_root_clk", "can1_post_div", base + 0x4740, 0);
- clks[IMX7D_CAN2_ROOT_CLK] = imx_clk_gate4("can2_root_clk", "can2_post_div", base + 0x4750, 0);
- clks[IMX7D_I2C1_ROOT_CLK] = imx_clk_gate4("i2c1_root_clk", "i2c1_post_div", base + 0x4880, 0);
- clks[IMX7D_I2C2_ROOT_CLK] = imx_clk_gate4("i2c2_root_clk", "i2c2_post_div", base + 0x4890, 0);
- clks[IMX7D_I2C3_ROOT_CLK] = imx_clk_gate4("i2c3_root_clk", "i2c3_post_div", base + 0x48a0, 0);
- clks[IMX7D_I2C4_ROOT_CLK] = imx_clk_gate4("i2c4_root_clk", "i2c4_post_div", base + 0x48b0, 0);
- clks[IMX7D_UART1_ROOT_CLK] = imx_clk_gate4("uart1_root_clk", "uart1_post_div", base + 0x4940, 0);
- clks[IMX7D_UART2_ROOT_CLK] = imx_clk_gate4("uart2_root_clk", "uart2_post_div", base + 0x4950, 0);
- clks[IMX7D_UART3_ROOT_CLK] = imx_clk_gate4("uart3_root_clk", "uart3_post_div", base + 0x4960, 0);
- clks[IMX7D_UART4_ROOT_CLK] = imx_clk_gate4("uart4_root_clk", "uart4_post_div", base + 0x4970, 0);
- clks[IMX7D_UART5_ROOT_CLK] = imx_clk_gate4("uart5_root_clk", "uart5_post_div", base + 0x4980, 0);
- clks[IMX7D_UART6_ROOT_CLK] = imx_clk_gate4("uart6_root_clk", "uart6_post_div", base + 0x4990, 0);
- clks[IMX7D_UART7_ROOT_CLK] = imx_clk_gate4("uart7_root_clk", "uart7_post_div", base + 0x49a0, 0);
- clks[IMX7D_ECSPI1_ROOT_CLK] = imx_clk_gate4("ecspi1_root_clk", "ecspi1_post_div", base + 0x4780, 0);
- clks[IMX7D_ECSPI2_ROOT_CLK] = imx_clk_gate4("ecspi2_root_clk", "ecspi2_post_div", base + 0x4790, 0);
- clks[IMX7D_ECSPI3_ROOT_CLK] = imx_clk_gate4("ecspi3_root_clk", "ecspi3_post_div", base + 0x47a0, 0);
- clks[IMX7D_ECSPI4_ROOT_CLK] = imx_clk_gate4("ecspi4_root_clk", "ecspi4_post_div", base + 0x47b0, 0);
- clks[IMX7D_PWM1_ROOT_CLK] = imx_clk_gate4("pwm1_root_clk", "pwm1_post_div", base + 0x4840, 0);
- clks[IMX7D_PWM2_ROOT_CLK] = imx_clk_gate4("pwm2_root_clk", "pwm2_post_div", base + 0x4850, 0);
- clks[IMX7D_PWM3_ROOT_CLK] = imx_clk_gate4("pwm3_root_clk", "pwm3_post_div", base + 0x4860, 0);
- clks[IMX7D_PWM4_ROOT_CLK] = imx_clk_gate4("pwm4_root_clk", "pwm4_post_div", base + 0x4870, 0);
- clks[IMX7D_FLEXTIMER1_ROOT_CLK] = imx_clk_gate4("flextimer1_root_clk", "flextimer1_post_div", base + 0x4800, 0);
- clks[IMX7D_FLEXTIMER2_ROOT_CLK] = imx_clk_gate4("flextimer2_root_clk", "flextimer2_post_div", base + 0x4810, 0);
- clks[IMX7D_SIM1_ROOT_CLK] = imx_clk_gate4("sim1_root_clk", "sim1_post_div", base + 0x4900, 0);
- clks[IMX7D_SIM2_ROOT_CLK] = imx_clk_gate4("sim2_root_clk", "sim2_post_div", base + 0x4910, 0);
- clks[IMX7D_GPT1_ROOT_CLK] = imx_clk_gate4("gpt1_root_clk", "gpt1_post_div", base + 0x47c0, 0);
- clks[IMX7D_GPT2_ROOT_CLK] = imx_clk_gate4("gpt2_root_clk", "gpt2_post_div", base + 0x47d0, 0);
- clks[IMX7D_GPT3_ROOT_CLK] = imx_clk_gate4("gpt3_root_clk", "gpt3_post_div", base + 0x47e0, 0);
- clks[IMX7D_GPT4_ROOT_CLK] = imx_clk_gate4("gpt4_root_clk", "gpt4_post_div", base + 0x47f0, 0);
- clks[IMX7D_TRACE_ROOT_CLK] = imx_clk_gate4("trace_root_clk", "trace_post_div", base + 0x4300, 0);
- clks[IMX7D_WDOG1_ROOT_CLK] = imx_clk_gate4("wdog1_root_clk", "wdog_post_div", base + 0x49c0, 0);
- clks[IMX7D_WDOG2_ROOT_CLK] = imx_clk_gate4("wdog2_root_clk", "wdog_post_div", base + 0x49d0, 0);
- clks[IMX7D_WDOG3_ROOT_CLK] = imx_clk_gate4("wdog3_root_clk", "wdog_post_div", base + 0x49e0, 0);
- clks[IMX7D_WDOG4_ROOT_CLK] = imx_clk_gate4("wdog4_root_clk", "wdog_post_div", base + 0x49f0, 0);
- clks[IMX7D_KPP_ROOT_CLK] = imx_clk_gate4("kpp_root_clk", "ipg_root_clk", base + 0x4aa0, 0);
- clks[IMX7D_CSI_MCLK_ROOT_CLK] = imx_clk_gate4("csi_mclk_root_clk", "csi_mclk_post_div", base + 0x4490, 0);
- clks[IMX7D_AUDIO_MCLK_ROOT_CLK] = imx_clk_gate4("audio_mclk_root_clk", "audio_mclk_post_div", base + 0x4790, 0);
- clks[IMX7D_WRCLK_ROOT_CLK] = imx_clk_gate4("wrclk_root_clk", "wrclk_post_div", base + 0x47a0, 0);
- clks[IMX7D_USB_CTRL_CLK] = imx_clk_gate4("usb_ctrl_clk", "ahb_root_clk", base + 0x4680, 0);
- clks[IMX7D_USB_PHY1_CLK] = imx_clk_gate4("usb_phy1_clk", "pll_usb1_main_clk", base + 0x46a0, 0);
- clks[IMX7D_USB_PHY2_CLK] = imx_clk_gate4("usb_phy2_clk", "pll_usb_main_clk", base + 0x46b0, 0);
- clks[IMX7D_ADC_ROOT_CLK] = imx_clk_gate4("adc_root_clk", "ipg_root_clk", base + 0x4200, 0);
-
- clks[IMX7D_GPT_3M_CLK] = imx_clk_fixed_factor("gpt_3m", "osc", 1, 8);
-
- clks[IMX7D_CLK_ARM] = imx_clk_cpu("arm", "arm_a7_root_clk",
- clks[IMX7D_ARM_A7_ROOT_CLK],
- clks[IMX7D_ARM_A7_ROOT_SRC],
- clks[IMX7D_PLL_ARM_MAIN_CLK],
- clks[IMX7D_PLL_SYS_MAIN_CLK]);
-
- imx_check_clocks(clks, ARRAY_SIZE(clks));
-
- clk_data.clks = clks;
- clk_data.clk_num = ARRAY_SIZE(clks);
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
-
- clk_set_parent(clks[IMX7D_PLL_ARM_MAIN_BYPASS], clks[IMX7D_PLL_ARM_MAIN]);
- clk_set_parent(clks[IMX7D_PLL_DRAM_MAIN_BYPASS], clks[IMX7D_PLL_DRAM_MAIN]);
- clk_set_parent(clks[IMX7D_PLL_SYS_MAIN_BYPASS], clks[IMX7D_PLL_SYS_MAIN]);
- clk_set_parent(clks[IMX7D_PLL_ENET_MAIN_BYPASS], clks[IMX7D_PLL_ENET_MAIN]);
- clk_set_parent(clks[IMX7D_PLL_AUDIO_MAIN_BYPASS], clks[IMX7D_PLL_AUDIO_MAIN]);
- clk_set_parent(clks[IMX7D_PLL_VIDEO_MAIN_BYPASS], clks[IMX7D_PLL_VIDEO_MAIN]);
-
- clk_set_parent(clks[IMX7D_MIPI_CSI_ROOT_SRC], clks[IMX7D_PLL_SYS_PFD3_CLK]);
+ hws[IMX7D_ARM_A7_ROOT_SRC] = imx_clk_hw_mux2("arm_a7_src", base + 0x8000, 24, 3, arm_a7_sel, ARRAY_SIZE(arm_a7_sel));
+ hws[IMX7D_ARM_M4_ROOT_SRC] = imx_clk_hw_mux2("arm_m4_src", base + 0x8080, 24, 3, arm_m4_sel, ARRAY_SIZE(arm_m4_sel));
+ hws[IMX7D_MAIN_AXI_ROOT_SRC] = imx_clk_hw_mux2("axi_src", base + 0x8800, 24, 3, axi_sel, ARRAY_SIZE(axi_sel));
+ hws[IMX7D_DISP_AXI_ROOT_SRC] = imx_clk_hw_mux2("disp_axi_src", base + 0x8880, 24, 3, disp_axi_sel, ARRAY_SIZE(disp_axi_sel));
+ hws[IMX7D_ENET_AXI_ROOT_SRC] = imx_clk_hw_mux2("enet_axi_src", base + 0x8900, 24, 3, enet_axi_sel, ARRAY_SIZE(enet_axi_sel));
+ hws[IMX7D_NAND_USDHC_BUS_ROOT_SRC] = imx_clk_hw_mux2("nand_usdhc_src", base + 0x8980, 24, 3, nand_usdhc_bus_sel, ARRAY_SIZE(nand_usdhc_bus_sel));
+ hws[IMX7D_AHB_CHANNEL_ROOT_SRC] = imx_clk_hw_mux2("ahb_src", base + 0x9000, 24, 3, ahb_channel_sel, ARRAY_SIZE(ahb_channel_sel));
+ hws[IMX7D_DRAM_PHYM_ROOT_SRC] = imx_clk_hw_mux2("dram_phym_src", base + 0x9800, 24, 1, dram_phym_sel, ARRAY_SIZE(dram_phym_sel));
+ hws[IMX7D_DRAM_ROOT_SRC] = imx_clk_hw_mux2("dram_src", base + 0x9880, 24, 1, dram_sel, ARRAY_SIZE(dram_sel));
+ hws[IMX7D_DRAM_PHYM_ALT_ROOT_SRC] = imx_clk_hw_mux2("dram_phym_alt_src", base + 0xa000, 24, 3, dram_phym_alt_sel, ARRAY_SIZE(dram_phym_alt_sel));
+ hws[IMX7D_DRAM_ALT_ROOT_SRC] = imx_clk_hw_mux2("dram_alt_src", base + 0xa080, 24, 3, dram_alt_sel, ARRAY_SIZE(dram_alt_sel));
+ hws[IMX7D_USB_HSIC_ROOT_SRC] = imx_clk_hw_mux2("usb_hsic_src", base + 0xa100, 24, 3, usb_hsic_sel, ARRAY_SIZE(usb_hsic_sel));
+ hws[IMX7D_PCIE_CTRL_ROOT_SRC] = imx_clk_hw_mux2("pcie_ctrl_src", base + 0xa180, 24, 3, pcie_ctrl_sel, ARRAY_SIZE(pcie_ctrl_sel));
+ hws[IMX7D_PCIE_PHY_ROOT_SRC] = imx_clk_hw_mux2("pcie_phy_src", base + 0xa200, 24, 3, pcie_phy_sel, ARRAY_SIZE(pcie_phy_sel));
+ hws[IMX7D_EPDC_PIXEL_ROOT_SRC] = imx_clk_hw_mux2("epdc_pixel_src", base + 0xa280, 24, 3, epdc_pixel_sel, ARRAY_SIZE(epdc_pixel_sel));
+ hws[IMX7D_LCDIF_PIXEL_ROOT_SRC] = imx_clk_hw_mux2("lcdif_pixel_src", base + 0xa300, 24, 3, lcdif_pixel_sel, ARRAY_SIZE(lcdif_pixel_sel));
+ hws[IMX7D_MIPI_DSI_ROOT_SRC] = imx_clk_hw_mux2("mipi_dsi_src", base + 0xa380, 24, 3, mipi_dsi_sel, ARRAY_SIZE(mipi_dsi_sel));
+ hws[IMX7D_MIPI_CSI_ROOT_SRC] = imx_clk_hw_mux2("mipi_csi_src", base + 0xa400, 24, 3, mipi_csi_sel, ARRAY_SIZE(mipi_csi_sel));
+ hws[IMX7D_MIPI_DPHY_ROOT_SRC] = imx_clk_hw_mux2("mipi_dphy_src", base + 0xa480, 24, 3, mipi_dphy_sel, ARRAY_SIZE(mipi_dphy_sel));
+ hws[IMX7D_SAI1_ROOT_SRC] = imx_clk_hw_mux2("sai1_src", base + 0xa500, 24, 3, sai1_sel, ARRAY_SIZE(sai1_sel));
+ hws[IMX7D_SAI2_ROOT_SRC] = imx_clk_hw_mux2("sai2_src", base + 0xa580, 24, 3, sai2_sel, ARRAY_SIZE(sai2_sel));
+ hws[IMX7D_SAI3_ROOT_SRC] = imx_clk_hw_mux2("sai3_src", base + 0xa600, 24, 3, sai3_sel, ARRAY_SIZE(sai3_sel));
+ hws[IMX7D_SPDIF_ROOT_SRC] = imx_clk_hw_mux2("spdif_src", base + 0xa680, 24, 3, spdif_sel, ARRAY_SIZE(spdif_sel));
+ hws[IMX7D_ENET1_REF_ROOT_SRC] = imx_clk_hw_mux2("enet1_ref_src", base + 0xa700, 24, 3, enet1_ref_sel, ARRAY_SIZE(enet1_ref_sel));
+ hws[IMX7D_ENET1_TIME_ROOT_SRC] = imx_clk_hw_mux2("enet1_time_src", base + 0xa780, 24, 3, enet1_time_sel, ARRAY_SIZE(enet1_time_sel));
+ hws[IMX7D_ENET2_REF_ROOT_SRC] = imx_clk_hw_mux2("enet2_ref_src", base + 0xa800, 24, 3, enet2_ref_sel, ARRAY_SIZE(enet2_ref_sel));
+ hws[IMX7D_ENET2_TIME_ROOT_SRC] = imx_clk_hw_mux2("enet2_time_src", base + 0xa880, 24, 3, enet2_time_sel, ARRAY_SIZE(enet2_time_sel));
+ hws[IMX7D_ENET_PHY_REF_ROOT_SRC] = imx_clk_hw_mux2("enet_phy_ref_src", base + 0xa900, 24, 3, enet_phy_ref_sel, ARRAY_SIZE(enet_phy_ref_sel));
+ hws[IMX7D_EIM_ROOT_SRC] = imx_clk_hw_mux2("eim_src", base + 0xa980, 24, 3, eim_sel, ARRAY_SIZE(eim_sel));
+ hws[IMX7D_NAND_ROOT_SRC] = imx_clk_hw_mux2("nand_src", base + 0xaa00, 24, 3, nand_sel, ARRAY_SIZE(nand_sel));
+ hws[IMX7D_QSPI_ROOT_SRC] = imx_clk_hw_mux2("qspi_src", base + 0xaa80, 24, 3, qspi_sel, ARRAY_SIZE(qspi_sel));
+ hws[IMX7D_USDHC1_ROOT_SRC] = imx_clk_hw_mux2("usdhc1_src", base + 0xab00, 24, 3, usdhc1_sel, ARRAY_SIZE(usdhc1_sel));
+ hws[IMX7D_USDHC2_ROOT_SRC] = imx_clk_hw_mux2("usdhc2_src", base + 0xab80, 24, 3, usdhc2_sel, ARRAY_SIZE(usdhc2_sel));
+ hws[IMX7D_USDHC3_ROOT_SRC] = imx_clk_hw_mux2("usdhc3_src", base + 0xac00, 24, 3, usdhc3_sel, ARRAY_SIZE(usdhc3_sel));
+ hws[IMX7D_CAN1_ROOT_SRC] = imx_clk_hw_mux2("can1_src", base + 0xac80, 24, 3, can1_sel, ARRAY_SIZE(can1_sel));
+ hws[IMX7D_CAN2_ROOT_SRC] = imx_clk_hw_mux2("can2_src", base + 0xad00, 24, 3, can2_sel, ARRAY_SIZE(can2_sel));
+ hws[IMX7D_I2C1_ROOT_SRC] = imx_clk_hw_mux2("i2c1_src", base + 0xad80, 24, 3, i2c1_sel, ARRAY_SIZE(i2c1_sel));
+ hws[IMX7D_I2C2_ROOT_SRC] = imx_clk_hw_mux2("i2c2_src", base + 0xae00, 24, 3, i2c2_sel, ARRAY_SIZE(i2c2_sel));
+ hws[IMX7D_I2C3_ROOT_SRC] = imx_clk_hw_mux2("i2c3_src", base + 0xae80, 24, 3, i2c3_sel, ARRAY_SIZE(i2c3_sel));
+ hws[IMX7D_I2C4_ROOT_SRC] = imx_clk_hw_mux2("i2c4_src", base + 0xaf00, 24, 3, i2c4_sel, ARRAY_SIZE(i2c4_sel));
+ hws[IMX7D_UART1_ROOT_SRC] = imx_clk_hw_mux2("uart1_src", base + 0xaf80, 24, 3, uart1_sel, ARRAY_SIZE(uart1_sel));
+ hws[IMX7D_UART2_ROOT_SRC] = imx_clk_hw_mux2("uart2_src", base + 0xb000, 24, 3, uart2_sel, ARRAY_SIZE(uart2_sel));
+ hws[IMX7D_UART3_ROOT_SRC] = imx_clk_hw_mux2("uart3_src", base + 0xb080, 24, 3, uart3_sel, ARRAY_SIZE(uart3_sel));
+ hws[IMX7D_UART4_ROOT_SRC] = imx_clk_hw_mux2("uart4_src", base + 0xb100, 24, 3, uart4_sel, ARRAY_SIZE(uart4_sel));
+ hws[IMX7D_UART5_ROOT_SRC] = imx_clk_hw_mux2("uart5_src", base + 0xb180, 24, 3, uart5_sel, ARRAY_SIZE(uart5_sel));
+ hws[IMX7D_UART6_ROOT_SRC] = imx_clk_hw_mux2("uart6_src", base + 0xb200, 24, 3, uart6_sel, ARRAY_SIZE(uart6_sel));
+ hws[IMX7D_UART7_ROOT_SRC] = imx_clk_hw_mux2("uart7_src", base + 0xb280, 24, 3, uart7_sel, ARRAY_SIZE(uart7_sel));
+ hws[IMX7D_ECSPI1_ROOT_SRC] = imx_clk_hw_mux2("ecspi1_src", base + 0xb300, 24, 3, ecspi1_sel, ARRAY_SIZE(ecspi1_sel));
+ hws[IMX7D_ECSPI2_ROOT_SRC] = imx_clk_hw_mux2("ecspi2_src", base + 0xb380, 24, 3, ecspi2_sel, ARRAY_SIZE(ecspi2_sel));
+ hws[IMX7D_ECSPI3_ROOT_SRC] = imx_clk_hw_mux2("ecspi3_src", base + 0xb400, 24, 3, ecspi3_sel, ARRAY_SIZE(ecspi3_sel));
+ hws[IMX7D_ECSPI4_ROOT_SRC] = imx_clk_hw_mux2("ecspi4_src", base + 0xb480, 24, 3, ecspi4_sel, ARRAY_SIZE(ecspi4_sel));
+ hws[IMX7D_PWM1_ROOT_SRC] = imx_clk_hw_mux2("pwm1_src", base + 0xb500, 24, 3, pwm1_sel, ARRAY_SIZE(pwm1_sel));
+ hws[IMX7D_PWM2_ROOT_SRC] = imx_clk_hw_mux2("pwm2_src", base + 0xb580, 24, 3, pwm2_sel, ARRAY_SIZE(pwm2_sel));
+ hws[IMX7D_PWM3_ROOT_SRC] = imx_clk_hw_mux2("pwm3_src", base + 0xb600, 24, 3, pwm3_sel, ARRAY_SIZE(pwm3_sel));
+ hws[IMX7D_PWM4_ROOT_SRC] = imx_clk_hw_mux2("pwm4_src", base + 0xb680, 24, 3, pwm4_sel, ARRAY_SIZE(pwm4_sel));
+ hws[IMX7D_FLEXTIMER1_ROOT_SRC] = imx_clk_hw_mux2("flextimer1_src", base + 0xb700, 24, 3, flextimer1_sel, ARRAY_SIZE(flextimer1_sel));
+ hws[IMX7D_FLEXTIMER2_ROOT_SRC] = imx_clk_hw_mux2("flextimer2_src", base + 0xb780, 24, 3, flextimer2_sel, ARRAY_SIZE(flextimer2_sel));
+ hws[IMX7D_SIM1_ROOT_SRC] = imx_clk_hw_mux2("sim1_src", base + 0xb800, 24, 3, sim1_sel, ARRAY_SIZE(sim1_sel));
+ hws[IMX7D_SIM2_ROOT_SRC] = imx_clk_hw_mux2("sim2_src", base + 0xb880, 24, 3, sim2_sel, ARRAY_SIZE(sim2_sel));
+ hws[IMX7D_GPT1_ROOT_SRC] = imx_clk_hw_mux2("gpt1_src", base + 0xb900, 24, 3, gpt1_sel, ARRAY_SIZE(gpt1_sel));
+ hws[IMX7D_GPT2_ROOT_SRC] = imx_clk_hw_mux2("gpt2_src", base + 0xb980, 24, 3, gpt2_sel, ARRAY_SIZE(gpt2_sel));
+ hws[IMX7D_GPT3_ROOT_SRC] = imx_clk_hw_mux2("gpt3_src", base + 0xba00, 24, 3, gpt3_sel, ARRAY_SIZE(gpt3_sel));
+ hws[IMX7D_GPT4_ROOT_SRC] = imx_clk_hw_mux2("gpt4_src", base + 0xba80, 24, 3, gpt4_sel, ARRAY_SIZE(gpt4_sel));
+ hws[IMX7D_TRACE_ROOT_SRC] = imx_clk_hw_mux2("trace_src", base + 0xbb00, 24, 3, trace_sel, ARRAY_SIZE(trace_sel));
+ hws[IMX7D_WDOG_ROOT_SRC] = imx_clk_hw_mux2("wdog_src", base + 0xbb80, 24, 3, wdog_sel, ARRAY_SIZE(wdog_sel));
+ hws[IMX7D_CSI_MCLK_ROOT_SRC] = imx_clk_hw_mux2("csi_mclk_src", base + 0xbc00, 24, 3, csi_mclk_sel, ARRAY_SIZE(csi_mclk_sel));
+ hws[IMX7D_AUDIO_MCLK_ROOT_SRC] = imx_clk_hw_mux2("audio_mclk_src", base + 0xbc80, 24, 3, audio_mclk_sel, ARRAY_SIZE(audio_mclk_sel));
+ hws[IMX7D_WRCLK_ROOT_SRC] = imx_clk_hw_mux2("wrclk_src", base + 0xbd00, 24, 3, wrclk_sel, ARRAY_SIZE(wrclk_sel));
+ hws[IMX7D_CLKO1_ROOT_SRC] = imx_clk_hw_mux2("clko1_src", base + 0xbd80, 24, 3, clko1_sel, ARRAY_SIZE(clko1_sel));
+ hws[IMX7D_CLKO2_ROOT_SRC] = imx_clk_hw_mux2("clko2_src", base + 0xbe00, 24, 3, clko2_sel, ARRAY_SIZE(clko2_sel));
+
+ hws[IMX7D_ARM_A7_ROOT_CG] = imx_clk_hw_gate3("arm_a7_cg", "arm_a7_src", base + 0x8000, 28);
+ hws[IMX7D_ARM_M4_ROOT_CG] = imx_clk_hw_gate3("arm_m4_cg", "arm_m4_src", base + 0x8080, 28);
+ hws[IMX7D_MAIN_AXI_ROOT_CG] = imx_clk_hw_gate3("axi_cg", "axi_src", base + 0x8800, 28);
+ hws[IMX7D_DISP_AXI_ROOT_CG] = imx_clk_hw_gate3("disp_axi_cg", "disp_axi_src", base + 0x8880, 28);
+ hws[IMX7D_ENET_AXI_ROOT_CG] = imx_clk_hw_gate3("enet_axi_cg", "enet_axi_src", base + 0x8900, 28);
+ hws[IMX7D_NAND_USDHC_BUS_ROOT_CG] = imx_clk_hw_gate3("nand_usdhc_cg", "nand_usdhc_src", base + 0x8980, 28);
+ hws[IMX7D_AHB_CHANNEL_ROOT_CG] = imx_clk_hw_gate3("ahb_cg", "ahb_src", base + 0x9000, 28);
+ hws[IMX7D_DRAM_PHYM_ROOT_CG] = imx_clk_hw_gate3("dram_phym_cg", "dram_phym_src", base + 0x9800, 28);
+ hws[IMX7D_DRAM_ROOT_CG] = imx_clk_hw_gate3("dram_cg", "dram_src", base + 0x9880, 28);
+ hws[IMX7D_DRAM_PHYM_ALT_ROOT_CG] = imx_clk_hw_gate3("dram_phym_alt_cg", "dram_phym_alt_src", base + 0xa000, 28);
+ hws[IMX7D_DRAM_ALT_ROOT_CG] = imx_clk_hw_gate3("dram_alt_cg", "dram_alt_src", base + 0xa080, 28);
+ hws[IMX7D_USB_HSIC_ROOT_CG] = imx_clk_hw_gate3("usb_hsic_cg", "usb_hsic_src", base + 0xa100, 28);
+ hws[IMX7D_PCIE_CTRL_ROOT_CG] = imx_clk_hw_gate3("pcie_ctrl_cg", "pcie_ctrl_src", base + 0xa180, 28);
+ hws[IMX7D_PCIE_PHY_ROOT_CG] = imx_clk_hw_gate3("pcie_phy_cg", "pcie_phy_src", base + 0xa200, 28);
+ hws[IMX7D_EPDC_PIXEL_ROOT_CG] = imx_clk_hw_gate3("epdc_pixel_cg", "epdc_pixel_src", base + 0xa280, 28);
+ hws[IMX7D_LCDIF_PIXEL_ROOT_CG] = imx_clk_hw_gate3("lcdif_pixel_cg", "lcdif_pixel_src", base + 0xa300, 28);
+ hws[IMX7D_MIPI_DSI_ROOT_CG] = imx_clk_hw_gate3("mipi_dsi_cg", "mipi_dsi_src", base + 0xa380, 28);
+ hws[IMX7D_MIPI_CSI_ROOT_CG] = imx_clk_hw_gate3("mipi_csi_cg", "mipi_csi_src", base + 0xa400, 28);
+ hws[IMX7D_MIPI_DPHY_ROOT_CG] = imx_clk_hw_gate3("mipi_dphy_cg", "mipi_dphy_src", base + 0xa480, 28);
+ hws[IMX7D_SAI1_ROOT_CG] = imx_clk_hw_gate3("sai1_cg", "sai1_src", base + 0xa500, 28);
+ hws[IMX7D_SAI2_ROOT_CG] = imx_clk_hw_gate3("sai2_cg", "sai2_src", base + 0xa580, 28);
+ hws[IMX7D_SAI3_ROOT_CG] = imx_clk_hw_gate3("sai3_cg", "sai3_src", base + 0xa600, 28);
+ hws[IMX7D_SPDIF_ROOT_CG] = imx_clk_hw_gate3("spdif_cg", "spdif_src", base + 0xa680, 28);
+ hws[IMX7D_ENET1_REF_ROOT_CG] = imx_clk_hw_gate3("enet1_ref_cg", "enet1_ref_src", base + 0xa700, 28);
+ hws[IMX7D_ENET1_TIME_ROOT_CG] = imx_clk_hw_gate3("enet1_time_cg", "enet1_time_src", base + 0xa780, 28);
+ hws[IMX7D_ENET2_REF_ROOT_CG] = imx_clk_hw_gate3("enet2_ref_cg", "enet2_ref_src", base + 0xa800, 28);
+ hws[IMX7D_ENET2_TIME_ROOT_CG] = imx_clk_hw_gate3("enet2_time_cg", "enet2_time_src", base + 0xa880, 28);
+ hws[IMX7D_ENET_PHY_REF_ROOT_CG] = imx_clk_hw_gate3("enet_phy_ref_cg", "enet_phy_ref_src", base + 0xa900, 28);
+ hws[IMX7D_EIM_ROOT_CG] = imx_clk_hw_gate3("eim_cg", "eim_src", base + 0xa980, 28);
+ hws[IMX7D_NAND_ROOT_CG] = imx_clk_hw_gate3("nand_cg", "nand_src", base + 0xaa00, 28);
+ hws[IMX7D_QSPI_ROOT_CG] = imx_clk_hw_gate3("qspi_cg", "qspi_src", base + 0xaa80, 28);
+ hws[IMX7D_USDHC1_ROOT_CG] = imx_clk_hw_gate3("usdhc1_cg", "usdhc1_src", base + 0xab00, 28);
+ hws[IMX7D_USDHC2_ROOT_CG] = imx_clk_hw_gate3("usdhc2_cg", "usdhc2_src", base + 0xab80, 28);
+ hws[IMX7D_USDHC3_ROOT_CG] = imx_clk_hw_gate3("usdhc3_cg", "usdhc3_src", base + 0xac00, 28);
+ hws[IMX7D_CAN1_ROOT_CG] = imx_clk_hw_gate3("can1_cg", "can1_src", base + 0xac80, 28);
+ hws[IMX7D_CAN2_ROOT_CG] = imx_clk_hw_gate3("can2_cg", "can2_src", base + 0xad00, 28);
+ hws[IMX7D_I2C1_ROOT_CG] = imx_clk_hw_gate3("i2c1_cg", "i2c1_src", base + 0xad80, 28);
+ hws[IMX7D_I2C2_ROOT_CG] = imx_clk_hw_gate3("i2c2_cg", "i2c2_src", base + 0xae00, 28);
+ hws[IMX7D_I2C3_ROOT_CG] = imx_clk_hw_gate3("i2c3_cg", "i2c3_src", base + 0xae80, 28);
+ hws[IMX7D_I2C4_ROOT_CG] = imx_clk_hw_gate3("i2c4_cg", "i2c4_src", base + 0xaf00, 28);
+ hws[IMX7D_UART1_ROOT_CG] = imx_clk_hw_gate3("uart1_cg", "uart1_src", base + 0xaf80, 28);
+ hws[IMX7D_UART2_ROOT_CG] = imx_clk_hw_gate3("uart2_cg", "uart2_src", base + 0xb000, 28);
+ hws[IMX7D_UART3_ROOT_CG] = imx_clk_hw_gate3("uart3_cg", "uart3_src", base + 0xb080, 28);
+ hws[IMX7D_UART4_ROOT_CG] = imx_clk_hw_gate3("uart4_cg", "uart4_src", base + 0xb100, 28);
+ hws[IMX7D_UART5_ROOT_CG] = imx_clk_hw_gate3("uart5_cg", "uart5_src", base + 0xb180, 28);
+ hws[IMX7D_UART6_ROOT_CG] = imx_clk_hw_gate3("uart6_cg", "uart6_src", base + 0xb200, 28);
+ hws[IMX7D_UART7_ROOT_CG] = imx_clk_hw_gate3("uart7_cg", "uart7_src", base + 0xb280, 28);
+ hws[IMX7D_ECSPI1_ROOT_CG] = imx_clk_hw_gate3("ecspi1_cg", "ecspi1_src", base + 0xb300, 28);
+ hws[IMX7D_ECSPI2_ROOT_CG] = imx_clk_hw_gate3("ecspi2_cg", "ecspi2_src", base + 0xb380, 28);
+ hws[IMX7D_ECSPI3_ROOT_CG] = imx_clk_hw_gate3("ecspi3_cg", "ecspi3_src", base + 0xb400, 28);
+ hws[IMX7D_ECSPI4_ROOT_CG] = imx_clk_hw_gate3("ecspi4_cg", "ecspi4_src", base + 0xb480, 28);
+ hws[IMX7D_PWM1_ROOT_CG] = imx_clk_hw_gate3("pwm1_cg", "pwm1_src", base + 0xb500, 28);
+ hws[IMX7D_PWM2_ROOT_CG] = imx_clk_hw_gate3("pwm2_cg", "pwm2_src", base + 0xb580, 28);
+ hws[IMX7D_PWM3_ROOT_CG] = imx_clk_hw_gate3("pwm3_cg", "pwm3_src", base + 0xb600, 28);
+ hws[IMX7D_PWM4_ROOT_CG] = imx_clk_hw_gate3("pwm4_cg", "pwm4_src", base + 0xb680, 28);
+ hws[IMX7D_FLEXTIMER1_ROOT_CG] = imx_clk_hw_gate3("flextimer1_cg", "flextimer1_src", base + 0xb700, 28);
+ hws[IMX7D_FLEXTIMER2_ROOT_CG] = imx_clk_hw_gate3("flextimer2_cg", "flextimer2_src", base + 0xb780, 28);
+ hws[IMX7D_SIM1_ROOT_CG] = imx_clk_hw_gate3("sim1_cg", "sim1_src", base + 0xb800, 28);
+ hws[IMX7D_SIM2_ROOT_CG] = imx_clk_hw_gate3("sim2_cg", "sim2_src", base + 0xb880, 28);
+ hws[IMX7D_GPT1_ROOT_CG] = imx_clk_hw_gate3("gpt1_cg", "gpt1_src", base + 0xb900, 28);
+ hws[IMX7D_GPT2_ROOT_CG] = imx_clk_hw_gate3("gpt2_cg", "gpt2_src", base + 0xb980, 28);
+ hws[IMX7D_GPT3_ROOT_CG] = imx_clk_hw_gate3("gpt3_cg", "gpt3_src", base + 0xbA00, 28);
+ hws[IMX7D_GPT4_ROOT_CG] = imx_clk_hw_gate3("gpt4_cg", "gpt4_src", base + 0xbA80, 28);
+ hws[IMX7D_TRACE_ROOT_CG] = imx_clk_hw_gate3("trace_cg", "trace_src", base + 0xbb00, 28);
+ hws[IMX7D_WDOG_ROOT_CG] = imx_clk_hw_gate3("wdog_cg", "wdog_src", base + 0xbb80, 28);
+ hws[IMX7D_CSI_MCLK_ROOT_CG] = imx_clk_hw_gate3("csi_mclk_cg", "csi_mclk_src", base + 0xbc00, 28);
+ hws[IMX7D_AUDIO_MCLK_ROOT_CG] = imx_clk_hw_gate3("audio_mclk_cg", "audio_mclk_src", base + 0xbc80, 28);
+ hws[IMX7D_WRCLK_ROOT_CG] = imx_clk_hw_gate3("wrclk_cg", "wrclk_src", base + 0xbd00, 28);
+ hws[IMX7D_CLKO1_ROOT_CG] = imx_clk_hw_gate3("clko1_cg", "clko1_src", base + 0xbd80, 28);
+ hws[IMX7D_CLKO2_ROOT_CG] = imx_clk_hw_gate3("clko2_cg", "clko2_src", base + 0xbe00, 28);
+
+ hws[IMX7D_MAIN_AXI_ROOT_PRE_DIV] = imx_clk_hw_divider2("axi_pre_div", "axi_cg", base + 0x8800, 16, 3);
+ hws[IMX7D_DISP_AXI_ROOT_PRE_DIV] = imx_clk_hw_divider2("disp_axi_pre_div", "disp_axi_cg", base + 0x8880, 16, 3);
+ hws[IMX7D_ENET_AXI_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet_axi_pre_div", "enet_axi_cg", base + 0x8900, 16, 3);
+ hws[IMX7D_NAND_USDHC_BUS_ROOT_PRE_DIV] = imx_clk_hw_divider2("nand_usdhc_pre_div", "nand_usdhc_cg", base + 0x8980, 16, 3);
+ hws[IMX7D_AHB_CHANNEL_ROOT_PRE_DIV] = imx_clk_hw_divider2("ahb_pre_div", "ahb_cg", base + 0x9000, 16, 3);
+ hws[IMX7D_DRAM_PHYM_ALT_ROOT_PRE_DIV] = imx_clk_hw_divider2("dram_phym_alt_pre_div", "dram_phym_alt_cg", base + 0xa000, 16, 3);
+ hws[IMX7D_DRAM_ALT_ROOT_PRE_DIV] = imx_clk_hw_divider2("dram_alt_pre_div", "dram_alt_cg", base + 0xa080, 16, 3);
+ hws[IMX7D_USB_HSIC_ROOT_PRE_DIV] = imx_clk_hw_divider2("usb_hsic_pre_div", "usb_hsic_cg", base + 0xa100, 16, 3);
+ hws[IMX7D_PCIE_CTRL_ROOT_PRE_DIV] = imx_clk_hw_divider2("pcie_ctrl_pre_div", "pcie_ctrl_cg", base + 0xa180, 16, 3);
+ hws[IMX7D_PCIE_PHY_ROOT_PRE_DIV] = imx_clk_hw_divider2("pcie_phy_pre_div", "pcie_phy_cg", base + 0xa200, 16, 3);
+ hws[IMX7D_EPDC_PIXEL_ROOT_PRE_DIV] = imx_clk_hw_divider2("epdc_pixel_pre_div", "epdc_pixel_cg", base + 0xa280, 16, 3);
+ hws[IMX7D_LCDIF_PIXEL_ROOT_PRE_DIV] = imx_clk_hw_divider2("lcdif_pixel_pre_div", "lcdif_pixel_cg", base + 0xa300, 16, 3);
+ hws[IMX7D_MIPI_DSI_ROOT_PRE_DIV] = imx_clk_hw_divider2("mipi_dsi_pre_div", "mipi_dsi_cg", base + 0xa380, 16, 3);
+ hws[IMX7D_MIPI_CSI_ROOT_PRE_DIV] = imx_clk_hw_divider2("mipi_csi_pre_div", "mipi_csi_cg", base + 0xa400, 16, 3);
+ hws[IMX7D_MIPI_DPHY_ROOT_PRE_DIV] = imx_clk_hw_divider2("mipi_dphy_pre_div", "mipi_dphy_cg", base + 0xa480, 16, 3);
+ hws[IMX7D_SAI1_ROOT_PRE_DIV] = imx_clk_hw_divider2("sai1_pre_div", "sai1_cg", base + 0xa500, 16, 3);
+ hws[IMX7D_SAI2_ROOT_PRE_DIV] = imx_clk_hw_divider2("sai2_pre_div", "sai2_cg", base + 0xa580, 16, 3);
+ hws[IMX7D_SAI3_ROOT_PRE_DIV] = imx_clk_hw_divider2("sai3_pre_div", "sai3_cg", base + 0xa600, 16, 3);
+ hws[IMX7D_SPDIF_ROOT_PRE_DIV] = imx_clk_hw_divider2("spdif_pre_div", "spdif_cg", base + 0xa680, 16, 3);
+ hws[IMX7D_ENET1_REF_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet1_ref_pre_div", "enet1_ref_cg", base + 0xa700, 16, 3);
+ hws[IMX7D_ENET1_TIME_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet1_time_pre_div", "enet1_time_cg", base + 0xa780, 16, 3);
+ hws[IMX7D_ENET2_REF_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet2_ref_pre_div", "enet2_ref_cg", base + 0xa800, 16, 3);
+ hws[IMX7D_ENET2_TIME_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet2_time_pre_div", "enet2_time_cg", base + 0xa880, 16, 3);
+ hws[IMX7D_ENET_PHY_REF_ROOT_PRE_DIV] = imx_clk_hw_divider2("enet_phy_ref_pre_div", "enet_phy_ref_cg", base + 0xa900, 16, 3);
+ hws[IMX7D_EIM_ROOT_PRE_DIV] = imx_clk_hw_divider2("eim_pre_div", "eim_cg", base + 0xa980, 16, 3);
+ hws[IMX7D_NAND_ROOT_PRE_DIV] = imx_clk_hw_divider2("nand_pre_div", "nand_cg", base + 0xaa00, 16, 3);
+ hws[IMX7D_QSPI_ROOT_PRE_DIV] = imx_clk_hw_divider2("qspi_pre_div", "qspi_cg", base + 0xaa80, 16, 3);
+ hws[IMX7D_USDHC1_ROOT_PRE_DIV] = imx_clk_hw_divider2("usdhc1_pre_div", "usdhc1_cg", base + 0xab00, 16, 3);
+ hws[IMX7D_USDHC2_ROOT_PRE_DIV] = imx_clk_hw_divider2("usdhc2_pre_div", "usdhc2_cg", base + 0xab80, 16, 3);
+ hws[IMX7D_USDHC3_ROOT_PRE_DIV] = imx_clk_hw_divider2("usdhc3_pre_div", "usdhc3_cg", base + 0xac00, 16, 3);
+ hws[IMX7D_CAN1_ROOT_PRE_DIV] = imx_clk_hw_divider2("can1_pre_div", "can1_cg", base + 0xac80, 16, 3);
+ hws[IMX7D_CAN2_ROOT_PRE_DIV] = imx_clk_hw_divider2("can2_pre_div", "can2_cg", base + 0xad00, 16, 3);
+ hws[IMX7D_I2C1_ROOT_PRE_DIV] = imx_clk_hw_divider2("i2c1_pre_div", "i2c1_cg", base + 0xad80, 16, 3);
+ hws[IMX7D_I2C2_ROOT_PRE_DIV] = imx_clk_hw_divider2("i2c2_pre_div", "i2c2_cg", base + 0xae00, 16, 3);
+ hws[IMX7D_I2C3_ROOT_PRE_DIV] = imx_clk_hw_divider2("i2c3_pre_div", "i2c3_cg", base + 0xae80, 16, 3);
+ hws[IMX7D_I2C4_ROOT_PRE_DIV] = imx_clk_hw_divider2("i2c4_pre_div", "i2c4_cg", base + 0xaf00, 16, 3);
+ hws[IMX7D_UART1_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart1_pre_div", "uart1_cg", base + 0xaf80, 16, 3);
+ hws[IMX7D_UART2_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart2_pre_div", "uart2_cg", base + 0xb000, 16, 3);
+ hws[IMX7D_UART3_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart3_pre_div", "uart3_cg", base + 0xb080, 16, 3);
+ hws[IMX7D_UART4_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart4_pre_div", "uart4_cg", base + 0xb100, 16, 3);
+ hws[IMX7D_UART5_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart5_pre_div", "uart5_cg", base + 0xb180, 16, 3);
+ hws[IMX7D_UART6_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart6_pre_div", "uart6_cg", base + 0xb200, 16, 3);
+ hws[IMX7D_UART7_ROOT_PRE_DIV] = imx_clk_hw_divider2("uart7_pre_div", "uart7_cg", base + 0xb280, 16, 3);
+ hws[IMX7D_ECSPI1_ROOT_PRE_DIV] = imx_clk_hw_divider2("ecspi1_pre_div", "ecspi1_cg", base + 0xb300, 16, 3);
+ hws[IMX7D_ECSPI2_ROOT_PRE_DIV] = imx_clk_hw_divider2("ecspi2_pre_div", "ecspi2_cg", base + 0xb380, 16, 3);
+ hws[IMX7D_ECSPI3_ROOT_PRE_DIV] = imx_clk_hw_divider2("ecspi3_pre_div", "ecspi3_cg", base + 0xb400, 16, 3);
+ hws[IMX7D_ECSPI4_ROOT_PRE_DIV] = imx_clk_hw_divider2("ecspi4_pre_div", "ecspi4_cg", base + 0xb480, 16, 3);
+ hws[IMX7D_PWM1_ROOT_PRE_DIV] = imx_clk_hw_divider2("pwm1_pre_div", "pwm1_cg", base + 0xb500, 16, 3);
+ hws[IMX7D_PWM2_ROOT_PRE_DIV] = imx_clk_hw_divider2("pwm2_pre_div", "pwm2_cg", base + 0xb580, 16, 3);
+ hws[IMX7D_PWM3_ROOT_PRE_DIV] = imx_clk_hw_divider2("pwm3_pre_div", "pwm3_cg", base + 0xb600, 16, 3);
+ hws[IMX7D_PWM4_ROOT_PRE_DIV] = imx_clk_hw_divider2("pwm4_pre_div", "pwm4_cg", base + 0xb680, 16, 3);
+ hws[IMX7D_FLEXTIMER1_ROOT_PRE_DIV] = imx_clk_hw_divider2("flextimer1_pre_div", "flextimer1_cg", base + 0xb700, 16, 3);
+ hws[IMX7D_FLEXTIMER2_ROOT_PRE_DIV] = imx_clk_hw_divider2("flextimer2_pre_div", "flextimer2_cg", base + 0xb780, 16, 3);
+ hws[IMX7D_SIM1_ROOT_PRE_DIV] = imx_clk_hw_divider2("sim1_pre_div", "sim1_cg", base + 0xb800, 16, 3);
+ hws[IMX7D_SIM2_ROOT_PRE_DIV] = imx_clk_hw_divider2("sim2_pre_div", "sim2_cg", base + 0xb880, 16, 3);
+ hws[IMX7D_GPT1_ROOT_PRE_DIV] = imx_clk_hw_divider2("gpt1_pre_div", "gpt1_cg", base + 0xb900, 16, 3);
+ hws[IMX7D_GPT2_ROOT_PRE_DIV] = imx_clk_hw_divider2("gpt2_pre_div", "gpt2_cg", base + 0xb980, 16, 3);
+ hws[IMX7D_GPT3_ROOT_PRE_DIV] = imx_clk_hw_divider2("gpt3_pre_div", "gpt3_cg", base + 0xba00, 16, 3);
+ hws[IMX7D_GPT4_ROOT_PRE_DIV] = imx_clk_hw_divider2("gpt4_pre_div", "gpt4_cg", base + 0xba80, 16, 3);
+ hws[IMX7D_TRACE_ROOT_PRE_DIV] = imx_clk_hw_divider2("trace_pre_div", "trace_cg", base + 0xbb00, 16, 3);
+ hws[IMX7D_WDOG_ROOT_PRE_DIV] = imx_clk_hw_divider2("wdog_pre_div", "wdog_cg", base + 0xbb80, 16, 3);
+ hws[IMX7D_CSI_MCLK_ROOT_PRE_DIV] = imx_clk_hw_divider2("csi_mclk_pre_div", "csi_mclk_cg", base + 0xbc00, 16, 3);
+ hws[IMX7D_AUDIO_MCLK_ROOT_PRE_DIV] = imx_clk_hw_divider2("audio_mclk_pre_div", "audio_mclk_cg", base + 0xbc80, 16, 3);
+ hws[IMX7D_WRCLK_ROOT_PRE_DIV] = imx_clk_hw_divider2("wrclk_pre_div", "wrclk_cg", base + 0xbd00, 16, 3);
+ hws[IMX7D_CLKO1_ROOT_PRE_DIV] = imx_clk_hw_divider2("clko1_pre_div", "clko1_cg", base + 0xbd80, 16, 3);
+ hws[IMX7D_CLKO2_ROOT_PRE_DIV] = imx_clk_hw_divider2("clko2_pre_div", "clko2_cg", base + 0xbe00, 16, 3);
+
+ hws[IMX7D_ARM_A7_ROOT_DIV] = imx_clk_hw_divider2("arm_a7_div", "arm_a7_cg", base + 0x8000, 0, 3);
+ hws[IMX7D_ARM_M4_ROOT_DIV] = imx_clk_hw_divider2("arm_m4_div", "arm_m4_cg", base + 0x8080, 0, 3);
+ hws[IMX7D_MAIN_AXI_ROOT_DIV] = imx_clk_hw_divider2("axi_post_div", "axi_pre_div", base + 0x8800, 0, 6);
+ hws[IMX7D_DISP_AXI_ROOT_DIV] = imx_clk_hw_divider2("disp_axi_post_div", "disp_axi_pre_div", base + 0x8880, 0, 6);
+ hws[IMX7D_ENET_AXI_ROOT_DIV] = imx_clk_hw_divider2("enet_axi_post_div", "enet_axi_pre_div", base + 0x8900, 0, 6);
+ hws[IMX7D_NAND_USDHC_BUS_ROOT_CLK] = imx_clk_hw_divider2("nand_usdhc_root_clk", "nand_usdhc_pre_div", base + 0x8980, 0, 6);
+ hws[IMX7D_AHB_CHANNEL_ROOT_DIV] = imx_clk_hw_divider2("ahb_root_clk", "ahb_pre_div", base + 0x9000, 0, 6);
+ hws[IMX7D_IPG_ROOT_CLK] = imx_clk_hw_divider_flags("ipg_root_clk", "ahb_root_clk", base + 0x9080, 0, 2, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE | CLK_SET_RATE_PARENT);
+ hws[IMX7D_DRAM_ROOT_DIV] = imx_clk_hw_divider2("dram_post_div", "dram_cg", base + 0x9880, 0, 3);
+ hws[IMX7D_DRAM_PHYM_ALT_ROOT_DIV] = imx_clk_hw_divider2("dram_phym_alt_post_div", "dram_phym_alt_pre_div", base + 0xa000, 0, 3);
+ hws[IMX7D_DRAM_ALT_ROOT_DIV] = imx_clk_hw_divider2("dram_alt_post_div", "dram_alt_pre_div", base + 0xa080, 0, 3);
+ hws[IMX7D_USB_HSIC_ROOT_DIV] = imx_clk_hw_divider2("usb_hsic_post_div", "usb_hsic_pre_div", base + 0xa100, 0, 6);
+ hws[IMX7D_PCIE_CTRL_ROOT_DIV] = imx_clk_hw_divider2("pcie_ctrl_post_div", "pcie_ctrl_pre_div", base + 0xa180, 0, 6);
+ hws[IMX7D_PCIE_PHY_ROOT_DIV] = imx_clk_hw_divider2("pcie_phy_post_div", "pcie_phy_pre_div", base + 0xa200, 0, 6);
+ hws[IMX7D_EPDC_PIXEL_ROOT_DIV] = imx_clk_hw_divider2("epdc_pixel_post_div", "epdc_pixel_pre_div", base + 0xa280, 0, 6);
+ hws[IMX7D_LCDIF_PIXEL_ROOT_DIV] = imx_clk_hw_divider2("lcdif_pixel_post_div", "lcdif_pixel_pre_div", base + 0xa300, 0, 6);
+ hws[IMX7D_MIPI_DSI_ROOT_DIV] = imx_clk_hw_divider2("mipi_dsi_post_div", "mipi_dsi_pre_div", base + 0xa380, 0, 6);
+ hws[IMX7D_MIPI_CSI_ROOT_DIV] = imx_clk_hw_divider2("mipi_csi_post_div", "mipi_csi_pre_div", base + 0xa400, 0, 6);
+ hws[IMX7D_MIPI_DPHY_ROOT_DIV] = imx_clk_hw_divider2("mipi_dphy_post_div", "mipi_dphy_pre_div", base + 0xa480, 0, 6);
+ hws[IMX7D_SAI1_ROOT_DIV] = imx_clk_hw_divider2("sai1_post_div", "sai1_pre_div", base + 0xa500, 0, 6);
+ hws[IMX7D_SAI2_ROOT_DIV] = imx_clk_hw_divider2("sai2_post_div", "sai2_pre_div", base + 0xa580, 0, 6);
+ hws[IMX7D_SAI3_ROOT_DIV] = imx_clk_hw_divider2("sai3_post_div", "sai3_pre_div", base + 0xa600, 0, 6);
+ hws[IMX7D_SPDIF_ROOT_DIV] = imx_clk_hw_divider2("spdif_post_div", "spdif_pre_div", base + 0xa680, 0, 6);
+ hws[IMX7D_ENET1_REF_ROOT_DIV] = imx_clk_hw_divider2("enet1_ref_post_div", "enet1_ref_pre_div", base + 0xa700, 0, 6);
+ hws[IMX7D_ENET1_TIME_ROOT_DIV] = imx_clk_hw_divider2("enet1_time_post_div", "enet1_time_pre_div", base + 0xa780, 0, 6);
+ hws[IMX7D_ENET2_REF_ROOT_DIV] = imx_clk_hw_divider2("enet2_ref_post_div", "enet2_ref_pre_div", base + 0xa800, 0, 6);
+ hws[IMX7D_ENET2_TIME_ROOT_DIV] = imx_clk_hw_divider2("enet2_time_post_div", "enet2_time_pre_div", base + 0xa880, 0, 6);
+ hws[IMX7D_ENET_PHY_REF_ROOT_CLK] = imx_clk_hw_divider2("enet_phy_ref_root_clk", "enet_phy_ref_pre_div", base + 0xa900, 0, 6);
+ hws[IMX7D_EIM_ROOT_DIV] = imx_clk_hw_divider2("eim_post_div", "eim_pre_div", base + 0xa980, 0, 6);
+ hws[IMX7D_NAND_ROOT_CLK] = imx_clk_hw_divider2("nand_root_clk", "nand_pre_div", base + 0xaa00, 0, 6);
+ hws[IMX7D_QSPI_ROOT_DIV] = imx_clk_hw_divider2("qspi_post_div", "qspi_pre_div", base + 0xaa80, 0, 6);
+ hws[IMX7D_USDHC1_ROOT_DIV] = imx_clk_hw_divider2("usdhc1_post_div", "usdhc1_pre_div", base + 0xab00, 0, 6);
+ hws[IMX7D_USDHC2_ROOT_DIV] = imx_clk_hw_divider2("usdhc2_post_div", "usdhc2_pre_div", base + 0xab80, 0, 6);
+ hws[IMX7D_USDHC3_ROOT_DIV] = imx_clk_hw_divider2("usdhc3_post_div", "usdhc3_pre_div", base + 0xac00, 0, 6);
+ hws[IMX7D_CAN1_ROOT_DIV] = imx_clk_hw_divider2("can1_post_div", "can1_pre_div", base + 0xac80, 0, 6);
+ hws[IMX7D_CAN2_ROOT_DIV] = imx_clk_hw_divider2("can2_post_div", "can2_pre_div", base + 0xad00, 0, 6);
+ hws[IMX7D_I2C1_ROOT_DIV] = imx_clk_hw_divider2("i2c1_post_div", "i2c1_pre_div", base + 0xad80, 0, 6);
+ hws[IMX7D_I2C2_ROOT_DIV] = imx_clk_hw_divider2("i2c2_post_div", "i2c2_pre_div", base + 0xae00, 0, 6);
+ hws[IMX7D_I2C3_ROOT_DIV] = imx_clk_hw_divider2("i2c3_post_div", "i2c3_pre_div", base + 0xae80, 0, 6);
+ hws[IMX7D_I2C4_ROOT_DIV] = imx_clk_hw_divider2("i2c4_post_div", "i2c4_pre_div", base + 0xaf00, 0, 6);
+ hws[IMX7D_UART1_ROOT_DIV] = imx_clk_hw_divider2("uart1_post_div", "uart1_pre_div", base + 0xaf80, 0, 6);
+ hws[IMX7D_UART2_ROOT_DIV] = imx_clk_hw_divider2("uart2_post_div", "uart2_pre_div", base + 0xb000, 0, 6);
+ hws[IMX7D_UART3_ROOT_DIV] = imx_clk_hw_divider2("uart3_post_div", "uart3_pre_div", base + 0xb080, 0, 6);
+ hws[IMX7D_UART4_ROOT_DIV] = imx_clk_hw_divider2("uart4_post_div", "uart4_pre_div", base + 0xb100, 0, 6);
+ hws[IMX7D_UART5_ROOT_DIV] = imx_clk_hw_divider2("uart5_post_div", "uart5_pre_div", base + 0xb180, 0, 6);
+ hws[IMX7D_UART6_ROOT_DIV] = imx_clk_hw_divider2("uart6_post_div", "uart6_pre_div", base + 0xb200, 0, 6);
+ hws[IMX7D_UART7_ROOT_DIV] = imx_clk_hw_divider2("uart7_post_div", "uart7_pre_div", base + 0xb280, 0, 6);
+ hws[IMX7D_ECSPI1_ROOT_DIV] = imx_clk_hw_divider2("ecspi1_post_div", "ecspi1_pre_div", base + 0xb300, 0, 6);
+ hws[IMX7D_ECSPI2_ROOT_DIV] = imx_clk_hw_divider2("ecspi2_post_div", "ecspi2_pre_div", base + 0xb380, 0, 6);
+ hws[IMX7D_ECSPI3_ROOT_DIV] = imx_clk_hw_divider2("ecspi3_post_div", "ecspi3_pre_div", base + 0xb400, 0, 6);
+ hws[IMX7D_ECSPI4_ROOT_DIV] = imx_clk_hw_divider2("ecspi4_post_div", "ecspi4_pre_div", base + 0xb480, 0, 6);
+ hws[IMX7D_PWM1_ROOT_DIV] = imx_clk_hw_divider2("pwm1_post_div", "pwm1_pre_div", base + 0xb500, 0, 6);
+ hws[IMX7D_PWM2_ROOT_DIV] = imx_clk_hw_divider2("pwm2_post_div", "pwm2_pre_div", base + 0xb580, 0, 6);
+ hws[IMX7D_PWM3_ROOT_DIV] = imx_clk_hw_divider2("pwm3_post_div", "pwm3_pre_div", base + 0xb600, 0, 6);
+ hws[IMX7D_PWM4_ROOT_DIV] = imx_clk_hw_divider2("pwm4_post_div", "pwm4_pre_div", base + 0xb680, 0, 6);
+ hws[IMX7D_FLEXTIMER1_ROOT_DIV] = imx_clk_hw_divider2("flextimer1_post_div", "flextimer1_pre_div", base + 0xb700, 0, 6);
+ hws[IMX7D_FLEXTIMER2_ROOT_DIV] = imx_clk_hw_divider2("flextimer2_post_div", "flextimer2_pre_div", base + 0xb780, 0, 6);
+ hws[IMX7D_SIM1_ROOT_DIV] = imx_clk_hw_divider2("sim1_post_div", "sim1_pre_div", base + 0xb800, 0, 6);
+ hws[IMX7D_SIM2_ROOT_DIV] = imx_clk_hw_divider2("sim2_post_div", "sim2_pre_div", base + 0xb880, 0, 6);
+ hws[IMX7D_GPT1_ROOT_DIV] = imx_clk_hw_divider2("gpt1_post_div", "gpt1_pre_div", base + 0xb900, 0, 6);
+ hws[IMX7D_GPT2_ROOT_DIV] = imx_clk_hw_divider2("gpt2_post_div", "gpt2_pre_div", base + 0xb980, 0, 6);
+ hws[IMX7D_GPT3_ROOT_DIV] = imx_clk_hw_divider2("gpt3_post_div", "gpt3_pre_div", base + 0xba00, 0, 6);
+ hws[IMX7D_GPT4_ROOT_DIV] = imx_clk_hw_divider2("gpt4_post_div", "gpt4_pre_div", base + 0xba80, 0, 6);
+ hws[IMX7D_TRACE_ROOT_DIV] = imx_clk_hw_divider2("trace_post_div", "trace_pre_div", base + 0xbb00, 0, 6);
+ hws[IMX7D_WDOG_ROOT_DIV] = imx_clk_hw_divider2("wdog_post_div", "wdog_pre_div", base + 0xbb80, 0, 6);
+ hws[IMX7D_CSI_MCLK_ROOT_DIV] = imx_clk_hw_divider2("csi_mclk_post_div", "csi_mclk_pre_div", base + 0xbc00, 0, 6);
+ hws[IMX7D_AUDIO_MCLK_ROOT_DIV] = imx_clk_hw_divider2("audio_mclk_post_div", "audio_mclk_pre_div", base + 0xbc80, 0, 6);
+ hws[IMX7D_WRCLK_ROOT_DIV] = imx_clk_hw_divider2("wrclk_post_div", "wrclk_pre_div", base + 0xbd00, 0, 6);
+ hws[IMX7D_CLKO1_ROOT_DIV] = imx_clk_hw_divider2("clko1_post_div", "clko1_pre_div", base + 0xbd80, 0, 6);
+ hws[IMX7D_CLKO2_ROOT_DIV] = imx_clk_hw_divider2("clko2_post_div", "clko2_pre_div", base + 0xbe00, 0, 6);
+
+ hws[IMX7D_ARM_A7_ROOT_CLK] = imx_clk_hw_gate2_flags("arm_a7_root_clk", "arm_a7_div", base + 0x4000, 0, CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_ARM_M4_ROOT_CLK] = imx_clk_hw_gate4("arm_m4_root_clk", "arm_m4_div", base + 0x4010, 0);
+ hws[IMX7D_MAIN_AXI_ROOT_CLK] = imx_clk_hw_gate2_flags("main_axi_root_clk", "axi_post_div", base + 0x4040, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_DISP_AXI_ROOT_CLK] = imx_clk_hw_gate4("disp_axi_root_clk", "disp_axi_post_div", base + 0x4050, 0);
+ hws[IMX7D_ENET_AXI_ROOT_CLK] = imx_clk_hw_gate4("enet_axi_root_clk", "enet_axi_post_div", base + 0x4060, 0);
+ hws[IMX7D_OCRAM_CLK] = imx_clk_hw_gate4("ocram_clk", "main_axi_root_clk", base + 0x4110, 0);
+ hws[IMX7D_OCRAM_S_CLK] = imx_clk_hw_gate4("ocram_s_clk", "ahb_root_clk", base + 0x4120, 0);
+ hws[IMX7D_DRAM_ROOT_CLK] = imx_clk_hw_gate2_flags("dram_root_clk", "dram_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_DRAM_PHYM_ROOT_CLK] = imx_clk_hw_gate2_flags("dram_phym_root_clk", "dram_phym_cg", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_DRAM_PHYM_ALT_ROOT_CLK] = imx_clk_hw_gate2_flags("dram_phym_alt_root_clk", "dram_phym_alt_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_DRAM_ALT_ROOT_CLK] = imx_clk_hw_gate2_flags("dram_alt_root_clk", "dram_alt_post_div", base + 0x4130, 0, CLK_IS_CRITICAL | CLK_OPS_PARENT_ENABLE);
+ hws[IMX7D_OCOTP_CLK] = imx_clk_hw_gate4("ocotp_clk", "ipg_root_clk", base + 0x4230, 0);
+ hws[IMX7D_SNVS_CLK] = imx_clk_hw_gate4("snvs_clk", "ipg_root_clk", base + 0x4250, 0);
+ hws[IMX7D_MU_ROOT_CLK] = imx_clk_hw_gate4("mu_root_clk", "ipg_root_clk", base + 0x4270, 0);
+ hws[IMX7D_CAAM_CLK] = imx_clk_hw_gate4("caam_clk", "ipg_root_clk", base + 0x4240, 0);
+ hws[IMX7D_USB_HSIC_ROOT_CLK] = imx_clk_hw_gate4("usb_hsic_root_clk", "usb_hsic_post_div", base + 0x4690, 0);
+ hws[IMX7D_SDMA_CORE_CLK] = imx_clk_hw_gate4("sdma_root_clk", "ahb_root_clk", base + 0x4480, 0);
+ hws[IMX7D_PCIE_CTRL_ROOT_CLK] = imx_clk_hw_gate4("pcie_ctrl_root_clk", "pcie_ctrl_post_div", base + 0x4600, 0);
+ hws[IMX7D_PCIE_PHY_ROOT_CLK] = imx_clk_hw_gate4("pcie_phy_root_clk", "pcie_phy_post_div", base + 0x4600, 0);
+ hws[IMX7D_EPDC_PIXEL_ROOT_CLK] = imx_clk_hw_gate4("epdc_pixel_root_clk", "epdc_pixel_post_div", base + 0x44a0, 0);
+ hws[IMX7D_LCDIF_PIXEL_ROOT_CLK] = imx_clk_hw_gate4("lcdif_pixel_root_clk", "lcdif_pixel_post_div", base + 0x44b0, 0);
+ hws[IMX7D_MIPI_DSI_ROOT_CLK] = imx_clk_hw_gate4("mipi_dsi_root_clk", "mipi_dsi_post_div", base + 0x4650, 0);
+ hws[IMX7D_MIPI_CSI_ROOT_CLK] = imx_clk_hw_gate4("mipi_csi_root_clk", "mipi_csi_post_div", base + 0x4640, 0);
+ hws[IMX7D_MIPI_DPHY_ROOT_CLK] = imx_clk_hw_gate4("mipi_dphy_root_clk", "mipi_dphy_post_div", base + 0x4660, 0);
+ hws[IMX7D_ENET1_IPG_ROOT_CLK] = imx_clk_hw_gate2_shared2("enet1_ipg_root_clk", "enet_axi_post_div", base + 0x4700, 0, &share_count_enet1);
+ hws[IMX7D_ENET1_TIME_ROOT_CLK] = imx_clk_hw_gate2_shared2("enet1_time_root_clk", "enet1_time_post_div", base + 0x4700, 0, &share_count_enet1);
+ hws[IMX7D_ENET2_IPG_ROOT_CLK] = imx_clk_hw_gate2_shared2("enet2_ipg_root_clk", "enet_axi_post_div", base + 0x4710, 0, &share_count_enet2);
+ hws[IMX7D_ENET2_TIME_ROOT_CLK] = imx_clk_hw_gate2_shared2("enet2_time_root_clk", "enet2_time_post_div", base + 0x4710, 0, &share_count_enet2);
+ hws[IMX7D_SAI1_ROOT_CLK] = imx_clk_hw_gate2_shared2("sai1_root_clk", "sai1_post_div", base + 0x48c0, 0, &share_count_sai1);
+ hws[IMX7D_SAI1_IPG_CLK] = imx_clk_hw_gate2_shared2("sai1_ipg_clk", "ipg_root_clk", base + 0x48c0, 0, &share_count_sai1);
+ hws[IMX7D_SAI2_ROOT_CLK] = imx_clk_hw_gate2_shared2("sai2_root_clk", "sai2_post_div", base + 0x48d0, 0, &share_count_sai2);
+ hws[IMX7D_SAI2_IPG_CLK] = imx_clk_hw_gate2_shared2("sai2_ipg_clk", "ipg_root_clk", base + 0x48d0, 0, &share_count_sai2);
+ hws[IMX7D_SAI3_ROOT_CLK] = imx_clk_hw_gate2_shared2("sai3_root_clk", "sai3_post_div", base + 0x48e0, 0, &share_count_sai3);
+ hws[IMX7D_SAI3_IPG_CLK] = imx_clk_hw_gate2_shared2("sai3_ipg_clk", "ipg_root_clk", base + 0x48e0, 0, &share_count_sai3);
+ hws[IMX7D_SPDIF_ROOT_CLK] = imx_clk_hw_gate4("spdif_root_clk", "spdif_post_div", base + 0x44d0, 0);
+ hws[IMX7D_EIM_ROOT_CLK] = imx_clk_hw_gate4("eim_root_clk", "eim_post_div", base + 0x4160, 0);
+ hws[IMX7D_NAND_RAWNAND_CLK] = imx_clk_hw_gate2_shared2("nand_rawnand_clk", "nand_root_clk", base + 0x4140, 0, &share_count_nand);
+ hws[IMX7D_NAND_USDHC_BUS_RAWNAND_CLK] = imx_clk_hw_gate2_shared2("nand_usdhc_rawnand_clk", "nand_usdhc_root_clk", base + 0x4140, 0, &share_count_nand);
+ hws[IMX7D_QSPI_ROOT_CLK] = imx_clk_hw_gate4("qspi_root_clk", "qspi_post_div", base + 0x4150, 0);
+ hws[IMX7D_USDHC1_ROOT_CLK] = imx_clk_hw_gate4("usdhc1_root_clk", "usdhc1_post_div", base + 0x46c0, 0);
+ hws[IMX7D_USDHC2_ROOT_CLK] = imx_clk_hw_gate4("usdhc2_root_clk", "usdhc2_post_div", base + 0x46d0, 0);
+ hws[IMX7D_USDHC3_ROOT_CLK] = imx_clk_hw_gate4("usdhc3_root_clk", "usdhc3_post_div", base + 0x46e0, 0);
+ hws[IMX7D_CAN1_ROOT_CLK] = imx_clk_hw_gate4("can1_root_clk", "can1_post_div", base + 0x4740, 0);
+ hws[IMX7D_CAN2_ROOT_CLK] = imx_clk_hw_gate4("can2_root_clk", "can2_post_div", base + 0x4750, 0);
+ hws[IMX7D_I2C1_ROOT_CLK] = imx_clk_hw_gate4("i2c1_root_clk", "i2c1_post_div", base + 0x4880, 0);
+ hws[IMX7D_I2C2_ROOT_CLK] = imx_clk_hw_gate4("i2c2_root_clk", "i2c2_post_div", base + 0x4890, 0);
+ hws[IMX7D_I2C3_ROOT_CLK] = imx_clk_hw_gate4("i2c3_root_clk", "i2c3_post_div", base + 0x48a0, 0);
+ hws[IMX7D_I2C4_ROOT_CLK] = imx_clk_hw_gate4("i2c4_root_clk", "i2c4_post_div", base + 0x48b0, 0);
+ hws[IMX7D_UART1_ROOT_CLK] = imx_clk_hw_gate4("uart1_root_clk", "uart1_post_div", base + 0x4940, 0);
+ hws[IMX7D_UART2_ROOT_CLK] = imx_clk_hw_gate4("uart2_root_clk", "uart2_post_div", base + 0x4950, 0);
+ hws[IMX7D_UART3_ROOT_CLK] = imx_clk_hw_gate4("uart3_root_clk", "uart3_post_div", base + 0x4960, 0);
+ hws[IMX7D_UART4_ROOT_CLK] = imx_clk_hw_gate4("uart4_root_clk", "uart4_post_div", base + 0x4970, 0);
+ hws[IMX7D_UART5_ROOT_CLK] = imx_clk_hw_gate4("uart5_root_clk", "uart5_post_div", base + 0x4980, 0);
+ hws[IMX7D_UART6_ROOT_CLK] = imx_clk_hw_gate4("uart6_root_clk", "uart6_post_div", base + 0x4990, 0);
+ hws[IMX7D_UART7_ROOT_CLK] = imx_clk_hw_gate4("uart7_root_clk", "uart7_post_div", base + 0x49a0, 0);
+ hws[IMX7D_ECSPI1_ROOT_CLK] = imx_clk_hw_gate4("ecspi1_root_clk", "ecspi1_post_div", base + 0x4780, 0);
+ hws[IMX7D_ECSPI2_ROOT_CLK] = imx_clk_hw_gate4("ecspi2_root_clk", "ecspi2_post_div", base + 0x4790, 0);
+ hws[IMX7D_ECSPI3_ROOT_CLK] = imx_clk_hw_gate4("ecspi3_root_clk", "ecspi3_post_div", base + 0x47a0, 0);
+ hws[IMX7D_ECSPI4_ROOT_CLK] = imx_clk_hw_gate4("ecspi4_root_clk", "ecspi4_post_div", base + 0x47b0, 0);
+ hws[IMX7D_PWM1_ROOT_CLK] = imx_clk_hw_gate4("pwm1_root_clk", "pwm1_post_div", base + 0x4840, 0);
+ hws[IMX7D_PWM2_ROOT_CLK] = imx_clk_hw_gate4("pwm2_root_clk", "pwm2_post_div", base + 0x4850, 0);
+ hws[IMX7D_PWM3_ROOT_CLK] = imx_clk_hw_gate4("pwm3_root_clk", "pwm3_post_div", base + 0x4860, 0);
+ hws[IMX7D_PWM4_ROOT_CLK] = imx_clk_hw_gate4("pwm4_root_clk", "pwm4_post_div", base + 0x4870, 0);
+ hws[IMX7D_FLEXTIMER1_ROOT_CLK] = imx_clk_hw_gate4("flextimer1_root_clk", "flextimer1_post_div", base + 0x4800, 0);
+ hws[IMX7D_FLEXTIMER2_ROOT_CLK] = imx_clk_hw_gate4("flextimer2_root_clk", "flextimer2_post_div", base + 0x4810, 0);
+ hws[IMX7D_SIM1_ROOT_CLK] = imx_clk_hw_gate4("sim1_root_clk", "sim1_post_div", base + 0x4900, 0);
+ hws[IMX7D_SIM2_ROOT_CLK] = imx_clk_hw_gate4("sim2_root_clk", "sim2_post_div", base + 0x4910, 0);
+ hws[IMX7D_GPT1_ROOT_CLK] = imx_clk_hw_gate4("gpt1_root_clk", "gpt1_post_div", base + 0x47c0, 0);
+ hws[IMX7D_GPT2_ROOT_CLK] = imx_clk_hw_gate4("gpt2_root_clk", "gpt2_post_div", base + 0x47d0, 0);
+ hws[IMX7D_GPT3_ROOT_CLK] = imx_clk_hw_gate4("gpt3_root_clk", "gpt3_post_div", base + 0x47e0, 0);
+ hws[IMX7D_GPT4_ROOT_CLK] = imx_clk_hw_gate4("gpt4_root_clk", "gpt4_post_div", base + 0x47f0, 0);
+ hws[IMX7D_TRACE_ROOT_CLK] = imx_clk_hw_gate4("trace_root_clk", "trace_post_div", base + 0x4300, 0);
+ hws[IMX7D_WDOG1_ROOT_CLK] = imx_clk_hw_gate4("wdog1_root_clk", "wdog_post_div", base + 0x49c0, 0);
+ hws[IMX7D_WDOG2_ROOT_CLK] = imx_clk_hw_gate4("wdog2_root_clk", "wdog_post_div", base + 0x49d0, 0);
+ hws[IMX7D_WDOG3_ROOT_CLK] = imx_clk_hw_gate4("wdog3_root_clk", "wdog_post_div", base + 0x49e0, 0);
+ hws[IMX7D_WDOG4_ROOT_CLK] = imx_clk_hw_gate4("wdog4_root_clk", "wdog_post_div", base + 0x49f0, 0);
+ hws[IMX7D_KPP_ROOT_CLK] = imx_clk_hw_gate4("kpp_root_clk", "ipg_root_clk", base + 0x4aa0, 0);
+ hws[IMX7D_CSI_MCLK_ROOT_CLK] = imx_clk_hw_gate4("csi_mclk_root_clk", "csi_mclk_post_div", base + 0x4490, 0);
+ hws[IMX7D_AUDIO_MCLK_ROOT_CLK] = imx_clk_hw_gate4("audio_mclk_root_clk", "audio_mclk_post_div", base + 0x4790, 0);
+ hws[IMX7D_WRCLK_ROOT_CLK] = imx_clk_hw_gate4("wrclk_root_clk", "wrclk_post_div", base + 0x47a0, 0);
+ hws[IMX7D_USB_CTRL_CLK] = imx_clk_hw_gate4("usb_ctrl_clk", "ahb_root_clk", base + 0x4680, 0);
+ hws[IMX7D_USB_PHY1_CLK] = imx_clk_hw_gate4("usb_phy1_clk", "pll_usb1_main_clk", base + 0x46a0, 0);
+ hws[IMX7D_USB_PHY2_CLK] = imx_clk_hw_gate4("usb_phy2_clk", "pll_usb_main_clk", base + 0x46b0, 0);
+ hws[IMX7D_ADC_ROOT_CLK] = imx_clk_hw_gate4("adc_root_clk", "ipg_root_clk", base + 0x4200, 0);
+
+ hws[IMX7D_GPT_3M_CLK] = imx_clk_hw_fixed_factor("gpt_3m", "osc", 1, 8);
+
+ hws[IMX7D_CLK_ARM] = imx_clk_hw_cpu("arm", "arm_a7_root_clk",
+ hws[IMX7D_ARM_A7_ROOT_CLK]->clk,
+ hws[IMX7D_ARM_A7_ROOT_SRC]->clk,
+ hws[IMX7D_PLL_ARM_MAIN_CLK]->clk,
+ hws[IMX7D_PLL_SYS_MAIN_CLK]->clk);
+
+ imx_check_clk_hws(hws, IMX7D_CLK_END);
+
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
+
+ clk_set_parent(hws[IMX7D_PLL_ARM_MAIN_BYPASS]->clk, hws[IMX7D_PLL_ARM_MAIN]->clk);
+ clk_set_parent(hws[IMX7D_PLL_DRAM_MAIN_BYPASS]->clk, hws[IMX7D_PLL_DRAM_MAIN]->clk);
+ clk_set_parent(hws[IMX7D_PLL_SYS_MAIN_BYPASS]->clk, hws[IMX7D_PLL_SYS_MAIN]->clk);
+ clk_set_parent(hws[IMX7D_PLL_ENET_MAIN_BYPASS]->clk, hws[IMX7D_PLL_ENET_MAIN]->clk);
+ clk_set_parent(hws[IMX7D_PLL_AUDIO_MAIN_BYPASS]->clk, hws[IMX7D_PLL_AUDIO_MAIN]->clk);
+ clk_set_parent(hws[IMX7D_PLL_VIDEO_MAIN_BYPASS]->clk, hws[IMX7D_PLL_VIDEO_MAIN]->clk);
+
+ clk_set_parent(hws[IMX7D_MIPI_CSI_ROOT_SRC]->clk, hws[IMX7D_PLL_SYS_PFD3_CLK]->clk);
/* use old gpt clk setting, gpt1 root clk must be twice as gpt counter freq */
- clk_set_parent(clks[IMX7D_GPT1_ROOT_SRC], clks[IMX7D_OSC_24M_CLK]);
+ clk_set_parent(hws[IMX7D_GPT1_ROOT_SRC]->clk, hws[IMX7D_OSC_24M_CLK]->clk);
/* Set clock rate for USBPHY, the USB_PLL at CCM is from USBOTG2 */
- clks[IMX7D_USB1_MAIN_480M_CLK] = imx_clk_fixed_factor("pll_usb1_main_clk", "osc", 20, 1);
- clks[IMX7D_USB_MAIN_480M_CLK] = imx_clk_fixed_factor("pll_usb_main_clk", "osc", 20, 1);
+ hws[IMX7D_USB1_MAIN_480M_CLK] = imx_clk_hw_fixed_factor("pll_usb1_main_clk", "osc", 20, 1);
+ hws[IMX7D_USB_MAIN_480M_CLK] = imx_clk_hw_fixed_factor("pll_usb_main_clk", "osc", 20, 1);
+
+ for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
+ int index = uart_clk_ids[i];
+
+ uart_clks[i] = &hws[index]->clk;
+ }
+
imx_register_uart_clocks(uart_clks);
clks[IMX7ULP_CLK_NIC0_DIV] = imx_clk_hw_divider_flags("nic0_clk", "nic_sel", base + 0x40, 24, 4, CLK_SET_RATE_PARENT | CLK_IS_CRITICAL);
clks[IMX7ULP_CLK_NIC1_DIV] = imx_clk_hw_divider_flags("nic1_clk", "nic0_clk", base + 0x40, 16, 4, CLK_SET_RATE_PARENT | CLK_IS_CRITICAL);
- clks[IMX7ULP_CLK_NIC1_BUS_DIV] = imx_clk_hw_divider_flags("nic1_bus_clk", "nic1_clk", base + 0x40, 4, 4, CLK_SET_RATE_PARENT | CLK_IS_CRITICAL);
+ clks[IMX7ULP_CLK_NIC1_BUS_DIV] = imx_clk_hw_divider_flags("nic1_bus_clk", "nic0_clk", base + 0x40, 4, 4, CLK_SET_RATE_PARENT | CLK_IS_CRITICAL);
clks[IMX7ULP_CLK_GPU_DIV] = imx_clk_hw_divider("gpu_clk", "nic0_clk", base + 0x40, 20, 4);
static const char *imx8mm_usb_phy_sels[] = {"osc_24m", "sys_pll1_100m", "sys_pll1_40m", "sys_pll2_100m",
"sys_pll2_200m", "clk_ext2", "clk_ext3", "audio_pll2_out", };
+static const char *imx8mm_gic_sels[] = {"osc_24m", "sys_pll2_200m", "sys_pll1_40m", "sys_pll2_100m",
+ "sys_pll1_800m", "clk_ext2", "clk_ext4", "audio_pll2_out" };
+
static const char *imx8mm_ecspi1_sels[] = {"osc_24m", "sys_pll2_200m", "sys_pll1_40m", "sys_pll1_160m",
"sys_pll1_800m", "sys_pll3_out", "sys_pll2_250m", "audio_pll2_out", };
"sys_pll2_1000m", "sys_pll3_out", "audio_pll2_out", "video_pll1_out", };
static const char *imx8mm_usdhc3_sels[] = {"osc_24m", "sys_pll1_400m", "sys_pll1_800m", "sys_pll2_500m",
- "sys_pll3_out", "sys_pll1_266m", "audio_pll2_clk", "sys_pll1_100m", };
+ "sys_pll3_out", "sys_pll1_266m", "audio_pll2_out", "sys_pll1_100m", };
static const char *imx8mm_csi1_core_sels[] = {"osc_24m", "sys_pll1_266m", "sys_pll2_250m", "sys_pll1_800m",
"sys_pll2_1000m", "sys_pll3_out", "audio_pll2_out", "video_pll1_out", };
"sys_pll2_1000m", "sys_pll3_out", "clk_ext3", "audio_pll2_out", };
static const char *imx8mm_vpu_h1_sels[] = {"osc_24m", "vpu_pll_out", "sys_pll1_800m", "sys_pll2_1000m",
- "audio_pll2_clk", "sys_pll2_125m", "sys_pll3_clk", "audio_pll1_out", };
+ "audio_pll2_out", "sys_pll2_125m", "sys_pll3_clk", "audio_pll1_out", };
static const char *imx8mm_dram_core_sels[] = {"dram_pll_out", "dram_alt_root", };
-static const char *imx8mm_clko1_sels[] = {"osc_24m", "sys_pll1_800m", "osc_27m", "sys_pll1_200m", "audio_pll2_clk",
+static const char *imx8mm_clko1_sels[] = {"osc_24m", "sys_pll1_800m", "osc_27m", "sys_pll1_200m", "audio_pll2_out",
"vpu_pll", "sys_pll1_80m", };
static struct clk *clks[IMX8MM_CLK_END];
/* IP */
clks[IMX8MM_CLK_DRAM_ALT] = imx8m_clk_composite("dram_alt", imx8mm_dram_alt_sels, base + 0xa000);
- clks[IMX8MM_CLK_DRAM_APB] = imx8m_clk_composite("dram_apb", imx8mm_dram_apb_sels, base + 0xa080);
+ clks[IMX8MM_CLK_DRAM_APB] = imx8m_clk_composite_critical("dram_apb", imx8mm_dram_apb_sels, base + 0xa080);
clks[IMX8MM_CLK_VPU_G1] = imx8m_clk_composite("vpu_g1", imx8mm_vpu_g1_sels, base + 0xa100);
clks[IMX8MM_CLK_VPU_G2] = imx8m_clk_composite("vpu_g2", imx8mm_vpu_g2_sels, base + 0xa180);
clks[IMX8MM_CLK_DISP_DTRC] = imx8m_clk_composite("disp_dtrc", imx8mm_disp_dtrc_sels, base + 0xa200);
clks[IMX8MM_CLK_UART4] = imx8m_clk_composite("uart4", imx8mm_uart4_sels, base + 0xb080);
clks[IMX8MM_CLK_USB_CORE_REF] = imx8m_clk_composite("usb_core_ref", imx8mm_usb_core_sels, base + 0xb100);
clks[IMX8MM_CLK_USB_PHY_REF] = imx8m_clk_composite("usb_phy_ref", imx8mm_usb_phy_sels, base + 0xb180);
+ clks[IMX8MM_CLK_GIC] = imx8m_clk_composite_critical("gic", imx8mm_gic_sels, base + 0xb200);
clks[IMX8MM_CLK_ECSPI1] = imx8m_clk_composite("ecspi1", imx8mm_ecspi1_sels, base + 0xb280);
clks[IMX8MM_CLK_ECSPI2] = imx8m_clk_composite("ecspi2", imx8mm_ecspi2_sels, base + 0xb300);
clks[IMX8MM_CLK_PWM1] = imx8m_clk_composite("pwm1", imx8mm_pwm1_sels, base + 0xb380);
clks[IMX8MM_CLK_ECSPI2_ROOT] = imx_clk_gate4("ecspi2_root_clk", "ecspi2", base + 0x4080, 0);
clks[IMX8MM_CLK_ECSPI3_ROOT] = imx_clk_gate4("ecspi3_root_clk", "ecspi3", base + 0x4090, 0);
clks[IMX8MM_CLK_ENET1_ROOT] = imx_clk_gate4("enet1_root_clk", "enet_axi", base + 0x40a0, 0);
+ clks[IMX8MM_CLK_GPIO1_ROOT] = imx_clk_gate4("gpio1_root_clk", "ipg_root", base + 0x40b0, 0);
+ clks[IMX8MM_CLK_GPIO2_ROOT] = imx_clk_gate4("gpio2_root_clk", "ipg_root", base + 0x40c0, 0);
+ clks[IMX8MM_CLK_GPIO3_ROOT] = imx_clk_gate4("gpio3_root_clk", "ipg_root", base + 0x40d0, 0);
+ clks[IMX8MM_CLK_GPIO4_ROOT] = imx_clk_gate4("gpio4_root_clk", "ipg_root", base + 0x40e0, 0);
+ clks[IMX8MM_CLK_GPIO5_ROOT] = imx_clk_gate4("gpio5_root_clk", "ipg_root", base + 0x40f0, 0);
clks[IMX8MM_CLK_GPT1_ROOT] = imx_clk_gate4("gpt1_root_clk", "gpt1", base + 0x4100, 0);
clks[IMX8MM_CLK_I2C1_ROOT] = imx_clk_gate4("i2c1_root_clk", "i2c1", base + 0x4170, 0);
clks[IMX8MM_CLK_I2C2_ROOT] = imx_clk_gate4("i2c2_root_clk", "i2c2", base + 0x4180, 0);
clks[IMX8MM_CLK_SAI5_IPG] = imx_clk_gate2_shared2("sai5_ipg_clk", "ipg_audio_root", base + 0x4370, 0, &share_count_sai5);
clks[IMX8MM_CLK_SAI6_ROOT] = imx_clk_gate2_shared2("sai6_root_clk", "sai6", base + 0x4380, 0, &share_count_sai6);
clks[IMX8MM_CLK_SAI6_IPG] = imx_clk_gate2_shared2("sai6_ipg_clk", "ipg_audio_root", base + 0x4380, 0, &share_count_sai6);
+ clks[IMX8MM_CLK_SNVS_ROOT] = imx_clk_gate4("snvs_root_clk", "ipg_root", base + 0x4470, 0);
clks[IMX8MM_CLK_UART1_ROOT] = imx_clk_gate4("uart1_root_clk", "uart1", base + 0x4490, 0);
clks[IMX8MM_CLK_UART2_ROOT] = imx_clk_gate4("uart2_root_clk", "uart2", base + 0x44a0, 0);
clks[IMX8MM_CLK_UART3_ROOT] = imx_clk_gate4("uart3_root_clk", "uart3", base + 0x44b0, 0);
static const char * const imx8mq_usb_phy_sels[] = {"osc_25m", "sys1_pll_100m", "sys1_pll_40m", "sys2_pll_100m",
"sys2_pll_200m", "clk_ext2", "clk_ext3", "audio_pll2_out", };
+static const char * const imx8mq_gic_sels[] = {"osc_25m", "sys2_pll_200m", "sys1_pll_40m", "sys2_pll_100m",
+ "sys2_pll_200m", "clk_ext2", "clk_ext3", "audio_pll2_out" };
+
static const char * const imx8mq_ecspi1_sels[] = {"osc_25m", "sys2_pll_200m", "sys1_pll_40m", "sys1_pll_160m",
"sys1_pll_800m", "sys3_pll2_out", "sys2_pll_250m", "audio_pll2_out", };
static struct clk_onecell_data clk_data;
+static struct clk ** const uart_clks[] = {
+ &clks[IMX8MQ_CLK_UART1_ROOT],
+ &clks[IMX8MQ_CLK_UART2_ROOT],
+ &clks[IMX8MQ_CLK_UART3_ROOT],
+ &clks[IMX8MQ_CLK_UART4_ROOT],
+ NULL
+};
+
static int imx8mq_clocks_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
void __iomem *base;
int err;
- int i;
clks[IMX8MQ_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
clks[IMX8MQ_CLK_32K] = of_clk_get_by_name(np, "ckil");
clks[IMX8MQ_SYS2_PLL_1000M] = imx_clk_fixed_factor("sys2_pll_1000m", "sys2_pll_out", 1, 1);
np = dev->of_node;
- base = of_iomap(np, 0);
- if (WARN_ON(!base))
- return -ENOMEM;
+ base = devm_platform_ioremap_resource(pdev, 0);
+ if (WARN_ON(IS_ERR(base)))
+ return PTR_ERR(base);
/* CORE */
clks[IMX8MQ_CLK_A53_SRC] = imx_clk_mux2("arm_a53_src", base + 0x8000, 24, 3, imx8mq_a53_sels, ARRAY_SIZE(imx8mq_a53_sels));
clks[IMX8MQ_CLK_UART4] = imx8m_clk_composite("uart4", imx8mq_uart4_sels, base + 0xb080);
clks[IMX8MQ_CLK_USB_CORE_REF] = imx8m_clk_composite("usb_core_ref", imx8mq_usb_core_sels, base + 0xb100);
clks[IMX8MQ_CLK_USB_PHY_REF] = imx8m_clk_composite("usb_phy_ref", imx8mq_usb_phy_sels, base + 0xb180);
+ clks[IMX8MQ_CLK_GIC] = imx8m_clk_composite_critical("gic", imx8mq_gic_sels, base + 0xb200);
clks[IMX8MQ_CLK_ECSPI1] = imx8m_clk_composite("ecspi1", imx8mq_ecspi1_sels, base + 0xb280);
clks[IMX8MQ_CLK_ECSPI2] = imx8m_clk_composite("ecspi2", imx8mq_ecspi2_sels, base + 0xb300);
clks[IMX8MQ_CLK_PWM1] = imx8m_clk_composite("pwm1", imx8mq_pwm1_sels, base + 0xb380);
clks[IMX8MQ_CLK_SAI5_IPG] = imx_clk_gate2_shared2("sai5_ipg_clk", "ipg_audio_root", base + 0x4370, 0, &share_count_sai5);
clks[IMX8MQ_CLK_SAI6_ROOT] = imx_clk_gate2_shared2("sai6_root_clk", "sai6", base + 0x4380, 0, &share_count_sai6);
clks[IMX8MQ_CLK_SAI6_IPG] = imx_clk_gate2_shared2("sai6_ipg_clk", "ipg_audio_root", base + 0x4380, 0, &share_count_sai6);
+ clks[IMX8MQ_CLK_SNVS_ROOT] = imx_clk_gate4("snvs_root_clk", "ipg_root", base + 0x4470, 0);
clks[IMX8MQ_CLK_UART1_ROOT] = imx_clk_gate4("uart1_root_clk", "uart1", base + 0x4490, 0);
clks[IMX8MQ_CLK_UART2_ROOT] = imx_clk_gate4("uart2_root_clk", "uart2", base + 0x44a0, 0);
clks[IMX8MQ_CLK_UART3_ROOT] = imx_clk_gate4("uart3_root_clk", "uart3", base + 0x44b0, 0);
clks[IMX8MQ_ARM_PLL_OUT],
clks[IMX8MQ_SYS1_PLL_800M]);
- for (i = 0; i < IMX8MQ_CLK_END; i++)
- if (IS_ERR(clks[i]))
- pr_err("i.MX8mq clk %u register failed with %ld\n",
- i, PTR_ERR(clks[i]));
+ imx_check_clocks(clks, ARRAY_SIZE(clks));
clk_data.clks = clks;
clk_data.clk_num = ARRAY_SIZE(clks);
err = of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data);
WARN_ON(err);
+ imx_register_uart_clocks(uart_clks);
+
return err;
}
.is_enabled = clk_pfd_is_enabled,
};
-struct clk *imx_clk_pfd(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_pfd(const char *name, const char *parent_name,
void __iomem *reg, u8 idx)
{
struct clk_pfd *pfd;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
pfd = kzalloc(sizeof(*pfd), GFP_KERNEL);
if (!pfd)
init.num_parents = 1;
pfd->hw.init = &init;
+ hw = &pfd->hw;
- clk = clk_register(NULL, &pfd->hw);
- if (IS_ERR(clk))
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(pfd);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
.recalc_rate = clk_pllv3_enet_recalc_rate,
};
-struct clk *imx_clk_pllv3(enum imx_pllv3_type type, const char *name,
+struct clk_hw *imx_clk_hw_pllv3(enum imx_pllv3_type type, const char *name,
const char *parent_name, void __iomem *base,
u32 div_mask)
{
struct clk_pllv3 *pll;
const struct clk_ops *ops;
- struct clk *clk;
+ struct clk_hw *hw;
struct clk_init_data init;
+ int ret;
pll = kzalloc(sizeof(*pll), GFP_KERNEL);
if (!pll)
init.num_parents = 1;
pll->hw.init = &init;
+ hw = &pll->hw;
- clk = clk_register(NULL, &pll->hw);
- if (IS_ERR(clk))
+ ret = clk_hw_register(NULL, hw);
+ if (ret) {
kfree(pll);
+ return ERR_PTR(ret);
+ }
- return clk;
+ return hw;
}
// SPDX-License-Identifier: GPL-2.0
#include <linux/clk.h>
+#include <linux/clk-provider.h>
#include <linux/err.h>
+#include <linux/io.h>
#include <linux/of.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include "clk.h"
+#define CCM_CCDR 0x4
+#define CCDR_MMDC_CH0_MASK BIT(17)
+#define CCDR_MMDC_CH1_MASK BIT(16)
+
DEFINE_SPINLOCK(imx_ccm_lock);
-void __init imx_check_clocks(struct clk *clks[], unsigned int count)
+void __init imx_mmdc_mask_handshake(void __iomem *ccm_base,
+ unsigned int chn)
+{
+ unsigned int reg;
+
+ reg = readl_relaxed(ccm_base + CCM_CCDR);
+ reg |= chn == 0 ? CCDR_MMDC_CH0_MASK : CCDR_MMDC_CH1_MASK;
+ writel_relaxed(reg, ccm_base + CCM_CCDR);
+}
+
+void imx_check_clocks(struct clk *clks[], unsigned int count)
{
unsigned i;
return clk;
}
+struct clk_hw * __init imx_obtain_fixed_clock_hw(
+ const char *name, unsigned long rate)
+{
+ struct clk *clk;
+
+ clk = imx_obtain_fixed_clock_from_dt(name);
+ if (IS_ERR(clk))
+ clk = imx_clk_fixed(name, rate);
+ return __clk_get_hw(clk);
+}
+
struct clk_hw * __init imx_obtain_fixed_clk_hw(struct device_node *np,
const char *name)
{
return;
}
-static int imx_keep_uart_clocks __initdata;
-static struct clk ** const *imx_uart_clocks __initdata;
+static int imx_keep_uart_clocks;
+static struct clk ** const *imx_uart_clocks;
static int __init imx_keep_uart_clocks_param(char *str)
{
__setup_param("earlyprintk", imx_keep_uart_earlyprintk,
imx_keep_uart_clocks_param, 0);
-void __init imx_register_uart_clocks(struct clk ** const clks[])
+void imx_register_uart_clocks(struct clk ** const clks[])
{
if (imx_keep_uart_clocks) {
int i;
void imx_check_clocks(struct clk *clks[], unsigned int count);
void imx_check_clk_hws(struct clk_hw *clks[], unsigned int count);
void imx_register_uart_clocks(struct clk ** const clks[]);
+void imx_register_uart_clocks_hws(struct clk_hw ** const hws[]);
+void imx_mmdc_mask_handshake(void __iomem *ccm_base, unsigned int chn);
extern void imx_cscmr1_fixup(u32 *val);
int flags;
};
+#define imx_clk_busy_divider(name, parent_name, reg, shift, width, busy_reg, busy_shift) \
+ imx_clk_hw_busy_divider(name, parent_name, reg, shift, width, busy_reg, busy_shift)->clk
+
+#define imx_clk_busy_mux(name, reg, shift, width, busy_reg, busy_shift, parent_names, num_parents) \
+ imx_clk_hw_busy_mux(name, reg, shift, width, busy_reg, busy_shift, parent_names, num_parents)->clk
+
+#define imx_clk_cpu(name, parent_name, div, mux, pll, step) \
+ imx_clk_hw_cpu(name, parent_name, div, mux, pll, step)->clk
+
+#define clk_register_gate2(dev, name, parent_name, flags, reg, bit_idx, \
+ cgr_val, clk_gate_flags, lock, share_count) \
+ clk_hw_register_gate2(dev, name, parent_name, flags, reg, bit_idx, \
+ cgr_val, clk_gate_flags, lock, share_count)->clk
+
+#define imx_clk_pllv3(type, name, parent_name, base, div_mask) \
+ imx_clk_hw_pllv3(type, name, parent_name, base, div_mask)->clk
+
+#define imx_clk_pfd(name, parent_name, reg, idx) \
+ imx_clk_hw_pfd(name, parent_name, reg, idx)->clk
+
+#define imx_clk_gate_exclusive(name, parent, reg, shift, exclusive_mask) \
+ imx_clk_hw_gate_exclusive(name, parent, reg, shift, exclusive_mask)->clk
+
+#define imx_clk_fixup_divider(name, parent, reg, shift, width, fixup) \
+ imx_clk_hw_fixup_divider(name, parent, reg, shift, width, fixup)->clk
+
+#define imx_clk_fixup_mux(name, reg, shift, width, parents, num_parents, fixup) \
+ imx_clk_hw_fixup_mux(name, reg, shift, width, parents, num_parents, fixup)->clk
+
+#define imx_clk_mux_ldb(name, reg, shift, width, parents, num_parents) \
+ imx_clk_hw_mux_ldb(name, reg, shift, width, parents, num_parents)->clk
+
+#define imx_clk_fixed_factor(name, parent, mult, div) \
+ imx_clk_hw_fixed_factor(name, parent, mult, div)->clk
+
+#define imx_clk_divider2(name, parent, reg, shift, width) \
+ imx_clk_hw_divider2(name, parent, reg, shift, width)->clk
+
+#define imx_clk_gate_dis(name, parent, reg, shift) \
+ imx_clk_hw_gate_dis(name, parent, reg, shift)->clk
+
+#define imx_clk_gate_dis_flags(name, parent, reg, shift, flags) \
+ imx_clk_hw_gate_dis_flags(name, parent, reg, shift, flags)->clk
+
+#define imx_clk_gate_flags(name, parent, reg, shift, flags) \
+ imx_clk_hw_gate_flags(name, parent, reg, shift, flags)->clk
+
+#define imx_clk_gate2(name, parent, reg, shift) \
+ imx_clk_hw_gate2(name, parent, reg, shift)->clk
+
+#define imx_clk_gate2_flags(name, parent, reg, shift, flags) \
+ imx_clk_hw_gate2_flags(name, parent, reg, shift, flags)->clk
+
+#define imx_clk_gate2_shared(name, parent, reg, shift, share_count) \
+ imx_clk_hw_gate2_shared(name, parent, reg, shift, share_count)->clk
+
+#define imx_clk_gate2_shared2(name, parent, reg, shift, share_count) \
+ imx_clk_hw_gate2_shared2(name, parent, reg, shift, share_count)->clk
+
+#define imx_clk_gate3(name, parent, reg, shift) \
+ imx_clk_hw_gate3(name, parent, reg, shift)->clk
+
+#define imx_clk_gate4(name, parent, reg, shift) \
+ imx_clk_hw_gate4(name, parent, reg, shift)->clk
+
+#define imx_clk_mux(name, reg, shift, width, parents, num_parents) \
+ imx_clk_hw_mux(name, reg, shift, width, parents, num_parents)->clk
+
struct clk *imx_clk_pll14xx(const char *name, const char *parent_name,
void __iomem *base, const struct imx_pll14xx_clk *pll_clk);
IMX_PLLV3_AV_IMX7,
};
-struct clk *imx_clk_pllv3(enum imx_pllv3_type type, const char *name,
+struct clk_hw *imx_clk_hw_pllv3(enum imx_pllv3_type type, const char *name,
const char *parent_name, void __iomem *base, u32 div_mask);
struct clk_hw *imx_clk_pllv4(const char *name, const char *parent_name,
void __iomem *base);
-struct clk *clk_register_gate2(struct device *dev, const char *name,
+struct clk_hw *clk_hw_register_gate2(struct device *dev, const char *name,
const char *parent_name, unsigned long flags,
void __iomem *reg, u8 bit_idx, u8 cgr_val,
u8 clk_gate_flags, spinlock_t *lock,
struct clk * imx_obtain_fixed_clock(
const char *name, unsigned long rate);
+struct clk_hw *imx_obtain_fixed_clock_hw(
+ const char *name, unsigned long rate);
+
struct clk_hw *imx_obtain_fixed_clk_hw(struct device_node *np,
const char *name);
-struct clk *imx_clk_gate_exclusive(const char *name, const char *parent,
+struct clk_hw *imx_clk_hw_gate_exclusive(const char *name, const char *parent,
void __iomem *reg, u8 shift, u32 exclusive_mask);
-struct clk *imx_clk_pfd(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_pfd(const char *name, const char *parent_name,
void __iomem *reg, u8 idx);
struct clk_hw *imx_clk_pfdv2(const char *name, const char *parent_name,
void __iomem *reg, u8 idx);
-struct clk *imx_clk_busy_divider(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_busy_divider(const char *name, const char *parent_name,
void __iomem *reg, u8 shift, u8 width,
void __iomem *busy_reg, u8 busy_shift);
-struct clk *imx_clk_busy_mux(const char *name, void __iomem *reg, u8 shift,
+struct clk_hw *imx_clk_hw_busy_mux(const char *name, void __iomem *reg, u8 shift,
u8 width, void __iomem *busy_reg, u8 busy_shift,
const char * const *parent_names, int num_parents);
bool rate_present, bool gate_present,
void __iomem *reg);
-struct clk *imx_clk_fixup_divider(const char *name, const char *parent,
+struct clk_hw *imx_clk_hw_fixup_divider(const char *name, const char *parent,
void __iomem *reg, u8 shift, u8 width,
void (*fixup)(u32 *val));
-struct clk *imx_clk_fixup_mux(const char *name, void __iomem *reg,
+struct clk_hw *imx_clk_hw_fixup_mux(const char *name, void __iomem *reg,
u8 shift, u8 width, const char * const *parents,
int num_parents, void (*fixup)(u32 *val));
return clk_hw_register_fixed_rate(NULL, name, NULL, 0, rate);
}
-static inline struct clk *imx_clk_mux_ldb(const char *name, void __iomem *reg,
+static inline struct clk_hw *imx_clk_hw_mux_ldb(const char *name, void __iomem *reg,
u8 shift, u8 width, const char * const *parents,
int num_parents)
{
- return clk_register_mux(NULL, name, parents, num_parents,
+ return clk_hw_register_mux(NULL, name, parents, num_parents,
CLK_SET_RATE_NO_REPARENT | CLK_SET_RATE_PARENT, reg,
shift, width, CLK_MUX_READ_ONLY, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_fixed_factor(const char *name,
+static inline struct clk_hw *imx_clk_hw_fixed_factor(const char *name,
const char *parent, unsigned int mult, unsigned int div)
{
- return clk_register_fixed_factor(NULL, name, parent,
+ return clk_hw_register_fixed_factor(NULL, name, parent,
CLK_SET_RATE_PARENT, mult, div);
}
reg, shift, width, 0, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_divider2(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_divider2(const char *name, const char *parent,
void __iomem *reg, u8 shift, u8 width)
{
- return clk_register_divider(NULL, name, parent,
+ return clk_hw_register_divider(NULL, name, parent,
CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
reg, shift, width, 0, &imx_ccm_lock);
}
shift, 0, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_gate_flags(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate_flags(const char *name, const char *parent,
void __iomem *reg, u8 shift, unsigned long flags)
{
- return clk_register_gate(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
shift, 0, &imx_ccm_lock);
}
shift, 0, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_gate_dis(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate_dis(const char *name, const char *parent,
void __iomem *reg, u8 shift)
{
- return clk_register_gate(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
shift, CLK_GATE_SET_TO_DISABLE, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_gate_dis_flags(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate_dis_flags(const char *name, const char *parent,
void __iomem *reg, u8 shift, unsigned long flags)
{
- return clk_register_gate(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
shift, CLK_GATE_SET_TO_DISABLE, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_gate2(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate2(const char *name, const char *parent,
void __iomem *reg, u8 shift)
{
- return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
shift, 0x3, 0, &imx_ccm_lock, NULL);
}
-static inline struct clk *imx_clk_gate2_flags(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate2_flags(const char *name, const char *parent,
void __iomem *reg, u8 shift, unsigned long flags)
{
- return clk_register_gate2(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate2(NULL, name, parent, flags | CLK_SET_RATE_PARENT, reg,
shift, 0x3, 0, &imx_ccm_lock, NULL);
}
-static inline struct clk *imx_clk_gate2_shared(const char *name,
+static inline struct clk_hw *imx_clk_hw_gate2_shared(const char *name,
const char *parent, void __iomem *reg, u8 shift,
unsigned int *share_count)
{
- return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
+ return clk_hw_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT, reg,
shift, 0x3, 0, &imx_ccm_lock, share_count);
}
-static inline struct clk *imx_clk_gate2_shared2(const char *name,
+static inline struct clk_hw *imx_clk_hw_gate2_shared2(const char *name,
const char *parent, void __iomem *reg, u8 shift,
unsigned int *share_count)
{
- return clk_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT |
+ return clk_hw_register_gate2(NULL, name, parent, CLK_SET_RATE_PARENT |
CLK_OPS_PARENT_ENABLE, reg, shift, 0x3, 0,
&imx_ccm_lock, share_count);
}
shift, cgr_val, 0, &imx_ccm_lock, NULL);
}
-static inline struct clk *imx_clk_gate3(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate3(const char *name, const char *parent,
void __iomem *reg, u8 shift)
{
- return clk_register_gate(NULL, name, parent,
+ return clk_hw_register_gate(NULL, name, parent,
CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
reg, shift, 0, &imx_ccm_lock);
}
reg, shift, 0, &imx_ccm_lock);
}
-static inline struct clk *imx_clk_gate4(const char *name, const char *parent,
+static inline struct clk_hw *imx_clk_hw_gate4(const char *name, const char *parent,
void __iomem *reg, u8 shift)
{
- return clk_register_gate2(NULL, name, parent,
+ return clk_hw_register_gate2(NULL, name, parent,
CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
reg, shift, 0x3, 0, &imx_ccm_lock, NULL);
}
reg, shift, 0x3, 0, &imx_ccm_lock, NULL);
}
-static inline struct clk *imx_clk_mux(const char *name, void __iomem *reg,
+static inline struct clk_hw *imx_clk_hw_mux(const char *name, void __iomem *reg,
u8 shift, u8 width, const char * const *parents,
int num_parents)
{
- return clk_register_mux(NULL, name, parents, num_parents,
+ return clk_hw_register_mux(NULL, name, parents, num_parents,
CLK_SET_RATE_NO_REPARENT, reg, shift,
width, 0, &imx_ccm_lock);
}
reg, shift, width, 0, &imx_ccm_lock);
}
-struct clk *imx_clk_cpu(const char *name, const char *parent_name,
+struct clk_hw *imx_clk_hw_cpu(const char *name, const char *parent_name,
struct clk *div, struct clk *mux, struct clk *pll,
struct clk *step);
# SPDX-License-Identifier: GPL-2.0-only
-obj-$(CONFIG_INGENIC_CGU_COMMON) += cgu.o
+obj-$(CONFIG_INGENIC_CGU_COMMON) += cgu.o pm.o
obj-$(CONFIG_INGENIC_CGU_JZ4740) += jz4740-cgu.o
obj-$(CONFIG_INGENIC_CGU_JZ4725B) += jz4725b-cgu.o
obj-$(CONFIG_INGENIC_CGU_JZ4770) += jz4770-cgu.o
div_reg = readl(cgu->base + clk_info->div.reg);
div = (div_reg >> clk_info->div.shift) &
GENMASK(clk_info->div.bits - 1, 0);
- div += 1;
- div *= clk_info->div.div;
+
+ if (clk_info->div.div_table)
+ div = clk_info->div.div_table[div];
+ else
+ div = (div + 1) * clk_info->div.div;
rate /= div;
} else if (clk_info->type & CGU_CLK_FIXDIV) {
return rate;
}
+static unsigned int
+ingenic_clk_calc_hw_div(const struct ingenic_cgu_clk_info *clk_info,
+ unsigned int div)
+{
+ unsigned int i;
+
+ for (i = 0; i < (1 << clk_info->div.bits)
+ && clk_info->div.div_table[i]; i++) {
+ if (clk_info->div.div_table[i] >= div)
+ return i;
+ }
+
+ return i - 1;
+}
+
static unsigned
ingenic_clk_calc_div(const struct ingenic_cgu_clk_info *clk_info,
unsigned long parent_rate, unsigned long req_rate)
{
- unsigned div;
+ unsigned int div, hw_div;
/* calculate the divide */
div = DIV_ROUND_UP(parent_rate, req_rate);
- /* and impose hardware constraints */
+ if (clk_info->div.div_table) {
+ hw_div = ingenic_clk_calc_hw_div(clk_info, div);
+
+ return clk_info->div.div_table[hw_div];
+ }
+
+ /* Impose hardware constraints */
div = min_t(unsigned, div, 1 << clk_info->div.bits);
div = max_t(unsigned, div, 1);
const struct ingenic_cgu_clk_info *clk_info;
const unsigned timeout = 100;
unsigned long rate, flags;
- unsigned div, i;
+ unsigned int hw_div, div, i;
u32 reg, mask;
int ret = 0;
if (rate != req_rate)
return -EINVAL;
+ if (clk_info->div.div_table)
+ hw_div = ingenic_clk_calc_hw_div(clk_info, div);
+ else
+ hw_div = ((div / clk_info->div.div) - 1);
+
spin_lock_irqsave(&cgu->lock, flags);
reg = readl(cgu->base + clk_info->div.reg);
/* update the divide */
mask = GENMASK(clk_info->div.bits - 1, 0);
reg &= ~(mask << clk_info->div.shift);
- reg |= ((div / clk_info->div.div) - 1) << clk_info->div.shift;
+ reg |= hw_div << clk_info->div.shift;
/* clear the stop bit */
if (clk_info->div.stop_bit != -1)
#define __DRIVERS_CLK_INGENIC_CGU_H__
#include <linux/bitops.h>
+#include <linux/clk-provider.h>
#include <linux/of.h>
#include <linux/spinlock.h>
* isn't one
* @busy_bit: the index of the busy bit within reg, or -1 if there isn't one
* @stop_bit: the index of the stop bit within reg, or -1 if there isn't one
+ * @div_table: optional table to map the value read from the register to the
+ * actual divider value
*/
struct ingenic_cgu_div_info {
unsigned reg;
s8 ce_bit;
s8 busy_bit;
s8 stop_bit;
+ const u8 *div_table;
};
/**
#include <linux/of.h>
#include <dt-bindings/clock/jz4725b-cgu.h>
#include "cgu.h"
+#include "pm.h"
/* CGU register offsets */
#define CGU_REG_CPCCR 0x00
0x0, 0x1, -1, 0x3,
};
+static const u8 jz4725b_cgu_cpccr_div_table[] = {
+ 1, 2, 3, 4, 6, 8,
+};
+
+static const u8 jz4725b_cgu_pll_half_div_table[] = {
+ 2, 1,
+};
+
static const struct ingenic_cgu_clk_info jz4725b_cgu_clocks[] = {
/* External clocks */
[JZ4725B_CLK_PLL_HALF] = {
"pll half", CGU_CLK_DIV,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 21, 1, 1, -1, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 21, 1, 1, -1, -1, -1,
+ jz4725b_cgu_pll_half_div_table,
+ },
},
[JZ4725B_CLK_CCLK] = {
"cclk", CGU_CLK_DIV,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1,
+ jz4725b_cgu_cpccr_div_table,
+ },
},
[JZ4725B_CLK_HCLK] = {
"hclk", CGU_CLK_DIV,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1,
+ jz4725b_cgu_cpccr_div_table,
+ },
},
[JZ4725B_CLK_PCLK] = {
"pclk", CGU_CLK_DIV,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1,
+ jz4725b_cgu_cpccr_div_table,
+ },
},
[JZ4725B_CLK_MCLK] = {
"mclk", CGU_CLK_DIV,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1,
+ jz4725b_cgu_cpccr_div_table,
+ },
},
[JZ4725B_CLK_IPU] = {
"ipu", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4725B_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 16, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 16, 1, 4, 22, -1, -1,
+ jz4725b_cgu_cpccr_div_table,
+ },
.gate = { CGU_REG_CLKGR, 13 },
},
retval = ingenic_cgu_register_clocks(cgu);
if (retval)
pr_err("%s: failed to register CGU Clocks\n", __func__);
+
+ ingenic_cgu_register_syscore_ops(cgu);
}
CLK_OF_DECLARE(jz4725b_cgu, "ingenic,jz4725b-cgu", jz4725b_cgu_init);
#include <linux/io.h>
#include <linux/of.h>
#include <dt-bindings/clock/jz4740-cgu.h>
-#include <asm/mach-jz4740/clock.h>
#include "cgu.h"
+#include "pm.h"
/* CGU register offsets */
#define CGU_REG_CPCCR 0x00
0x0, 0x1, -1, 0x3,
};
+static const u8 jz4740_cgu_cpccr_div_table[] = {
+ 1, 2, 3, 4, 6, 8, 12, 16, 24, 32,
+};
+
static const struct ingenic_cgu_clk_info jz4740_cgu_clocks[] = {
/* External clocks */
[JZ4740_CLK_CCLK] = {
"cclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1,
+ jz4740_cgu_cpccr_div_table,
+ },
},
[JZ4740_CLK_HCLK] = {
"hclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1,
+ jz4740_cgu_cpccr_div_table,
+ },
},
[JZ4740_CLK_PCLK] = {
"pclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1,
+ jz4740_cgu_cpccr_div_table,
+ },
},
[JZ4740_CLK_MCLK] = {
"mclk", CGU_CLK_DIV,
.parents = { JZ4740_CLK_PLL, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1,
+ jz4740_cgu_cpccr_div_table,
+ },
},
[JZ4740_CLK_LCD] = {
"lcd", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4740_CLK_PLL_HALF, -1, -1, -1 },
- .div = { CGU_REG_CPCCR, 16, 1, 5, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 16, 1, 5, 22, -1, -1,
+ jz4740_cgu_cpccr_div_table,
+ },
.gate = { CGU_REG_CLKGR, 10 },
},
retval = ingenic_cgu_register_clocks(cgu);
if (retval)
pr_err("%s: failed to register CGU Clocks\n", __func__);
-}
-CLK_OF_DECLARE(jz4740_cgu, "ingenic,jz4740-cgu", jz4740_cgu_init);
-
-void jz4740_clock_set_wait_mode(enum jz4740_wait_mode mode)
-{
- uint32_t lcr = readl(cgu->base + CGU_REG_LCR);
-
- switch (mode) {
- case JZ4740_WAIT_MODE_IDLE:
- lcr &= ~LCR_SLEEP;
- break;
-
- case JZ4740_WAIT_MODE_SLEEP:
- lcr |= LCR_SLEEP;
- break;
- }
-
- writel(lcr, cgu->base + CGU_REG_LCR);
-}
-void jz4740_clock_udc_disable_auto_suspend(void)
-{
- uint32_t clkgr = readl(cgu->base + CGU_REG_CLKGR);
-
- clkgr &= ~CLKGR_UDC;
- writel(clkgr, cgu->base + CGU_REG_CLKGR);
-}
-EXPORT_SYMBOL_GPL(jz4740_clock_udc_disable_auto_suspend);
-
-void jz4740_clock_udc_enable_auto_suspend(void)
-{
- uint32_t clkgr = readl(cgu->base + CGU_REG_CLKGR);
-
- clkgr |= CLKGR_UDC;
- writel(clkgr, cgu->base + CGU_REG_CLKGR);
-}
-EXPORT_SYMBOL_GPL(jz4740_clock_udc_enable_auto_suspend);
-
-#define JZ_CLOCK_GATE_UART0 BIT(0)
-#define JZ_CLOCK_GATE_TCU BIT(1)
-#define JZ_CLOCK_GATE_DMAC BIT(12)
-
-void jz4740_clock_suspend(void)
-{
- uint32_t clkgr, cppcr;
-
- clkgr = readl(cgu->base + CGU_REG_CLKGR);
- clkgr |= JZ_CLOCK_GATE_TCU | JZ_CLOCK_GATE_DMAC | JZ_CLOCK_GATE_UART0;
- writel(clkgr, cgu->base + CGU_REG_CLKGR);
-
- cppcr = readl(cgu->base + CGU_REG_CPPCR);
- cppcr &= ~BIT(jz4740_cgu_clocks[JZ4740_CLK_PLL].pll.enable_bit);
- writel(cppcr, cgu->base + CGU_REG_CPPCR);
-}
-
-void jz4740_clock_resume(void)
-{
- uint32_t clkgr, cppcr, stable;
-
- cppcr = readl(cgu->base + CGU_REG_CPPCR);
- cppcr |= BIT(jz4740_cgu_clocks[JZ4740_CLK_PLL].pll.enable_bit);
- writel(cppcr, cgu->base + CGU_REG_CPPCR);
-
- stable = BIT(jz4740_cgu_clocks[JZ4740_CLK_PLL].pll.stable_bit);
- do {
- cppcr = readl(cgu->base + CGU_REG_CPPCR);
- } while (!(cppcr & stable));
-
- clkgr = readl(cgu->base + CGU_REG_CLKGR);
- clkgr &= ~JZ_CLOCK_GATE_TCU;
- clkgr &= ~JZ_CLOCK_GATE_DMAC;
- clkgr &= ~JZ_CLOCK_GATE_UART0;
- writel(clkgr, cgu->base + CGU_REG_CLKGR);
+ ingenic_cgu_register_syscore_ops(cgu);
}
+CLK_OF_DECLARE(jz4740_cgu, "ingenic,jz4740-cgu", jz4740_cgu_init);
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/of.h>
-#include <linux/syscore_ops.h>
#include <dt-bindings/clock/jz4770-cgu.h>
#include "cgu.h"
+#include "pm.h"
/*
* CPM registers offset address definition
#define CGU_REG_MSC2CDR 0xA8
#define CGU_REG_BCHCDR 0xAC
-/* bits within the LCR register */
-#define LCR_LPM BIT(0) /* Low Power Mode */
-
/* bits within the OPCR register */
#define OPCR_SPENDH BIT(5) /* UHC PHY suspend */
0x0, 0x1, -1, 0x2, -1, -1, -1, 0x3,
};
+static const u8 jz4770_cgu_cpccr_div_table[] = {
+ 1, 2, 3, 4, 6, 8, 12,
+};
+
static const struct ingenic_cgu_clk_info jz4770_cgu_clocks[] = {
/* External clocks */
[JZ4770_CLK_CCLK] = {
"cclk", CGU_CLK_DIV,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 0, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
},
[JZ4770_CLK_H0CLK] = {
"h0clk", CGU_CLK_DIV,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 4, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
},
[JZ4770_CLK_H1CLK] = {
"h1clk", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 24, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 24, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
.gate = { CGU_REG_CLKGR1, 7 },
},
[JZ4770_CLK_H2CLK] = {
"h2clk", CGU_CLK_DIV,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 16, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 16, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
},
[JZ4770_CLK_C1CLK] = {
"c1clk", CGU_CLK_DIV | CGU_CLK_GATE,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 12, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
.gate = { CGU_REG_OPCR, 31, true }, // disable CCLK stop on idle
},
[JZ4770_CLK_PCLK] = {
"pclk", CGU_CLK_DIV,
.parents = { JZ4770_CLK_PLL0, },
- .div = { CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1 },
+ .div = {
+ CGU_REG_CPCCR, 8, 1, 4, 22, -1, -1,
+ jz4770_cgu_cpccr_div_table,
+ },
},
/* Those divided clocks can connect to PLL0 or PLL1 */
},
};
-#if IS_ENABLED(CONFIG_PM_SLEEP)
-static int jz4770_cgu_pm_suspend(void)
-{
- u32 val;
-
- val = readl(cgu->base + CGU_REG_LCR);
- writel(val | LCR_LPM, cgu->base + CGU_REG_LCR);
- return 0;
-}
-
-static void jz4770_cgu_pm_resume(void)
-{
- u32 val;
-
- val = readl(cgu->base + CGU_REG_LCR);
- writel(val & ~LCR_LPM, cgu->base + CGU_REG_LCR);
-}
-
-static struct syscore_ops jz4770_cgu_pm_ops = {
- .suspend = jz4770_cgu_pm_suspend,
- .resume = jz4770_cgu_pm_resume,
-};
-#endif /* CONFIG_PM_SLEEP */
-
static void __init jz4770_cgu_init(struct device_node *np)
{
int retval;
if (retval)
pr_err("%s: failed to register CGU Clocks\n", __func__);
-#if IS_ENABLED(CONFIG_PM_SLEEP)
- register_syscore_ops(&jz4770_cgu_pm_ops);
-#endif
+ ingenic_cgu_register_syscore_ops(cgu);
}
/* We only probe via devicetree, no need for a platform driver */
#include <linux/of.h>
#include <dt-bindings/clock/jz4780-cgu.h>
#include "cgu.h"
+#include "pm.h"
/* CGU register offsets */
#define CGU_REG_CLOCKCONTROL 0x00
pr_err("%s: failed to register CGU Clocks\n", __func__);
return;
}
+
+ ingenic_cgu_register_syscore_ops(cgu);
}
CLK_OF_DECLARE(jz4780_cgu, "ingenic,jz4780-cgu", jz4780_cgu_init);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2019 Paul Cercueil <paul@crapouillou.net>
+ */
+
+#include "cgu.h"
+#include "pm.h"
+
+#include <linux/io.h>
+#include <linux/syscore_ops.h>
+
+#define CGU_REG_LCR 0x04
+
+#define LCR_LOW_POWER_MODE BIT(0)
+
+static void __iomem * __maybe_unused ingenic_cgu_base;
+
+static int __maybe_unused ingenic_cgu_pm_suspend(void)
+{
+ u32 val = readl(ingenic_cgu_base + CGU_REG_LCR);
+
+ writel(val | LCR_LOW_POWER_MODE, ingenic_cgu_base + CGU_REG_LCR);
+
+ return 0;
+}
+
+static void __maybe_unused ingenic_cgu_pm_resume(void)
+{
+ u32 val = readl(ingenic_cgu_base + CGU_REG_LCR);
+
+ writel(val & ~LCR_LOW_POWER_MODE, ingenic_cgu_base + CGU_REG_LCR);
+}
+
+static struct syscore_ops __maybe_unused ingenic_cgu_pm_ops = {
+ .suspend = ingenic_cgu_pm_suspend,
+ .resume = ingenic_cgu_pm_resume,
+};
+
+void ingenic_cgu_register_syscore_ops(struct ingenic_cgu *cgu)
+{
+ if (IS_ENABLED(CONFIG_PM_SLEEP)) {
+ ingenic_cgu_base = cgu->base;
+ register_syscore_ops(&ingenic_cgu_pm_ops);
+ }
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2019 Paul Cercueil <paul@crapouillou.net>
+ */
+#ifndef DRIVERS_CLK_INGENIC_PM_H
+#define DRIVERS_CLK_INGENIC_PM_H
+
+struct ingenic_cgu;
+
+void ingenic_cgu_register_syscore_ops(struct ingenic_cgu *cgu);
+
+#endif /* DRIVERS_CLK_INGENIC_PM_H */
This adds the clock driver support over TI System Control Interface.
If you wish to use clock resources from the PMMC firmware, say Y.
Otherwise, say N.
+
+config TI_SCI_CLK_PROBE_FROM_FW
+ bool "Probe available clocks from firmware"
+ depends on TI_SCI_CLK
+ default n
+ help
+ Forces the TI SCI clock driver to probe available clocks from the
+ firmware. By default, only the used clocks are probed from DT.
+ This is mostly only useful for debugging purposes, and will
+ increase the boot time of the device. If you want the clocks probed
+ from firmware, say Y. Otherwise, say N.
#include <linux/slab.h>
#include <linux/soc/ti/ti_sci_protocol.h>
#include <linux/bsearch.h>
+#include <linux/list_sort.h>
#define SCI_CLK_SSC_ENABLE BIT(0)
#define SCI_CLK_ALLOW_FREQ_CHANGE BIT(1)
* @num_parents: Number of parents for this clock
* @provider: Master clock provider
* @flags: Flags for the clock
+ * @node: Link for handling clocks probed via DT
*/
struct sci_clk {
struct clk_hw hw;
u16 dev_id;
- u8 clk_id;
- u8 num_parents;
+ u32 clk_id;
+ u32 num_parents;
struct sci_clk_provider *provider;
u8 flags;
+ struct list_head node;
};
#define to_sci_clk(_hw) container_of(_hw, struct sci_clk, hw)
static u8 sci_clk_get_parent(struct clk_hw *hw)
{
struct sci_clk *clk = to_sci_clk(hw);
- u8 parent_id;
+ u32 parent_id = 0;
int ret;
ret = clk->provider->ops->get_parent(clk->provider->sci, clk->dev_id,
- clk->clk_id, &parent_id);
+ clk->clk_id, (void *)&parent_id);
if (ret) {
dev_err(clk->provider->dev,
"get-parent failed for dev=%d, clk=%d, ret=%d\n",
return 0;
}
- return parent_id - clk->clk_id - 1;
+ parent_id = parent_id - clk->clk_id - 1;
+
+ return (u8)parent_id;
}
/**
int i;
int ret = 0;
- name = kasprintf(GFP_KERNEL, "%s:%d:%d", dev_name(provider->dev),
- sci_clk->dev_id, sci_clk->clk_id);
+ name = kasprintf(GFP_KERNEL, "clk:%d:%d", sci_clk->dev_id,
+ sci_clk->clk_id);
init.name = name;
for (i = 0; i < sci_clk->num_parents; i++) {
char *parent_name;
- parent_name = kasprintf(GFP_KERNEL, "%s:%d:%d",
- dev_name(provider->dev),
+ parent_name = kasprintf(GFP_KERNEL, "clk:%d:%d",
sci_clk->dev_id,
sci_clk->clk_id + 1 + i);
if (!parent_name) {
};
MODULE_DEVICE_TABLE(of, ti_sci_clk_of_match);
-/**
- * ti_sci_clk_probe - Probe function for the TI SCI clock driver
- * @pdev: platform device pointer to be probed
- *
- * Probes the TI SCI clock device. Allocates a new clock provider
- * and registers this to the common clock framework. Also applies
- * any required flags to the identified clocks via clock lists
- * supplied from DT. Returns 0 for success, negative error value
- * for failure.
- */
-static int ti_sci_clk_probe(struct platform_device *pdev)
+#ifdef CONFIG_TI_SCI_CLK_PROBE_FROM_FW
+static int ti_sci_scan_clocks_from_fw(struct sci_clk_provider *provider)
{
- struct device *dev = &pdev->dev;
- struct device_node *np = dev->of_node;
- struct sci_clk_provider *provider;
- const struct ti_sci_handle *handle;
int ret;
int num_clks = 0;
struct sci_clk **clks = NULL;
int max_clks = 0;
int clk_id = 0;
int dev_id = 0;
- u8 num_parents;
+ u32 num_parents = 0;
int gap_size = 0;
-
- handle = devm_ti_sci_get_handle(dev);
- if (IS_ERR(handle))
- return PTR_ERR(handle);
-
- provider = devm_kzalloc(dev, sizeof(*provider), GFP_KERNEL);
- if (!provider)
- return -ENOMEM;
-
- provider->sci = handle;
- provider->ops = &handle->ops.clk_ops;
- provider->dev = dev;
+ struct device *dev = provider->dev;
while (1) {
ret = provider->ops->get_num_parents(provider->sci, dev_id,
- clk_id, &num_parents);
+ clk_id,
+ (void *)&num_parents);
if (ret) {
gap_size++;
if (!clk_id) {
devm_kfree(dev, clks);
+ return 0;
+}
+
+#else
+
+static int _cmp_sci_clk_list(void *priv, struct list_head *a,
+ struct list_head *b)
+{
+ struct sci_clk *ca = container_of(a, struct sci_clk, node);
+ struct sci_clk *cb = container_of(b, struct sci_clk, node);
+
+ return _cmp_sci_clk(ca, &cb);
+}
+
+static int ti_sci_scan_clocks_from_dt(struct sci_clk_provider *provider)
+{
+ struct device *dev = provider->dev;
+ struct device_node *np = NULL;
+ int ret;
+ int index;
+ struct of_phandle_args args;
+ struct list_head clks;
+ struct sci_clk *sci_clk, *prev;
+ int num_clks = 0;
+ int num_parents;
+ int clk_id;
+ const char * const clk_names[] = {
+ "clocks", "assigned-clocks", "assigned-clock-parents", NULL
+ };
+ const char * const *clk_name;
+
+ INIT_LIST_HEAD(&clks);
+
+ clk_name = clk_names;
+
+ while (*clk_name) {
+ np = of_find_node_with_property(np, *clk_name);
+ if (!np) {
+ clk_name++;
+ break;
+ }
+
+ if (!of_device_is_available(np))
+ continue;
+
+ index = 0;
+
+ do {
+ ret = of_parse_phandle_with_args(np, *clk_name,
+ "#clock-cells", index,
+ &args);
+ if (ret)
+ break;
+
+ if (args.args_count == 2 && args.np == dev->of_node) {
+ sci_clk = devm_kzalloc(dev, sizeof(*sci_clk),
+ GFP_KERNEL);
+ if (!sci_clk)
+ return -ENOMEM;
+
+ sci_clk->dev_id = args.args[0];
+ sci_clk->clk_id = args.args[1];
+ sci_clk->provider = provider;
+ provider->ops->get_num_parents(provider->sci,
+ sci_clk->dev_id,
+ sci_clk->clk_id,
+ (void *)&sci_clk->num_parents);
+ list_add_tail(&sci_clk->node, &clks);
+
+ num_clks++;
+
+ num_parents = sci_clk->num_parents;
+ if (num_parents == 1)
+ num_parents = 0;
+
+ /*
+ * Linux kernel has inherent limitation
+ * of 255 clock parents at the moment.
+ * Right now, it is not expected that
+ * any mux clock from sci-clk driver
+ * would exceed that limit either, but
+ * the ABI basically provides that
+ * possibility. Print out a warning if
+ * this happens for any clock.
+ */
+ if (num_parents >= 255) {
+ dev_warn(dev, "too many parents for dev=%d, clk=%d (%d), cropping to 255.\n",
+ sci_clk->dev_id,
+ sci_clk->clk_id, num_parents);
+ num_parents = 255;
+ }
+
+ clk_id = args.args[1] + 1;
+
+ while (num_parents--) {
+ sci_clk = devm_kzalloc(dev,
+ sizeof(*sci_clk),
+ GFP_KERNEL);
+ if (!sci_clk)
+ return -ENOMEM;
+ sci_clk->dev_id = args.args[0];
+ sci_clk->clk_id = clk_id++;
+ sci_clk->provider = provider;
+ list_add_tail(&sci_clk->node, &clks);
+
+ num_clks++;
+ }
+ }
+
+ index++;
+ } while (args.np);
+ }
+
+ list_sort(NULL, &clks, _cmp_sci_clk_list);
+
+ provider->clocks = devm_kmalloc_array(dev, num_clks, sizeof(sci_clk),
+ GFP_KERNEL);
+ if (!provider->clocks)
+ return -ENOMEM;
+
+ num_clks = 0;
+ prev = NULL;
+
+ list_for_each_entry(sci_clk, &clks, node) {
+ if (prev && prev->dev_id == sci_clk->dev_id &&
+ prev->clk_id == sci_clk->clk_id)
+ continue;
+
+ provider->clocks[num_clks++] = sci_clk;
+ prev = sci_clk;
+ }
+
+ provider->num_clocks = num_clks;
+
+ return 0;
+}
+#endif
+
+/**
+ * ti_sci_clk_probe - Probe function for the TI SCI clock driver
+ * @pdev: platform device pointer to be probed
+ *
+ * Probes the TI SCI clock device. Allocates a new clock provider
+ * and registers this to the common clock framework. Also applies
+ * any required flags to the identified clocks via clock lists
+ * supplied from DT. Returns 0 for success, negative error value
+ * for failure.
+ */
+static int ti_sci_clk_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ struct sci_clk_provider *provider;
+ const struct ti_sci_handle *handle;
+ int ret;
+
+ handle = devm_ti_sci_get_handle(dev);
+ if (IS_ERR(handle))
+ return PTR_ERR(handle);
+
+ provider = devm_kzalloc(dev, sizeof(*provider), GFP_KERNEL);
+ if (!provider)
+ return -ENOMEM;
+
+ provider->sci = handle;
+ provider->ops = &handle->ops.clk_ops;
+ provider->dev = dev;
+
+#ifdef CONFIG_TI_SCI_CLK_PROBE_FROM_FW
+ ret = ti_sci_scan_clocks_from_fw(provider);
+ if (ret) {
+ dev_err(dev, "scan clocks from FW failed: %d\n", ret);
+ return ret;
+ }
+#else
+ ret = ti_sci_scan_clocks_from_dt(provider);
+ if (ret) {
+ dev_err(dev, "scan clocks from DT failed: %d\n", ret);
+ return ret;
+ }
+#endif
+
ret = ti_sci_init_clocks(provider);
if (ret) {
pr_err("ti-sci-init-clocks failed.\n");
help
This driver supports MediaTek MT8516 clocks.
+config COMMON_CLK_MT8516_AUDSYS
+ bool "Clock driver for MediaTek MT8516 audsys"
+ depends on COMMON_CLK_MT8516
+ help
+ This driver supports MediaTek MT8516 audsys clocks.
+
endmenu
obj-$(CONFIG_COMMON_CLK_MT8183_VDECSYS) += clk-mt8183-vdec.o
obj-$(CONFIG_COMMON_CLK_MT8183_VENCSYS) += clk-mt8183-venc.o
obj-$(CONFIG_COMMON_CLK_MT8516) += clk-mt8516.o
+obj-$(CONFIG_COMMON_CLK_MT8516_AUDSYS) += clk-mt8516-aud.o
"syspll_d5"
};
-static const char * const sspm_parents[] = {
- "clk26m",
- "univpll_d2_d4",
- "syspll_d2_d2",
- "univpll_d2_d2",
- "syspll_d3"
-};
-
static const char * const dpi0_parents[] = {
"clk26m",
"tvdpll_d2",
MUX_GATE_CLR_SET_UPD(CLK_TOP_MUX_ATB, "atb_sel",
atb_parents, 0xa0,
0xa4, 0xa8, 0, 2, 7, 0x004, 24),
- MUX_GATE_CLR_SET_UPD(CLK_TOP_MUX_SSPM, "sspm_sel",
- sspm_parents, 0xa0,
- 0xa4, 0xa8, 8, 3, 15, 0x004, 25),
MUX_GATE_CLR_SET_UPD(CLK_TOP_MUX_DPI0, "dpi0_sel",
dpi0_parents, 0xa0,
0xa4, 0xa8, 16, 4, 23, 0x004, 26),
"fufs_sel", 13),
GATE_INFRA2(CLK_INFRA_MD32_BCLK, "infra_md32_bclk",
"axi_sel", 14),
- GATE_INFRA2(CLK_INFRA_SSPM, "infra_sspm",
- "sspm_sel", 15),
GATE_INFRA2(CLK_INFRA_UNIPRO_MBIST, "infra_unipro_mbist",
"axi_sel", 16),
- GATE_INFRA2(CLK_INFRA_SSPM_BUS_HCLK, "infra_sspm_bus_hclk",
- "axi_sel", 17),
GATE_INFRA2(CLK_INFRA_I2C5, "infra_i2c5",
"i2c_sel", 18),
GATE_INFRA2(CLK_INFRA_I2C5_ARBITER, "infra_i2c5_arbiter",
"msdc50_0_sel", 1),
GATE_INFRA3(CLK_INFRA_MSDC2_SELF, "infra_msdc2_self",
"msdc50_0_sel", 2),
- GATE_INFRA3(CLK_INFRA_SSPM_26M_SELF, "infra_sspm_26m_self",
- "f_f26m_ck", 3),
- GATE_INFRA3(CLK_INFRA_SSPM_32K_SELF, "infra_sspm_32k_self",
- "f_f26m_ck", 4),
GATE_INFRA3(CLK_INFRA_UFS_AXI, "infra_ufs_axi",
"axi_sel", 5),
GATE_INFRA3(CLK_INFRA_I2C6, "infra_i2c6",
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: James Liao <jamesjj.liao@mediatek.com>
+ * Fabien Parent <fparent@baylibre.com>
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+#include "clk-mtk.h"
+#include "clk-gate.h"
+
+#include <dt-bindings/clock/mt8516-clk.h>
+
+static const struct mtk_gate_regs aud_cg_regs = {
+ .set_ofs = 0x0,
+ .clr_ofs = 0x0,
+ .sta_ofs = 0x0,
+};
+
+#define GATE_AUD(_id, _name, _parent, _shift) { \
+ .id = _id, \
+ .name = _name, \
+ .parent_name = _parent, \
+ .regs = &aud_cg_regs, \
+ .shift = _shift, \
+ .ops = &mtk_clk_gate_ops_no_setclr, \
+ }
+
+static const struct mtk_gate aud_clks[] __initconst = {
+ GATE_AUD(CLK_AUD_AFE, "aud_afe", "clk26m_ck", 2),
+ GATE_AUD(CLK_AUD_I2S, "aud_i2s", "i2s_infra_bck", 6),
+ GATE_AUD(CLK_AUD_22M, "aud_22m", "rg_aud_engen1", 8),
+ GATE_AUD(CLK_AUD_24M, "aud_24m", "rg_aud_engen2", 9),
+ GATE_AUD(CLK_AUD_INTDIR, "aud_intdir", "rg_aud_spdif_in", 15),
+ GATE_AUD(CLK_AUD_APLL2_TUNER, "aud_apll2_tuner", "rg_aud_engen2", 18),
+ GATE_AUD(CLK_AUD_APLL_TUNER, "aud_apll_tuner", "rg_aud_engen1", 19),
+ GATE_AUD(CLK_AUD_HDMI, "aud_hdmi", "apll12_div4", 20),
+ GATE_AUD(CLK_AUD_SPDF, "aud_spdf", "apll12_div6", 21),
+ GATE_AUD(CLK_AUD_ADC, "aud_adc", "aud_afe", 24),
+ GATE_AUD(CLK_AUD_DAC, "aud_dac", "aud_afe", 25),
+ GATE_AUD(CLK_AUD_DAC_PREDIS, "aud_dac_predis", "aud_afe", 26),
+ GATE_AUD(CLK_AUD_TML, "aud_tml", "aud_afe", 27),
+};
+
+static void __init mtk_audsys_init(struct device_node *node)
+{
+ struct clk_onecell_data *clk_data;
+ int r;
+
+ clk_data = mtk_alloc_clk_data(CLK_AUD_NR_CLK);
+
+ mtk_clk_register_gates(node, aud_clks, ARRAY_SIZE(aud_clks), clk_data);
+
+ r = of_clk_add_provider(node, of_clk_src_onecell_get, clk_data);
+ if (r)
+ pr_err("%s(): could not register clock provider: %d\n",
+ __func__, r);
+
+}
+CLK_OF_DECLARE(mtk_audsys, "mediatek,mt8516-audsys", mtk_audsys_init);
"nfi1x_ck"
};
-static const char * const ddrphycfg_parents[] __initconst = {
- "clk26m_ck",
- "mainpll_d16"
-};
-
static const char * const usb_78m_parents[] __initconst = {
"clk_null",
"clk26m_ck",
.shift = 16,
.width = 9,
},
- .ssen = {
- .reg_off = HHI_MPLL_CNTL,
- .shift = 25,
- .width = 1,
- },
.misc = {
.reg_off = HHI_PLL_TOP_MISC,
.shift = 0,
.shift = 16,
.width = 9,
},
+ .ssen = {
+ .reg_off = HHI_MPLL_CNTL,
+ .shift = 25,
+ .width = 1,
+ },
.misc = {
.reg_off = HHI_PLL_TOP_MISC,
.shift = 2,
else
__acquire(mpll->lock);
- /* Enable and set the fractional part */
+ /* Set the fractional part */
meson_parm_write(clk->map, &mpll->sdm, sdm);
- meson_parm_write(clk->map, &mpll->sdm_en, 1);
-
- /* Set additional fractional part enable if required */
- if (MESON_PARM_APPLICABLE(&mpll->ssen))
- meson_parm_write(clk->map, &mpll->ssen, 1);
/* Set the integer divider part */
meson_parm_write(clk->map, &mpll->n2, n2);
- /* Set the magic misc bit if required */
- if (MESON_PARM_APPLICABLE(&mpll->misc))
- meson_parm_write(clk->map, &mpll->misc, 1);
-
if (mpll->lock)
spin_unlock_irqrestore(mpll->lock, flags);
else
return 0;
}
+static void mpll_init(struct clk_hw *hw)
+{
+ struct clk_regmap *clk = to_clk_regmap(hw);
+ struct meson_clk_mpll_data *mpll = meson_clk_mpll_data(clk);
+
+ if (mpll->init_count)
+ regmap_multi_reg_write(clk->map, mpll->init_regs,
+ mpll->init_count);
+
+ /* Enable the fractional part */
+ meson_parm_write(clk->map, &mpll->sdm_en, 1);
+
+ /* Set spread spectrum if possible */
+ if (MESON_PARM_APPLICABLE(&mpll->ssen)) {
+ unsigned int ss =
+ mpll->flags & CLK_MESON_MPLL_SPREAD_SPECTRUM ? 1 : 0;
+ meson_parm_write(clk->map, &mpll->ssen, ss);
+ }
+
+ /* Set the magic misc bit if required */
+ if (MESON_PARM_APPLICABLE(&mpll->misc))
+ meson_parm_write(clk->map, &mpll->misc, 1);
+}
+
const struct clk_ops meson_clk_mpll_ro_ops = {
.recalc_rate = mpll_recalc_rate,
.round_rate = mpll_round_rate,
.recalc_rate = mpll_recalc_rate,
.round_rate = mpll_round_rate,
.set_rate = mpll_set_rate,
+ .init = mpll_init,
};
EXPORT_SYMBOL_GPL(meson_clk_mpll_ops);
struct parm n2;
struct parm ssen;
struct parm misc;
+ const struct reg_sequence *init_regs;
+ unsigned int init_count;
spinlock_t *lock;
u8 flags;
};
#define CLK_MESON_MPLL_ROUND_CLOSEST BIT(0)
+#define CLK_MESON_MPLL_SPREAD_SPECTRUM BIT(1)
extern const struct clk_ops meson_clk_mpll_ro_ops;
extern const struct clk_ops meson_clk_mpll_ops;
},
};
+static struct clk_regmap g12b_sys1_pll_dco = {
+ .data = &(struct meson_clk_pll_data){
+ .en = {
+ .reg_off = HHI_SYS1_PLL_CNTL0,
+ .shift = 28,
+ .width = 1,
+ },
+ .m = {
+ .reg_off = HHI_SYS1_PLL_CNTL0,
+ .shift = 0,
+ .width = 8,
+ },
+ .n = {
+ .reg_off = HHI_SYS1_PLL_CNTL0,
+ .shift = 10,
+ .width = 5,
+ },
+ .l = {
+ .reg_off = HHI_SYS1_PLL_CNTL0,
+ .shift = 31,
+ .width = 1,
+ },
+ .rst = {
+ .reg_off = HHI_SYS1_PLL_CNTL0,
+ .shift = 29,
+ .width = 1,
+ },
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "sys1_pll_dco",
+ .ops = &meson_clk_pll_ro_ops,
+ .parent_names = (const char *[]){ IN_PREFIX "xtal" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_regmap g12b_sys1_pll = {
+ .data = &(struct clk_regmap_div_data){
+ .offset = HHI_SYS1_PLL_CNTL0,
+ .shift = 16,
+ .width = 3,
+ .flags = CLK_DIVIDER_POWER_OF_TWO,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "sys1_pll",
+ .ops = &clk_regmap_divider_ro_ops,
+ .parent_names = (const char *[]){ "sys1_pll_dco" },
+ .num_parents = 1,
+ },
+};
+
static struct clk_regmap g12a_sys_pll_div16_en = {
.data = &(struct clk_regmap_gate_data){
.offset = HHI_SYS_CPU_CLK_CNTL1,
},
};
+static struct clk_regmap g12b_sys1_pll_div16_en = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .bit_idx = 24,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "sys1_pll_div16_en",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "sys1_pll" },
+ .num_parents = 1,
+ /*
+ * This clock is used to debug the sys_pll range
+ * Linux should not change it at runtime
+ */
+ },
+};
+
static struct clk_fixed_factor g12a_sys_pll_div16 = {
.mult = 1,
.div = 16,
},
};
+static struct clk_fixed_factor g12b_sys1_pll_div16 = {
+ .mult = 1,
+ .div = 16,
+ .hw.init = &(struct clk_init_data){
+ .name = "sys1_pll_div16",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "sys1_pll_div16_en" },
+ .num_parents = 1,
+ },
+};
+
/* Datasheet names this field as "premux0" */
static struct clk_regmap g12a_cpu_clk_premux0 = {
.data = &(struct clk_regmap_mux_data){
},
};
+/* Datasheet names this field as "Final_mux_sel" */
+static struct clk_regmap g12b_cpu_clk = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPU_CLK_CNTL0,
+ .mask = 0x1,
+ .shift = 11,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpu_clk",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpu_clk_dyn",
+ "sys1_pll" },
+ .num_parents = 2,
+ },
+};
+
+/* Datasheet names this field as "premux0" */
+static struct clk_regmap g12b_cpub_clk_premux0 = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x3,
+ .shift = 0,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn0_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ IN_PREFIX "xtal",
+ "fclk_div2",
+ "fclk_div3" },
+ .num_parents = 3,
+ },
+};
+
+/* Datasheet names this field as "mux0_divn_tcnt" */
+static struct clk_regmap g12b_cpub_clk_mux0_div = {
+ .data = &(struct clk_regmap_div_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .shift = 4,
+ .width = 6,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn0_div",
+ .ops = &clk_regmap_divider_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn0_sel" },
+ .num_parents = 1,
+ },
+};
+
+/* Datasheet names this field as "postmux0" */
+static struct clk_regmap g12b_cpub_clk_postmux0 = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x1,
+ .shift = 2,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn0",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn0_sel",
+ "cpub_clk_dyn0_div" },
+ .num_parents = 2,
+ },
+};
+
+/* Datasheet names this field as "premux1" */
+static struct clk_regmap g12b_cpub_clk_premux1 = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x3,
+ .shift = 16,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn1_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ IN_PREFIX "xtal",
+ "fclk_div2",
+ "fclk_div3" },
+ .num_parents = 3,
+ },
+};
+
+/* Datasheet names this field as "Mux1_divn_tcnt" */
+static struct clk_regmap g12b_cpub_clk_mux1_div = {
+ .data = &(struct clk_regmap_div_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .shift = 20,
+ .width = 6,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn1_div",
+ .ops = &clk_regmap_divider_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn1_sel" },
+ .num_parents = 1,
+ },
+};
+
+/* Datasheet names this field as "postmux1" */
+static struct clk_regmap g12b_cpub_clk_postmux1 = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x1,
+ .shift = 18,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn1",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn1_sel",
+ "cpub_clk_dyn1_div" },
+ .num_parents = 2,
+ },
+};
+
+/* Datasheet names this field as "Final_dyn_mux_sel" */
+static struct clk_regmap g12b_cpub_clk_dyn = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x1,
+ .shift = 10,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_dyn",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn0",
+ "cpub_clk_dyn1" },
+ .num_parents = 2,
+ },
+};
+
+/* Datasheet names this field as "Final_mux_sel" */
+static struct clk_regmap g12b_cpub_clk = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL,
+ .mask = 0x1,
+ .shift = 11,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_dyn",
+ "sys_pll" },
+ .num_parents = 2,
+ },
+};
+
static struct clk_regmap g12a_cpu_clk_div16_en = {
.data = &(struct clk_regmap_gate_data){
.offset = HHI_SYS_CPU_CLK_CNTL1,
},
};
+static struct clk_regmap g12b_cpub_clk_div16_en = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .bit_idx = 1,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cpub_clk_div16_en",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ /*
+ * This clock is used to debug the cpu_clk range
+ * Linux should not change it at runtime
+ */
+ },
+};
+
static struct clk_fixed_factor g12a_cpu_clk_div16 = {
.mult = 1,
.div = 16,
},
};
+static struct clk_fixed_factor g12b_cpub_clk_div16 = {
+ .mult = 1,
+ .div = 16,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div16",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk_div16_en" },
+ .num_parents = 1,
+ },
+};
+
static struct clk_regmap g12a_cpu_clk_apb_div = {
.data = &(struct clk_regmap_div_data){
.offset = HHI_SYS_CPU_CLK_CNTL1,
},
};
-static struct clk_regmap g12a_cpu_clk_trace = {
+static struct clk_regmap g12a_cpu_clk_trace = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPU_CLK_CNTL1,
+ .bit_idx = 23,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cpu_clk_trace",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "cpu_clk_trace_div" },
+ .num_parents = 1,
+ /*
+ * This clock is set by the ROM monitor code,
+ * Linux should not change it at runtime
+ */
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div2 = {
+ .mult = 1,
+ .div = 2,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div2",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div3 = {
+ .mult = 1,
+ .div = 3,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div3",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div4 = {
+ .mult = 1,
+ .div = 4,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div4",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div5 = {
+ .mult = 1,
+ .div = 5,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div5",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div6 = {
+ .mult = 1,
+ .div = 6,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div6",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div7 = {
+ .mult = 1,
+ .div = 7,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div7",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_fixed_factor g12b_cpub_clk_div8 = {
+ .mult = 1,
+ .div = 8,
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_div8",
+ .ops = &clk_fixed_factor_ops,
+ .parent_names = (const char *[]){ "cpub_clk" },
+ .num_parents = 1,
+ },
+};
+
+static u32 mux_table_cpub[] = { 1, 2, 3, 4, 5, 6, 7 };
+static struct clk_regmap g12b_cpub_clk_apb_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .mask = 7,
+ .shift = 3,
+ .table = mux_table_cpub,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_apb_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_div2",
+ "cpub_clk_div3",
+ "cpub_clk_div4",
+ "cpub_clk_div5",
+ "cpub_clk_div6",
+ "cpub_clk_div7",
+ "cpub_clk_div8" },
+ .num_parents = 7,
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_apb = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .bit_idx = 16,
+ .flags = CLK_GATE_SET_TO_DISABLE,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cpub_clk_apb",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_apb_sel" },
+ .num_parents = 1,
+ /*
+ * This clock is set by the ROM monitor code,
+ * Linux should not change it at runtime
+ */
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_atb_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .mask = 7,
+ .shift = 6,
+ .table = mux_table_cpub,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_atb_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_div2",
+ "cpub_clk_div3",
+ "cpub_clk_div4",
+ "cpub_clk_div5",
+ "cpub_clk_div6",
+ "cpub_clk_div7",
+ "cpub_clk_div8" },
+ .num_parents = 7,
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_atb = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .bit_idx = 17,
+ .flags = CLK_GATE_SET_TO_DISABLE,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cpub_clk_atb",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_atb_sel" },
+ .num_parents = 1,
+ /*
+ * This clock is set by the ROM monitor code,
+ * Linux should not change it at runtime
+ */
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_axi_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .mask = 7,
+ .shift = 9,
+ .table = mux_table_cpub,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_axi_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_div2",
+ "cpub_clk_div3",
+ "cpub_clk_div4",
+ "cpub_clk_div5",
+ "cpub_clk_div6",
+ "cpub_clk_div7",
+ "cpub_clk_div8" },
+ .num_parents = 7,
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_axi = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .bit_idx = 18,
+ .flags = CLK_GATE_SET_TO_DISABLE,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cpub_clk_axi",
+ .ops = &clk_regmap_gate_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_axi_sel" },
+ .num_parents = 1,
+ /*
+ * This clock is set by the ROM monitor code,
+ * Linux should not change it at runtime
+ */
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_trace_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
+ .mask = 7,
+ .shift = 20,
+ .table = mux_table_cpub,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cpub_clk_trace_sel",
+ .ops = &clk_regmap_mux_ro_ops,
+ .parent_names = (const char *[]){ "cpub_clk_div2",
+ "cpub_clk_div3",
+ "cpub_clk_div4",
+ "cpub_clk_div5",
+ "cpub_clk_div6",
+ "cpub_clk_div7",
+ "cpub_clk_div8" },
+ .num_parents = 7,
+ },
+};
+
+static struct clk_regmap g12b_cpub_clk_trace = {
.data = &(struct clk_regmap_gate_data){
- .offset = HHI_SYS_CPU_CLK_CNTL1,
+ .offset = HHI_SYS_CPUB_CLK_CNTL1,
.bit_idx = 23,
+ .flags = CLK_GATE_SET_TO_DISABLE,
},
.hw.init = &(struct clk_init_data) {
- .name = "cpu_clk_trace",
+ .name = "cpub_clk_trace",
.ops = &clk_regmap_gate_ro_ops,
- .parent_names = (const char *[]){ "cpu_clk_trace_div" },
+ .parent_names = (const char *[]){ "cpub_clk_trace_sel" },
.num_parents = 1,
/*
* This clock is set by the ROM monitor code,
.ops = &clk_regmap_gate_ops,
.parent_names = (const char *[]){ "fclk_div3_div" },
.num_parents = 1,
+ /*
+ * This clock is used by the resident firmware and is required
+ * by the platform to operate correctly.
+ * Until the following condition are met, we need this clock to
+ * be marked as critical:
+ * a) Mark the clock used by a firmware resource, if possible
+ * b) CCF has a clock hand-off mechanism to make the sure the
+ * clock stays on until the proper driver comes along
+ */
+ .flags = CLK_IS_CRITICAL,
},
};
},
};
+static const struct reg_sequence g12a_mpll0_init_regs[] = {
+ { .reg = HHI_MPLL_CNTL2, .def = 0x40000033 },
+};
+
static struct clk_regmap g12a_mpll0_div = {
.data = &(struct meson_clk_mpll_data){
.sdm = {
.width = 1,
},
.lock = &meson_clk_lock,
+ .init_regs = g12a_mpll0_init_regs,
+ .init_count = ARRAY_SIZE(g12a_mpll0_init_regs),
},
.hw.init = &(struct clk_init_data){
.name = "mpll0_div",
},
};
+static const struct reg_sequence g12a_mpll1_init_regs[] = {
+ { .reg = HHI_MPLL_CNTL4, .def = 0x40000033 },
+};
+
static struct clk_regmap g12a_mpll1_div = {
.data = &(struct meson_clk_mpll_data){
.sdm = {
.width = 1,
},
.lock = &meson_clk_lock,
+ .init_regs = g12a_mpll1_init_regs,
+ .init_count = ARRAY_SIZE(g12a_mpll1_init_regs),
},
.hw.init = &(struct clk_init_data){
.name = "mpll1_div",
},
};
+static const struct reg_sequence g12a_mpll2_init_regs[] = {
+ { .reg = HHI_MPLL_CNTL6, .def = 0x40000033 },
+};
+
static struct clk_regmap g12a_mpll2_div = {
.data = &(struct meson_clk_mpll_data){
.sdm = {
.width = 1,
},
.lock = &meson_clk_lock,
+ .init_regs = g12a_mpll2_init_regs,
+ .init_count = ARRAY_SIZE(g12a_mpll2_init_regs),
},
.hw.init = &(struct clk_init_data){
.name = "mpll2_div",
},
};
+static const struct reg_sequence g12a_mpll3_init_regs[] = {
+ { .reg = HHI_MPLL_CNTL8, .def = 0x40000033 },
+};
+
static struct clk_regmap g12a_mpll3_div = {
.data = &(struct meson_clk_mpll_data){
.sdm = {
.width = 1,
},
.lock = &meson_clk_lock,
+ .init_regs = g12a_mpll3_init_regs,
+ .init_count = ARRAY_SIZE(g12a_mpll3_init_regs),
},
.hw.init = &(struct clk_init_data){
.name = "mpll3_div",
},
};
+static struct clk_regmap g12a_ts_div = {
+ .data = &(struct clk_regmap_div_data){
+ .offset = HHI_TS_CLK_CNTL,
+ .shift = 0,
+ .width = 8,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "ts_div",
+ .ops = &clk_regmap_divider_ro_ops,
+ .parent_names = (const char *[]){ "xtal" },
+ .num_parents = 1,
+ },
+};
+
+static struct clk_regmap g12a_ts = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_TS_CLK_CNTL,
+ .bit_idx = 8,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "ts",
+ .ops = &clk_regmap_gate_ops,
+ .parent_names = (const char *[]){ "ts_div" },
+ .num_parents = 1,
+ },
+};
+
/* Everything Else (EE) domain gates */
static MESON_GATE(g12a_ddr, HHI_GCLK_MPEG0, 0);
static MESON_GATE(g12a_dos, HHI_GCLK_MPEG0, 1);
[CLKID_VDEC_HEVCF_SEL] = &g12a_vdec_hevcf_sel.hw,
[CLKID_VDEC_HEVCF_DIV] = &g12a_vdec_hevcf_div.hw,
[CLKID_VDEC_HEVCF] = &g12a_vdec_hevcf.hw,
+ [CLKID_TS_DIV] = &g12a_ts_div.hw,
+ [CLKID_TS] = &g12a_ts.hw,
+ [NR_CLKS] = NULL,
+ },
+ .num = NR_CLKS,
+};
+
+static struct clk_hw_onecell_data g12b_hw_onecell_data = {
+ .hws = {
+ [CLKID_SYS_PLL] = &g12a_sys_pll.hw,
+ [CLKID_FIXED_PLL] = &g12a_fixed_pll.hw,
+ [CLKID_FCLK_DIV2] = &g12a_fclk_div2.hw,
+ [CLKID_FCLK_DIV3] = &g12a_fclk_div3.hw,
+ [CLKID_FCLK_DIV4] = &g12a_fclk_div4.hw,
+ [CLKID_FCLK_DIV5] = &g12a_fclk_div5.hw,
+ [CLKID_FCLK_DIV7] = &g12a_fclk_div7.hw,
+ [CLKID_FCLK_DIV2P5] = &g12a_fclk_div2p5.hw,
+ [CLKID_GP0_PLL] = &g12a_gp0_pll.hw,
+ [CLKID_MPEG_SEL] = &g12a_mpeg_clk_sel.hw,
+ [CLKID_MPEG_DIV] = &g12a_mpeg_clk_div.hw,
+ [CLKID_CLK81] = &g12a_clk81.hw,
+ [CLKID_MPLL0] = &g12a_mpll0.hw,
+ [CLKID_MPLL1] = &g12a_mpll1.hw,
+ [CLKID_MPLL2] = &g12a_mpll2.hw,
+ [CLKID_MPLL3] = &g12a_mpll3.hw,
+ [CLKID_DDR] = &g12a_ddr.hw,
+ [CLKID_DOS] = &g12a_dos.hw,
+ [CLKID_AUDIO_LOCKER] = &g12a_audio_locker.hw,
+ [CLKID_MIPI_DSI_HOST] = &g12a_mipi_dsi_host.hw,
+ [CLKID_ETH_PHY] = &g12a_eth_phy.hw,
+ [CLKID_ISA] = &g12a_isa.hw,
+ [CLKID_PL301] = &g12a_pl301.hw,
+ [CLKID_PERIPHS] = &g12a_periphs.hw,
+ [CLKID_SPICC0] = &g12a_spicc_0.hw,
+ [CLKID_I2C] = &g12a_i2c.hw,
+ [CLKID_SANA] = &g12a_sana.hw,
+ [CLKID_SD] = &g12a_sd.hw,
+ [CLKID_RNG0] = &g12a_rng0.hw,
+ [CLKID_UART0] = &g12a_uart0.hw,
+ [CLKID_SPICC1] = &g12a_spicc_1.hw,
+ [CLKID_HIU_IFACE] = &g12a_hiu_reg.hw,
+ [CLKID_MIPI_DSI_PHY] = &g12a_mipi_dsi_phy.hw,
+ [CLKID_ASSIST_MISC] = &g12a_assist_misc.hw,
+ [CLKID_SD_EMMC_A] = &g12a_emmc_a.hw,
+ [CLKID_SD_EMMC_B] = &g12a_emmc_b.hw,
+ [CLKID_SD_EMMC_C] = &g12a_emmc_c.hw,
+ [CLKID_AUDIO_CODEC] = &g12a_audio_codec.hw,
+ [CLKID_AUDIO] = &g12a_audio.hw,
+ [CLKID_ETH] = &g12a_eth_core.hw,
+ [CLKID_DEMUX] = &g12a_demux.hw,
+ [CLKID_AUDIO_IFIFO] = &g12a_audio_ififo.hw,
+ [CLKID_ADC] = &g12a_adc.hw,
+ [CLKID_UART1] = &g12a_uart1.hw,
+ [CLKID_G2D] = &g12a_g2d.hw,
+ [CLKID_RESET] = &g12a_reset.hw,
+ [CLKID_PCIE_COMB] = &g12a_pcie_comb.hw,
+ [CLKID_PARSER] = &g12a_parser.hw,
+ [CLKID_USB] = &g12a_usb_general.hw,
+ [CLKID_PCIE_PHY] = &g12a_pcie_phy.hw,
+ [CLKID_AHB_ARB0] = &g12a_ahb_arb0.hw,
+ [CLKID_AHB_DATA_BUS] = &g12a_ahb_data_bus.hw,
+ [CLKID_AHB_CTRL_BUS] = &g12a_ahb_ctrl_bus.hw,
+ [CLKID_HTX_HDCP22] = &g12a_htx_hdcp22.hw,
+ [CLKID_HTX_PCLK] = &g12a_htx_pclk.hw,
+ [CLKID_BT656] = &g12a_bt656.hw,
+ [CLKID_USB1_DDR_BRIDGE] = &g12a_usb1_to_ddr.hw,
+ [CLKID_MMC_PCLK] = &g12a_mmc_pclk.hw,
+ [CLKID_UART2] = &g12a_uart2.hw,
+ [CLKID_VPU_INTR] = &g12a_vpu_intr.hw,
+ [CLKID_GIC] = &g12a_gic.hw,
+ [CLKID_SD_EMMC_A_CLK0_SEL] = &g12a_sd_emmc_a_clk0_sel.hw,
+ [CLKID_SD_EMMC_A_CLK0_DIV] = &g12a_sd_emmc_a_clk0_div.hw,
+ [CLKID_SD_EMMC_A_CLK0] = &g12a_sd_emmc_a_clk0.hw,
+ [CLKID_SD_EMMC_B_CLK0_SEL] = &g12a_sd_emmc_b_clk0_sel.hw,
+ [CLKID_SD_EMMC_B_CLK0_DIV] = &g12a_sd_emmc_b_clk0_div.hw,
+ [CLKID_SD_EMMC_B_CLK0] = &g12a_sd_emmc_b_clk0.hw,
+ [CLKID_SD_EMMC_C_CLK0_SEL] = &g12a_sd_emmc_c_clk0_sel.hw,
+ [CLKID_SD_EMMC_C_CLK0_DIV] = &g12a_sd_emmc_c_clk0_div.hw,
+ [CLKID_SD_EMMC_C_CLK0] = &g12a_sd_emmc_c_clk0.hw,
+ [CLKID_MPLL0_DIV] = &g12a_mpll0_div.hw,
+ [CLKID_MPLL1_DIV] = &g12a_mpll1_div.hw,
+ [CLKID_MPLL2_DIV] = &g12a_mpll2_div.hw,
+ [CLKID_MPLL3_DIV] = &g12a_mpll3_div.hw,
+ [CLKID_FCLK_DIV2_DIV] = &g12a_fclk_div2_div.hw,
+ [CLKID_FCLK_DIV3_DIV] = &g12a_fclk_div3_div.hw,
+ [CLKID_FCLK_DIV4_DIV] = &g12a_fclk_div4_div.hw,
+ [CLKID_FCLK_DIV5_DIV] = &g12a_fclk_div5_div.hw,
+ [CLKID_FCLK_DIV7_DIV] = &g12a_fclk_div7_div.hw,
+ [CLKID_FCLK_DIV2P5_DIV] = &g12a_fclk_div2p5_div.hw,
+ [CLKID_HIFI_PLL] = &g12a_hifi_pll.hw,
+ [CLKID_VCLK2_VENCI0] = &g12a_vclk2_venci0.hw,
+ [CLKID_VCLK2_VENCI1] = &g12a_vclk2_venci1.hw,
+ [CLKID_VCLK2_VENCP0] = &g12a_vclk2_vencp0.hw,
+ [CLKID_VCLK2_VENCP1] = &g12a_vclk2_vencp1.hw,
+ [CLKID_VCLK2_VENCT0] = &g12a_vclk2_venct0.hw,
+ [CLKID_VCLK2_VENCT1] = &g12a_vclk2_venct1.hw,
+ [CLKID_VCLK2_OTHER] = &g12a_vclk2_other.hw,
+ [CLKID_VCLK2_ENCI] = &g12a_vclk2_enci.hw,
+ [CLKID_VCLK2_ENCP] = &g12a_vclk2_encp.hw,
+ [CLKID_DAC_CLK] = &g12a_dac_clk.hw,
+ [CLKID_AOCLK] = &g12a_aoclk_gate.hw,
+ [CLKID_IEC958] = &g12a_iec958_gate.hw,
+ [CLKID_ENC480P] = &g12a_enc480p.hw,
+ [CLKID_RNG1] = &g12a_rng1.hw,
+ [CLKID_VCLK2_ENCT] = &g12a_vclk2_enct.hw,
+ [CLKID_VCLK2_ENCL] = &g12a_vclk2_encl.hw,
+ [CLKID_VCLK2_VENCLMMC] = &g12a_vclk2_venclmmc.hw,
+ [CLKID_VCLK2_VENCL] = &g12a_vclk2_vencl.hw,
+ [CLKID_VCLK2_OTHER1] = &g12a_vclk2_other1.hw,
+ [CLKID_FIXED_PLL_DCO] = &g12a_fixed_pll_dco.hw,
+ [CLKID_SYS_PLL_DCO] = &g12a_sys_pll_dco.hw,
+ [CLKID_GP0_PLL_DCO] = &g12a_gp0_pll_dco.hw,
+ [CLKID_HIFI_PLL_DCO] = &g12a_hifi_pll_dco.hw,
+ [CLKID_DMA] = &g12a_dma.hw,
+ [CLKID_EFUSE] = &g12a_efuse.hw,
+ [CLKID_ROM_BOOT] = &g12a_rom_boot.hw,
+ [CLKID_RESET_SEC] = &g12a_reset_sec.hw,
+ [CLKID_SEC_AHB_APB3] = &g12a_sec_ahb_apb3.hw,
+ [CLKID_MPLL_PREDIV] = &g12a_mpll_prediv.hw,
+ [CLKID_VPU_0_SEL] = &g12a_vpu_0_sel.hw,
+ [CLKID_VPU_0_DIV] = &g12a_vpu_0_div.hw,
+ [CLKID_VPU_0] = &g12a_vpu_0.hw,
+ [CLKID_VPU_1_SEL] = &g12a_vpu_1_sel.hw,
+ [CLKID_VPU_1_DIV] = &g12a_vpu_1_div.hw,
+ [CLKID_VPU_1] = &g12a_vpu_1.hw,
+ [CLKID_VPU] = &g12a_vpu.hw,
+ [CLKID_VAPB_0_SEL] = &g12a_vapb_0_sel.hw,
+ [CLKID_VAPB_0_DIV] = &g12a_vapb_0_div.hw,
+ [CLKID_VAPB_0] = &g12a_vapb_0.hw,
+ [CLKID_VAPB_1_SEL] = &g12a_vapb_1_sel.hw,
+ [CLKID_VAPB_1_DIV] = &g12a_vapb_1_div.hw,
+ [CLKID_VAPB_1] = &g12a_vapb_1.hw,
+ [CLKID_VAPB_SEL] = &g12a_vapb_sel.hw,
+ [CLKID_VAPB] = &g12a_vapb.hw,
+ [CLKID_HDMI_PLL_DCO] = &g12a_hdmi_pll_dco.hw,
+ [CLKID_HDMI_PLL_OD] = &g12a_hdmi_pll_od.hw,
+ [CLKID_HDMI_PLL_OD2] = &g12a_hdmi_pll_od2.hw,
+ [CLKID_HDMI_PLL] = &g12a_hdmi_pll.hw,
+ [CLKID_VID_PLL] = &g12a_vid_pll_div.hw,
+ [CLKID_VID_PLL_SEL] = &g12a_vid_pll_sel.hw,
+ [CLKID_VID_PLL_DIV] = &g12a_vid_pll.hw,
+ [CLKID_VCLK_SEL] = &g12a_vclk_sel.hw,
+ [CLKID_VCLK2_SEL] = &g12a_vclk2_sel.hw,
+ [CLKID_VCLK_INPUT] = &g12a_vclk_input.hw,
+ [CLKID_VCLK2_INPUT] = &g12a_vclk2_input.hw,
+ [CLKID_VCLK_DIV] = &g12a_vclk_div.hw,
+ [CLKID_VCLK2_DIV] = &g12a_vclk2_div.hw,
+ [CLKID_VCLK] = &g12a_vclk.hw,
+ [CLKID_VCLK2] = &g12a_vclk2.hw,
+ [CLKID_VCLK_DIV1] = &g12a_vclk_div1.hw,
+ [CLKID_VCLK_DIV2_EN] = &g12a_vclk_div2_en.hw,
+ [CLKID_VCLK_DIV4_EN] = &g12a_vclk_div4_en.hw,
+ [CLKID_VCLK_DIV6_EN] = &g12a_vclk_div6_en.hw,
+ [CLKID_VCLK_DIV12_EN] = &g12a_vclk_div12_en.hw,
+ [CLKID_VCLK2_DIV1] = &g12a_vclk2_div1.hw,
+ [CLKID_VCLK2_DIV2_EN] = &g12a_vclk2_div2_en.hw,
+ [CLKID_VCLK2_DIV4_EN] = &g12a_vclk2_div4_en.hw,
+ [CLKID_VCLK2_DIV6_EN] = &g12a_vclk2_div6_en.hw,
+ [CLKID_VCLK2_DIV12_EN] = &g12a_vclk2_div12_en.hw,
+ [CLKID_VCLK_DIV2] = &g12a_vclk_div2.hw,
+ [CLKID_VCLK_DIV4] = &g12a_vclk_div4.hw,
+ [CLKID_VCLK_DIV6] = &g12a_vclk_div6.hw,
+ [CLKID_VCLK_DIV12] = &g12a_vclk_div12.hw,
+ [CLKID_VCLK2_DIV2] = &g12a_vclk2_div2.hw,
+ [CLKID_VCLK2_DIV4] = &g12a_vclk2_div4.hw,
+ [CLKID_VCLK2_DIV6] = &g12a_vclk2_div6.hw,
+ [CLKID_VCLK2_DIV12] = &g12a_vclk2_div12.hw,
+ [CLKID_CTS_ENCI_SEL] = &g12a_cts_enci_sel.hw,
+ [CLKID_CTS_ENCP_SEL] = &g12a_cts_encp_sel.hw,
+ [CLKID_CTS_VDAC_SEL] = &g12a_cts_vdac_sel.hw,
+ [CLKID_HDMI_TX_SEL] = &g12a_hdmi_tx_sel.hw,
+ [CLKID_CTS_ENCI] = &g12a_cts_enci.hw,
+ [CLKID_CTS_ENCP] = &g12a_cts_encp.hw,
+ [CLKID_CTS_VDAC] = &g12a_cts_vdac.hw,
+ [CLKID_HDMI_TX] = &g12a_hdmi_tx.hw,
+ [CLKID_HDMI_SEL] = &g12a_hdmi_sel.hw,
+ [CLKID_HDMI_DIV] = &g12a_hdmi_div.hw,
+ [CLKID_HDMI] = &g12a_hdmi.hw,
+ [CLKID_MALI_0_SEL] = &g12a_mali_0_sel.hw,
+ [CLKID_MALI_0_DIV] = &g12a_mali_0_div.hw,
+ [CLKID_MALI_0] = &g12a_mali_0.hw,
+ [CLKID_MALI_1_SEL] = &g12a_mali_1_sel.hw,
+ [CLKID_MALI_1_DIV] = &g12a_mali_1_div.hw,
+ [CLKID_MALI_1] = &g12a_mali_1.hw,
+ [CLKID_MALI] = &g12a_mali.hw,
+ [CLKID_MPLL_50M_DIV] = &g12a_mpll_50m_div.hw,
+ [CLKID_MPLL_50M] = &g12a_mpll_50m.hw,
+ [CLKID_SYS_PLL_DIV16_EN] = &g12a_sys_pll_div16_en.hw,
+ [CLKID_SYS_PLL_DIV16] = &g12a_sys_pll_div16.hw,
+ [CLKID_CPU_CLK_DYN0_SEL] = &g12a_cpu_clk_premux0.hw,
+ [CLKID_CPU_CLK_DYN0_DIV] = &g12a_cpu_clk_mux0_div.hw,
+ [CLKID_CPU_CLK_DYN0] = &g12a_cpu_clk_postmux0.hw,
+ [CLKID_CPU_CLK_DYN1_SEL] = &g12a_cpu_clk_premux1.hw,
+ [CLKID_CPU_CLK_DYN1_DIV] = &g12a_cpu_clk_mux1_div.hw,
+ [CLKID_CPU_CLK_DYN1] = &g12a_cpu_clk_postmux1.hw,
+ [CLKID_CPU_CLK_DYN] = &g12a_cpu_clk_dyn.hw,
+ [CLKID_CPU_CLK] = &g12b_cpu_clk.hw,
+ [CLKID_CPU_CLK_DIV16_EN] = &g12a_cpu_clk_div16_en.hw,
+ [CLKID_CPU_CLK_DIV16] = &g12a_cpu_clk_div16.hw,
+ [CLKID_CPU_CLK_APB_DIV] = &g12a_cpu_clk_apb_div.hw,
+ [CLKID_CPU_CLK_APB] = &g12a_cpu_clk_apb.hw,
+ [CLKID_CPU_CLK_ATB_DIV] = &g12a_cpu_clk_atb_div.hw,
+ [CLKID_CPU_CLK_ATB] = &g12a_cpu_clk_atb.hw,
+ [CLKID_CPU_CLK_AXI_DIV] = &g12a_cpu_clk_axi_div.hw,
+ [CLKID_CPU_CLK_AXI] = &g12a_cpu_clk_axi.hw,
+ [CLKID_CPU_CLK_TRACE_DIV] = &g12a_cpu_clk_trace_div.hw,
+ [CLKID_CPU_CLK_TRACE] = &g12a_cpu_clk_trace.hw,
+ [CLKID_PCIE_PLL_DCO] = &g12a_pcie_pll_dco.hw,
+ [CLKID_PCIE_PLL_DCO_DIV2] = &g12a_pcie_pll_dco_div2.hw,
+ [CLKID_PCIE_PLL_OD] = &g12a_pcie_pll_od.hw,
+ [CLKID_PCIE_PLL] = &g12a_pcie_pll.hw,
+ [CLKID_VDEC_1_SEL] = &g12a_vdec_1_sel.hw,
+ [CLKID_VDEC_1_DIV] = &g12a_vdec_1_div.hw,
+ [CLKID_VDEC_1] = &g12a_vdec_1.hw,
+ [CLKID_VDEC_HEVC_SEL] = &g12a_vdec_hevc_sel.hw,
+ [CLKID_VDEC_HEVC_DIV] = &g12a_vdec_hevc_div.hw,
+ [CLKID_VDEC_HEVC] = &g12a_vdec_hevc.hw,
+ [CLKID_VDEC_HEVCF_SEL] = &g12a_vdec_hevcf_sel.hw,
+ [CLKID_VDEC_HEVCF_DIV] = &g12a_vdec_hevcf_div.hw,
+ [CLKID_VDEC_HEVCF] = &g12a_vdec_hevcf.hw,
+ [CLKID_TS_DIV] = &g12a_ts_div.hw,
+ [CLKID_TS] = &g12a_ts.hw,
+ [CLKID_SYS1_PLL_DCO] = &g12b_sys1_pll_dco.hw,
+ [CLKID_SYS1_PLL] = &g12b_sys1_pll.hw,
+ [CLKID_SYS1_PLL_DIV16_EN] = &g12b_sys1_pll_div16_en.hw,
+ [CLKID_SYS1_PLL_DIV16] = &g12b_sys1_pll_div16.hw,
+ [CLKID_CPUB_CLK_DYN0_SEL] = &g12b_cpub_clk_premux0.hw,
+ [CLKID_CPUB_CLK_DYN0_DIV] = &g12b_cpub_clk_mux0_div.hw,
+ [CLKID_CPUB_CLK_DYN0] = &g12b_cpub_clk_postmux0.hw,
+ [CLKID_CPUB_CLK_DYN1_SEL] = &g12b_cpub_clk_premux1.hw,
+ [CLKID_CPUB_CLK_DYN1_DIV] = &g12b_cpub_clk_mux1_div.hw,
+ [CLKID_CPUB_CLK_DYN1] = &g12b_cpub_clk_postmux1.hw,
+ [CLKID_CPUB_CLK_DYN] = &g12b_cpub_clk_dyn.hw,
+ [CLKID_CPUB_CLK] = &g12b_cpub_clk.hw,
+ [CLKID_CPUB_CLK_DIV16_EN] = &g12b_cpub_clk_div16_en.hw,
+ [CLKID_CPUB_CLK_DIV16] = &g12b_cpub_clk_div16.hw,
+ [CLKID_CPUB_CLK_DIV2] = &g12b_cpub_clk_div2.hw,
+ [CLKID_CPUB_CLK_DIV3] = &g12b_cpub_clk_div3.hw,
+ [CLKID_CPUB_CLK_DIV4] = &g12b_cpub_clk_div4.hw,
+ [CLKID_CPUB_CLK_DIV5] = &g12b_cpub_clk_div5.hw,
+ [CLKID_CPUB_CLK_DIV6] = &g12b_cpub_clk_div6.hw,
+ [CLKID_CPUB_CLK_DIV7] = &g12b_cpub_clk_div7.hw,
+ [CLKID_CPUB_CLK_DIV8] = &g12b_cpub_clk_div8.hw,
+ [CLKID_CPUB_CLK_APB_SEL] = &g12b_cpub_clk_apb_sel.hw,
+ [CLKID_CPUB_CLK_APB] = &g12b_cpub_clk_apb.hw,
+ [CLKID_CPUB_CLK_ATB_SEL] = &g12b_cpub_clk_atb_sel.hw,
+ [CLKID_CPUB_CLK_ATB] = &g12b_cpub_clk_atb.hw,
+ [CLKID_CPUB_CLK_AXI_SEL] = &g12b_cpub_clk_axi_sel.hw,
+ [CLKID_CPUB_CLK_AXI] = &g12b_cpub_clk_axi.hw,
+ [CLKID_CPUB_CLK_TRACE_SEL] = &g12b_cpub_clk_trace_sel.hw,
+ [CLKID_CPUB_CLK_TRACE] = &g12b_cpub_clk_trace.hw,
[NR_CLKS] = NULL,
},
.num = NR_CLKS,
&g12a_vdec_hevcf_sel,
&g12a_vdec_hevcf_div,
&g12a_vdec_hevcf,
+ &g12a_ts_div,
+ &g12a_ts,
+ &g12b_cpu_clk,
+ &g12b_sys1_pll_dco,
+ &g12b_sys1_pll,
+ &g12b_sys1_pll_div16_en,
+ &g12b_cpub_clk_premux0,
+ &g12b_cpub_clk_mux0_div,
+ &g12b_cpub_clk_postmux0,
+ &g12b_cpub_clk_premux1,
+ &g12b_cpub_clk_mux1_div,
+ &g12b_cpub_clk_postmux1,
+ &g12b_cpub_clk_dyn,
+ &g12b_cpub_clk,
+ &g12b_cpub_clk_div16_en,
+ &g12b_cpub_clk_apb_sel,
+ &g12b_cpub_clk_apb,
+ &g12b_cpub_clk_atb_sel,
+ &g12b_cpub_clk_atb,
+ &g12b_cpub_clk_axi_sel,
+ &g12b_cpub_clk_axi,
+ &g12b_cpub_clk_trace_sel,
+ &g12b_cpub_clk_trace,
+};
+
+static const struct reg_sequence g12a_init_regs[] = {
+ { .reg = HHI_MPLL_CNTL0, .def = 0x00000543 },
};
static const struct meson_eeclkc_data g12a_clkc_data = {
.regmap_clks = g12a_clk_regmaps,
.regmap_clk_num = ARRAY_SIZE(g12a_clk_regmaps),
- .hw_onecell_data = &g12a_hw_onecell_data
+ .hw_onecell_data = &g12a_hw_onecell_data,
+ .init_regs = g12a_init_regs,
+ .init_count = ARRAY_SIZE(g12a_init_regs),
+};
+
+static const struct meson_eeclkc_data g12b_clkc_data = {
+ .regmap_clks = g12a_clk_regmaps,
+ .regmap_clk_num = ARRAY_SIZE(g12a_clk_regmaps),
+ .hw_onecell_data = &g12b_hw_onecell_data
};
static const struct of_device_id clkc_match_table[] = {
{ .compatible = "amlogic,g12a-clkc", .data = &g12a_clkc_data },
+ { .compatible = "amlogic,g12b-clkc", .data = &g12b_clkc_data },
{}
};
#define HHI_VDEC4_CLK_CNTL 0x1EC
#define HHI_HDCP22_CLK_CNTL 0x1F0
#define HHI_VAPBCLK_CNTL 0x1F4
+#define HHI_SYS_CPUB_CLK_CNTL1 0x200
+#define HHI_SYS_CPUB_CLK_CNTL 0x208
#define HHI_VPU_CLKB_CNTL 0x20C
#define HHI_GEN_CLK_CNTL 0x228
#define HHI_VDIN_MEAS_CLK_CNTL 0x250
#define HHI_HDMI_PLL_CNTL5 0x334
#define HHI_HDMI_PLL_CNTL6 0x338
#define HHI_SPICC_CLK_CNTL 0x3dc
+#define HHI_SYS1_PLL_CNTL0 0x380
+#define HHI_SYS1_PLL_CNTL1 0x384
+#define HHI_SYS1_PLL_CNTL2 0x388
+#define HHI_SYS1_PLL_CNTL3 0x38c
+#define HHI_SYS1_PLL_CNTL4 0x390
+#define HHI_SYS1_PLL_CNTL5 0x394
+#define HHI_SYS1_PLL_CNTL6 0x398
/*
* CLKID index values
#define CLKID_VDEC_HEVC_DIV 206
#define CLKID_VDEC_HEVCF_SEL 208
#define CLKID_VDEC_HEVCF_DIV 209
+#define CLKID_TS_DIV 211
+#define CLKID_SYS1_PLL_DCO 213
+#define CLKID_SYS1_PLL 214
+#define CLKID_SYS1_PLL_DIV16_EN 215
+#define CLKID_SYS1_PLL_DIV16 216
+#define CLKID_CPUB_CLK_DYN0_SEL 217
+#define CLKID_CPUB_CLK_DYN0_DIV 218
+#define CLKID_CPUB_CLK_DYN0 219
+#define CLKID_CPUB_CLK_DYN1_SEL 220
+#define CLKID_CPUB_CLK_DYN1_DIV 221
+#define CLKID_CPUB_CLK_DYN1 222
+#define CLKID_CPUB_CLK_DYN 223
+#define CLKID_CPUB_CLK 224
+#define CLKID_CPUB_CLK_DIV16_EN 225
+#define CLKID_CPUB_CLK_DIV16 226
+#define CLKID_CPUB_CLK_DIV2 227
+#define CLKID_CPUB_CLK_DIV3 228
+#define CLKID_CPUB_CLK_DIV4 229
+#define CLKID_CPUB_CLK_DIV5 230
+#define CLKID_CPUB_CLK_DIV6 231
+#define CLKID_CPUB_CLK_DIV7 232
+#define CLKID_CPUB_CLK_DIV8 233
+#define CLKID_CPUB_CLK_APB_SEL 234
+#define CLKID_CPUB_CLK_APB 235
+#define CLKID_CPUB_CLK_ATB_SEL 236
+#define CLKID_CPUB_CLK_ATB 237
+#define CLKID_CPUB_CLK_AXI_SEL 238
+#define CLKID_CPUB_CLK_AXI 239
+#define CLKID_CPUB_CLK_TRACE_SEL 240
+#define CLKID_CPUB_CLK_TRACE 241
-#define NR_CLKS 211
+#define NR_CLKS 242
/* include the CLKIDs that have been made part of the DT binding */
#include <dt-bindings/clock/g12a-clkc.h>
.shift = 16,
.width = 9,
},
- .ssen = {
- .reg_off = HHI_MPLL_CNTL,
- .shift = 25,
- .width = 1,
- },
.lock = &meson_clk_lock,
},
.hw.init = &(struct clk_init_data){
return PTR_ERR(map);
}
+ if (data->init_count)
+ regmap_multi_reg_write(map, data->init_regs, data->init_count);
+
input = meson_clk_hw_register_input(dev, "xtal", IN_PREFIX "xtal", 0);
if (IS_ERR(input)) {
ret = PTR_ERR(input);
struct meson_eeclkc_data {
struct clk_regmap *const *regmap_clks;
unsigned int regmap_clk_num;
+ const struct reg_sequence *init_regs;
+ unsigned int init_count;
struct clk_hw_onecell_data *hw_onecell_data;
};
},
};
+/* TODO: the clock at index 0 is "DDR_PLL" which we don't support yet */
+static const char * const meson8b_cts_amclk_parent_names[] = {
+ "mpll0", "mpll1", "mpll2"
+};
+
+static u32 meson8b_cts_amclk_mux_table[] = { 1, 2, 3 };
+
+static struct clk_regmap meson8b_cts_amclk_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_AUD_CLK_CNTL,
+ .mask = 0x3,
+ .shift = 9,
+ .table = meson8b_cts_amclk_mux_table,
+ .flags = CLK_MUX_ROUND_CLOSEST,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cts_amclk_sel",
+ .ops = &clk_regmap_mux_ops,
+ .parent_names = meson8b_cts_amclk_parent_names,
+ .num_parents = ARRAY_SIZE(meson8b_cts_amclk_parent_names),
+ },
+};
+
+static struct clk_regmap meson8b_cts_amclk_div = {
+ .data = &(struct clk_regmap_div_data) {
+ .offset = HHI_AUD_CLK_CNTL,
+ .shift = 0,
+ .width = 8,
+ .flags = CLK_DIVIDER_ROUND_CLOSEST,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cts_amclk_div",
+ .ops = &clk_regmap_divider_ops,
+ .parent_names = (const char *[]){ "cts_amclk_sel" },
+ .num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
+ },
+};
+
+static struct clk_regmap meson8b_cts_amclk = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_AUD_CLK_CNTL,
+ .bit_idx = 8,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cts_amclk",
+ .ops = &clk_regmap_gate_ops,
+ .parent_names = (const char *[]){ "cts_amclk_div" },
+ .num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
+ },
+};
+
+/* TODO: the clock at index 0 is "DDR_PLL" which we don't support yet */
+static const char * const meson8b_cts_mclk_i958_parent_names[] = {
+ "mpll0", "mpll1", "mpll2"
+};
+
+static u32 meson8b_cts_mclk_i958_mux_table[] = { 1, 2, 3 };
+
+static struct clk_regmap meson8b_cts_mclk_i958_sel = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_AUD_CLK_CNTL2,
+ .mask = 0x3,
+ .shift = 25,
+ .table = meson8b_cts_mclk_i958_mux_table,
+ .flags = CLK_MUX_ROUND_CLOSEST,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cts_mclk_i958_sel",
+ .ops = &clk_regmap_mux_ops,
+ .parent_names = meson8b_cts_mclk_i958_parent_names,
+ .num_parents = ARRAY_SIZE(meson8b_cts_mclk_i958_parent_names),
+ },
+};
+
+static struct clk_regmap meson8b_cts_mclk_i958_div = {
+ .data = &(struct clk_regmap_div_data){
+ .offset = HHI_AUD_CLK_CNTL2,
+ .shift = 16,
+ .width = 8,
+ .flags = CLK_DIVIDER_ROUND_CLOSEST,
+ },
+ .hw.init = &(struct clk_init_data) {
+ .name = "cts_mclk_i958_div",
+ .ops = &clk_regmap_divider_ops,
+ .parent_names = (const char *[]){ "cts_mclk_i958_sel" },
+ .num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
+ },
+};
+
+static struct clk_regmap meson8b_cts_mclk_i958 = {
+ .data = &(struct clk_regmap_gate_data){
+ .offset = HHI_AUD_CLK_CNTL2,
+ .bit_idx = 24,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cts_mclk_i958",
+ .ops = &clk_regmap_gate_ops,
+ .parent_names = (const char *[]){ "cts_mclk_i958_div" },
+ .num_parents = 1,
+ .flags = CLK_SET_RATE_PARENT,
+ },
+};
+
+static struct clk_regmap meson8b_cts_i958 = {
+ .data = &(struct clk_regmap_mux_data){
+ .offset = HHI_AUD_CLK_CNTL2,
+ .mask = 0x1,
+ .shift = 27,
+ },
+ .hw.init = &(struct clk_init_data){
+ .name = "cts_i958",
+ .ops = &clk_regmap_mux_ops,
+ .parent_names = (const char *[]){ "cts_amclk",
+ "cts_mclk_i958" },
+ .num_parents = 2,
+ /*
+ * The parent is specific to origin of the audio data. Let the
+ * consumer choose the appropriate parent.
+ */
+ .flags = CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ },
+};
+
/* Everything Else (EE) domain gates */
static MESON_GATE(meson8b_ddr, HHI_GCLK_MPEG0, 0);
[CLKID_VDEC_HEVC_DIV] = &meson8b_vdec_hevc_div.hw,
[CLKID_VDEC_HEVC_EN] = &meson8b_vdec_hevc_en.hw,
[CLKID_VDEC_HEVC] = &meson8b_vdec_hevc.hw,
+ [CLKID_CTS_AMCLK_SEL] = &meson8b_cts_amclk_sel.hw,
+ [CLKID_CTS_AMCLK_DIV] = &meson8b_cts_amclk_div.hw,
+ [CLKID_CTS_AMCLK] = &meson8b_cts_amclk.hw,
+ [CLKID_CTS_MCLK_I958_SEL] = &meson8b_cts_mclk_i958_sel.hw,
+ [CLKID_CTS_MCLK_I958_DIV] = &meson8b_cts_mclk_i958_div.hw,
+ [CLKID_CTS_MCLK_I958] = &meson8b_cts_mclk_i958.hw,
+ [CLKID_CTS_I958] = &meson8b_cts_i958.hw,
[CLK_NR_CLKS] = NULL,
},
.num = CLK_NR_CLKS,
[CLKID_VDEC_HEVC_DIV] = &meson8b_vdec_hevc_div.hw,
[CLKID_VDEC_HEVC_EN] = &meson8b_vdec_hevc_en.hw,
[CLKID_VDEC_HEVC] = &meson8b_vdec_hevc.hw,
+ [CLKID_CTS_AMCLK_SEL] = &meson8b_cts_amclk_sel.hw,
+ [CLKID_CTS_AMCLK_DIV] = &meson8b_cts_amclk_div.hw,
+ [CLKID_CTS_AMCLK] = &meson8b_cts_amclk.hw,
+ [CLKID_CTS_MCLK_I958_SEL] = &meson8b_cts_mclk_i958_sel.hw,
+ [CLKID_CTS_MCLK_I958_DIV] = &meson8b_cts_mclk_i958_div.hw,
+ [CLKID_CTS_MCLK_I958] = &meson8b_cts_mclk_i958.hw,
+ [CLKID_CTS_I958] = &meson8b_cts_i958.hw,
[CLK_NR_CLKS] = NULL,
},
.num = CLK_NR_CLKS,
[CLKID_VDEC_HEVC_DIV] = &meson8b_vdec_hevc_div.hw,
[CLKID_VDEC_HEVC_EN] = &meson8b_vdec_hevc_en.hw,
[CLKID_VDEC_HEVC] = &meson8b_vdec_hevc.hw,
+ [CLKID_CTS_AMCLK_SEL] = &meson8b_cts_amclk_sel.hw,
+ [CLKID_CTS_AMCLK_DIV] = &meson8b_cts_amclk_div.hw,
+ [CLKID_CTS_AMCLK] = &meson8b_cts_amclk.hw,
+ [CLKID_CTS_MCLK_I958_SEL] = &meson8b_cts_mclk_i958_sel.hw,
+ [CLKID_CTS_MCLK_I958_DIV] = &meson8b_cts_mclk_i958_div.hw,
+ [CLKID_CTS_MCLK_I958] = &meson8b_cts_mclk_i958.hw,
+ [CLKID_CTS_I958] = &meson8b_cts_i958.hw,
[CLK_NR_CLKS] = NULL,
},
.num = CLK_NR_CLKS,
&meson8b_vdec_hevc_div,
&meson8b_vdec_hevc_en,
&meson8b_vdec_hevc,
+ &meson8b_cts_amclk,
+ &meson8b_cts_amclk_sel,
+ &meson8b_cts_amclk_div,
+ &meson8b_cts_mclk_i958_sel,
+ &meson8b_cts_mclk_i958_div,
+ &meson8b_cts_mclk_i958,
+ &meson8b_cts_i958,
};
static const struct meson8b_clk_reset_line {
#define HHI_SYS_CPU_CLK_CNTL1 0x15c /* 0x57 offset in data sheet */
#define HHI_VID_CLK_DIV 0x164 /* 0x59 offset in data sheet */
#define HHI_MPEG_CLK_CNTL 0x174 /* 0x5d offset in data sheet */
+#define HHI_AUD_CLK_CNTL 0x178 /* 0x5e offset in data sheet */
#define HHI_VID_CLK_CNTL 0x17c /* 0x5f offset in data sheet */
+#define HHI_AUD_CLK_CNTL2 0x190 /* 0x64 offset in data sheet */
#define HHI_VID_CLK_CNTL2 0x194 /* 0x65 offset in data sheet */
#define HHI_VID_DIVIDER_CNTL 0x198 /* 0x66 offset in data sheet */
#define HHI_SYS_CPU_CLK_CNTL0 0x19c /* 0x67 offset in data sheet */
#define CLKID_VDEC_HEVC_SEL 203
#define CLKID_VDEC_HEVC_DIV 204
#define CLKID_VDEC_HEVC_EN 205
+#define CLKID_CTS_AMCLK_SEL 207
+#define CLKID_CTS_AMCLK_DIV 208
+#define CLKID_CTS_MCLK_I958_SEL 210
+#define CLKID_CTS_MCLK_I958_DIV 211
-#define CLK_NR_CLKS 207
+#define CLK_NR_CLKS 214
/*
* include the CLKID and RESETID that have
struct mmp_clk_factor_masks *masks = factor->masks;
int i;
unsigned long val;
- unsigned long prev_rate, rate = 0;
+ unsigned long rate = 0;
unsigned long flags = 0;
for (i = 0; i < factor->ftbl_cnt; i++) {
- prev_rate = rate;
rate = (((prate / 10000) * factor->ftbl[i].den) /
(factor->ftbl[i].num * factor->masks->factor)) * 10000;
if (rate > drate)
}
}
+static u32 __init mv98dx1135_get_tclk_freq(void __iomem *sar)
+{
+ return 166666667;
+}
+
static const struct coreclk_soc_desc kirkwood_coreclks = {
.get_tclk_freq = kirkwood_get_tclk_freq,
.get_cpu_freq = kirkwood_get_cpu_freq,
.num_ratios = ARRAY_SIZE(kirkwood_coreclk_ratios),
};
+static const struct coreclk_soc_desc mv98dx1135_coreclks = {
+ .get_tclk_freq = mv98dx1135_get_tclk_freq,
+ .get_cpu_freq = kirkwood_get_cpu_freq,
+ .get_clk_ratio = kirkwood_get_clk_ratio,
+ .ratios = kirkwood_coreclk_ratios,
+ .num_ratios = ARRAY_SIZE(kirkwood_coreclk_ratios),
+};
+
/*
* Clock Gating Control
*/
if (of_device_is_compatible(np, "marvell,mv88f6180-core-clock"))
mvebu_coreclk_setup(np, &mv88f6180_coreclks);
+ else if (of_device_is_compatible(np, "marvell,mv98dx1135-core-clock"))
+ mvebu_coreclk_setup(np, &mv98dx1135_coreclks);
else
mvebu_coreclk_setup(np, &kirkwood_coreclks);
kirkwood_clk_init);
CLK_OF_DECLARE(mv88f6180_clk, "marvell,mv88f6180-core-clock",
kirkwood_clk_init);
+CLK_OF_DECLARE(98dx1135_clk, "marvell,mv98dx1135-core-clock",
+ kirkwood_clk_init);
"gpll0_early_div"
};
-static const struct parent_map gcc_xo_gpll0_gpll2_gpll3_gpll0_early_div_map[] = {
- { P_XO, 0 },
- { P_GPLL0, 1 },
- { P_GPLL2, 2 },
- { P_GPLL3, 3 },
- { P_GPLL0_EARLY_DIV, 6 }
-};
-
-static const char * const gcc_xo_gpll0_gpll2_gpll3_gpll0_early_div[] = {
- "xo",
- "gpll0",
- "gpll2",
- "gpll3",
- "gpll0_early_div"
-};
-
static const struct parent_map gcc_xo_gpll0_gpll1_early_div_gpll1_gpll4_gpll0_early_div_map[] = {
{ P_XO, 0 },
{ P_GPLL0, 1 },
"gpll0_early_div"
};
-static const struct parent_map gcc_xo_gpll0_gpll2_gpll3_gpll1_gpll4_gpll0_early_div_map[] = {
- { P_XO, 0 },
- { P_GPLL0, 1 },
- { P_GPLL2, 2 },
- { P_GPLL3, 3 },
- { P_GPLL1, 4 },
- { P_GPLL4, 5 },
- { P_GPLL0_EARLY_DIV, 6 }
-};
-
-static const char * const gcc_xo_gpll0_gpll2_gpll3_gpll1_gpll4_gpll0_early_div[] = {
- "xo",
- "gpll0",
- "gpll2",
- "gpll3",
- "gpll1",
- "gpll4",
- "gpll0_early_div"
-};
-
static struct clk_fixed_factor xo = {
.mult = 1,
.div = 1,
[GCC_PCIE_0_PHY_BCR] = { 0x3e004 },
[GCC_PCIE_0_LINK_DOWN_BCR] = { 0x3e038 },
[GCC_PCIEPHY_0_PHY_BCR] = { 0x3e03c },
+ [GCC_PCIE_0_AXI_MASTER_STICKY_ARES] = { 0x3e040, 6},
+ [GCC_PCIE_0_AHB_ARES] = { 0x3e040, 5 },
+ [GCC_PCIE_0_AXI_SLAVE_ARES] = { 0x3e040, 4 },
+ [GCC_PCIE_0_AXI_MASTER_ARES] = { 0x3e040, 3 },
+ [GCC_PCIE_0_CORE_STICKY_ARES] = { 0x3e040, 2 },
+ [GCC_PCIE_0_SLEEP_ARES] = { 0x3e040, 1 },
+ [GCC_PCIE_0_PIPE_ARES] = { 0x3e040, 0 },
[GCC_EMAC_BCR] = { 0x4e000 },
};
udelay(1);
}
- return gdsc_poll_status(sc, status);
+ ret = gdsc_poll_status(sc, status);
+ WARN(ret, "%s status stuck at 'o%s'", sc->pd.name, status ? "ff" : "n");
+ return ret;
}
static inline int gdsc_deassert_reset(struct gdsc *sc)
* @div: divisor value (1-64)
* @src_shift: Shift to access the register bits to select the parent clock
* @src_width: Number of register bits to select the parent clock (may be 0)
- * @parents: Array to map from valid parent clocks indices to hardware indices
* @nb: Notifier block to save/restore clock state for system resume
+ * @parents: Array to map from valid parent clocks indices to hardware indices
*/
struct div6_clock {
struct clk_hw hw;
unsigned int div;
u32 src_shift;
u32 src_width;
- u8 *parents;
struct notifier_block nb;
+ u8 parents[];
};
#define to_div6_clock(_hw) container_of(_hw, struct div6_clock, hw)
struct clk *clk;
unsigned int i;
- clock = kzalloc(sizeof(*clock), GFP_KERNEL);
+ clock = kzalloc(struct_size(clock, parents, num_parents), GFP_KERNEL);
if (!clock)
return ERR_PTR(-ENOMEM);
- clock->parents = kmalloc_array(num_parents, sizeof(*clock->parents),
- GFP_KERNEL);
- if (!clock->parents) {
- clk = ERR_PTR(-ENOMEM);
- goto free_clock;
- }
-
clock->reg = reg;
/*
pr_err("%s: invalid number of parents for DIV6 clock %s\n",
__func__, name);
clk = ERR_PTR(-EINVAL);
- goto free_parents;
+ goto free_clock;
}
/* Filter out invalid parents */
clk = clk_register(NULL, &clock->hw);
if (IS_ERR(clk))
- goto free_parents;
+ goto free_clock;
if (notifiers) {
clock->nb.notifier_call = cpg_div6_clock_notifier_call;
return clk;
-free_parents:
- kfree(clock->parents);
free_clock:
kfree(clock);
return clk;
/**
* struct mstp_clock_group - MSTP gating clocks group
*
- * @data: clocks in this group
+ * @data: clock specifier translation for clocks in this group
* @smstpcr: module stop control register
* @mstpsr: module stop status register (optional)
* @lock: protects writes to SMSTPCR
* @width_8bit: registers are 8-bit, not 32-bit
+ * @clks: clocks in this group
*/
struct mstp_clock_group {
struct clk_onecell_data data;
void __iomem *mstpsr;
spinlock_t lock;
bool width_8bit;
+ struct clk *clks[];
};
/**
struct clk **clks;
unsigned int i;
- group = kzalloc(sizeof(*group), GFP_KERNEL);
- clks = kmalloc_array(MSTP_MAX_CLOCKS, sizeof(*clks), GFP_KERNEL);
- if (group == NULL || clks == NULL) {
+ group = kzalloc(struct_size(group, clks, MSTP_MAX_CLOCKS), GFP_KERNEL);
+ if (group == NULL) {
kfree(group);
- kfree(clks);
return;
}
+ clks = group->clks;
spin_lock_init(&group->lock);
group->data.clks = clks;
if (group->smstpcr == NULL) {
pr_err("%s: failed to remap SMSTPCR\n", __func__);
kfree(group);
- kfree(clks);
return;
}
return PTR_ERR(clk);
error = pm_clk_create(dev);
- if (error) {
- dev_err(dev, "pm_clk_create failed %d\n", error);
+ if (error)
goto fail_put;
- }
error = pm_clk_add_clk(dev, clk);
- if (error) {
- dev_err(dev, "pm_clk_add_clk %pC failed %d\n", clk, error);
+ if (error)
goto fail_destroy;
- }
return 0;
};
static const struct mssr_mod_clk r8a774a1_mod_clks[] __initconst = {
+ DEF_MOD("tmu4", 121, R8A774A1_CLK_S0D6),
+ DEF_MOD("tmu3", 122, R8A774A1_CLK_S3D2),
+ DEF_MOD("tmu2", 123, R8A774A1_CLK_S3D2),
+ DEF_MOD("tmu1", 124, R8A774A1_CLK_S3D2),
+ DEF_MOD("tmu0", 125, R8A774A1_CLK_CP),
DEF_MOD("fdp1-0", 119, R8A774A1_CLK_S0D1),
DEF_MOD("scif5", 202, R8A774A1_CLK_S3D4),
DEF_MOD("scif4", 203, R8A774A1_CLK_S3D4),
DEF_MOD("cmt2", 301, R8A7795_CLK_R),
DEF_MOD("cmt1", 302, R8A7795_CLK_R),
DEF_MOD("cmt0", 303, R8A7795_CLK_R),
+ DEF_MOD("tpu0", 304, R8A7795_CLK_S3D4),
DEF_MOD("scif2", 310, R8A7795_CLK_S3D4),
DEF_MOD("sdif3", 311, R8A7795_CLK_SD3),
DEF_MOD("sdif2", 312, R8A7795_CLK_SD2),
DEF_MOD("ehci0", 703, R8A7795_CLK_S3D2),
DEF_MOD("hsusb", 704, R8A7795_CLK_S3D2),
DEF_MOD("hsusb3", 705, R8A7795_CLK_S3D2),
+ DEF_MOD("cmm3", 708, R8A7795_CLK_S2D1),
+ DEF_MOD("cmm2", 709, R8A7795_CLK_S2D1),
+ DEF_MOD("cmm1", 710, R8A7795_CLK_S2D1),
+ DEF_MOD("cmm0", 711, R8A7795_CLK_S2D1),
DEF_MOD("csi21", 713, R8A7795_CLK_CSI0), /* ES1.x */
DEF_MOD("csi20", 714, R8A7795_CLK_CSI0),
DEF_MOD("csi41", 715, R8A7795_CLK_CSI0),
DEF_MOD("cmt2", 301, R8A7796_CLK_R),
DEF_MOD("cmt1", 302, R8A7796_CLK_R),
DEF_MOD("cmt0", 303, R8A7796_CLK_R),
+ DEF_MOD("tpu0", 304, R8A7796_CLK_S3D4),
DEF_MOD("scif2", 310, R8A7796_CLK_S3D4),
DEF_MOD("sdif3", 311, R8A7796_CLK_SD3),
DEF_MOD("sdif2", 312, R8A7796_CLK_SD2),
DEF_MOD("ehci1", 702, R8A7796_CLK_S3D2),
DEF_MOD("ehci0", 703, R8A7796_CLK_S3D2),
DEF_MOD("hsusb", 704, R8A7796_CLK_S3D2),
+ DEF_MOD("cmm2", 709, R8A7796_CLK_S2D1),
+ DEF_MOD("cmm1", 710, R8A7796_CLK_S2D1),
+ DEF_MOD("cmm0", 711, R8A7796_CLK_S2D1),
DEF_MOD("csi20", 714, R8A7796_CLK_CSI0),
DEF_MOD("csi40", 716, R8A7796_CLK_CSI0),
DEF_MOD("du2", 722, R8A7796_CLK_S2D1),
DEF_MOD("cmt2", 301, R8A77965_CLK_R),
DEF_MOD("cmt1", 302, R8A77965_CLK_R),
DEF_MOD("cmt0", 303, R8A77965_CLK_R),
+ DEF_MOD("tpu0", 304, R8A77965_CLK_S3D4),
DEF_MOD("scif2", 310, R8A77965_CLK_S3D4),
DEF_MOD("sdif3", 311, R8A77965_CLK_SD3),
DEF_MOD("sdif2", 312, R8A77965_CLK_SD2),
DEF_MOD("ehci1", 702, R8A77965_CLK_S3D2),
DEF_MOD("ehci0", 703, R8A77965_CLK_S3D2),
DEF_MOD("hsusb", 704, R8A77965_CLK_S3D2),
+ DEF_MOD("cmm3", 708, R8A77965_CLK_S2D1),
+ DEF_MOD("cmm1", 710, R8A77965_CLK_S2D1),
+ DEF_MOD("cmm0", 711, R8A77965_CLK_S2D1),
DEF_MOD("csi20", 714, R8A77965_CLK_CSI0),
DEF_MOD("csi40", 716, R8A77965_CLK_CSI0),
DEF_MOD("du3", 721, R8A77965_CLK_S2D1),
DEF_MOD("ehci0", 703, R8A77990_CLK_S3D2),
DEF_MOD("hsusb", 704, R8A77990_CLK_S3D2),
+ DEF_MOD("cmm1", 710, R8A77990_CLK_S1D1),
+ DEF_MOD("cmm0", 711, R8A77990_CLK_S1D1),
DEF_MOD("csi40", 716, R8A77990_CLK_CSI0),
DEF_MOD("du1", 723, R8A77990_CLK_S1D1),
DEF_MOD("du0", 724, R8A77990_CLK_S1D1),
DEF_MOD("vspbs", 627, R8A77995_CLK_S0D1),
DEF_MOD("ehci0", 703, R8A77995_CLK_S3D2),
DEF_MOD("hsusb", 704, R8A77995_CLK_S3D2),
+ DEF_MOD("cmm1", 710, R8A77995_CLK_S1D1),
+ DEF_MOD("cmm0", 711, R8A77995_CLK_S1D1),
DEF_MOD("du1", 723, R8A77995_CLK_S1D1),
DEF_MOD("du0", 724, R8A77995_CLK_S1D1),
DEF_MOD("lvds", 727, R8A77995_CLK_S2D1),
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/platform_device.h>
+#include <linux/pm_clock.h>
+#include <linux/pm_domain.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <dt-bindings/clock/r9a06g032-sysctrl.h>
/* This is used to describe a clock for instantiation */
struct r9a06g032_clkdesc {
const char *name;
+ uint32_t managed: 1;
uint32_t type: 3;
uint32_t index: 8;
uint32_t source : 8; /* source index + 1 (0 == none) */
#define D_GATE(_idx, _n, _src, ...) \
{ .type = K_GATE, .index = R9A06G032_##_idx, \
.source = 1 + R9A06G032_##_src, .name = _n, \
- .gate = I_GATE(__VA_ARGS__), }
+ .gate = I_GATE(__VA_ARGS__) }
+#define D_MODULE(_idx, _n, _src, ...) \
+ { .type = K_GATE, .index = R9A06G032_##_idx, \
+ .source = 1 + R9A06G032_##_src, .name = _n, \
+ .managed = 1, .gate = I_GATE(__VA_ARGS__) }
#define D_ROOT(_idx, _n, _mul, _div) \
{ .type = K_FFC, .index = R9A06G032_##_idx, .name = _n, \
.div = _div, .mul = _mul }
#define R9A06G032_CLOCK_COUNT (R9A06G032_UART_GROUP_34567 + 1)
-static const struct r9a06g032_clkdesc r9a06g032_clocks[] __initconst = {
+static const struct r9a06g032_clkdesc r9a06g032_clocks[] = {
D_ROOT(CLKOUT, "clkout", 25, 1),
D_ROOT(CLK_PLL_USB, "clk_pll_usb", 12, 10),
D_FFC(CLKOUT_D10, "clkout_d10", CLKOUT, 10),
D_GATE(CLK_P6_PG2, "clk_p6_pg2", DIV_P6_PG, 0x8a3, 0x8a4, 0x8a5, 0, 0xb61, 0, 0),
D_GATE(CLK_P6_PG3, "clk_p6_pg3", DIV_P6_PG, 0x8a6, 0x8a7, 0x8a8, 0, 0xb62, 0, 0),
D_GATE(CLK_P6_PG4, "clk_p6_pg4", DIV_P6_PG, 0x8a9, 0x8aa, 0x8ab, 0, 0xb63, 0, 0),
- D_GATE(CLK_PCI_USB, "clk_pci_usb", CLKOUT_D40, 0xe6, 0, 0, 0, 0, 0, 0),
+ D_MODULE(CLK_PCI_USB, "clk_pci_usb", CLKOUT_D40, 0xe6, 0, 0, 0, 0, 0, 0),
D_GATE(CLK_QSPI0, "clk_qspi0", DIV_QSPI0, 0x2a4, 0x2a5, 0, 0, 0, 0, 0),
D_GATE(CLK_QSPI1, "clk_qspi1", DIV_QSPI1, 0x484, 0x485, 0, 0, 0, 0, 0),
D_GATE(CLK_RGMII_REF, "clk_rgmii_ref", CLKOUT_D8, 0x340, 0, 0, 0, 0, 0, 0),
D_GATE(CLK_SPI5, "clk_spi5", DIV_P4_PG, 0x822, 0x823, 0, 0, 0, 0, 0),
D_GATE(CLK_SWITCH, "clk_switch", DIV_SWITCH, 0x982, 0x983, 0, 0, 0, 0, 0),
D_DIV(DIV_MOTOR, "div_motor", CLKOUT_D5, 84, 2, 8),
- D_GATE(HCLK_ECAT125, "hclk_ecat125", CLKOUT_D8, 0x400, 0x401, 0, 0x402, 0, 0x440, 0x441),
- D_GATE(HCLK_PINCONFIG, "hclk_pinconfig", CLKOUT_D40, 0x740, 0x741, 0x742, 0, 0xae0, 0, 0),
- D_GATE(HCLK_SERCOS, "hclk_sercos", CLKOUT_D10, 0x420, 0x422, 0, 0x421, 0, 0x460, 0x461),
- D_GATE(HCLK_SGPIO2, "hclk_sgpio2", DIV_P5_PG, 0x8c3, 0x8c4, 0x8c5, 0, 0xb41, 0, 0),
- D_GATE(HCLK_SGPIO3, "hclk_sgpio3", DIV_P5_PG, 0x8c6, 0x8c7, 0x8c8, 0, 0xb42, 0, 0),
- D_GATE(HCLK_SGPIO4, "hclk_sgpio4", DIV_P5_PG, 0x8c9, 0x8ca, 0x8cb, 0, 0xb43, 0, 0),
- D_GATE(HCLK_TIMER0, "hclk_timer0", CLKOUT_D40, 0x743, 0x744, 0x745, 0, 0xae1, 0, 0),
- D_GATE(HCLK_TIMER1, "hclk_timer1", CLKOUT_D40, 0x746, 0x747, 0x748, 0, 0xae2, 0, 0),
- D_GATE(HCLK_USBF, "hclk_usbf", CLKOUT_D8, 0xe3, 0, 0, 0xe4, 0, 0x102, 0x103),
- D_GATE(HCLK_USBH, "hclk_usbh", CLKOUT_D8, 0xe0, 0xe1, 0, 0xe2, 0, 0x100, 0x101),
- D_GATE(HCLK_USBPM, "hclk_usbpm", CLKOUT_D8, 0xe5, 0, 0, 0, 0, 0, 0),
+ D_MODULE(HCLK_ECAT125, "hclk_ecat125", CLKOUT_D8, 0x400, 0x401, 0, 0x402, 0, 0x440, 0x441),
+ D_MODULE(HCLK_PINCONFIG, "hclk_pinconfig", CLKOUT_D40, 0x740, 0x741, 0x742, 0, 0xae0, 0, 0),
+ D_MODULE(HCLK_SERCOS, "hclk_sercos", CLKOUT_D10, 0x420, 0x422, 0, 0x421, 0, 0x460, 0x461),
+ D_MODULE(HCLK_SGPIO2, "hclk_sgpio2", DIV_P5_PG, 0x8c3, 0x8c4, 0x8c5, 0, 0xb41, 0, 0),
+ D_MODULE(HCLK_SGPIO3, "hclk_sgpio3", DIV_P5_PG, 0x8c6, 0x8c7, 0x8c8, 0, 0xb42, 0, 0),
+ D_MODULE(HCLK_SGPIO4, "hclk_sgpio4", DIV_P5_PG, 0x8c9, 0x8ca, 0x8cb, 0, 0xb43, 0, 0),
+ D_MODULE(HCLK_TIMER0, "hclk_timer0", CLKOUT_D40, 0x743, 0x744, 0x745, 0, 0xae1, 0, 0),
+ D_MODULE(HCLK_TIMER1, "hclk_timer1", CLKOUT_D40, 0x746, 0x747, 0x748, 0, 0xae2, 0, 0),
+ D_MODULE(HCLK_USBF, "hclk_usbf", CLKOUT_D8, 0xe3, 0, 0, 0xe4, 0, 0x102, 0x103),
+ D_MODULE(HCLK_USBH, "hclk_usbh", CLKOUT_D8, 0xe0, 0xe1, 0, 0xe2, 0, 0x100, 0x101),
+ D_MODULE(HCLK_USBPM, "hclk_usbpm", CLKOUT_D8, 0xe5, 0, 0, 0, 0, 0, 0),
D_GATE(CLK_48_PG_F, "clk_48_pg_f", CLK_48, 0x78c, 0x78d, 0, 0x78e, 0, 0xb04, 0xb05),
D_GATE(CLK_48_PG4, "clk_48_pg4", CLK_48, 0x789, 0x78a, 0x78b, 0, 0xb03, 0, 0),
D_FFC(CLK_DDRPHY_PLLCLK_D4, "clk_ddrphy_pllclk_d4", CLK_DDRPHY_PLLCLK, 4),
D_FFC(CLK_REF_SYNC_D8, "clk_ref_sync_d8", CLK_REF_SYNC, 8),
D_FFC(CLK_SERCOS100_D2, "clk_sercos100_d2", CLK_SERCOS100, 2),
D_DIV(DIV_CA7, "div_ca7", CLK_REF_SYNC, 57, 1, 4, 1, 2, 4),
- D_GATE(HCLK_CAN0, "hclk_can0", CLK_48, 0x783, 0x784, 0x785, 0, 0xb01, 0, 0),
- D_GATE(HCLK_CAN1, "hclk_can1", CLK_48, 0x786, 0x787, 0x788, 0, 0xb02, 0, 0),
- D_GATE(HCLK_DELTASIGMA, "hclk_deltasigma", DIV_MOTOR, 0x1ef, 0x1f0, 0x1f1, 0, 0, 0, 0),
- D_GATE(HCLK_PWMPTO, "hclk_pwmpto", DIV_MOTOR, 0x1ec, 0x1ed, 0x1ee, 0, 0, 0, 0),
- D_GATE(HCLK_RSV, "hclk_rsv", CLK_48, 0x780, 0x781, 0x782, 0, 0xb00, 0, 0),
- D_GATE(HCLK_SGPIO0, "hclk_sgpio0", DIV_MOTOR, 0x1e0, 0x1e1, 0x1e2, 0, 0, 0, 0),
- D_GATE(HCLK_SGPIO1, "hclk_sgpio1", DIV_MOTOR, 0x1e3, 0x1e4, 0x1e5, 0, 0, 0, 0),
+ D_MODULE(HCLK_CAN0, "hclk_can0", CLK_48, 0x783, 0x784, 0x785, 0, 0xb01, 0, 0),
+ D_MODULE(HCLK_CAN1, "hclk_can1", CLK_48, 0x786, 0x787, 0x788, 0, 0xb02, 0, 0),
+ D_MODULE(HCLK_DELTASIGMA, "hclk_deltasigma", DIV_MOTOR, 0x1ef, 0x1f0, 0x1f1, 0, 0, 0, 0),
+ D_MODULE(HCLK_PWMPTO, "hclk_pwmpto", DIV_MOTOR, 0x1ec, 0x1ed, 0x1ee, 0, 0, 0, 0),
+ D_MODULE(HCLK_RSV, "hclk_rsv", CLK_48, 0x780, 0x781, 0x782, 0, 0xb00, 0, 0),
+ D_MODULE(HCLK_SGPIO0, "hclk_sgpio0", DIV_MOTOR, 0x1e0, 0x1e1, 0x1e2, 0, 0, 0, 0),
+ D_MODULE(HCLK_SGPIO1, "hclk_sgpio1", DIV_MOTOR, 0x1e3, 0x1e4, 0x1e5, 0, 0, 0, 0),
D_DIV(RTOS_MDC, "rtos_mdc", CLK_REF_SYNC, 100, 80, 640, 80, 160, 320, 640),
D_GATE(CLK_CM3, "clk_cm3", CLK_REF_SYNC_D4, 0xba0, 0xba1, 0, 0xba2, 0, 0xbc0, 0xbc1),
D_GATE(CLK_DDRC, "clk_ddrc", CLK_DDRPHY_PLLCLK_D4, 0x323, 0x324, 0, 0, 0, 0, 0),
D_GATE(CLK_HSR50, "clk_hsr50", CLK_HSR100_D2, 0x484, 0x485, 0, 0, 0, 0, 0),
D_GATE(CLK_HW_RTOS, "clk_hw_rtos", CLK_REF_SYNC_D4, 0xc60, 0xc61, 0, 0, 0, 0, 0),
D_GATE(CLK_SERCOS50, "clk_sercos50", CLK_SERCOS100_D2, 0x424, 0x423, 0, 0, 0, 0, 0),
- D_GATE(HCLK_ADC, "hclk_adc", CLK_REF_SYNC_D8, 0x1af, 0x1b0, 0x1b1, 0, 0, 0, 0),
- D_GATE(HCLK_CM3, "hclk_cm3", CLK_REF_SYNC_D4, 0xc20, 0xc21, 0xc22, 0, 0, 0, 0),
- D_GATE(HCLK_CRYPTO_EIP150, "hclk_crypto_eip150", CLK_REF_SYNC_D4, 0x123, 0x124, 0x125, 0, 0x142, 0, 0),
- D_GATE(HCLK_CRYPTO_EIP93, "hclk_crypto_eip93", CLK_REF_SYNC_D4, 0x120, 0x121, 0, 0x122, 0, 0x140, 0x141),
- D_GATE(HCLK_DDRC, "hclk_ddrc", CLK_REF_SYNC_D4, 0x320, 0x322, 0, 0x321, 0, 0x3a0, 0x3a1),
- D_GATE(HCLK_DMA0, "hclk_dma0", CLK_REF_SYNC_D4, 0x260, 0x261, 0x262, 0x263, 0x2c0, 0x2c1, 0x2c2),
- D_GATE(HCLK_DMA1, "hclk_dma1", CLK_REF_SYNC_D4, 0x264, 0x265, 0x266, 0x267, 0x2c3, 0x2c4, 0x2c5),
- D_GATE(HCLK_GMAC0, "hclk_gmac0", CLK_REF_SYNC_D4, 0x360, 0x361, 0x362, 0x363, 0x3c0, 0x3c1, 0x3c2),
- D_GATE(HCLK_GMAC1, "hclk_gmac1", CLK_REF_SYNC_D4, 0x380, 0x381, 0x382, 0x383, 0x3e0, 0x3e1, 0x3e2),
- D_GATE(HCLK_GPIO0, "hclk_gpio0", CLK_REF_SYNC_D4, 0x212, 0x213, 0x214, 0, 0, 0, 0),
- D_GATE(HCLK_GPIO1, "hclk_gpio1", CLK_REF_SYNC_D4, 0x215, 0x216, 0x217, 0, 0, 0, 0),
- D_GATE(HCLK_GPIO2, "hclk_gpio2", CLK_REF_SYNC_D4, 0x229, 0x22a, 0x22b, 0, 0, 0, 0),
- D_GATE(HCLK_HSR, "hclk_hsr", CLK_HSR100_D2, 0x480, 0x482, 0, 0x481, 0, 0x4c0, 0x4c1),
- D_GATE(HCLK_I2C0, "hclk_i2c0", CLK_REF_SYNC_D8, 0x1a9, 0x1aa, 0x1ab, 0, 0, 0, 0),
- D_GATE(HCLK_I2C1, "hclk_i2c1", CLK_REF_SYNC_D8, 0x1ac, 0x1ad, 0x1ae, 0, 0, 0, 0),
- D_GATE(HCLK_LCD, "hclk_lcd", CLK_REF_SYNC_D4, 0x7a0, 0x7a1, 0x7a2, 0, 0xb20, 0, 0),
- D_GATE(HCLK_MSEBI_M, "hclk_msebi_m", CLK_REF_SYNC_D4, 0x164, 0x165, 0x166, 0, 0x183, 0, 0),
- D_GATE(HCLK_MSEBI_S, "hclk_msebi_s", CLK_REF_SYNC_D4, 0x160, 0x161, 0x162, 0x163, 0x180, 0x181, 0x182),
- D_GATE(HCLK_NAND, "hclk_nand", CLK_REF_SYNC_D4, 0x280, 0x281, 0x282, 0x283, 0x2e0, 0x2e1, 0x2e2),
- D_GATE(HCLK_PG_I, "hclk_pg_i", CLK_REF_SYNC_D4, 0x7ac, 0x7ad, 0, 0x7ae, 0, 0xb24, 0xb25),
- D_GATE(HCLK_PG19, "hclk_pg19", CLK_REF_SYNC_D4, 0x22c, 0x22d, 0x22e, 0, 0, 0, 0),
- D_GATE(HCLK_PG20, "hclk_pg20", CLK_REF_SYNC_D4, 0x22f, 0x230, 0x231, 0, 0, 0, 0),
- D_GATE(HCLK_PG3, "hclk_pg3", CLK_REF_SYNC_D4, 0x7a6, 0x7a7, 0x7a8, 0, 0xb22, 0, 0),
- D_GATE(HCLK_PG4, "hclk_pg4", CLK_REF_SYNC_D4, 0x7a9, 0x7aa, 0x7ab, 0, 0xb23, 0, 0),
- D_GATE(HCLK_QSPI0, "hclk_qspi0", CLK_REF_SYNC_D4, 0x2a0, 0x2a1, 0x2a2, 0x2a3, 0x300, 0x301, 0x302),
- D_GATE(HCLK_QSPI1, "hclk_qspi1", CLK_REF_SYNC_D4, 0x480, 0x481, 0x482, 0x483, 0x4c0, 0x4c1, 0x4c2),
- D_GATE(HCLK_ROM, "hclk_rom", CLK_REF_SYNC_D4, 0xaa0, 0xaa1, 0xaa2, 0, 0xb80, 0, 0),
- D_GATE(HCLK_RTC, "hclk_rtc", CLK_REF_SYNC_D8, 0xa00, 0, 0, 0, 0, 0, 0),
- D_GATE(HCLK_SDIO0, "hclk_sdio0", CLK_REF_SYNC_D4, 0x60, 0x61, 0x62, 0x63, 0x80, 0x81, 0x82),
- D_GATE(HCLK_SDIO1, "hclk_sdio1", CLK_REF_SYNC_D4, 0x640, 0x641, 0x642, 0x643, 0x660, 0x661, 0x662),
- D_GATE(HCLK_SEMAP, "hclk_semap", CLK_REF_SYNC_D4, 0x7a3, 0x7a4, 0x7a5, 0, 0xb21, 0, 0),
- D_GATE(HCLK_SPI0, "hclk_spi0", CLK_REF_SYNC_D4, 0x200, 0x201, 0x202, 0, 0, 0, 0),
- D_GATE(HCLK_SPI1, "hclk_spi1", CLK_REF_SYNC_D4, 0x203, 0x204, 0x205, 0, 0, 0, 0),
- D_GATE(HCLK_SPI2, "hclk_spi2", CLK_REF_SYNC_D4, 0x206, 0x207, 0x208, 0, 0, 0, 0),
- D_GATE(HCLK_SPI3, "hclk_spi3", CLK_REF_SYNC_D4, 0x209, 0x20a, 0x20b, 0, 0, 0, 0),
- D_GATE(HCLK_SPI4, "hclk_spi4", CLK_REF_SYNC_D4, 0x20c, 0x20d, 0x20e, 0, 0, 0, 0),
- D_GATE(HCLK_SPI5, "hclk_spi5", CLK_REF_SYNC_D4, 0x20f, 0x210, 0x211, 0, 0, 0, 0),
- D_GATE(HCLK_SWITCH, "hclk_switch", CLK_REF_SYNC_D4, 0x980, 0, 0x981, 0, 0, 0, 0),
- D_GATE(HCLK_SWITCH_RG, "hclk_switch_rg", CLK_REF_SYNC_D4, 0xc40, 0xc41, 0xc42, 0, 0, 0, 0),
- D_GATE(HCLK_UART0, "hclk_uart0", CLK_REF_SYNC_D8, 0x1a0, 0x1a1, 0x1a2, 0, 0, 0, 0),
- D_GATE(HCLK_UART1, "hclk_uart1", CLK_REF_SYNC_D8, 0x1a3, 0x1a4, 0x1a5, 0, 0, 0, 0),
- D_GATE(HCLK_UART2, "hclk_uart2", CLK_REF_SYNC_D8, 0x1a6, 0x1a7, 0x1a8, 0, 0, 0, 0),
- D_GATE(HCLK_UART3, "hclk_uart3", CLK_REF_SYNC_D4, 0x218, 0x219, 0x21a, 0, 0, 0, 0),
- D_GATE(HCLK_UART4, "hclk_uart4", CLK_REF_SYNC_D4, 0x21b, 0x21c, 0x21d, 0, 0, 0, 0),
- D_GATE(HCLK_UART5, "hclk_uart5", CLK_REF_SYNC_D4, 0x220, 0x221, 0x222, 0, 0, 0, 0),
- D_GATE(HCLK_UART6, "hclk_uart6", CLK_REF_SYNC_D4, 0x223, 0x224, 0x225, 0, 0, 0, 0),
- D_GATE(HCLK_UART7, "hclk_uart7", CLK_REF_SYNC_D4, 0x226, 0x227, 0x228, 0, 0, 0, 0),
+ D_MODULE(HCLK_ADC, "hclk_adc", CLK_REF_SYNC_D8, 0x1af, 0x1b0, 0x1b1, 0, 0, 0, 0),
+ D_MODULE(HCLK_CM3, "hclk_cm3", CLK_REF_SYNC_D4, 0xc20, 0xc21, 0xc22, 0, 0, 0, 0),
+ D_MODULE(HCLK_CRYPTO_EIP150, "hclk_crypto_eip150", CLK_REF_SYNC_D4, 0x123, 0x124, 0x125, 0, 0x142, 0, 0),
+ D_MODULE(HCLK_CRYPTO_EIP93, "hclk_crypto_eip93", CLK_REF_SYNC_D4, 0x120, 0x121, 0, 0x122, 0, 0x140, 0x141),
+ D_MODULE(HCLK_DDRC, "hclk_ddrc", CLK_REF_SYNC_D4, 0x320, 0x322, 0, 0x321, 0, 0x3a0, 0x3a1),
+ D_MODULE(HCLK_DMA0, "hclk_dma0", CLK_REF_SYNC_D4, 0x260, 0x261, 0x262, 0x263, 0x2c0, 0x2c1, 0x2c2),
+ D_MODULE(HCLK_DMA1, "hclk_dma1", CLK_REF_SYNC_D4, 0x264, 0x265, 0x266, 0x267, 0x2c3, 0x2c4, 0x2c5),
+ D_MODULE(HCLK_GMAC0, "hclk_gmac0", CLK_REF_SYNC_D4, 0x360, 0x361, 0x362, 0x363, 0x3c0, 0x3c1, 0x3c2),
+ D_MODULE(HCLK_GMAC1, "hclk_gmac1", CLK_REF_SYNC_D4, 0x380, 0x381, 0x382, 0x383, 0x3e0, 0x3e1, 0x3e2),
+ D_MODULE(HCLK_GPIO0, "hclk_gpio0", CLK_REF_SYNC_D4, 0x212, 0x213, 0x214, 0, 0, 0, 0),
+ D_MODULE(HCLK_GPIO1, "hclk_gpio1", CLK_REF_SYNC_D4, 0x215, 0x216, 0x217, 0, 0, 0, 0),
+ D_MODULE(HCLK_GPIO2, "hclk_gpio2", CLK_REF_SYNC_D4, 0x229, 0x22a, 0x22b, 0, 0, 0, 0),
+ D_MODULE(HCLK_HSR, "hclk_hsr", CLK_HSR100_D2, 0x480, 0x482, 0, 0x481, 0, 0x4c0, 0x4c1),
+ D_MODULE(HCLK_I2C0, "hclk_i2c0", CLK_REF_SYNC_D8, 0x1a9, 0x1aa, 0x1ab, 0, 0, 0, 0),
+ D_MODULE(HCLK_I2C1, "hclk_i2c1", CLK_REF_SYNC_D8, 0x1ac, 0x1ad, 0x1ae, 0, 0, 0, 0),
+ D_MODULE(HCLK_LCD, "hclk_lcd", CLK_REF_SYNC_D4, 0x7a0, 0x7a1, 0x7a2, 0, 0xb20, 0, 0),
+ D_MODULE(HCLK_MSEBI_M, "hclk_msebi_m", CLK_REF_SYNC_D4, 0x164, 0x165, 0x166, 0, 0x183, 0, 0),
+ D_MODULE(HCLK_MSEBI_S, "hclk_msebi_s", CLK_REF_SYNC_D4, 0x160, 0x161, 0x162, 0x163, 0x180, 0x181, 0x182),
+ D_MODULE(HCLK_NAND, "hclk_nand", CLK_REF_SYNC_D4, 0x280, 0x281, 0x282, 0x283, 0x2e0, 0x2e1, 0x2e2),
+ D_MODULE(HCLK_PG_I, "hclk_pg_i", CLK_REF_SYNC_D4, 0x7ac, 0x7ad, 0, 0x7ae, 0, 0xb24, 0xb25),
+ D_MODULE(HCLK_PG19, "hclk_pg19", CLK_REF_SYNC_D4, 0x22c, 0x22d, 0x22e, 0, 0, 0, 0),
+ D_MODULE(HCLK_PG20, "hclk_pg20", CLK_REF_SYNC_D4, 0x22f, 0x230, 0x231, 0, 0, 0, 0),
+ D_MODULE(HCLK_PG3, "hclk_pg3", CLK_REF_SYNC_D4, 0x7a6, 0x7a7, 0x7a8, 0, 0xb22, 0, 0),
+ D_MODULE(HCLK_PG4, "hclk_pg4", CLK_REF_SYNC_D4, 0x7a9, 0x7aa, 0x7ab, 0, 0xb23, 0, 0),
+ D_MODULE(HCLK_QSPI0, "hclk_qspi0", CLK_REF_SYNC_D4, 0x2a0, 0x2a1, 0x2a2, 0x2a3, 0x300, 0x301, 0x302),
+ D_MODULE(HCLK_QSPI1, "hclk_qspi1", CLK_REF_SYNC_D4, 0x480, 0x481, 0x482, 0x483, 0x4c0, 0x4c1, 0x4c2),
+ D_MODULE(HCLK_ROM, "hclk_rom", CLK_REF_SYNC_D4, 0xaa0, 0xaa1, 0xaa2, 0, 0xb80, 0, 0),
+ D_MODULE(HCLK_RTC, "hclk_rtc", CLK_REF_SYNC_D8, 0xa00, 0, 0, 0, 0, 0, 0),
+ D_MODULE(HCLK_SDIO0, "hclk_sdio0", CLK_REF_SYNC_D4, 0x60, 0x61, 0x62, 0x63, 0x80, 0x81, 0x82),
+ D_MODULE(HCLK_SDIO1, "hclk_sdio1", CLK_REF_SYNC_D4, 0x640, 0x641, 0x642, 0x643, 0x660, 0x661, 0x662),
+ D_MODULE(HCLK_SEMAP, "hclk_semap", CLK_REF_SYNC_D4, 0x7a3, 0x7a4, 0x7a5, 0, 0xb21, 0, 0),
+ D_MODULE(HCLK_SPI0, "hclk_spi0", CLK_REF_SYNC_D4, 0x200, 0x201, 0x202, 0, 0, 0, 0),
+ D_MODULE(HCLK_SPI1, "hclk_spi1", CLK_REF_SYNC_D4, 0x203, 0x204, 0x205, 0, 0, 0, 0),
+ D_MODULE(HCLK_SPI2, "hclk_spi2", CLK_REF_SYNC_D4, 0x206, 0x207, 0x208, 0, 0, 0, 0),
+ D_MODULE(HCLK_SPI3, "hclk_spi3", CLK_REF_SYNC_D4, 0x209, 0x20a, 0x20b, 0, 0, 0, 0),
+ D_MODULE(HCLK_SPI4, "hclk_spi4", CLK_REF_SYNC_D4, 0x20c, 0x20d, 0x20e, 0, 0, 0, 0),
+ D_MODULE(HCLK_SPI5, "hclk_spi5", CLK_REF_SYNC_D4, 0x20f, 0x210, 0x211, 0, 0, 0, 0),
+ D_MODULE(HCLK_SWITCH, "hclk_switch", CLK_REF_SYNC_D4, 0x980, 0, 0x981, 0, 0, 0, 0),
+ D_MODULE(HCLK_SWITCH_RG, "hclk_switch_rg", CLK_REF_SYNC_D4, 0xc40, 0xc41, 0xc42, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART0, "hclk_uart0", CLK_REF_SYNC_D8, 0x1a0, 0x1a1, 0x1a2, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART1, "hclk_uart1", CLK_REF_SYNC_D8, 0x1a3, 0x1a4, 0x1a5, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART2, "hclk_uart2", CLK_REF_SYNC_D8, 0x1a6, 0x1a7, 0x1a8, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART3, "hclk_uart3", CLK_REF_SYNC_D4, 0x218, 0x219, 0x21a, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART4, "hclk_uart4", CLK_REF_SYNC_D4, 0x21b, 0x21c, 0x21d, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART5, "hclk_uart5", CLK_REF_SYNC_D4, 0x220, 0x221, 0x222, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART6, "hclk_uart6", CLK_REF_SYNC_D4, 0x223, 0x224, 0x225, 0, 0, 0, 0),
+ D_MODULE(HCLK_UART7, "hclk_uart7", CLK_REF_SYNC_D4, 0x226, 0x227, 0x228, 0, 0, 0, 0),
/*
* These are not hardware clocks, but are needed to handle the special
* case where we have a 'selector bit' that doesn't just change the
#define to_r9a06g032_gate(_hw) container_of(_hw, struct r9a06g032_clk_gate, hw)
+static int create_add_module_clock(struct of_phandle_args *clkspec,
+ struct device *dev)
+{
+ struct clk *clk;
+ int error;
+
+ clk = of_clk_get_from_provider(clkspec);
+ if (IS_ERR(clk))
+ return PTR_ERR(clk);
+
+ error = pm_clk_create(dev);
+ if (error) {
+ clk_put(clk);
+ return error;
+ }
+
+ error = pm_clk_add_clk(dev, clk);
+ if (error) {
+ pm_clk_destroy(dev);
+ clk_put(clk);
+ }
+
+ return error;
+}
+
+static int r9a06g032_attach_dev(struct generic_pm_domain *pd,
+ struct device *dev)
+{
+ struct device_node *np = dev->of_node;
+ struct of_phandle_args clkspec;
+ int i = 0;
+ int error;
+ int index;
+
+ while (!of_parse_phandle_with_args(np, "clocks", "#clock-cells", i,
+ &clkspec)) {
+ if (clkspec.np != pd->dev.of_node)
+ continue;
+
+ index = clkspec.args[0];
+ if (index < R9A06G032_CLOCK_COUNT &&
+ r9a06g032_clocks[index].managed) {
+ error = create_add_module_clock(&clkspec, dev);
+ of_node_put(clkspec.np);
+ if (error)
+ return error;
+ }
+ i++;
+ }
+
+ return 0;
+}
+
+static void r9a06g032_detach_dev(struct generic_pm_domain *unused, struct device *dev)
+{
+ if (!pm_clk_no_clocks(dev))
+ pm_clk_destroy(dev);
+}
+
+static int r9a06g032_add_clk_domain(struct device *dev)
+{
+ struct device_node *np = dev->of_node;
+ struct generic_pm_domain *pd;
+
+ pd = devm_kzalloc(dev, sizeof(*pd), GFP_KERNEL);
+ if (!pd)
+ return -ENOMEM;
+
+ pd->name = np->name;
+ pd->flags = GENPD_FLAG_PM_CLK | GENPD_FLAG_ACTIVE_WAKEUP;
+ pd->attach_dev = r9a06g032_attach_dev;
+ pd->detach_dev = r9a06g032_detach_dev;
+ pm_genpd_init(pd, &pm_domain_always_on_gov, false);
+
+ of_genpd_add_provider_simple(np, pd);
+ return 0;
+}
+
static void
r9a06g032_clk_gate_set(struct r9a06g032_priv *clocks,
struct r9a06g032_gate *g, int on)
if (error)
return error;
- return devm_add_action_or_reset(dev,
+ error = devm_add_action_or_reset(dev,
r9a06g032_clocks_del_clk_provider, np);
+ if (error)
+ return error;
+
+ return r9a06g032_add_clk_domain(dev);
}
static const struct of_device_id r9a06g032_match[] = {
* @dev: CPG/MSSR device
* @base: CPG/MSSR register block base address
* @rmw_lock: protects RMW register accesses
- * @clks: Array containing all Core and Module Clocks
+ * @np: Device node in DT for this CPG/MSSR module
* @num_core_clks: Number of Core Clocks in clks[]
* @num_mod_clks: Number of Module Clocks in clks[]
* @last_dt_core_clk: ID of the last Core Clock exported to DT
+ * @stbyctrl: This device has Standby Control Registers
* @notifiers: Notifier chain to save/restore clock state for system resume
* @smstpcr_saved[].mask: Mask of SMSTPCR[] bits under our control
* @smstpcr_saved[].val: Saved values of SMSTPCR[]
- * @stbyctrl: This device has Standby Control Registers
+ * @clks: Array containing all Core and Module Clocks
*/
struct cpg_mssr_priv {
#ifdef CONFIG_RESET_CONTROLLER
spinlock_t rmw_lock;
struct device_node *np;
- struct clk **clks;
unsigned int num_core_clks;
unsigned int num_mod_clks;
unsigned int last_dt_core_clk;
u32 mask;
u32 val;
} smstpcr_saved[ARRAY_SIZE(smstpcr)];
+
+ struct clk *clks[];
};
static struct cpg_mssr_priv *cpg_mssr_priv;
struct cpg_mssr_clk_domain {
struct generic_pm_domain genpd;
- struct device_node *np;
unsigned int num_core_pm_clks;
- unsigned int core_pm_clks[0];
+ unsigned int core_pm_clks[];
};
static struct cpg_mssr_clk_domain *cpg_mssr_clk_domain;
{
unsigned int i;
- if (clkspec->np != pd->np || clkspec->args_count != 2)
+ if (clkspec->np != pd->genpd.dev.of_node || clkspec->args_count != 2)
return false;
switch (clkspec->args[0]) {
return PTR_ERR(clk);
error = pm_clk_create(dev);
- if (error) {
- dev_err(dev, "pm_clk_create failed %d\n", error);
+ if (error)
goto fail_put;
- }
error = pm_clk_add_clk(dev, clk);
- if (error) {
- dev_err(dev, "pm_clk_add_clk %pC failed %d\n", clk, error);
+ if (error)
goto fail_destroy;
- }
return 0;
if (!pd)
return -ENOMEM;
- pd->np = np;
pd->num_core_pm_clks = num_core_pm_clks;
memcpy(pd->core_pm_clks, core_pm_clks, pm_size);
const struct cpg_mssr_info *info)
{
struct cpg_mssr_priv *priv;
- struct clk **clks = NULL;
unsigned int nclks, i;
int error;
return error;
}
- priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ nclks = info->num_total_core_clks + info->num_hw_mod_clks;
+ priv = kzalloc(struct_size(priv, clks, nclks), GFP_KERNEL);
if (!priv)
return -ENOMEM;
goto out_err;
}
- nclks = info->num_total_core_clks + info->num_hw_mod_clks;
- clks = kmalloc_array(nclks, sizeof(*clks), GFP_KERNEL);
- if (!clks) {
- error = -ENOMEM;
- goto out_err;
- }
-
cpg_mssr_priv = priv;
- priv->clks = clks;
priv->num_core_clks = info->num_total_core_clks;
priv->num_mod_clks = info->num_hw_mod_clks;
priv->last_dt_core_clk = info->last_dt_core_clk;
priv->stbyctrl = info->stbyctrl;
for (i = 0; i < nclks; i++)
- clks[i] = ERR_PTR(-ENOENT);
+ priv->clks[i] = ERR_PTR(-ENOENT);
error = of_clk_add_provider(np, cpg_mssr_clk_src_twocell_get, priv);
if (error)
return 0;
out_err:
- kfree(clks);
if (priv->base)
iounmap(priv->base);
kfree(priv);
static int rockchip_mmc_get_phase(struct clk_hw *hw)
{
struct rockchip_mmc_clock *mmc_clock = to_mmc_clock(hw);
- unsigned long rate = clk_get_rate(hw->clk);
+ unsigned long rate = clk_hw_get_rate(hw);
u32 raw_value;
u16 degrees;
u32 delay_num = 0;
/* See the comment for rockchip_mmc_set_phase below */
- if (!rate) {
- pr_err("%s: invalid clk rate\n", __func__);
+ if (!rate)
return -EINVAL;
- }
raw_value = readl(mmc_clock->reg) >> (mmc_clock->shift);
degrees = (raw_value & ROCKCHIP_MMC_DEGREE_MASK) * 90;
if (raw_value & ROCKCHIP_MMC_DELAY_SEL) {
- /* degrees/delaynum * 10000 */
+ /* degrees/delaynum * 1000000 */
unsigned long factor = (ROCKCHIP_MMC_DELAY_ELEMENT_PSEC / 10) *
- 36 * (rate / 1000000);
+ 36 * (rate / 10000);
delay_num = (raw_value & ROCKCHIP_MMC_DELAYNUM_MASK);
delay_num >>= ROCKCHIP_MMC_DELAYNUM_OFFSET;
- degrees += DIV_ROUND_CLOSEST(delay_num * factor, 10000);
+ degrees += DIV_ROUND_CLOSEST(delay_num * factor, 1000000);
}
return degrees % 360;
static int rockchip_mmc_set_phase(struct clk_hw *hw, int degrees)
{
struct rockchip_mmc_clock *mmc_clock = to_mmc_clock(hw);
- unsigned long rate = clk_get_rate(hw->clk);
+ unsigned long rate = clk_hw_get_rate(hw);
u8 nineties, remainder;
u8 delay_num;
u32 raw_value;
GATE(ACLK_GIC, "aclk_gic", "aclk_bus_pre", CLK_IGNORE_UNUSED, PX30_CLKGATE_CON(13), 12, GFLAGS),
GATE(ACLK_DCF, "aclk_dcf", "aclk_bus_pre", 0, PX30_CLKGATE_CON(13), 15, GFLAGS),
+ /* aclk_dmac is controlled by sgrf_soc_con1[11]. */
+ SGRF_GATE(ACLK_DMAC, "aclk_dmac", "aclk_bus_pre"),
+
GATE(0, "hclk_bus_niu", "hclk_bus_pre", CLK_IGNORE_UNUSED, PX30_CLKGATE_CON(13), 9, GFLAGS),
GATE(0, "hclk_rom", "hclk_bus_pre", CLK_IGNORE_UNUSED, PX30_CLKGATE_CON(13), 14, GFLAGS),
GATE(HCLK_PDM, "hclk_pdm", "hclk_bus_pre", 0, PX30_CLKGATE_CON(14), 1, GFLAGS),
{
struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
- struct clk *clk;
reg_base = of_iomap(np, 0);
if (!reg_base) {
return;
}
- /* aclk_dmac is controlled by sgrf_soc_con1[11]. */
- clk = clk_register_fixed_factor(NULL, "aclk_dmac", "aclk_bus_pre", 0, 1, 1);
- if (IS_ERR(clk))
- pr_warn("%s: could not register clock aclk_dmac: %ld\n",
- __func__, PTR_ERR(clk));
- else
- rockchip_clk_add_lookup(ctx, clk, ACLK_DMAC);
-
rockchip_clk_register_plls(ctx, px30_pll_clks,
ARRAY_SIZE(px30_pll_clks),
PX30_GRF_SOC_STATUS0);
RK3228_CPUCLK_RATE(1608000000, 1, 7),
RK3228_CPUCLK_RATE(1512000000, 1, 7),
RK3228_CPUCLK_RATE(1488000000, 1, 5),
+ RK3228_CPUCLK_RATE(1464000000, 1, 5),
RK3228_CPUCLK_RATE(1416000000, 1, 5),
RK3228_CPUCLK_RATE(1392000000, 1, 5),
RK3228_CPUCLK_RATE(1296000000, 1, 5),
RK2928_CLKGATE_CON(4), 0, GFLAGS),
/* PD_MISC */
- MUX(0, "hdmiphy", mux_hdmiphy_p, CLK_SET_RATE_PARENT,
+ MUX(SCLK_HDMI_PHY, "hdmiphy", mux_hdmiphy_p, CLK_SET_RATE_PARENT,
RK2928_MISC_CON, 13, 1, MFLAGS),
MUX(0, "usb480m_phy", mux_usb480m_phy_p, CLK_SET_RATE_PARENT,
RK2928_MISC_CON, 14, 1, MFLAGS),
RK3066_PLL_RATE( 160000000, 1, 80, 12),
RK3066_PLL_RATE( 157500000, 1, 105, 16),
RK3066_PLL_RATE( 126000000, 1, 84, 16),
- RK3066_PLL_RATE( 48000000, 1, 64, 32),
{ /* sentinel */ },
};
GATE(PCLK_GRF, "pclk_grf", "pclk_pd_alive", CLK_IGNORE_UNUSED, RK3288_CLKGATE_CON(14), 11, GFLAGS),
GATE(0, "pclk_alive_niu", "pclk_pd_alive", 0, RK3288_CLKGATE_CON(14), 12, GFLAGS),
+ /* Watchdog pclk is controlled by RK3288_SGRF_SOC_CON0[1]. */
+ SGRF_GATE(PCLK_WDT, "pclk_wdt", "pclk_pd_alive"),
+
/* pclk_pd_pmu gates */
GATE(PCLK_PMU, "pclk_pmu", "pclk_pd_pmu", CLK_IGNORE_UNUSED, RK3288_CLKGATE_CON(17), 0, GFLAGS),
GATE(0, "pclk_intmem1", "pclk_pd_pmu", CLK_IGNORE_UNUSED, RK3288_CLKGATE_CON(17), 1, GFLAGS),
static void __init rk3288_clk_init(struct device_node *np)
{
struct rockchip_clk_provider *ctx;
- struct clk *clk;
rk3288_cru_base = of_iomap(np, 0);
if (!rk3288_cru_base) {
return;
}
- /* Watchdog pclk is controlled by RK3288_SGRF_SOC_CON0[1]. */
- clk = clk_register_fixed_factor(NULL, "pclk_wdt", "pclk_pd_alive", 0, 1, 1);
- if (IS_ERR(clk))
- pr_warn("%s: could not register clock pclk_wdt: %ld\n",
- __func__, PTR_ERR(clk));
- else
- rockchip_clk_add_lookup(ctx, clk, PCLK_WDT);
-
rockchip_clk_register_plls(ctx, rk3288_pll_clks,
ARRAY_SIZE(rk3288_pll_clks),
RK3288_GRF_SOC_STATUS1);
GATE(PCLK_SARADC, "pclk_saradc", "pclk_bus", 0, RK3328_CLKGATE_CON(17), 15, GFLAGS),
GATE(0, "pclk_pmu", "pclk_bus", CLK_IGNORE_UNUSED, RK3328_CLKGATE_CON(28), 3, GFLAGS),
+ /* Watchdog pclk is controlled from the secure GRF */
+ SGRF_GATE(PCLK_WDT, "pclk_wdt", "pclk_bus"),
+
GATE(PCLK_USB3PHY_OTG, "pclk_usb3phy_otg", "pclk_phy_pre", 0, RK3328_CLKGATE_CON(28), 1, GFLAGS),
GATE(PCLK_USB3PHY_PIPE, "pclk_usb3phy_pipe", "pclk_phy_pre", 0, RK3328_CLKGATE_CON(28), 2, GFLAGS),
GATE(PCLK_USB3_GRF, "pclk_usb3_grf", "pclk_phy_pre", CLK_IGNORE_UNUSED, RK3328_CLKGATE_CON(17), 2, GFLAGS),
GATE(PCLK_GPIO2, "pclk_gpio2", "pclk_pd_alive", 0, RK3368_CLKGATE_CON(22), 2, GFLAGS),
GATE(PCLK_GPIO1, "pclk_gpio1", "pclk_pd_alive", 0, RK3368_CLKGATE_CON(22), 1, GFLAGS),
+ /* Watchdog pclk is controlled by sgrf_soc_con3[7]. */
+ SGRF_GATE(PCLK_WDT, "pclk_wdt", "pclk_pd_alive"),
+
/*
* pclk_vio gates
* pclk_vio comes from the exactly same source as hclk_vio
{
struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
- struct clk *clk;
reg_base = of_iomap(np, 0);
if (!reg_base) {
return;
}
- /* Watchdog pclk is controlled by sgrf_soc_con3[7]. */
- clk = clk_register_fixed_factor(NULL, "pclk_wdt", "pclk_pd_alive", 0, 1, 1);
- if (IS_ERR(clk))
- pr_warn("%s: could not register clock pclk_wdt: %ld\n",
- __func__, PTR_ERR(clk));
- else
- rockchip_clk_add_lookup(ctx, clk, PCLK_WDT);
-
rockchip_clk_register_plls(ctx, rk3368_pll_clks,
ARRAY_SIZE(rk3368_pll_clks),
RK3368_GRF_SOC_STATUS0);
GATE(PCLK_PMU_INTR_ARB, "pclk_pmu_intr_arb", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 9, GFLAGS),
GATE(PCLK_SGRF, "pclk_sgrf", "pclk_alive", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(31), 10, GFLAGS),
+ /* Watchdog pclk is controlled by RK3399 SECURE_GRF_SOC_CON3[8]. */
+ SGRF_GATE(PCLK_WDT, "pclk_wdt", "pclk_alive"),
+
GATE(SCLK_MIPIDPHY_REF, "clk_mipidphy_ref", "xin24m", 0, RK3399_CLKGATE_CON(11), 14, GFLAGS),
GATE(SCLK_DPHY_PLL, "clk_dphy_pll", "clk_mipidphy_ref", CLK_IGNORE_UNUSED, RK3399_CLKGATE_CON(21), 0, GFLAGS),
{
struct rockchip_clk_provider *ctx;
void __iomem *reg_base;
- struct clk *clk;
reg_base = of_iomap(np, 0);
if (!reg_base) {
return;
}
- /* Watchdog pclk is controlled by RK3399 SECURE_GRF_SOC_CON3[8]. */
- clk = clk_register_fixed_factor(NULL, "pclk_wdt", "pclk_alive", 0, 1, 1);
- if (IS_ERR(clk))
- pr_warn("%s: could not register clock pclk_wdt: %ld\n",
- __func__, PTR_ERR(clk));
- else
- rockchip_clk_add_lookup(ctx, clk, PCLK_WDT);
-
rockchip_clk_register_plls(ctx, rk3399_pll_clks,
ARRAY_SIZE(rk3399_pll_clks), -1);
.gate_offset = -1, \
}
+/* SGRF clocks are only accessible from secure mode, so not controllable */
+#define SGRF_GATE(_id, cname, pname) \
+ FACTOR(_id, cname, pname, 0, 1, 1)
+
struct rockchip_clk_provider *rockchip_clk_init(struct device_node *np,
void __iomem *base, unsigned long nr_clks);
void rockchip_clk_of_add_provider(struct device_node *np,
/* list of gate clocks supported in exynos4x12 soc */
static const struct samsung_gate_clock exynos4x12_gate_clks[] __initconst = {
+ GATE(CLK_ASYNC_G3D, "async_g3d", "aclk200", GATE_IP_LEFTBUS, 6, 0, 0),
GATE(CLK_AUDSS, "audss", "sclk_epll", E4X12_GATE_IP_MAUDIO, 0, 0, 0),
GATE(CLK_MDNIE0, "mdnie0", "aclk160", GATE_IP_LCD0, 2, 0, 0),
GATE(CLK_ROTATOR, "rotator", "aclk200", E4X12_GATE_IP_IMAGE, 1, 0, 0),
#define SRC_CDREX 0x20200
#define DIV_CDREX0 0x20500
#define DIV_CDREX1 0x20504
+#define GATE_BUS_CDREX0 0x20700
+#define GATE_BUS_CDREX1 0x20704
#define KPLL_LOCK 0x28000
#define KPLL_CON0 0x28100
#define SRC_KFC 0x28200
DIV_CDREX1,
SRC_KFC,
DIV_KFC0,
+ GATE_BUS_CDREX0,
+ GATE_BUS_CDREX1,
};
static const unsigned long exynos5800_clk_regs[] __initconst = {
PNAME(mout_group14_5800_p) = { "dout_aclk550_cam", "dout_sclk_sw" };
PNAME(mout_group15_5800_p) = { "dout_osc_div", "mout_sw_aclk550_cam" };
PNAME(mout_group16_5800_p) = { "dout_osc_div", "mout_mau_epll_clk" };
+PNAME(mout_mx_mspll_ccore_phy_p) = { "sclk_bpll", "mout_sclk_dpll",
+ "mout_sclk_mpll", "ff_dout_spll2",
+ "mout_sclk_spll", "mout_sclk_epll"};
/* fixed rate clocks generated outside the soc */
static struct samsung_fixed_rate_clock
static const struct samsung_fixed_factor_clock
exynos5800_fixed_factor_clks[] __initconst = {
FFACTOR(0, "ff_dout_epll2", "mout_sclk_epll", 1, 2, 0),
- FFACTOR(0, "ff_dout_spll2", "mout_sclk_spll", 1, 2, 0),
+ FFACTOR(CLK_FF_DOUT_SPLL2, "ff_dout_spll2", "mout_sclk_spll", 1, 2, 0),
};
static const struct samsung_mux_clock exynos5800_mux_clks[] __initconst = {
MUX(0, "mout_aclk300_disp1", mout_group5_5800_p, SRC_TOP2, 24, 2),
MUX(0, "mout_aclk300_gscl", mout_group5_5800_p, SRC_TOP2, 28, 2),
+ MUX(CLK_MOUT_MX_MSPLL_CCORE_PHY, "mout_mx_mspll_ccore_phy",
+ mout_mx_mspll_ccore_phy_p, SRC_TOP7, 0, 3),
+
MUX(CLK_MOUT_MX_MSPLL_CCORE, "mout_mx_mspll_ccore",
- mout_mx_mspll_ccore_p, SRC_TOP7, 16, 2),
+ mout_mx_mspll_ccore_p, SRC_TOP7, 16, 3),
MUX_F(CLK_MOUT_MAU_EPLL, "mout_mau_epll_clk", mout_mau_epll_clk_5800_p,
SRC_TOP7, 20, 2, CLK_SET_RATE_PARENT, 0),
- MUX(0, "sclk_bpll", mout_bpll_p, SRC_TOP7, 24, 1),
+ MUX(CLK_SCLK_BPLL, "sclk_bpll", mout_bpll_p, SRC_TOP7, 24, 1),
MUX(0, "mout_epll2", mout_epll2_5800_p, SRC_TOP7, 28, 1),
MUX(0, "mout_aclk550_cam", mout_group3_5800_p, SRC_TOP8, 16, 3),
MUX(0, "mout_sclk_mpll", mout_mpll_p, SRC_TOP6, 0, 1),
MUX(CLK_MOUT_VPLL, "mout_sclk_vpll", mout_vpll_p, SRC_TOP6, 4, 1),
- MUX(0, "mout_sclk_spll", mout_spll_p, SRC_TOP6, 8, 1),
+ MUX(CLK_MOUT_SCLK_SPLL, "mout_sclk_spll", mout_spll_p, SRC_TOP6, 8, 1),
MUX(0, "mout_sclk_ipll", mout_ipll_p, SRC_TOP6, 12, 1),
MUX(0, "mout_sclk_rpll", mout_rpll_p, SRC_TOP6, 16, 1),
MUX_F(CLK_MOUT_EPLL, "mout_sclk_epll", mout_epll_p, SRC_TOP6, 20, 1,
"mout_aclk400_disp1", DIV_TOP2, 4, 3),
/* CDREX Block */
- DIV(CLK_DOUT_PCLK_CDREX, "dout_pclk_cdrex", "dout_aclk_cdrex1",
- DIV_CDREX0, 28, 3),
+ /*
+ * The three clocks below are controlled using the same register and
+ * bits. They are put into one because there is a need of
+ * synchronization between the BUS and DREXs (two external memory
+ * interfaces).
+ * They are put here to show this HW assumption and for clock
+ * information summary completeness.
+ */
+ DIV_F(CLK_DOUT_PCLK_CDREX, "dout_pclk_cdrex", "dout_aclk_cdrex1",
+ DIV_CDREX0, 28, 3, CLK_GET_RATE_NOCACHE, 0),
+ DIV_F(CLK_DOUT_PCLK_DREX0, "dout_pclk_drex0", "dout_cclk_drex0",
+ DIV_CDREX0, 28, 3, CLK_GET_RATE_NOCACHE, 0),
+ DIV_F(CLK_DOUT_PCLK_DREX1, "dout_pclk_drex1", "dout_cclk_drex0",
+ DIV_CDREX0, 28, 3, CLK_GET_RATE_NOCACHE, 0),
+
DIV_F(CLK_DOUT_SCLK_CDREX, "dout_sclk_cdrex", "mout_mclk_cdrex",
DIV_CDREX0, 24, 3, CLK_SET_RATE_PARENT, 0),
DIV(CLK_DOUT_ACLK_CDREX1, "dout_aclk_cdrex1", "dout_clk2x_phy0",
GATE_TOP_SCLK_ISP, 12, CLK_SET_RATE_PARENT, 0),
GATE(CLK_G3D, "g3d", "mout_user_aclk_g3d", GATE_IP_G3D, 9, 0, 0),
+
+ /* CDREX */
+ GATE(CLK_CLKM_PHY0, "clkm_phy0", "dout_sclk_cdrex",
+ GATE_BUS_CDREX0, 0, 0, 0),
+ GATE(CLK_CLKM_PHY1, "clkm_phy1", "dout_sclk_cdrex",
+ GATE_BUS_CDREX0, 1, 0, 0),
+ GATE(0, "mx_mspll_ccore_phy", "mout_mx_mspll_ccore_phy",
+ SRC_MASK_TOP7, 0, CLK_IGNORE_UNUSED, 0),
+
+ GATE(CLK_ACLK_PPMU_DREX1_1, "aclk_ppmu_drex1_1", "dout_aclk_cdrex1",
+ GATE_BUS_CDREX1, 12, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_ACLK_PPMU_DREX1_0, "aclk_ppmu_drex1_0", "dout_aclk_cdrex1",
+ GATE_BUS_CDREX1, 13, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_ACLK_PPMU_DREX0_1, "aclk_ppmu_drex0_1", "dout_aclk_cdrex1",
+ GATE_BUS_CDREX1, 14, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_ACLK_PPMU_DREX0_0, "aclk_ppmu_drex0_0", "dout_aclk_cdrex1",
+ GATE_BUS_CDREX1, 15, CLK_IGNORE_UNUSED, 0),
+
+ GATE(CLK_PCLK_PPMU_DREX1_1, "pclk_ppmu_drex1_1", "dout_pclk_cdrex",
+ GATE_BUS_CDREX1, 26, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_PCLK_PPMU_DREX1_0, "pclk_ppmu_drex1_0", "dout_pclk_cdrex",
+ GATE_BUS_CDREX1, 27, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_PCLK_PPMU_DREX0_1, "pclk_ppmu_drex0_1", "dout_pclk_cdrex",
+ GATE_BUS_CDREX1, 28, CLK_IGNORE_UNUSED, 0),
+ GATE(CLK_PCLK_PPMU_DREX0_0, "pclk_ppmu_drex0_0", "dout_pclk_cdrex",
+ GATE_BUS_CDREX1, 29, CLK_IGNORE_UNUSED, 0),
};
static const struct samsung_div_clock exynos5x_disp_div_clks[] __initconst = {
PLL_35XX_RATE(24 * MHZ, 200000000, 200, 3, 3),
};
+static const struct samsung_pll_rate_table exynos5422_bpll_rate_table[] = {
+ PLL_35XX_RATE(24 * MHZ, 825000000, 275, 4, 1),
+ PLL_35XX_RATE(24 * MHZ, 728000000, 182, 3, 1),
+ PLL_35XX_RATE(24 * MHZ, 633000000, 211, 4, 1),
+ PLL_35XX_RATE(24 * MHZ, 543000000, 181, 2, 2),
+ PLL_35XX_RATE(24 * MHZ, 413000000, 413, 6, 2),
+ PLL_35XX_RATE(24 * MHZ, 275000000, 275, 3, 3),
+ PLL_35XX_RATE(24 * MHZ, 206000000, 206, 3, 3),
+ PLL_35XX_RATE(24 * MHZ, 165000000, 110, 2, 3),
+};
+
static const struct samsung_pll_rate_table exynos5420_epll_24mhz_tbl[] = {
PLL_36XX_RATE(24 * MHZ, 600000000U, 100, 2, 1, 0),
PLL_36XX_RATE(24 * MHZ, 400000000U, 200, 3, 2, 0),
exynos5x_plls[apll].rate_table = exynos5420_pll2550x_24mhz_tbl;
exynos5x_plls[epll].rate_table = exynos5420_epll_24mhz_tbl;
exynos5x_plls[kpll].rate_table = exynos5420_pll2550x_24mhz_tbl;
- exynos5x_plls[bpll].rate_table = exynos5420_pll2550x_24mhz_tbl;
}
+ if (soc == EXYNOS5420)
+ exynos5x_plls[bpll].rate_table = exynos5420_pll2550x_24mhz_tbl;
+ else
+ exynos5x_plls[bpll].rate_table = exynos5422_bpll_rate_table;
+
samsung_clk_register_pll(ctx, exynos5x_plls, ARRAY_SIZE(exynos5x_plls),
reg_base);
samsung_clk_register_fixed_rate(ctx, exynos5x_fixed_rate_clks,
data->nr_clk_save = info->nr_clk_regs;
data->clk_suspend = info->suspend_regs;
data->nr_clk_suspend = info->nr_suspend_regs;
- data->nr_pclks = of_count_phandle_with_args(dev->of_node, "clocks",
- "#clock-cells");
+ data->nr_pclks = of_clk_get_parent_count(dev->of_node);
+
if (data->nr_pclks > 0) {
data->pclks = devm_kcalloc(dev, sizeof(struct clk *),
data->nr_pclks, GFP_KERNEL);
8, 0, 0, 0, 0, 0, 0},
{ STRATIX10_SPI_M_CLK, "spi_m_clk", "l4_mp_clk", NULL, 1, 0, 0xA4,
9, 0, 0, 0, 0, 0, 0},
- { STRATIX10_NAND_CLK, "nand_clk", "l4_main_clk", NULL, 1, 0, 0xA4,
+ { STRATIX10_NAND_X_CLK, "nand_x_clk", "l4_mp_clk", NULL, 1, 0, 0xA4,
10, 0, 0, 0, 0, 0, 0},
+ { STRATIX10_NAND_CLK, "nand_clk", "nand_x_clk", NULL, 1, 0, 0xA4,
+ 10, 0, 0, 0, 0, 0, 4},
+ { STRATIX10_NAND_ECC_CLK, "nand_ecc_clk", "nand_x_clk", NULL, 1, 0, 0xA4,
+ 10, 0, 0, 0, 0, 0, 4},
};
static int s10_clk_register_c_perip(const struct stratix10_perip_c_clock *clks,
void __iomem *base;
struct device_node *node = pdev->dev.of_node;
struct regmap *regmap;
+ struct resource *res;
if (of_find_property(node, "sprd,syscon", NULL)) {
regmap = syscon_regmap_lookup_by_phandle(node, "sprd,syscon");
return PTR_ERR(regmap);
}
} else {
- base = of_iomap(node, 0);
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
regmap = devm_regmap_init_mmio(&pdev->dev, base,
&sprdclk_regmap_config);
- if (IS_ERR_OR_NULL(regmap)) {
+ if (IS_ERR(regmap)) {
pr_err("failed to init regmap\n");
return PTR_ERR(regmap);
}
{
const struct of_device_id *match;
const struct sprd_clk_desc *desc;
+ int ret;
match = of_match_node(sprd_sc9860_clk_ids, pdev->dev.of_node);
if (!match) {
}
desc = match->data;
- sprd_clk_regmap_init(pdev, desc);
+ ret = sprd_clk_regmap_init(pdev, desc);
+ if (ret)
+ return ret;
return sprd_clk_probe(&pdev->dev, desc->hw_clks);
}
},
};
-static CLK_FIXED_FACTOR(pll_periph_clk, "pll-periph", "pll-periph-base",
- 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph_clk, "pll-periph",
+ &pll_periph_base_clk.common.hw,
+ 2, 1, CLK_SET_RATE_PARENT);
/* Not documented on A10 */
static struct ccu_div pll_periph_sata_clk = {
&out_b_clk.common
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* Post-divider for pll-audio is hardcoded to 1 */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video0_2x_clk, "pll-video0-2x",
- "pll-video0", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video1_2x_clk, "pll-video1-2x",
- "pll-video1", 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video0_2x_clk, "pll-video0-2x",
+ &pll_video0_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video1_2x_clk, "pll-video1-2x",
+ &pll_video1_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
static struct clk_hw_onecell_data sun4i_a10_hw_clks = {
0x1a0, 0, 3, BIT(31), CLK_SET_RATE_PARENT);
/* Fixed Factor clocks */
-static CLK_FIXED_FACTOR(osc12M_clk, "osc12M", "osc24M", 2, 1, 0);
+static CLK_FIXED_FACTOR_FW_NAME(osc12M_clk, "osc12M", "hosc", 2, 1, 0);
+
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph0_2x_clk, "pll-periph0-2x",
- "pll-periph0", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_periph1_2x_clk, "pll-periph1-2x",
- "pll-periph1", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video0_2x_clk, "pll-video0-2x",
- "pll-video0", 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph0_2x_clk, "pll-periph0-2x",
+ &pll_periph0_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_periph1_2x_clk, "pll-periph1-2x",
+ &pll_periph1_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video0_2x_clk, "pll-video0-2x",
+ &pll_video0_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
static struct ccu_common *sun50i_a64_ccu_clks[] = {
&pll_cpux_clk.common,
},
};
-static CLK_FIXED_FACTOR(r_ahb_clk, "r-ahb", "ar100", 1, 1, 0);
+static CLK_FIXED_FACTOR_HW(r_ahb_clk, "r-ahb", &ar100_clk.common.hw, 1, 1, 0);
static struct ccu_div r_apb1_clk = {
.div = _SUNXI_CCU_DIV(0, 2),
static SUNXI_CCU_GATE(r_apb1_ir_clk, "r-apb1-ir", "r-apb1",
0x1cc, BIT(0), 0);
static SUNXI_CCU_GATE(r_apb1_w1_clk, "r-apb1-w1", "r-apb1",
- 0x1cc, BIT(0), 0);
+ 0x1ec, BIT(0), 0);
/* Information of IR(RX) mod clock is gathered from BSP source code */
static const char * const r_mod0_default_parents[] = { "osc32k", "osc24M" };
static SUNXI_CCU_GATE(bus_ehci3_clk, "bus-ehci3", "ahb3", 0xa8c, BIT(7), 0);
static SUNXI_CCU_GATE(bus_otg_clk, "bus-otg", "ahb3", 0xa8c, BIT(8), 0);
-static CLK_FIXED_FACTOR(pcie_ref_100m_clk, "pcie-ref-100M",
- "pll-periph0-4x", 24, 1, 0);
+static struct clk_fixed_factor pll_periph0_4x_clk;
+static CLK_FIXED_FACTOR_HW(pcie_ref_100m_clk, "pcie-ref-100M",
+ &pll_periph0_4x_clk.hw, 24, 1, 0);
static SUNXI_CCU_GATE(pcie_ref_clk, "pcie-ref", "pcie-ref-100M",
0xab0, BIT(31), 0);
static SUNXI_CCU_GATE(pcie_ref_out_clk, "pcie-ref-out", "pcie-ref",
static SUNXI_CCU_GATE(bus_hdcp_clk, "bus-hdcp", "ahb3", 0xc4c, BIT(0), 0);
/* Fixed factor clocks */
-static CLK_FIXED_FACTOR(osc12M_clk, "osc12M", "osc24M", 2, 1, 0);
+static CLK_FIXED_FACTOR_FW_NAME(osc12M_clk, "osc12M", "hosc", 2, 1, 0);
+
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
/*
* The divider of pll-audio is fixed to 8 now, as pll-audio-4x has a
* fixed post-divider 2.
*/
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 8, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 4, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-
-static CLK_FIXED_FACTOR(pll_periph0_4x_clk, "pll-periph0-4x",
- "pll-periph0", 1, 4, 0);
-static CLK_FIXED_FACTOR(pll_periph0_2x_clk, "pll-periph0-2x",
- "pll-periph0", 1, 2, 0);
-
-static CLK_FIXED_FACTOR(pll_periph1_4x_clk, "pll-periph1-4x",
- "pll-periph1", 1, 4, 0);
-static CLK_FIXED_FACTOR(pll_periph1_2x_clk, "pll-periph1-2x",
- "pll-periph1", 1, 2, 0);
-
-static CLK_FIXED_FACTOR(pll_video0_4x_clk, "pll-video0-4x",
- "pll-video0", 1, 4, CLK_SET_RATE_PARENT);
-
-static CLK_FIXED_FACTOR(pll_video1_4x_clk, "pll-video1-4x",
- "pll-video1", 1, 4, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 8, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 4, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+
+static const struct clk_hw *pll_periph0_parents[] = {
+ &pll_periph0_clk.common.hw
+};
+static CLK_FIXED_FACTOR_HWS(pll_periph0_4x_clk, "pll-periph0-4x",
+ pll_periph0_parents,
+ 1, 4, 0);
+static CLK_FIXED_FACTOR_HWS(pll_periph0_2x_clk, "pll-periph0-2x",
+ pll_periph0_parents,
+ 1, 2, 0);
+
+static const struct clk_hw *pll_periph1_parents[] = {
+ &pll_periph1_clk.common.hw
+};
+static CLK_FIXED_FACTOR_HWS(pll_periph1_4x_clk, "pll-periph1-4x",
+ pll_periph1_parents,
+ 1, 4, 0);
+static CLK_FIXED_FACTOR_HWS(pll_periph1_2x_clk, "pll-periph1-2x",
+ pll_periph1_parents,
+ 1, 2, 0);
+
+static CLK_FIXED_FACTOR_HW(pll_video0_4x_clk, "pll-video0-4x",
+ &pll_video0_clk.common.hw,
+ 1, 4, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video1_4x_clk, "pll-video1-4x",
+ &pll_video1_clk.common.hw,
+ 1, 4, CLK_SET_RATE_PARENT);
static struct ccu_common *sun50i_h6_ccu_clks[] = {
&pll_cpux_clk.common,
&iep_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video0_2x_clk, "pll-video0-2x",
- "pll-video0", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video1_2x_clk, "pll-video1-2x",
- "pll-video1", 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video0_2x_clk, "pll-video0-2x",
+ &pll_video0_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video1_2x_clk, "pll-video1-2x",
+ &pll_video1_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
static struct clk_hw_onecell_data sun5i_a10s_hw_clks = {
.hws = {
&out_c_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph_2x_clk, "pll-periph-2x",
- "pll-periph", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video0_2x_clk, "pll-video0-2x",
- "pll-video0", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video1_2x_clk, "pll-video1-2x",
- "pll-video1", 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph_2x_clk, "pll-periph-2x",
+ &pll_periph_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video0_2x_clk, "pll-video0-2x",
+ &pll_video0_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video1_2x_clk, "pll-video1-2x",
+ &pll_video1_clk.common.hw,
+ 1, 2, CLK_SET_RATE_PARENT);
static struct clk_hw_onecell_data sun6i_a31_hw_clks = {
.hws = {
&ats_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph_2x_clk, "pll-periph-2x",
- "pll-periph", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video_2x_clk, "pll-video-2x",
- "pll-video", 1, 2, 0);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph_2x_clk, "pll-periph-2x",
+ &pll_periph_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video_2x_clk, "pll-video-2x",
+ &pll_video_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data sun8i_a23_hw_clks = {
.hws = {
&ats_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph_2x_clk, "pll-periph-2x",
- "pll-periph", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video_2x_clk, "pll-video-2x",
- "pll-video", 1, 2, 0);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph_2x_clk, "pll-periph-2x",
+ &pll_periph_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video_2x_clk, "pll-video-2x",
+ &pll_video_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data sun8i_a33_hw_clks = {
.hws = {
&gpu_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 1 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph0_2x_clk, "pll-periph0-2x",
- "pll-periph0", 1, 2, 0);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph0_2x_clk, "pll-periph0-2x",
+ &pll_periph0_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data sun8i_h3_hw_clks = {
.hws = {
#include "ccu-sun8i-r.h"
-static const char * const ar100_parents[] = { "osc32k", "osc24M",
- "pll-periph0", "iosc" };
-static const char * const a83t_ar100_parents[] = { "osc16M-d512", "osc24M",
- "pll-periph0", "iosc" };
+static const struct clk_parent_data ar100_parents[] = {
+ { .fw_name = "losc" },
+ { .fw_name = "hosc" },
+ { .fw_name = "pll-periph" },
+ { .fw_name = "iosc" },
+};
+
static const struct ccu_mux_var_prediv ar100_predivs[] = {
{ .index = 2, .shift = 8, .width = 5 },
};
.common = {
.reg = 0x00,
.features = CCU_FEATURE_VARIABLE_PREDIV,
- .hw.init = CLK_HW_INIT_PARENTS("ar100",
- ar100_parents,
- &ccu_div_ops,
- 0),
- },
-};
-
-static struct ccu_div a83t_ar100_clk = {
- .div = _SUNXI_CCU_DIV_FLAGS(4, 2, CLK_DIVIDER_POWER_OF_TWO),
-
- .mux = {
- .shift = 16,
- .width = 2,
-
- .var_predivs = ar100_predivs,
- .n_var_predivs = ARRAY_SIZE(ar100_predivs),
- },
-
- .common = {
- .reg = 0x00,
- .features = CCU_FEATURE_VARIABLE_PREDIV,
- .hw.init = CLK_HW_INIT_PARENTS("ar100",
- a83t_ar100_parents,
- &ccu_div_ops,
- 0),
+ .hw.init = CLK_HW_INIT_PARENTS_DATA("ar100",
+ ar100_parents,
+ &ccu_div_ops,
+ 0),
},
};
-static CLK_FIXED_FACTOR(ahb0_clk, "ahb0", "ar100", 1, 1, 0);
+static CLK_FIXED_FACTOR_HW(ahb0_clk, "ahb0", &ar100_clk.common.hw, 1, 1, 0);
static struct ccu_div apb0_clk = {
.div = _SUNXI_CCU_DIV_FLAGS(0, 2, CLK_DIVIDER_POWER_OF_TWO),
.common = {
.reg = 0x0c,
- .hw.init = CLK_HW_INIT("apb0",
- "ahb0",
- &ccu_div_ops,
- 0),
+ .hw.init = CLK_HW_INIT_HW("apb0",
+ &ahb0_clk.hw,
+ &ccu_div_ops,
+ 0),
},
};
static SUNXI_CCU_M(a83t_apb0_clk, "apb0", "ahb0", 0x0c, 0, 2, 0);
-static SUNXI_CCU_GATE(apb0_pio_clk, "apb0-pio", "apb0",
- 0x28, BIT(0), 0);
-static SUNXI_CCU_GATE(apb0_ir_clk, "apb0-ir", "apb0",
- 0x28, BIT(1), 0);
-static SUNXI_CCU_GATE(apb0_timer_clk, "apb0-timer", "apb0",
- 0x28, BIT(2), 0);
-static SUNXI_CCU_GATE(apb0_rsb_clk, "apb0-rsb", "apb0",
- 0x28, BIT(3), 0);
-static SUNXI_CCU_GATE(apb0_uart_clk, "apb0-uart", "apb0",
- 0x28, BIT(4), 0);
-static SUNXI_CCU_GATE(apb0_i2c_clk, "apb0-i2c", "apb0",
- 0x28, BIT(6), 0);
-static SUNXI_CCU_GATE(apb0_twd_clk, "apb0-twd", "apb0",
- 0x28, BIT(7), 0);
+/*
+ * Define the parent as an array that can be reused to save space
+ * instead of having compound literals for each gate. Also have it
+ * non-const so we can change it on the A83T.
+ */
+static const struct clk_hw *apb0_gate_parent[] = { &apb0_clk.common.hw };
+static SUNXI_CCU_GATE_HWS(apb0_pio_clk, "apb0-pio",
+ apb0_gate_parent, 0x28, BIT(0), 0);
+static SUNXI_CCU_GATE_HWS(apb0_ir_clk, "apb0-ir",
+ apb0_gate_parent, 0x28, BIT(1), 0);
+static SUNXI_CCU_GATE_HWS(apb0_timer_clk, "apb0-timer",
+ apb0_gate_parent, 0x28, BIT(2), 0);
+static SUNXI_CCU_GATE_HWS(apb0_rsb_clk, "apb0-rsb",
+ apb0_gate_parent, 0x28, BIT(3), 0);
+static SUNXI_CCU_GATE_HWS(apb0_uart_clk, "apb0-uart",
+ apb0_gate_parent, 0x28, BIT(4), 0);
+static SUNXI_CCU_GATE_HWS(apb0_i2c_clk, "apb0-i2c",
+ apb0_gate_parent, 0x28, BIT(6), 0);
+static SUNXI_CCU_GATE_HWS(apb0_twd_clk, "apb0-twd",
+ apb0_gate_parent, 0x28, BIT(7), 0);
static const char * const r_mod0_default_parents[] = { "osc32k", "osc24M" };
static SUNXI_CCU_MP_WITH_MUX_GATE(ir_clk, "ir",
BIT(31), /* gate */
0);
-static const char *const a83t_r_mod0_parents[] = { "osc16M", "osc24M" };
+static const struct clk_parent_data a83t_r_mod0_parents[] = {
+ { .fw_name = "iosc" },
+ { .fw_name = "hosc" },
+};
static const struct ccu_mux_fixed_prediv a83t_ir_predivs[] = {
{ .index = 0, .div = 16 },
};
.common = {
.reg = 0x54,
.features = CCU_FEATURE_VARIABLE_PREDIV,
- .hw.init = CLK_HW_INIT_PARENTS("ir",
- a83t_r_mod0_parents,
- &ccu_mp_ops,
- 0),
+ .hw.init = CLK_HW_INIT_PARENTS_DATA("ir",
+ a83t_r_mod0_parents,
+ &ccu_mp_ops,
+ 0),
},
};
static struct ccu_common *sun8i_a83t_r_ccu_clks[] = {
- &a83t_ar100_clk.common,
+ &ar100_clk.common,
&a83t_apb0_clk.common,
&apb0_pio_clk.common,
&apb0_ir_clk.common,
static struct clk_hw_onecell_data sun8i_a83t_r_hw_clks = {
.hws = {
- [CLK_AR100] = &a83t_ar100_clk.common.hw,
+ [CLK_AR100] = &ar100_clk.common.hw,
[CLK_AHB0] = &ahb0_clk.hw,
[CLK_APB0] = &a83t_apb0_clk.common.hw,
[CLK_APB0_PIO] = &apb0_pio_clk.common.hw,
static void __init sun8i_a83t_r_ccu_setup(struct device_node *node)
{
+ /* Fix apb0 bus gate parents here */
+ apb0_gate_parent[0] = &a83t_apb0_clk.common.hw;
+
sunxi_r_ccu_init(node, &sun8i_a83t_r_ccu_desc);
}
CLK_OF_DECLARE(sun8i_a83t_r_ccu, "allwinner,sun8i-a83t-r-ccu",
};
/* Fixed Factor clocks */
-static CLK_FIXED_FACTOR(osc12M_clk, "osc12M", "osc24M", 2, 1, 0);
+static CLK_FIXED_FACTOR_FW_NAME(osc12M_clk, "osc12M", "hosc", 2, 1, 0);
+
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
/* We hardcode the divider to 4 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 4, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph0_2x_clk, "pll-periph0-2x",
- "pll-periph0", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_periph1_2x_clk, "pll-periph1-2x",
- "pll-periph1", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video0_2x_clk, "pll-video0-2x",
- "pll-video0", 1, 2, 0);
-static CLK_FIXED_FACTOR(pll_video1_2x_clk, "pll-video1-2x",
- "pll-video1", 1, 2, 0);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 4, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph0_2x_clk, "pll-periph0-2x",
+ &pll_periph0_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_periph1_2x_clk, "pll-periph1-2x",
+ &pll_periph1_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video0_2x_clk, "pll-video0-2x",
+ &pll_video0_clk.common.hw,
+ 1, 2, 0);
+static CLK_FIXED_FACTOR_HW(pll_video1_2x_clk, "pll-video1-2x",
+ &pll_video1_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data sun8i_r40_hw_clks = {
.hws = {
&mipi_csi_clk.common,
};
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
/* We hardcode the divider to 4 for now */
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 4, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_periph0_2x_clk, "pll-periph0-2x",
- "pll-periph0", 1, 2, 0);
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 4, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_periph0_2x_clk, "pll-periph0-2x",
+ &pll_periph0_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data sun8i_v3s_hw_clks = {
.hws = {
#include "ccu-sun9i-a80-usb.h"
-static SUNXI_CCU_GATE(bus_hci0_clk, "bus-hci0", "bus-usb", 0x0, BIT(1), 0);
-static SUNXI_CCU_GATE(usb_ohci0_clk, "usb-ohci0", "osc24M", 0x0, BIT(2), 0);
-static SUNXI_CCU_GATE(bus_hci1_clk, "bus-hci1", "bus-usb", 0x0, BIT(3), 0);
-static SUNXI_CCU_GATE(bus_hci2_clk, "bus-hci2", "bus-usb", 0x0, BIT(5), 0);
-static SUNXI_CCU_GATE(usb_ohci2_clk, "usb-ohci2", "osc24M", 0x0, BIT(6), 0);
-
-static SUNXI_CCU_GATE(usb0_phy_clk, "usb0-phy", "osc24M", 0x4, BIT(1), 0);
-static SUNXI_CCU_GATE(usb1_hsic_clk, "usb1-hsic", "osc24M", 0x4, BIT(2), 0);
-static SUNXI_CCU_GATE(usb1_phy_clk, "usb1-phy", "osc24M", 0x4, BIT(3), 0);
-static SUNXI_CCU_GATE(usb2_hsic_clk, "usb2-hsic", "osc24M", 0x4, BIT(4), 0);
-static SUNXI_CCU_GATE(usb2_phy_clk, "usb2-phy", "osc24M", 0x4, BIT(5), 0);
-static SUNXI_CCU_GATE(usb_hsic_clk, "usb-hsic", "osc24M", 0x4, BIT(10), 0);
+static const struct clk_parent_data clk_parent_hosc[] = {
+ { .fw_name = "hosc" },
+};
+
+static const struct clk_parent_data clk_parent_bus[] = {
+ { .fw_name = "bus" },
+};
+
+static SUNXI_CCU_GATE_DATA(bus_hci0_clk, "bus-hci0", clk_parent_bus, 0x0, BIT(1), 0);
+static SUNXI_CCU_GATE_DATA(usb_ohci0_clk, "usb-ohci0", clk_parent_hosc, 0x0, BIT(2), 0);
+static SUNXI_CCU_GATE_DATA(bus_hci1_clk, "bus-hci1", clk_parent_bus, 0x0, BIT(3), 0);
+static SUNXI_CCU_GATE_DATA(bus_hci2_clk, "bus-hci2", clk_parent_bus, 0x0, BIT(5), 0);
+static SUNXI_CCU_GATE_DATA(usb_ohci2_clk, "usb-ohci2", clk_parent_hosc, 0x0, BIT(6), 0);
+
+static SUNXI_CCU_GATE_DATA(usb0_phy_clk, "usb0-phy", clk_parent_hosc, 0x4, BIT(1), 0);
+static SUNXI_CCU_GATE_DATA(usb1_hsic_clk, "usb1-hsic", clk_parent_hosc, 0x4, BIT(2), 0);
+static SUNXI_CCU_GATE_DATA(usb1_phy_clk, "usb1-phy", clk_parent_hosc, 0x4, BIT(3), 0);
+static SUNXI_CCU_GATE_DATA(usb2_hsic_clk, "usb2-hsic", clk_parent_hosc, 0x4, BIT(4), 0);
+static SUNXI_CCU_GATE_DATA(usb2_phy_clk, "usb2-phy", clk_parent_hosc, 0x4, BIT(5), 0);
+static SUNXI_CCU_GATE_DATA(usb_hsic_clk, "usb-hsic", clk_parent_hosc, 0x4, BIT(10), 0);
static struct ccu_common *sun9i_a80_usb_clks[] = {
&bus_hci0_clk.common,
&avs_clk.common,
};
-static CLK_FIXED_FACTOR(pll_audio_clk, "pll-audio",
- "pll-audio-base", 4, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_2x_clk, "pll-audio-2x",
- "pll-audio-base", 2, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_4x_clk, "pll-audio-4x",
- "pll-audio-base", 1, 1, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_audio_8x_clk, "pll-audio-8x",
- "pll-audio-base", 1, 2, CLK_SET_RATE_PARENT);
-static CLK_FIXED_FACTOR(pll_video_2x_clk, "pll-video-2x",
- "pll-video", 1, 2, 0);
+static const struct clk_hw *clk_parent_pll_audio[] = {
+ &pll_audio_base_clk.common.hw
+};
+
+static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
+ clk_parent_pll_audio,
+ 4, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
+ clk_parent_pll_audio,
+ 2, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_4x_clk, "pll-audio-4x",
+ clk_parent_pll_audio,
+ 1, 1, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HWS(pll_audio_8x_clk, "pll-audio-8x",
+ clk_parent_pll_audio,
+ 1, 2, CLK_SET_RATE_PARENT);
+static CLK_FIXED_FACTOR_HW(pll_video_2x_clk, "pll-video-2x",
+ &pll_video_clk.common.hw,
+ 1, 2, 0);
static struct clk_hw_onecell_data suniv_hw_clks = {
.hws = {
if (!hw)
continue;
- ret = clk_hw_register(NULL, hw);
+ ret = of_clk_hw_register(node, hw);
if (ret) {
pr_err("Couldn't register clock %d - %s\n",
i, clk_hw_get_name(hw));
} \
}
+#define SUNXI_CCU_GATE_HW(_struct, _name, _parent, _reg, _gate, _flags) \
+ struct ccu_gate _struct = { \
+ .enable = _gate, \
+ .common = { \
+ .reg = _reg, \
+ .hw.init = CLK_HW_INIT_HW(_name, \
+ _parent, \
+ &ccu_gate_ops, \
+ _flags), \
+ } \
+ }
+
+#define SUNXI_CCU_GATE_FW(_struct, _name, _parent, _reg, _gate, _flags) \
+ struct ccu_gate _struct = { \
+ .enable = _gate, \
+ .common = { \
+ .reg = _reg, \
+ .hw.init = CLK_HW_INIT_FW_NAME(_name, \
+ _parent, \
+ &ccu_gate_ops, \
+ _flags), \
+ } \
+ }
+
+/*
+ * The following two macros allow the re-use of the data structure
+ * holding the parent info.
+ */
+#define SUNXI_CCU_GATE_HWS(_struct, _name, _parent, _reg, _gate, _flags) \
+ struct ccu_gate _struct = { \
+ .enable = _gate, \
+ .common = { \
+ .reg = _reg, \
+ .hw.init = CLK_HW_INIT_HWS(_name, \
+ _parent, \
+ &ccu_gate_ops, \
+ _flags), \
+ } \
+ }
+
+#define SUNXI_CCU_GATE_DATA(_struct, _name, _data, _reg, _gate, _flags) \
+ struct ccu_gate _struct = { \
+ .enable = _gate, \
+ .common = { \
+ .reg = _reg, \
+ .hw.init = \
+ CLK_HW_INIT_PARENTS_DATA(_name, \
+ _data, \
+ &ccu_gate_ops, \
+ _flags), \
+ } \
+ }
+
static inline struct ccu_gate *hw_to_ccu_gate(struct clk_hw *hw)
{
struct ccu_common *common = hw_to_ccu_common(hw);
if (endp) {
derived_name = kstrndup(clk_name, endp - clk_name,
GFP_KERNEL);
+ if (!derived_name)
+ return NULL;
factors.name = derived_name;
} else {
factors.name = clk_name;
pllre->params->defaults_set = true;
if (val & PLL_ENABLE) {
- pr_warn("PLL_RE already enabled. Postponing set full defaults\n");
-
/*
* PLL is ON: check if defaults already set, then set those
* that can be updated in flight.
_pll_misc_chk_default(clk_base, pllre->params, 0, val,
~mask & PLLRE_MISC0_WRITE_MASK);
- /* Enable lock detect */
+ /* The PLL doesn't work if it's in IDDQ. */
val = readl_relaxed(clk_base + pllre->params->ext_misc_reg[0]);
+ if (val & PLLRE_MISC0_IDDQ)
+ pr_warn("unexpected IDDQ bit set for enabled clock\n");
+
+ /* Enable lock detect */
val &= ~mask;
val |= PLLRE_MISC0_DEFAULT_VALUE & mask;
writel_relaxed(val, clk_base + pllre->params->ext_misc_reg[0]);
udelay(1);
+ if (!pllre->params->defaults_set)
+ pr_warn("PLL_RE already enabled. Postponing set full defaults\n");
+
return;
}
};
static struct tegra_clk_pll_freq_table pll_u_freq_table[] = {
- { 12000000, 480000000, 40, 1, 0, 0 },
- { 13000000, 480000000, 36, 1, 0, 0 }, /* actual: 468.0 MHz */
- { 38400000, 480000000, 25, 2, 0, 0 },
+ { 12000000, 480000000, 40, 1, 1, 0 },
+ { 13000000, 480000000, 36, 1, 1, 0 }, /* actual: 468.0 MHz */
+ { 38400000, 480000000, 25, 2, 1, 0 },
{ 0, 0, 0, 0, 0, 0 },
};
{ TEGRA210_CLK_DFLL_SOC, TEGRA210_CLK_PLL_P, 51000000, 1 },
{ TEGRA210_CLK_DFLL_REF, TEGRA210_CLK_PLL_P, 51000000, 1 },
{ TEGRA210_CLK_SBC4, TEGRA210_CLK_PLL_P, 12000000, 1 },
- { TEGRA210_CLK_PLL_RE_VCO, TEGRA210_CLK_CLK_MAX, 672000000, 1 },
+ { TEGRA210_CLK_PLL_U_OUT1, TEGRA210_CLK_CLK_MAX, 48000000, 1 },
{ TEGRA210_CLK_XUSB_GATE, TEGRA210_CLK_CLK_MAX, 0, 1 },
{ TEGRA210_CLK_XUSB_SS_SRC, TEGRA210_CLK_PLL_U_480M, 120000000, 0 },
{ TEGRA210_CLK_XUSB_FS_SRC, TEGRA210_CLK_PLL_U_48M, 48000000, 0 },
{ TEGRA210_CLK_PLL_DP, TEGRA210_CLK_CLK_MAX, 270000000, 0 },
{ TEGRA210_CLK_SOC_THERM, TEGRA210_CLK_PLL_P, 51000000, 0 },
{ TEGRA210_CLK_CCLK_G, TEGRA210_CLK_CLK_MAX, 0, 1 },
- { TEGRA210_CLK_PLL_U_OUT1, TEGRA210_CLK_CLK_MAX, 48000000, 1 },
{ TEGRA210_CLK_PLL_U_OUT2, TEGRA210_CLK_CLK_MAX, 60000000, 1 },
{ TEGRA210_CLK_SPDIF_IN_SYNC, TEGRA210_CLK_CLK_MAX, 24576000, 0 },
{ TEGRA210_CLK_I2S0_SYNC, TEGRA210_CLK_CLK_MAX, 24576000, 0 },
return 0;
}
-static const struct clk_div_table *
-_get_div_table_from_setup(struct ti_clk_divider *setup, u8 *width)
-{
- const struct clk_div_table *table = NULL;
-
- ti_clk_parse_divider_data(setup->dividers, setup->num_dividers,
- setup->max_div, setup->flags, width,
- &table);
-
- return table;
-}
-
-struct clk_hw *ti_clk_build_component_div(struct ti_clk_divider *setup)
-{
- struct clk_omap_divider *div;
- struct clk_omap_reg *reg;
- int ret;
-
- if (!setup)
- return NULL;
-
- div = kzalloc(sizeof(*div), GFP_KERNEL);
- if (!div)
- return ERR_PTR(-ENOMEM);
-
- reg = (struct clk_omap_reg *)&div->reg;
- reg->index = setup->module;
- reg->offset = setup->reg;
-
- if (setup->flags & CLKF_INDEX_STARTS_AT_ONE)
- div->flags |= CLK_DIVIDER_ONE_BASED;
-
- if (setup->flags & CLKF_INDEX_POWER_OF_TWO)
- div->flags |= CLK_DIVIDER_POWER_OF_TWO;
-
- div->table = _get_div_table_from_setup(setup, &div->width);
- if (IS_ERR(div->table)) {
- ret = PTR_ERR(div->table);
- kfree(div);
- return ERR_PTR(ret);
- }
-
-
- div->shift = setup->bit_shift;
- div->latch = -EINVAL;
-
- return &div->hw;
-}
-
-struct clk *ti_clk_register_divider(struct ti_clk *setup)
-{
- struct ti_clk_divider *div = setup->data;
- struct clk_omap_reg reg = {
- .index = div->module,
- .offset = div->reg,
- };
- u8 width;
- u32 flags = 0;
- u8 div_flags = 0;
- const struct clk_div_table *table;
- struct clk *clk;
-
- if (div->flags & CLKF_INDEX_STARTS_AT_ONE)
- div_flags |= CLK_DIVIDER_ONE_BASED;
-
- if (div->flags & CLKF_INDEX_POWER_OF_TWO)
- div_flags |= CLK_DIVIDER_POWER_OF_TWO;
-
- if (div->flags & CLKF_SET_RATE_PARENT)
- flags |= CLK_SET_RATE_PARENT;
-
- table = _get_div_table_from_setup(div, &width);
- if (IS_ERR(table))
- return (struct clk *)table;
-
- clk = _register_divider(NULL, setup->name, div->parent,
- flags, ®, div->bit_shift,
- width, -EINVAL, div_flags, table);
-
- if (IS_ERR(clk))
- kfree(table);
-
- return clk;
-}
-
static struct clk_div_table *
__init ti_clk_get_div_table(struct device_node *node)
{
return clk;
}
-struct clk_hw *ti_clk_build_component_gate(struct ti_clk_gate *setup)
-{
- struct clk_hw_omap *gate;
- struct clk_omap_reg *reg;
- const struct clk_hw_omap_ops *ops = &clkhwops_wait;
-
- if (!setup)
- return NULL;
-
- gate = kzalloc(sizeof(*gate), GFP_KERNEL);
- if (!gate)
- return ERR_PTR(-ENOMEM);
-
- reg = (struct clk_omap_reg *)&gate->enable_reg;
- reg->index = setup->module;
- reg->offset = setup->reg;
-
- gate->enable_bit = setup->bit_shift;
-
- if (setup->flags & CLKF_NO_WAIT)
- ops = NULL;
-
- if (setup->flags & CLKF_INTERFACE)
- ops = &clkhwops_iclk_wait;
-
- gate->ops = ops;
-
- return &gate->hw;
-}
-
static void __init _of_ti_gate_clk_setup(struct device_node *node,
const struct clk_ops *ops,
const struct clk_hw_omap_ops *hw_ops)
return clk;
}
-struct clk *ti_clk_register_mux(struct ti_clk *setup)
-{
- struct ti_clk_mux *mux;
- u32 flags;
- u8 mux_flags = 0;
- struct clk_omap_reg reg;
- u32 mask;
-
- mux = setup->data;
- flags = CLK_SET_RATE_NO_REPARENT;
-
- mask = mux->num_parents;
- if (!(mux->flags & CLKF_INDEX_STARTS_AT_ONE))
- mask--;
-
- mask = (1 << fls(mask)) - 1;
- reg.index = mux->module;
- reg.offset = mux->reg;
- reg.ptr = NULL;
-
- if (mux->flags & CLKF_INDEX_STARTS_AT_ONE)
- mux_flags |= CLK_MUX_INDEX_ONE;
-
- if (mux->flags & CLKF_SET_RATE_PARENT)
- flags |= CLK_SET_RATE_PARENT;
-
- return _register_mux(NULL, setup->name, mux->parents, mux->num_parents,
- flags, ®, mux->bit_shift, mask, -EINVAL,
- mux_flags, NULL);
-}
-
/**
* of_mux_clk_setup - Setup function for simple mux rate clock
* @node: DT node for the clock
static int bmips_cpufreq_init(struct cpufreq_policy *policy)
{
struct cpufreq_frequency_table *freq_table;
- int ret;
freq_table = bmips_cpufreq_get_freq_table(policy);
if (IS_ERR(freq_table)) {
- ret = PTR_ERR(freq_table);
- pr_err("%s: couldn't determine frequency table (%d).\n",
- BMIPS_CPUFREQ_NAME, ret);
- return ret;
+ pr_err("%s: couldn't determine frequency table (%ld).\n",
+ BMIPS_CPUFREQ_NAME, PTR_ERR(freq_table));
+ return PTR_ERR(freq_table);
}
- ret = cpufreq_generic_init(policy, freq_table, TRANSITION_LATENCY);
- if (ret)
- bmips_cpufreq_exit(policy);
- else
- pr_info("%s: registered\n", BMIPS_CPUFREQ_NAME);
+ cpufreq_generic_init(policy, freq_table, TRANSITION_LATENCY);
+ pr_info("%s: registered\n", BMIPS_CPUFREQ_NAME);
- return ret;
+ return 0;
}
static struct cpufreq_driver bmips_cpufreq_driver = {
#include <linux/kernel_stat.h>
#include <linux/module.h>
#include <linux/mutex.h>
+#include <linux/pm_qos.h>
#include <linux/slab.h>
#include <linux/suspend.h>
#include <linux/syscore_ops.h>
* - set policies transition latency
* - policy->cpus with all possible CPUs
*/
-int cpufreq_generic_init(struct cpufreq_policy *policy,
+void cpufreq_generic_init(struct cpufreq_policy *policy,
struct cpufreq_frequency_table *table,
unsigned int transition_latency)
{
* share the clock and voltage and clock.
*/
cpumask_setall(policy->cpus);
-
- return 0;
}
EXPORT_SYMBOL_GPL(cpufreq_generic_init);
static ssize_t store_##file_name \
(struct cpufreq_policy *policy, const char *buf, size_t count) \
{ \
- int ret, temp; \
- struct cpufreq_policy new_policy; \
+ unsigned long val; \
+ int ret; \
\
- memcpy(&new_policy, policy, sizeof(*policy)); \
- new_policy.min = policy->user_policy.min; \
- new_policy.max = policy->user_policy.max; \
- \
- ret = sscanf(buf, "%u", &new_policy.object); \
+ ret = sscanf(buf, "%lu", &val); \
if (ret != 1) \
return -EINVAL; \
\
- temp = new_policy.object; \
- ret = cpufreq_set_policy(policy, &new_policy); \
- if (!ret) \
- policy->user_policy.object = temp; \
- \
- return ret ? ret : count; \
+ ret = dev_pm_qos_update_request(policy->object##_freq_req, val);\
+ return ret >= 0 ? count : ret; \
}
store_one(scaling_min_freq, min);
{
struct device *dev = get_cpu_device(cpu);
- if (!dev)
+ if (unlikely(!dev))
return;
if (cpumask_test_and_set_cpu(cpu, policy->real_cpus))
return ret;
}
-static void refresh_frequency_limits(struct cpufreq_policy *policy)
+void refresh_frequency_limits(struct cpufreq_policy *policy)
{
- struct cpufreq_policy new_policy = *policy;
-
- pr_debug("updating policy for CPU %u\n", policy->cpu);
+ struct cpufreq_policy new_policy;
- new_policy.min = policy->user_policy.min;
- new_policy.max = policy->user_policy.max;
+ if (!policy_is_inactive(policy)) {
+ new_policy = *policy;
+ pr_debug("updating policy for CPU %u\n", policy->cpu);
- cpufreq_set_policy(policy, &new_policy);
+ cpufreq_set_policy(policy, &new_policy);
+ }
}
+EXPORT_SYMBOL(refresh_frequency_limits);
static void handle_update(struct work_struct *work)
{
container_of(work, struct cpufreq_policy, update);
pr_debug("handle_update for cpu %u called\n", policy->cpu);
+ down_write(&policy->rwsem);
refresh_frequency_limits(policy);
+ up_write(&policy->rwsem);
+}
+
+static int cpufreq_notifier_min(struct notifier_block *nb, unsigned long freq,
+ void *data)
+{
+ struct cpufreq_policy *policy = container_of(nb, struct cpufreq_policy, nb_min);
+
+ schedule_work(&policy->update);
+ return 0;
+}
+
+static int cpufreq_notifier_max(struct notifier_block *nb, unsigned long freq,
+ void *data)
+{
+ struct cpufreq_policy *policy = container_of(nb, struct cpufreq_policy, nb_max);
+
+ schedule_work(&policy->update);
+ return 0;
+}
+
+static void cpufreq_policy_put_kobj(struct cpufreq_policy *policy)
+{
+ struct kobject *kobj;
+ struct completion *cmp;
+
+ down_write(&policy->rwsem);
+ cpufreq_stats_free_table(policy);
+ kobj = &policy->kobj;
+ cmp = &policy->kobj_unregister;
+ up_write(&policy->rwsem);
+ kobject_put(kobj);
+
+ /*
+ * We need to make sure that the underlying kobj is
+ * actually not referenced anymore by anybody before we
+ * proceed with unloading.
+ */
+ pr_debug("waiting for dropping of refcount\n");
+ wait_for_completion(cmp);
+ pr_debug("wait complete\n");
}
static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
{
struct cpufreq_policy *policy;
+ struct device *dev = get_cpu_device(cpu);
int ret;
+ if (!dev)
+ return NULL;
+
policy = kzalloc(sizeof(*policy), GFP_KERNEL);
if (!policy)
return NULL;
ret = kobject_init_and_add(&policy->kobj, &ktype_cpufreq,
cpufreq_global_kobject, "policy%u", cpu);
if (ret) {
- pr_err("%s: failed to init policy->kobj: %d\n", __func__, ret);
+ dev_err(dev, "%s: failed to init policy->kobj: %d\n", __func__, ret);
/*
* The entire policy object will be freed below, but the extra
* memory allocated for the kobject name needs to be freed by
goto err_free_real_cpus;
}
+ policy->nb_min.notifier_call = cpufreq_notifier_min;
+ policy->nb_max.notifier_call = cpufreq_notifier_max;
+
+ ret = dev_pm_qos_add_notifier(dev, &policy->nb_min,
+ DEV_PM_QOS_MIN_FREQUENCY);
+ if (ret) {
+ dev_err(dev, "Failed to register MIN QoS notifier: %d (%*pbl)\n",
+ ret, cpumask_pr_args(policy->cpus));
+ goto err_kobj_remove;
+ }
+
+ ret = dev_pm_qos_add_notifier(dev, &policy->nb_max,
+ DEV_PM_QOS_MAX_FREQUENCY);
+ if (ret) {
+ dev_err(dev, "Failed to register MAX QoS notifier: %d (%*pbl)\n",
+ ret, cpumask_pr_args(policy->cpus));
+ goto err_min_qos_notifier;
+ }
+
INIT_LIST_HEAD(&policy->policy_list);
init_rwsem(&policy->rwsem);
spin_lock_init(&policy->transition_lock);
policy->cpu = cpu;
return policy;
+err_min_qos_notifier:
+ dev_pm_qos_remove_notifier(dev, &policy->nb_min,
+ DEV_PM_QOS_MIN_FREQUENCY);
+err_kobj_remove:
+ cpufreq_policy_put_kobj(policy);
err_free_real_cpus:
free_cpumask_var(policy->real_cpus);
err_free_rcpumask:
return NULL;
}
-static void cpufreq_policy_put_kobj(struct cpufreq_policy *policy)
-{
- struct kobject *kobj;
- struct completion *cmp;
-
- down_write(&policy->rwsem);
- cpufreq_stats_free_table(policy);
- kobj = &policy->kobj;
- cmp = &policy->kobj_unregister;
- up_write(&policy->rwsem);
- kobject_put(kobj);
-
- /*
- * We need to make sure that the underlying kobj is
- * actually not referenced anymore by anybody before we
- * proceed with unloading.
- */
- pr_debug("waiting for dropping of refcount\n");
- wait_for_completion(cmp);
- pr_debug("wait complete\n");
-}
-
static void cpufreq_policy_free(struct cpufreq_policy *policy)
{
+ struct device *dev = get_cpu_device(policy->cpu);
unsigned long flags;
int cpu;
per_cpu(cpufreq_cpu_data, cpu) = NULL;
write_unlock_irqrestore(&cpufreq_driver_lock, flags);
+ dev_pm_qos_remove_notifier(dev, &policy->nb_max,
+ DEV_PM_QOS_MAX_FREQUENCY);
+ dev_pm_qos_remove_notifier(dev, &policy->nb_min,
+ DEV_PM_QOS_MIN_FREQUENCY);
+ dev_pm_qos_remove_request(policy->max_freq_req);
+ dev_pm_qos_remove_request(policy->min_freq_req);
+ kfree(policy->min_freq_req);
+
cpufreq_policy_put_kobj(policy);
free_cpumask_var(policy->real_cpus);
free_cpumask_var(policy->related_cpus);
cpumask_and(policy->cpus, policy->cpus, cpu_online_mask);
if (new_policy) {
- policy->user_policy.min = policy->min;
- policy->user_policy.max = policy->max;
+ struct device *dev = get_cpu_device(cpu);
for_each_cpu(j, policy->related_cpus) {
per_cpu(cpufreq_cpu_data, j) = policy;
add_cpu_dev_symlink(policy, j);
}
- } else {
- policy->min = policy->user_policy.min;
- policy->max = policy->user_policy.max;
+
+ policy->min_freq_req = kzalloc(2 * sizeof(*policy->min_freq_req),
+ GFP_KERNEL);
+ if (!policy->min_freq_req)
+ goto out_destroy_policy;
+
+ ret = dev_pm_qos_add_request(dev, policy->min_freq_req,
+ DEV_PM_QOS_MIN_FREQUENCY,
+ policy->min);
+ if (ret < 0) {
+ /*
+ * So we don't call dev_pm_qos_remove_request() for an
+ * uninitialized request.
+ */
+ kfree(policy->min_freq_req);
+ policy->min_freq_req = NULL;
+
+ dev_err(dev, "Failed to add min-freq constraint (%d)\n",
+ ret);
+ goto out_destroy_policy;
+ }
+
+ /*
+ * This must be initialized right here to avoid calling
+ * dev_pm_qos_remove_request() on uninitialized request in case
+ * of errors.
+ */
+ policy->max_freq_req = policy->min_freq_req + 1;
+
+ ret = dev_pm_qos_add_request(dev, policy->max_freq_req,
+ DEV_PM_QOS_MAX_FREQUENCY,
+ policy->max);
+ if (ret < 0) {
+ policy->max_freq_req = NULL;
+ dev_err(dev, "Failed to add max-freq constraint (%d)\n",
+ ret);
+ goto out_destroy_policy;
+ }
}
if (cpufreq_driver->get && has_target()) {
struct cpufreq_policy *new_policy)
{
struct cpufreq_governor *old_gov;
+ struct device *cpu_dev = get_cpu_device(policy->cpu);
int ret;
pr_debug("setting new policy for CPU %u: %u - %u kHz\n",
memcpy(&new_policy->cpuinfo, &policy->cpuinfo, sizeof(policy->cpuinfo));
/*
- * This check works well when we store new min/max freq attributes,
- * because new_policy is a copy of policy with one field updated.
- */
- if (new_policy->min > new_policy->max)
- return -EINVAL;
+ * PM QoS framework collects all the requests from users and provide us
+ * the final aggregated value here.
+ */
+ new_policy->min = dev_pm_qos_read_value(cpu_dev, DEV_PM_QOS_MIN_FREQUENCY);
+ new_policy->max = dev_pm_qos_read_value(cpu_dev, DEV_PM_QOS_MAX_FREQUENCY);
/* verify the cpu speed can be set within this limit */
ret = cpufreq_driver->verify(new_policy);
if (ret)
return ret;
+ /*
+ * The notifier-chain shall be removed once all the users of
+ * CPUFREQ_ADJUST are moved to use the QoS framework.
+ */
/* adjust if necessary - all reasons */
blocking_notifier_call_chain(&cpufreq_policy_notifier_list,
CPUFREQ_ADJUST, new_policy);
* @cpu: CPU to re-evaluate the policy for.
*
* Update the current frequency for the cpufreq policy of @cpu and use
- * cpufreq_set_policy() to re-apply the min and max limits saved in the
- * user_policy sub-structure of that policy, which triggers the evaluation
- * of policy notifiers and the cpufreq driver's ->verify() callback for the
- * policy in question, among other things.
+ * cpufreq_set_policy() to re-apply the min and max limits, which triggers the
+ * evaluation of policy notifiers and the cpufreq driver's ->verify() callback
+ * for the policy in question, among other things.
*/
void cpufreq_update_policy(unsigned int cpu)
{
break;
}
- down_write(&policy->rwsem);
- policy->user_policy.max = policy->max;
- cpufreq_governor_limits(policy);
- up_write(&policy->rwsem);
+ ret = dev_pm_qos_update_request(policy->max_freq_req, policy->max);
+ if (ret)
+ break;
}
return ret;
* Setting the latency to 2000 us to accommodate addition of drivers
* to pre/post change notification list.
*/
- return cpufreq_generic_init(policy, freq_table, 2000 * 1000);
+ cpufreq_generic_init(policy, freq_table, 2000 * 1000);
+ return 0;
}
static struct cpufreq_driver davinci_driver = {
* According to datasheet minimum speed grading is not supported for
* consumer parts so clamp to 1 to avoid warning for "no OPPs"
*
- * Applies to 8mq and 8mm.
+ * Applies to i.MX8M series SoCs.
*/
if (mkt_segment == 0 && speed_grade == 0 && (
of_machine_is_compatible("fsl,imx8mm") ||
+ of_machine_is_compatible("fsl,imx8mn") ||
of_machine_is_compatible("fsl,imx8mq")))
speed_grade = 1;
static int imx6q_cpufreq_init(struct cpufreq_policy *policy)
{
- int ret;
-
policy->clk = clks[ARM].clk;
- ret = cpufreq_generic_init(policy, freq_table, transition_latency);
+ cpufreq_generic_init(policy, freq_table, transition_latency);
policy->suspend_freq = max_freq;
dev_pm_opp_of_register_em(policy->cpus);
- return ret;
+ return 0;
}
static struct cpufreq_driver imx6q_cpufreq_driver = {
static void intel_pstate_update_max_freq(unsigned int cpu)
{
struct cpufreq_policy *policy = cpufreq_cpu_acquire(cpu);
- struct cpufreq_policy new_policy;
struct cpudata *cpudata;
if (!policy)
policy->cpuinfo.max_freq = global.turbo_disabled_mf ?
cpudata->pstate.max_freq : cpudata->pstate.turbo_freq;
- memcpy(&new_policy, policy, sizeof(*policy));
- new_policy.max = min(policy->user_policy.max, policy->cpuinfo.max_freq);
- new_policy.min = min(policy->user_policy.min, new_policy.max);
-
- cpufreq_set_policy(policy, &new_policy);
+ refresh_frequency_limits(policy);
cpufreq_cpu_release(policy);
}
/* Module init and exit code */
static int kirkwood_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, kirkwood_freq_table, 5000);
+ cpufreq_generic_init(policy, kirkwood_freq_table, 5000);
+ return 0;
}
static struct cpufreq_driver kirkwood_cpufreq_driver = {
struct device *cpu_dev = get_cpu_device(policy->cpu);
struct cpufreq_frequency_table *freq_tbl;
unsigned int pll_freq, freq;
- int steps, i, ret;
+ int steps, i;
pll_freq = clk_get_rate(cpufreq->pll_clk) / 1000;
freq_tbl[i].frequency = CPUFREQ_TABLE_END;
policy->clk = cpufreq->clk;
- ret = cpufreq_generic_init(policy, freq_tbl, 0);
- if (ret)
- kfree(freq_tbl);
+ cpufreq_generic_init(policy, freq_tbl, 0);
- return ret;
+ return 0;
}
static int ls1x_cpufreq_exit(struct cpufreq_policy *policy)
}
policy->clk = cpuclk;
- return cpufreq_generic_init(policy, &loongson2_clockmod_table[0], 0);
+ cpufreq_generic_init(policy, &loongson2_clockmod_table[0], 0);
+ return 0;
}
static int loongson2_cpufreq_exit(struct cpufreq_policy *policy)
static int maple_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, maple_cpu_freqs, 12000);
+ cpufreq_generic_init(policy, maple_cpu_freqs, 12000);
+ return 0;
}
static struct cpufreq_driver maple_cpufreq_driver = {
dev_err(mpu_dev,
"%s: cpu%d: failed creating freq table[%d]\n",
__func__, policy->cpu, result);
- goto fail;
+ clk_put(policy->clk);
+ return result;
}
}
atomic_inc_return(&freq_table_users);
/* FIXME: what's the actual transition time? */
- result = cpufreq_generic_init(policy, freq_table, 300 * 1000);
- if (!result) {
- dev_pm_opp_of_register_em(policy->cpus);
- return 0;
- }
+ cpufreq_generic_init(policy, freq_table, 300 * 1000);
+ dev_pm_opp_of_register_em(policy->cpus);
- freq_table_free();
-fail:
- clk_put(policy->clk);
- return result;
+ return 0;
}
static int omap_cpu_exit(struct cpufreq_policy *policy)
policy->cur = pas_freqs[cur_astate].frequency;
ppc_proc_freq = policy->cur * 1000ul;
- return cpufreq_generic_init(policy, pas_freqs, get_gizmo_latency());
+ cpufreq_generic_init(policy, pas_freqs, get_gizmo_latency());
+ return 0;
out_unmap_sdcpwr:
iounmap(sdcpwr_mapbase);
static int pmac_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, pmac_cpu_freqs, transition_latency);
+ cpufreq_generic_init(policy, pmac_cpu_freqs, transition_latency);
+ return 0;
}
static u32 read_gpio(struct device_node *np)
static int g5_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, g5_cpu_freqs, transition_latency);
+ cpufreq_generic_init(policy, g5_cpu_freqs, transition_latency);
+ return 0;
}
static struct cpufreq_driver g5_cpufreq_driver = {
/* Datasheet says PLL stabalisation time must be at least 300us,
* so but add some fudge. (reference in LOCKCON0 register description)
*/
- ret = cpufreq_generic_init(policy, s3c_freq->freq_table,
+ cpufreq_generic_init(policy, s3c_freq->freq_table,
(500 * 1000) + s3c_freq->regulator_latency);
- if (ret)
- goto err_freq_table;
-
register_reboot_notifier(&s3c2416_cpufreq_reboot_notifier);
return 0;
-err_freq_table:
#ifdef CONFIG_ARM_S3C2416_CPUFREQ_VCORESCALE
- regulator_put(s3c_freq->vddarm);
err_vddarm:
-#endif
clk_put(s3c_freq->armclk);
+#endif
err_armclk:
clk_put(s3c_freq->hclk);
err_hclk:
static int s3c64xx_cpufreq_driver_init(struct cpufreq_policy *policy)
{
- int ret;
struct cpufreq_frequency_table *freq;
if (policy->cpu != 0)
#ifdef CONFIG_REGULATOR
vddarm = regulator_get(NULL, "vddarm");
if (IS_ERR(vddarm)) {
- ret = PTR_ERR(vddarm);
- pr_err("Failed to obtain VDDARM: %d\n", ret);
+ pr_err("Failed to obtain VDDARM: %ld\n", PTR_ERR(vddarm));
pr_err("Only frequency scaling available\n");
vddarm = NULL;
} else {
* the PLLs, which we don't currently) is ~300us worst case,
* but add some fudge.
*/
- ret = cpufreq_generic_init(policy, s3c64xx_freq_table,
+ cpufreq_generic_init(policy, s3c64xx_freq_table,
(500 * 1000) + regulator_latency);
- if (ret != 0) {
- pr_err("Failed to configure frequency table: %d\n",
- ret);
- regulator_put(vddarm);
- clk_put(policy->clk);
- }
-
- return ret;
+ return 0;
}
static struct cpufreq_driver s3c64xx_cpufreq_driver = {
s5pv210_dram_conf[1].freq = clk_get_rate(dmc1_clk);
policy->suspend_freq = SLEEP_FREQ;
- return cpufreq_generic_init(policy, s5pv210_freq_table, 40000);
+ cpufreq_generic_init(policy, s5pv210_freq_table, 40000);
+ return 0;
out_dmc1:
clk_put(dmc0_clk);
static int __init sa1100_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, sa11x0_freq_table, 0);
+ cpufreq_generic_init(policy, sa11x0_freq_table, 0);
+ return 0;
}
static struct cpufreq_driver sa1100_driver __refdata = {
static int __init sa1110_cpu_init(struct cpufreq_policy *policy)
{
- return cpufreq_generic_init(policy, sa11x0_freq_table, 0);
+ cpufreq_generic_init(policy, sa11x0_freq_table, 0);
+ return 0;
}
/* sa1110_driver needs __refdata because it must remain after init registers
static int spear_cpufreq_init(struct cpufreq_policy *policy)
{
policy->clk = spear_cpufreq.clk;
- return cpufreq_generic_init(policy, spear_cpufreq.freq_tbl,
+ cpufreq_generic_init(policy, spear_cpufreq.freq_tbl,
spear_cpufreq.transition_latency);
+ return 0;
}
static struct cpufreq_driver spear_cpufreq_driver = {
static int tegra_cpu_init(struct cpufreq_policy *policy)
{
struct tegra20_cpufreq *cpufreq = cpufreq_get_driver_data();
- int ret;
clk_prepare_enable(cpufreq->cpu_clk);
/* FIXME: what's the actual transition time? */
- ret = cpufreq_generic_init(policy, freq_table, 300 * 1000);
- if (ret) {
- clk_disable_unprepare(cpufreq->cpu_clk);
- return ret;
- }
-
+ cpufreq_generic_init(policy, freq_table, 300 * 1000);
policy->clk = cpufreq->cpu_clk;
policy->suspend_freq = freq_table[0].frequency;
return 0;
{
int global_req = pm_qos_request(PM_QOS_CPU_DMA_LATENCY);
struct device *device = get_cpu_device(cpu);
- int device_req = dev_pm_qos_raw_read_value(device);
+ int device_req = dev_pm_qos_raw_resume_latency(device);
return device_req < global_req ? device_req : global_req;
}
* keysize in CBC and ECB mode.
* Add support also for DES and 3DES in CBC and ECB mode.
*
- * You could find the datasheet in Documentation/arm/sunxi/README
+ * You could find the datasheet in Documentation/arm/sunxi.rst
*/
#include "sun4i-ss.h"
*
* Core file which registers crypto algorithms supported by the SS.
*
- * You could find a link for the datasheet in Documentation/arm/sunxi/README
+ * You could find a link for the datasheet in Documentation/arm/sunxi.rst
*/
#include <linux/clk.h>
#include <linux/crypto.h>
*
* This file add support for MD5 and SHA1.
*
- * You could find the datasheet in Documentation/arm/sunxi/README
+ * You could find the datasheet in Documentation/arm/sunxi.rst
*/
#include "sun4i-ss.h"
#include <linux/scatterlist.h>
* Support MD5 and SHA1 hash algorithms.
* Support DES and 3DES
*
- * You could find the datasheet in Documentation/arm/sunxi/README
+ * You could find the datasheet in Documentation/arm/sunxi.rst
*/
#include <linux/clk.h>
}
static DEVICE_ATTR_RO(target_node);
+static unsigned long long dev_dax_resource(struct dev_dax *dev_dax)
+{
+ struct dax_region *dax_region = dev_dax->region;
+
+ return dax_region->res.start;
+}
+
+static ssize_t resource_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct dev_dax *dev_dax = to_dev_dax(dev);
+
+ return sprintf(buf, "%#llx\n", dev_dax_resource(dev_dax));
+}
+static DEVICE_ATTR_RO(resource);
+
static ssize_t modalias_show(struct device *dev, struct device_attribute *attr,
char *buf)
{
if (a == &dev_attr_target_node.attr && dev_dax_target_node(dev_dax) < 0)
return 0;
+ if (a == &dev_attr_resource.attr)
+ return 0400;
return a->mode;
}
&dev_attr_modalias.attr,
&dev_attr_size.attr,
&dev_attr_target_node.attr,
+ &dev_attr_resource.attr,
NULL,
};
* No 'host' or dax_operations since there is no access to this
* device outside of mmap of the resulting character device.
*/
- dax_dev = alloc_dax(dev_dax, NULL, NULL);
+ dax_dev = alloc_dax(dev_dax, NULL, NULL, DAXDEV_F_SYNC);
if (!dax_dev)
goto err;
* @target_node: effective numa node if dev_dax memory range is onlined
* @dev - device core
* @pgmap - pgmap for memmap setup / lifetime (driver owned)
+ * @dax_mem_res: physical address range of hotadded DAX memory
*/
struct dev_dax {
struct dax_region *region;
int target_node;
struct device dev;
struct dev_pagemap pgmap;
+ struct resource *dax_kmem_res;
};
static inline struct dev_dax *to_dev_dax(struct device *dev)
new_res->name = dev_name(dev);
rc = add_memory(numa_node, new_res->start, resource_size(new_res));
- if (rc)
+ if (rc) {
+ release_resource(new_res);
+ kfree(new_res);
return rc;
+ }
+ dev_dax->dax_kmem_res = new_res;
return 0;
}
+#ifdef CONFIG_MEMORY_HOTREMOVE
+static int dev_dax_kmem_remove(struct device *dev)
+{
+ struct dev_dax *dev_dax = to_dev_dax(dev);
+ struct resource *res = dev_dax->dax_kmem_res;
+ resource_size_t kmem_start = res->start;
+ resource_size_t kmem_size = resource_size(res);
+ int rc;
+
+ /*
+ * We have one shot for removing memory, if some memory blocks were not
+ * offline prior to calling this function remove_memory() will fail, and
+ * there is no way to hotremove this memory until reboot because device
+ * unbind will succeed even if we return failure.
+ */
+ rc = remove_memory(dev_dax->target_node, kmem_start, kmem_size);
+ if (rc) {
+ dev_err(dev,
+ "DAX region %pR cannot be hotremoved until the next reboot\n",
+ res);
+ return rc;
+ }
+
+ /* Release and free dax resources */
+ release_resource(res);
+ kfree(res);
+ dev_dax->dax_kmem_res = NULL;
+
+ return 0;
+}
+#else
static int dev_dax_kmem_remove(struct device *dev)
{
/*
- * Purposely leak the request_mem_region() for the device-dax
- * range and return '0' to ->remove() attempts. The removal of
- * the device from the driver always succeeds, but the region
- * is permanently pinned as reserved by the unreleased
+ * Without hotremove purposely leak the request_mem_region() for the
+ * device-dax range and return '0' to ->remove() attempts. The removal
+ * of the device from the driver always succeeds, but the region is
+ * permanently pinned as reserved by the unreleased
* request_mem_region().
*/
return 0;
}
+#endif /* CONFIG_MEMORY_HOTREMOVE */
static struct dax_device_driver device_dax_kmem_driver = {
.drv = {
#include <linux/pagemap.h>
#include <linux/module.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/magic.h>
#include <linux/genhd.h>
#include <linux/pfn_t.h>
DAXDEV_ALIVE,
/* gate whether dax_flush() calls the low level flush routine */
DAXDEV_WRITE_CACHE,
+ /* flag to check if device supports synchronous flush */
+ DAXDEV_SYNC,
};
/**
}
EXPORT_SYMBOL_GPL(dax_write_cache_enabled);
+bool __dax_synchronous(struct dax_device *dax_dev)
+{
+ return test_bit(DAXDEV_SYNC, &dax_dev->flags);
+}
+EXPORT_SYMBOL_GPL(__dax_synchronous);
+
+void __set_dax_synchronous(struct dax_device *dax_dev)
+{
+ set_bit(DAXDEV_SYNC, &dax_dev->flags);
+}
+EXPORT_SYMBOL_GPL(__set_dax_synchronous);
+
bool dax_alive(struct dax_device *dax_dev)
{
lockdep_assert_held(&dax_srcu);
.drop_inode = generic_delete_inode,
};
-static struct dentry *dax_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int dax_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "dax:", &dax_sops, NULL, DAXFS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, DAXFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->ops = &dax_sops;
+ return 0;
}
static struct file_system_type dax_fs_type = {
- .name = "dax",
- .mount = dax_mount,
- .kill_sb = kill_anon_super,
+ .name = "dax",
+ .init_fs_context = dax_init_fs_context,
+ .kill_sb = kill_anon_super,
};
static int dax_test(struct inode *inode, void *data)
}
struct dax_device *alloc_dax(void *private, const char *__host,
- const struct dax_operations *ops)
+ const struct dax_operations *ops, unsigned long flags)
{
struct dax_device *dax_dev;
const char *host;
dax_add_host(dax_dev, host);
dax_dev->ops = ops;
dax_dev->private = private;
+ if (flags & DAXDEV_F_SYNC)
+ set_dax_synchronous(dax_dev);
+
return dax_dev;
err_dev:
if (!dax_cache)
return -ENOMEM;
- rc = register_filesystem(&dax_fs_type);
- if (rc)
- goto err_register_fs;
-
dax_mnt = kern_mount(&dax_fs_type);
if (IS_ERR(dax_mnt)) {
rc = PTR_ERR(dax_mnt);
return 0;
err_mount:
- unregister_filesystem(&dax_fs_type);
- err_register_fs:
kmem_cache_destroy(dax_cache);
return rc;
static void dax_fs_exit(void)
{
kern_unmount(dax_mnt);
- unregister_filesystem(&dax_fs_type);
kmem_cache_destroy(dax_cache);
}
associated with a buffer. When a job is submitted to the GPU a fence
is attached to the buffer and is transferred via userspace, using Sync
Files fds, to the DRM driver for example. More details at
- Documentation/sync_file.txt.
+ Documentation/driver-api/sync_file.rst.
config SW_SYNC
bool "Sync File Validation Framework"
#include <linux/reservation.h>
#include <linux/mm.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <uapi/linux/dma-buf.h>
#include <uapi/linux/magic.h>
static struct vfsmount *dma_buf_mnt;
-static struct dentry *dma_buf_fs_mount(struct file_system_type *fs_type,
- int flags, const char *name, void *data)
+static int dma_buf_fs_init_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "dmabuf:", NULL, &dma_buf_dentry_ops,
- DMA_BUF_MAGIC);
+ struct pseudo_fs_context *ctx;
+
+ ctx = init_pseudo(fc, DMA_BUF_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->dops = &dma_buf_dentry_ops;
+ return 0;
}
static struct file_system_type dma_buf_fs_type = {
.name = "dmabuf",
- .mount = dma_buf_fs_mount,
+ .init_fs_context = dma_buf_fs_init_context,
.kill_sb = kill_anon_super,
};
depends on MICROBLAZE || NIOS2 || ARCH_ZYNQ || ARCH_ZYNQMP || ARCH_SOCFPGA || COMPILE_TEST
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
+ select REGMAP_MMIO
help
Enable support for the Analog Devices AXI-DMAC peripheral. This DMA
controller is often used in Analog Device's reference designs for FPGA
config TEGRA210_ADMA
tristate "NVIDIA Tegra210 ADMA support"
- depends on (ARCH_TEGRA_210_SOC || COMPILE_TEST) && PM_CLK
+ depends on (ARCH_TEGRA_210_SOC || COMPILE_TEST)
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
help
source "drivers/dma/dw/Kconfig"
+source "drivers/dma/dw-edma/Kconfig"
+
source "drivers/dma/hsu/Kconfig"
source "drivers/dma/sh/Kconfig"
obj-$(CONFIG_DMA_SUN6I) += sun6i-dma.o
obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/
obj-$(CONFIG_DW_DMAC_CORE) += dw/
+obj-$(CONFIG_DW_EDMA) += dw-edma/
obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o
obj-$(CONFIG_FSL_DMA) += fsldma.o
obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o
static void init_pl08x_debugfs(struct pl08x_driver_data *pl08x)
{
/* Expose a simple debugfs interface to view all clocks */
- (void) debugfs_create_file(dev_name(&pl08x->adev->dev),
- S_IFREG | S_IRUGO, NULL, pl08x,
- &pl08x_debugfs_fops);
+ debugfs_create_file(dev_name(&pl08x->adev->dev), S_IFREG | S_IRUGO,
+ NULL, pl08x, &pl08x_debugfs_fops);
}
#else
struct at_xdmac_desc *desc;
struct dma_async_tx_descriptor *txd;
- desc = list_first_entry(&atchan->xfers_list, struct at_xdmac_desc, xfer_node);
- txd = &desc->tx_dma_desc;
+ if (!list_empty(&atchan->xfers_list)) {
+ desc = list_first_entry(&atchan->xfers_list,
+ struct at_xdmac_desc, xfer_node);
+ txd = &desc->tx_dma_desc;
- if (txd->flags & DMA_PREP_INTERRUPT)
- dmaengine_desc_get_callback_invoke(txd, NULL);
+ if (txd->flags & DMA_PREP_INTERRUPT)
+ dmaengine_desc_get_callback_invoke(txd, NULL);
+ }
}
static void at_xdmac_handle_error(struct at_xdmac_chan *atchan)
struct list_head reqs_free_list;
/* DebugFS directory entries */
struct dentry *root;
- struct dentry *stats;
};
/* ====== Command helper routines ===== */
/* Create debugfs root entry */
sba->root = debugfs_create_dir(dev_name(sba->dev), NULL);
- if (IS_ERR_OR_NULL(sba->root)) {
- dev_err(sba->dev, "failed to create debugfs root entry\n");
- sba->root = NULL;
- goto skip_debugfs;
- }
/* Create debugfs stats entry */
- sba->stats = debugfs_create_devm_seqfile(sba->dev, "stats", sba->root,
- sba_debugfs_stats_show);
- if (IS_ERR_OR_NULL(sba->stats))
- dev_err(sba->dev, "failed to create debugfs stats file\n");
+ debugfs_create_devm_seqfile(sba->dev, "stats", sba->root,
+ sba_debugfs_stats_show);
+
skip_debugfs:
/* Register DMA device with Linux async framework */
dma_dentry = debugfs_create_dir("dma", NULL);
- (void) debugfs_create_file("status",
- S_IFREG | S_IRUGO,
- dma_dentry, NULL,
- &coh901318_debugfs_status_operations);
+ debugfs_create_file("status", S_IFREG | S_IRUGO, dma_dentry, NULL,
+ &coh901318_debugfs_status_operations);
return 0;
}
/*
* Driver for the Analog Devices AXI-DMAC core
*
- * Copyright 2013-2015 Analog Devices Inc.
+ * Copyright 2013-2019 Analog Devices Inc.
* Author: Lars-Peter Clausen <lars@metafoo.de>
*/
#include <linux/of.h>
#include <linux/of_dma.h>
#include <linux/platform_device.h>
+#include <linux/regmap.h>
#include <linux/slab.h>
+#include <linux/fpga/adi-axi-common.h>
#include <dt-bindings/dma/axi-dmac.h>
#define AXI_DMAC_REG_STATUS 0x430
#define AXI_DMAC_REG_CURRENT_SRC_ADDR 0x434
#define AXI_DMAC_REG_CURRENT_DEST_ADDR 0x438
+#define AXI_DMAC_REG_PARTIAL_XFER_LEN 0x44c
+#define AXI_DMAC_REG_PARTIAL_XFER_ID 0x450
#define AXI_DMAC_CTRL_ENABLE BIT(0)
#define AXI_DMAC_CTRL_PAUSE BIT(1)
#define AXI_DMAC_IRQ_EOT BIT(1)
#define AXI_DMAC_FLAG_CYCLIC BIT(0)
+#define AXI_DMAC_FLAG_LAST BIT(1)
+#define AXI_DMAC_FLAG_PARTIAL_REPORT BIT(2)
+
+#define AXI_DMAC_FLAG_PARTIAL_XFER_DONE BIT(31)
/* The maximum ID allocated by the hardware is 31 */
#define AXI_DMAC_SG_UNUSED 32U
unsigned int dest_stride;
unsigned int src_stride;
unsigned int id;
+ unsigned int partial_len;
bool schedule_when_free;
};
struct axi_dmac_desc {
struct virt_dma_desc vdesc;
bool cyclic;
+ bool have_partial_xfer;
unsigned int num_submitted;
unsigned int num_completed;
unsigned int dest_type;
unsigned int max_length;
- unsigned int align_mask;
+ unsigned int address_align_mask;
+ unsigned int length_align_mask;
+ bool hw_partial_xfer;
bool hw_cyclic;
bool hw_2d;
};
{
if (len == 0)
return false;
- if ((len & chan->align_mask) != 0) /* Not aligned */
+ if ((len & chan->length_align_mask) != 0) /* Not aligned */
return false;
return true;
}
static bool axi_dmac_check_addr(struct axi_dmac_chan *chan, dma_addr_t addr)
{
- if ((addr & chan->align_mask) != 0) /* Not aligned */
+ if ((addr & chan->address_align_mask) != 0) /* Not aligned */
return false;
return true;
}
}
desc->num_submitted++;
- if (desc->num_submitted == desc->num_sgs) {
+ if (desc->num_submitted == desc->num_sgs ||
+ desc->have_partial_xfer) {
if (desc->cyclic)
desc->num_submitted = 0; /* Start again */
else
chan->next_desc = NULL;
+ flags |= AXI_DMAC_FLAG_LAST;
} else {
chan->next_desc = desc;
}
desc->num_sgs == 1)
flags |= AXI_DMAC_FLAG_CYCLIC;
+ if (chan->hw_partial_xfer)
+ flags |= AXI_DMAC_FLAG_PARTIAL_REPORT;
+
axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, sg->x_len - 1);
axi_dmac_write(dmac, AXI_DMAC_REG_Y_LENGTH, sg->y_len - 1);
axi_dmac_write(dmac, AXI_DMAC_REG_FLAGS, flags);
struct axi_dmac_desc, vdesc.node);
}
+static inline unsigned int axi_dmac_total_sg_bytes(struct axi_dmac_chan *chan,
+ struct axi_dmac_sg *sg)
+{
+ if (chan->hw_2d)
+ return sg->x_len * sg->y_len;
+ else
+ return sg->x_len;
+}
+
+static void axi_dmac_dequeue_partial_xfers(struct axi_dmac_chan *chan)
+{
+ struct axi_dmac *dmac = chan_to_axi_dmac(chan);
+ struct axi_dmac_desc *desc;
+ struct axi_dmac_sg *sg;
+ u32 xfer_done, len, id, i;
+ bool found_sg;
+
+ do {
+ len = axi_dmac_read(dmac, AXI_DMAC_REG_PARTIAL_XFER_LEN);
+ id = axi_dmac_read(dmac, AXI_DMAC_REG_PARTIAL_XFER_ID);
+
+ found_sg = false;
+ list_for_each_entry(desc, &chan->active_descs, vdesc.node) {
+ for (i = 0; i < desc->num_sgs; i++) {
+ sg = &desc->sg[i];
+ if (sg->id == AXI_DMAC_SG_UNUSED)
+ continue;
+ if (sg->id == id) {
+ desc->have_partial_xfer = true;
+ sg->partial_len = len;
+ found_sg = true;
+ break;
+ }
+ }
+ if (found_sg)
+ break;
+ }
+
+ if (found_sg) {
+ dev_dbg(dmac->dma_dev.dev,
+ "Found partial segment id=%u, len=%u\n",
+ id, len);
+ } else {
+ dev_warn(dmac->dma_dev.dev,
+ "Not found partial segment id=%u, len=%u\n",
+ id, len);
+ }
+
+ /* Check if we have any more partial transfers */
+ xfer_done = axi_dmac_read(dmac, AXI_DMAC_REG_TRANSFER_DONE);
+ xfer_done = !(xfer_done & AXI_DMAC_FLAG_PARTIAL_XFER_DONE);
+
+ } while (!xfer_done);
+}
+
+static void axi_dmac_compute_residue(struct axi_dmac_chan *chan,
+ struct axi_dmac_desc *active)
+{
+ struct dmaengine_result *rslt = &active->vdesc.tx_result;
+ unsigned int start = active->num_completed - 1;
+ struct axi_dmac_sg *sg;
+ unsigned int i, total;
+
+ rslt->result = DMA_TRANS_NOERROR;
+ rslt->residue = 0;
+
+ /*
+ * We get here if the last completed segment is partial, which
+ * means we can compute the residue from that segment onwards
+ */
+ for (i = start; i < active->num_sgs; i++) {
+ sg = &active->sg[i];
+ total = axi_dmac_total_sg_bytes(chan, sg);
+ rslt->residue += (total - sg->partial_len);
+ }
+}
+
static bool axi_dmac_transfer_done(struct axi_dmac_chan *chan,
unsigned int completed_transfers)
{
if (!active)
return false;
+ if (chan->hw_partial_xfer &&
+ (completed_transfers & AXI_DMAC_FLAG_PARTIAL_XFER_DONE))
+ axi_dmac_dequeue_partial_xfers(chan);
+
do {
sg = &active->sg[active->num_completed];
if (sg->id == AXI_DMAC_SG_UNUSED) /* Not yet submitted */
start_next = true;
}
+ if (sg->partial_len)
+ axi_dmac_compute_residue(chan, active);
+
if (active->cyclic)
vchan_cyclic_callback(&active->vdesc);
- if (active->num_completed == active->num_sgs) {
+ if (active->num_completed == active->num_sgs ||
+ sg->partial_len) {
if (active->cyclic) {
active->num_completed = 0; /* wrap around */
} else {
num_segments = DIV_ROUND_UP(period_len, chan->max_length);
segment_size = DIV_ROUND_UP(period_len, num_segments);
/* Take care of alignment */
- segment_size = ((segment_size - 1) | chan->align_mask) + 1;
+ segment_size = ((segment_size - 1) | chan->length_align_mask) + 1;
for (i = 0; i < num_periods; i++) {
len = period_len;
desc->sg[0].y_len = 1;
}
+ if (flags & DMA_CYCLIC)
+ desc->cyclic = true;
+
return vchan_tx_prep(&chan->vchan, &desc->vdesc, flags);
}
kfree(container_of(vdesc, struct axi_dmac_desc, vdesc));
}
+static bool axi_dmac_regmap_rdwr(struct device *dev, unsigned int reg)
+{
+ switch (reg) {
+ case AXI_DMAC_REG_IRQ_MASK:
+ case AXI_DMAC_REG_IRQ_SOURCE:
+ case AXI_DMAC_REG_IRQ_PENDING:
+ case AXI_DMAC_REG_CTRL:
+ case AXI_DMAC_REG_TRANSFER_ID:
+ case AXI_DMAC_REG_START_TRANSFER:
+ case AXI_DMAC_REG_FLAGS:
+ case AXI_DMAC_REG_DEST_ADDRESS:
+ case AXI_DMAC_REG_SRC_ADDRESS:
+ case AXI_DMAC_REG_X_LENGTH:
+ case AXI_DMAC_REG_Y_LENGTH:
+ case AXI_DMAC_REG_DEST_STRIDE:
+ case AXI_DMAC_REG_SRC_STRIDE:
+ case AXI_DMAC_REG_TRANSFER_DONE:
+ case AXI_DMAC_REG_ACTIVE_TRANSFER_ID:
+ case AXI_DMAC_REG_STATUS:
+ case AXI_DMAC_REG_CURRENT_SRC_ADDR:
+ case AXI_DMAC_REG_CURRENT_DEST_ADDR:
+ case AXI_DMAC_REG_PARTIAL_XFER_LEN:
+ case AXI_DMAC_REG_PARTIAL_XFER_ID:
+ return true;
+ default:
+ return false;
+ }
+}
+
+static const struct regmap_config axi_dmac_regmap_config = {
+ .reg_bits = 32,
+ .val_bits = 32,
+ .reg_stride = 4,
+ .max_register = AXI_DMAC_REG_PARTIAL_XFER_ID,
+ .readable_reg = axi_dmac_regmap_rdwr,
+ .writeable_reg = axi_dmac_regmap_rdwr,
+};
+
/*
* The configuration stored in the devicetree matches the configuration
* parameters of the peripheral instance and allows the driver to know which
return ret;
chan->dest_width = val / 8;
- chan->align_mask = max(chan->dest_width, chan->src_width) - 1;
+ chan->address_align_mask = max(chan->dest_width, chan->src_width) - 1;
if (axi_dmac_dest_is_mem(chan) && axi_dmac_src_is_mem(chan))
chan->direction = DMA_MEM_TO_MEM;
return 0;
}
-static void axi_dmac_detect_caps(struct axi_dmac *dmac)
+static int axi_dmac_detect_caps(struct axi_dmac *dmac)
{
struct axi_dmac_chan *chan = &dmac->chan;
+ unsigned int version;
+
+ version = axi_dmac_read(dmac, ADI_AXI_REG_VERSION);
axi_dmac_write(dmac, AXI_DMAC_REG_FLAGS, AXI_DMAC_FLAG_CYCLIC);
if (axi_dmac_read(dmac, AXI_DMAC_REG_FLAGS) == AXI_DMAC_FLAG_CYCLIC)
chan->max_length = axi_dmac_read(dmac, AXI_DMAC_REG_X_LENGTH);
if (chan->max_length != UINT_MAX)
chan->max_length++;
+
+ axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, 0xffffffff);
+ if (axi_dmac_read(dmac, AXI_DMAC_REG_DEST_ADDRESS) == 0 &&
+ chan->dest_type == AXI_DMAC_BUS_TYPE_AXI_MM) {
+ dev_err(dmac->dma_dev.dev,
+ "Destination memory-mapped interface not supported.");
+ return -ENODEV;
+ }
+
+ axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, 0xffffffff);
+ if (axi_dmac_read(dmac, AXI_DMAC_REG_SRC_ADDRESS) == 0 &&
+ chan->src_type == AXI_DMAC_BUS_TYPE_AXI_MM) {
+ dev_err(dmac->dma_dev.dev,
+ "Source memory-mapped interface not supported.");
+ return -ENODEV;
+ }
+
+ if (version >= ADI_AXI_PCORE_VER(4, 2, 'a'))
+ chan->hw_partial_xfer = true;
+
+ if (version >= ADI_AXI_PCORE_VER(4, 1, 'a')) {
+ axi_dmac_write(dmac, AXI_DMAC_REG_X_LENGTH, 0x00);
+ chan->length_align_mask =
+ axi_dmac_read(dmac, AXI_DMAC_REG_X_LENGTH);
+ } else {
+ chan->length_align_mask = chan->address_align_mask;
+ }
+
+ return 0;
}
static int axi_dmac_probe(struct platform_device *pdev)
if (ret < 0)
return ret;
- axi_dmac_detect_caps(dmac);
+ ret = axi_dmac_detect_caps(dmac);
+ if (ret)
+ goto err_clk_disable;
+
+ dma_dev->copy_align = (dmac->chan.address_align_mask + 1);
axi_dmac_write(dmac, AXI_DMAC_REG_IRQ_MASK, 0x00);
platform_set_drvdata(pdev, dmac);
+ devm_regmap_init_mmio(&pdev->dev, dmac->base, &axi_dmac_regmap_config);
+
return 0;
err_unregister_of:
};
struct jz4780_dma_filter_data {
- struct device_node *of_node;
uint32_t transfer_type;
int channel;
};
struct jz4780_dma_dev *jzdma = jz4780_dma_chan_parent(jzchan);
struct jz4780_dma_filter_data *data = param;
- if (jzdma->dma_device.dev->of_node != data->of_node)
- return false;
if (data->channel > -1) {
if (data->channel != jzchan->id)
if (dma_spec->args_count != 2)
return NULL;
- data.of_node = ofdma->of_node;
data.transfer_type = dma_spec->args[0];
data.channel = dma_spec->args[1];
return dma_get_slave_channel(
&jzdma->chan[data.channel].vchan.chan);
} else {
- return dma_request_channel(mask, jz4780_dma_filter_fn, &data);
+ return __dma_request_channel(&mask, jz4780_dma_filter_fn, &data,
+ ofdma->of_node);
}
}
/* --- sysfs implementation --- */
/**
- * dev_to_dma_chan - convert a device pointer to the its sysfs container object
+ * dev_to_dma_chan - convert a device pointer to its sysfs container object
* @dev - device node
*
* Must be called under dma_list_mutex
* @mask: capabilities that the channel must satisfy
* @fn: optional callback to disposition available channels
* @fn_param: opaque parameter to pass to dma_filter_fn
+ * @np: device node to look for DMA channels
*
* Returns pointer to appropriate DMA channel on success or NULL.
*/
struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
- dma_filter_fn fn, void *fn_param)
+ dma_filter_fn fn, void *fn_param,
+ struct device_node *np)
{
struct dma_device *device, *_d;
struct dma_chan *chan = NULL;
/* Find a channel */
mutex_lock(&dma_list_mutex);
list_for_each_entry_safe(device, _d, &dma_device_list, global_node) {
+ /* Finds a DMA controller with matching device node */
+ if (np && device->dev->of_node && np != device->dev->of_node)
+ continue;
+
chan = find_candidate(device, mask, fn, fn_param);
if (!IS_ERR(chan))
break;
chan = acpi_dma_request_slave_chan_by_name(dev, name);
if (chan) {
- /* Valid channel found or requester need to be deferred */
+ /* Valid channel found or requester needs to be deferred */
if (!IS_ERR(chan) || PTR_ERR(chan) == -EPROBE_DEFER)
return chan;
}
if (!mask)
return ERR_PTR(-ENODEV);
- chan = __dma_request_channel(mask, NULL, NULL);
+ chan = __dma_request_channel(mask, NULL, NULL, NULL);
if (!chan) {
mutex_lock(&dma_list_mutex);
if (list_empty(&dma_device_list))
static int timeout = 3000;
module_param(timeout, uint, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(timeout, "Transfer Timeout in msec (default: 3000), "
- "Pass -1 for infinite timeout");
+ "Pass 0xFFFFFFFF (4294967295) for maximum timeout");
static bool noverify;
module_param(noverify, bool, S_IRUGO | S_IWUSR);
* @iterations: iterations before stopping test
* @xor_sources: number of xor source buffers
* @pq_sources: number of p+q source buffers
- * @timeout: transfer timeout in msec, -1 for infinite timeout
+ * @timeout: transfer timeout in msec, 0 - 0xFFFFFFFF (4294967295)
*/
struct dmatest_params {
unsigned int buf_size;
unsigned int iterations;
unsigned int xor_sources;
unsigned int pq_sources;
- int timeout;
+ unsigned int timeout;
bool noverify;
bool norandom;
int alignment;
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+
+config DW_EDMA
+ tristate "Synopsys DesignWare eDMA controller driver"
+ depends on PCI && PCI_MSI
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ help
+ Support the Synopsys DesignWare eDMA controller, normally
+ implemented on endpoints SoCs.
+
+config DW_EDMA_PCIE
+ tristate "Synopsys DesignWare eDMA PCIe driver"
+ depends on PCI && PCI_MSI
+ select DW_EDMA
+ help
+ Provides a glue-logic between the Synopsys DesignWare
+ eDMA controller and an endpoint PCIe device. This also serves
+ as a reference design to whom desires to use this IP.
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_DW_EDMA) += dw-edma.o
+dw-edma-$(CONFIG_DEBUG_FS) := dw-edma-v0-debugfs.o
+dw-edma-objs := dw-edma-core.o \
+ dw-edma-v0-core.o $(dw-edma-y)
+obj-$(CONFIG_DW_EDMA_PCIE) += dw-edma-pcie.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA core driver
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/pm_runtime.h>
+#include <linux/dmaengine.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/dma/edma.h>
+#include <linux/pci.h>
+
+#include "dw-edma-core.h"
+#include "dw-edma-v0-core.h"
+#include "../dmaengine.h"
+#include "../virt-dma.h"
+
+static inline
+struct device *dchan2dev(struct dma_chan *dchan)
+{
+ return &dchan->dev->device;
+}
+
+static inline
+struct device *chan2dev(struct dw_edma_chan *chan)
+{
+ return &chan->vc.chan.dev->device;
+}
+
+static inline
+struct dw_edma_desc *vd2dw_edma_desc(struct virt_dma_desc *vd)
+{
+ return container_of(vd, struct dw_edma_desc, vd);
+}
+
+static struct dw_edma_burst *dw_edma_alloc_burst(struct dw_edma_chunk *chunk)
+{
+ struct dw_edma_burst *burst;
+
+ burst = kzalloc(sizeof(*burst), GFP_NOWAIT);
+ if (unlikely(!burst))
+ return NULL;
+
+ INIT_LIST_HEAD(&burst->list);
+ if (chunk->burst) {
+ /* Create and add new element into the linked list */
+ chunk->bursts_alloc++;
+ list_add_tail(&burst->list, &chunk->burst->list);
+ } else {
+ /* List head */
+ chunk->bursts_alloc = 0;
+ chunk->burst = burst;
+ }
+
+ return burst;
+}
+
+static struct dw_edma_chunk *dw_edma_alloc_chunk(struct dw_edma_desc *desc)
+{
+ struct dw_edma_chan *chan = desc->chan;
+ struct dw_edma *dw = chan->chip->dw;
+ struct dw_edma_chunk *chunk;
+
+ chunk = kzalloc(sizeof(*chunk), GFP_NOWAIT);
+ if (unlikely(!chunk))
+ return NULL;
+
+ INIT_LIST_HEAD(&chunk->list);
+ chunk->chan = chan;
+ /* Toggling change bit (CB) in each chunk, this is a mechanism to
+ * inform the eDMA HW block that this is a new linked list ready
+ * to be consumed.
+ * - Odd chunks originate CB equal to 0
+ * - Even chunks originate CB equal to 1
+ */
+ chunk->cb = !(desc->chunks_alloc % 2);
+ chunk->ll_region.paddr = dw->ll_region.paddr + chan->ll_off;
+ chunk->ll_region.vaddr = dw->ll_region.vaddr + chan->ll_off;
+
+ if (desc->chunk) {
+ /* Create and add new element into the linked list */
+ desc->chunks_alloc++;
+ list_add_tail(&chunk->list, &desc->chunk->list);
+ if (!dw_edma_alloc_burst(chunk)) {
+ kfree(chunk);
+ return NULL;
+ }
+ } else {
+ /* List head */
+ chunk->burst = NULL;
+ desc->chunks_alloc = 0;
+ desc->chunk = chunk;
+ }
+
+ return chunk;
+}
+
+static struct dw_edma_desc *dw_edma_alloc_desc(struct dw_edma_chan *chan)
+{
+ struct dw_edma_desc *desc;
+
+ desc = kzalloc(sizeof(*desc), GFP_NOWAIT);
+ if (unlikely(!desc))
+ return NULL;
+
+ desc->chan = chan;
+ if (!dw_edma_alloc_chunk(desc)) {
+ kfree(desc);
+ return NULL;
+ }
+
+ return desc;
+}
+
+static void dw_edma_free_burst(struct dw_edma_chunk *chunk)
+{
+ struct dw_edma_burst *child, *_next;
+
+ /* Remove all the list elements */
+ list_for_each_entry_safe(child, _next, &chunk->burst->list, list) {
+ list_del(&child->list);
+ kfree(child);
+ chunk->bursts_alloc--;
+ }
+
+ /* Remove the list head */
+ kfree(child);
+ chunk->burst = NULL;
+}
+
+static void dw_edma_free_chunk(struct dw_edma_desc *desc)
+{
+ struct dw_edma_chunk *child, *_next;
+
+ if (!desc->chunk)
+ return;
+
+ /* Remove all the list elements */
+ list_for_each_entry_safe(child, _next, &desc->chunk->list, list) {
+ dw_edma_free_burst(child);
+ list_del(&child->list);
+ kfree(child);
+ desc->chunks_alloc--;
+ }
+
+ /* Remove the list head */
+ kfree(child);
+ desc->chunk = NULL;
+}
+
+static void dw_edma_free_desc(struct dw_edma_desc *desc)
+{
+ dw_edma_free_chunk(desc);
+ kfree(desc);
+}
+
+static void vchan_free_desc(struct virt_dma_desc *vdesc)
+{
+ dw_edma_free_desc(vd2dw_edma_desc(vdesc));
+}
+
+static void dw_edma_start_transfer(struct dw_edma_chan *chan)
+{
+ struct dw_edma_chunk *child;
+ struct dw_edma_desc *desc;
+ struct virt_dma_desc *vd;
+
+ vd = vchan_next_desc(&chan->vc);
+ if (!vd)
+ return;
+
+ desc = vd2dw_edma_desc(vd);
+ if (!desc)
+ return;
+
+ child = list_first_entry_or_null(&desc->chunk->list,
+ struct dw_edma_chunk, list);
+ if (!child)
+ return;
+
+ dw_edma_v0_core_start(child, !desc->xfer_sz);
+ desc->xfer_sz += child->ll_region.sz;
+ dw_edma_free_burst(child);
+ list_del(&child->list);
+ kfree(child);
+ desc->chunks_alloc--;
+}
+
+static int dw_edma_device_config(struct dma_chan *dchan,
+ struct dma_slave_config *config)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+
+ memcpy(&chan->config, config, sizeof(*config));
+ chan->configured = true;
+
+ return 0;
+}
+
+static int dw_edma_device_pause(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ int err = 0;
+
+ if (!chan->configured)
+ err = -EPERM;
+ else if (chan->status != EDMA_ST_BUSY)
+ err = -EPERM;
+ else if (chan->request != EDMA_REQ_NONE)
+ err = -EPERM;
+ else
+ chan->request = EDMA_REQ_PAUSE;
+
+ return err;
+}
+
+static int dw_edma_device_resume(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ int err = 0;
+
+ if (!chan->configured) {
+ err = -EPERM;
+ } else if (chan->status != EDMA_ST_PAUSE) {
+ err = -EPERM;
+ } else if (chan->request != EDMA_REQ_NONE) {
+ err = -EPERM;
+ } else {
+ chan->status = EDMA_ST_BUSY;
+ dw_edma_start_transfer(chan);
+ }
+
+ return err;
+}
+
+static int dw_edma_device_terminate_all(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ int err = 0;
+ LIST_HEAD(head);
+
+ if (!chan->configured) {
+ /* Do nothing */
+ } else if (chan->status == EDMA_ST_PAUSE) {
+ chan->status = EDMA_ST_IDLE;
+ chan->configured = false;
+ } else if (chan->status == EDMA_ST_IDLE) {
+ chan->configured = false;
+ } else if (dw_edma_v0_core_ch_status(chan) == DMA_COMPLETE) {
+ /*
+ * The channel is in a false BUSY state, probably didn't
+ * receive or lost an interrupt
+ */
+ chan->status = EDMA_ST_IDLE;
+ chan->configured = false;
+ } else if (chan->request > EDMA_REQ_PAUSE) {
+ err = -EPERM;
+ } else {
+ chan->request = EDMA_REQ_STOP;
+ }
+
+ return err;
+}
+
+static void dw_edma_device_issue_pending(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&chan->vc.lock, flags);
+ if (chan->configured && chan->request == EDMA_REQ_NONE &&
+ chan->status == EDMA_ST_IDLE && vchan_issue_pending(&chan->vc)) {
+ chan->status = EDMA_ST_BUSY;
+ dw_edma_start_transfer(chan);
+ }
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
+}
+
+static enum dma_status
+dw_edma_device_tx_status(struct dma_chan *dchan, dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ struct dw_edma_desc *desc;
+ struct virt_dma_desc *vd;
+ unsigned long flags;
+ enum dma_status ret;
+ u32 residue = 0;
+
+ ret = dma_cookie_status(dchan, cookie, txstate);
+ if (ret == DMA_COMPLETE)
+ return ret;
+
+ if (ret == DMA_IN_PROGRESS && chan->status == EDMA_ST_PAUSE)
+ ret = DMA_PAUSED;
+
+ if (!txstate)
+ goto ret_residue;
+
+ spin_lock_irqsave(&chan->vc.lock, flags);
+ vd = vchan_find_desc(&chan->vc, cookie);
+ if (vd) {
+ desc = vd2dw_edma_desc(vd);
+ if (desc)
+ residue = desc->alloc_sz - desc->xfer_sz;
+ }
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
+
+ret_residue:
+ dma_set_residue(txstate, residue);
+
+ return ret;
+}
+
+static struct dma_async_tx_descriptor *
+dw_edma_device_transfer(struct dw_edma_transfer *xfer)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(xfer->dchan);
+ enum dma_transfer_direction direction = xfer->direction;
+ phys_addr_t src_addr, dst_addr;
+ struct scatterlist *sg = NULL;
+ struct dw_edma_chunk *chunk;
+ struct dw_edma_burst *burst;
+ struct dw_edma_desc *desc;
+ u32 cnt;
+ int i;
+
+ if ((direction == DMA_MEM_TO_DEV && chan->dir == EDMA_DIR_WRITE) ||
+ (direction == DMA_DEV_TO_MEM && chan->dir == EDMA_DIR_READ))
+ return NULL;
+
+ if (xfer->cyclic) {
+ if (!xfer->xfer.cyclic.len || !xfer->xfer.cyclic.cnt)
+ return NULL;
+ } else {
+ if (xfer->xfer.sg.len < 1)
+ return NULL;
+ }
+
+ if (!chan->configured)
+ return NULL;
+
+ desc = dw_edma_alloc_desc(chan);
+ if (unlikely(!desc))
+ goto err_alloc;
+
+ chunk = dw_edma_alloc_chunk(desc);
+ if (unlikely(!chunk))
+ goto err_alloc;
+
+ src_addr = chan->config.src_addr;
+ dst_addr = chan->config.dst_addr;
+
+ if (xfer->cyclic) {
+ cnt = xfer->xfer.cyclic.cnt;
+ } else {
+ cnt = xfer->xfer.sg.len;
+ sg = xfer->xfer.sg.sgl;
+ }
+
+ for (i = 0; i < cnt; i++) {
+ if (!xfer->cyclic && !sg)
+ break;
+
+ if (chunk->bursts_alloc == chan->ll_max) {
+ chunk = dw_edma_alloc_chunk(desc);
+ if (unlikely(!chunk))
+ goto err_alloc;
+ }
+
+ burst = dw_edma_alloc_burst(chunk);
+ if (unlikely(!burst))
+ goto err_alloc;
+
+ if (xfer->cyclic)
+ burst->sz = xfer->xfer.cyclic.len;
+ else
+ burst->sz = sg_dma_len(sg);
+
+ chunk->ll_region.sz += burst->sz;
+ desc->alloc_sz += burst->sz;
+
+ if (direction == DMA_DEV_TO_MEM) {
+ burst->sar = src_addr;
+ if (xfer->cyclic) {
+ burst->dar = xfer->xfer.cyclic.paddr;
+ } else {
+ burst->dar = sg_dma_address(sg);
+ /* Unlike the typical assumption by other
+ * drivers/IPs the peripheral memory isn't
+ * a FIFO memory, in this case, it's a
+ * linear memory and that why the source
+ * and destination addresses are increased
+ * by the same portion (data length)
+ */
+ src_addr += sg_dma_len(sg);
+ }
+ } else {
+ burst->dar = dst_addr;
+ if (xfer->cyclic) {
+ burst->sar = xfer->xfer.cyclic.paddr;
+ } else {
+ burst->sar = sg_dma_address(sg);
+ /* Unlike the typical assumption by other
+ * drivers/IPs the peripheral memory isn't
+ * a FIFO memory, in this case, it's a
+ * linear memory and that why the source
+ * and destination addresses are increased
+ * by the same portion (data length)
+ */
+ dst_addr += sg_dma_len(sg);
+ }
+ }
+
+ if (!xfer->cyclic)
+ sg = sg_next(sg);
+ }
+
+ return vchan_tx_prep(&chan->vc, &desc->vd, xfer->flags);
+
+err_alloc:
+ if (desc)
+ dw_edma_free_desc(desc);
+
+ return NULL;
+}
+
+static struct dma_async_tx_descriptor *
+dw_edma_device_prep_slave_sg(struct dma_chan *dchan, struct scatterlist *sgl,
+ unsigned int len,
+ enum dma_transfer_direction direction,
+ unsigned long flags, void *context)
+{
+ struct dw_edma_transfer xfer;
+
+ xfer.dchan = dchan;
+ xfer.direction = direction;
+ xfer.xfer.sg.sgl = sgl;
+ xfer.xfer.sg.len = len;
+ xfer.flags = flags;
+ xfer.cyclic = false;
+
+ return dw_edma_device_transfer(&xfer);
+}
+
+static struct dma_async_tx_descriptor *
+dw_edma_device_prep_dma_cyclic(struct dma_chan *dchan, dma_addr_t paddr,
+ size_t len, size_t count,
+ enum dma_transfer_direction direction,
+ unsigned long flags)
+{
+ struct dw_edma_transfer xfer;
+
+ xfer.dchan = dchan;
+ xfer.direction = direction;
+ xfer.xfer.cyclic.paddr = paddr;
+ xfer.xfer.cyclic.len = len;
+ xfer.xfer.cyclic.cnt = count;
+ xfer.flags = flags;
+ xfer.cyclic = true;
+
+ return dw_edma_device_transfer(&xfer);
+}
+
+static void dw_edma_done_interrupt(struct dw_edma_chan *chan)
+{
+ struct dw_edma_desc *desc;
+ struct virt_dma_desc *vd;
+ unsigned long flags;
+
+ dw_edma_v0_core_clear_done_int(chan);
+
+ spin_lock_irqsave(&chan->vc.lock, flags);
+ vd = vchan_next_desc(&chan->vc);
+ if (vd) {
+ switch (chan->request) {
+ case EDMA_REQ_NONE:
+ desc = vd2dw_edma_desc(vd);
+ if (desc->chunks_alloc) {
+ chan->status = EDMA_ST_BUSY;
+ dw_edma_start_transfer(chan);
+ } else {
+ list_del(&vd->node);
+ vchan_cookie_complete(vd);
+ chan->status = EDMA_ST_IDLE;
+ }
+ break;
+
+ case EDMA_REQ_STOP:
+ list_del(&vd->node);
+ vchan_cookie_complete(vd);
+ chan->request = EDMA_REQ_NONE;
+ chan->status = EDMA_ST_IDLE;
+ break;
+
+ case EDMA_REQ_PAUSE:
+ chan->request = EDMA_REQ_NONE;
+ chan->status = EDMA_ST_PAUSE;
+ break;
+
+ default:
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
+}
+
+static void dw_edma_abort_interrupt(struct dw_edma_chan *chan)
+{
+ struct virt_dma_desc *vd;
+ unsigned long flags;
+
+ dw_edma_v0_core_clear_abort_int(chan);
+
+ spin_lock_irqsave(&chan->vc.lock, flags);
+ vd = vchan_next_desc(&chan->vc);
+ if (vd) {
+ list_del(&vd->node);
+ vchan_cookie_complete(vd);
+ }
+ spin_unlock_irqrestore(&chan->vc.lock, flags);
+ chan->request = EDMA_REQ_NONE;
+ chan->status = EDMA_ST_IDLE;
+}
+
+static irqreturn_t dw_edma_interrupt(int irq, void *data, bool write)
+{
+ struct dw_edma_irq *dw_irq = data;
+ struct dw_edma *dw = dw_irq->dw;
+ unsigned long total, pos, val;
+ unsigned long off;
+ u32 mask;
+
+ if (write) {
+ total = dw->wr_ch_cnt;
+ off = 0;
+ mask = dw_irq->wr_mask;
+ } else {
+ total = dw->rd_ch_cnt;
+ off = dw->wr_ch_cnt;
+ mask = dw_irq->rd_mask;
+ }
+
+ val = dw_edma_v0_core_status_done_int(dw, write ?
+ EDMA_DIR_WRITE :
+ EDMA_DIR_READ);
+ val &= mask;
+ for_each_set_bit(pos, &val, total) {
+ struct dw_edma_chan *chan = &dw->chan[pos + off];
+
+ dw_edma_done_interrupt(chan);
+ }
+
+ val = dw_edma_v0_core_status_abort_int(dw, write ?
+ EDMA_DIR_WRITE :
+ EDMA_DIR_READ);
+ val &= mask;
+ for_each_set_bit(pos, &val, total) {
+ struct dw_edma_chan *chan = &dw->chan[pos + off];
+
+ dw_edma_abort_interrupt(chan);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static inline irqreturn_t dw_edma_interrupt_write(int irq, void *data)
+{
+ return dw_edma_interrupt(irq, data, true);
+}
+
+static inline irqreturn_t dw_edma_interrupt_read(int irq, void *data)
+{
+ return dw_edma_interrupt(irq, data, false);
+}
+
+static irqreturn_t dw_edma_interrupt_common(int irq, void *data)
+{
+ dw_edma_interrupt(irq, data, true);
+ dw_edma_interrupt(irq, data, false);
+
+ return IRQ_HANDLED;
+}
+
+static int dw_edma_alloc_chan_resources(struct dma_chan *dchan)
+{
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+
+ if (chan->status != EDMA_ST_IDLE)
+ return -EBUSY;
+
+ pm_runtime_get(chan->chip->dev);
+
+ return 0;
+}
+
+static void dw_edma_free_chan_resources(struct dma_chan *dchan)
+{
+ unsigned long timeout = jiffies + msecs_to_jiffies(5000);
+ struct dw_edma_chan *chan = dchan2dw_edma_chan(dchan);
+ int ret;
+
+ while (time_before(jiffies, timeout)) {
+ ret = dw_edma_device_terminate_all(dchan);
+ if (!ret)
+ break;
+
+ if (time_after_eq(jiffies, timeout))
+ return;
+
+ cpu_relax();
+ }
+
+ pm_runtime_put(chan->chip->dev);
+}
+
+static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
+ u32 wr_alloc, u32 rd_alloc)
+{
+ struct dw_edma_region *dt_region;
+ struct device *dev = chip->dev;
+ struct dw_edma *dw = chip->dw;
+ struct dw_edma_chan *chan;
+ size_t ll_chunk, dt_chunk;
+ struct dw_edma_irq *irq;
+ struct dma_device *dma;
+ u32 i, j, cnt, ch_cnt;
+ u32 alloc, off_alloc;
+ int err = 0;
+ u32 pos;
+
+ ch_cnt = dw->wr_ch_cnt + dw->rd_ch_cnt;
+ ll_chunk = dw->ll_region.sz;
+ dt_chunk = dw->dt_region.sz;
+
+ /* Calculate linked list chunk for each channel */
+ ll_chunk /= roundup_pow_of_two(ch_cnt);
+
+ /* Calculate linked list chunk for each channel */
+ dt_chunk /= roundup_pow_of_two(ch_cnt);
+
+ if (write) {
+ i = 0;
+ cnt = dw->wr_ch_cnt;
+ dma = &dw->wr_edma;
+ alloc = wr_alloc;
+ off_alloc = 0;
+ } else {
+ i = dw->wr_ch_cnt;
+ cnt = dw->rd_ch_cnt;
+ dma = &dw->rd_edma;
+ alloc = rd_alloc;
+ off_alloc = wr_alloc;
+ }
+
+ INIT_LIST_HEAD(&dma->channels);
+ for (j = 0; (alloc || dw->nr_irqs == 1) && j < cnt; j++, i++) {
+ chan = &dw->chan[i];
+
+ dt_region = devm_kzalloc(dev, sizeof(*dt_region), GFP_KERNEL);
+ if (!dt_region)
+ return -ENOMEM;
+
+ chan->vc.chan.private = dt_region;
+
+ chan->chip = chip;
+ chan->id = j;
+ chan->dir = write ? EDMA_DIR_WRITE : EDMA_DIR_READ;
+ chan->configured = false;
+ chan->request = EDMA_REQ_NONE;
+ chan->status = EDMA_ST_IDLE;
+
+ chan->ll_off = (ll_chunk * i);
+ chan->ll_max = (ll_chunk / EDMA_LL_SZ) - 1;
+
+ chan->dt_off = (dt_chunk * i);
+
+ dev_vdbg(dev, "L. List:\tChannel %s[%u] off=0x%.8lx, max_cnt=%u\n",
+ write ? "write" : "read", j,
+ chan->ll_off, chan->ll_max);
+
+ if (dw->nr_irqs == 1)
+ pos = 0;
+ else
+ pos = off_alloc + (j % alloc);
+
+ irq = &dw->irq[pos];
+
+ if (write)
+ irq->wr_mask |= BIT(j);
+ else
+ irq->rd_mask |= BIT(j);
+
+ irq->dw = dw;
+ memcpy(&chan->msi, &irq->msi, sizeof(chan->msi));
+
+ dev_vdbg(dev, "MSI:\t\tChannel %s[%u] addr=0x%.8x%.8x, data=0x%.8x\n",
+ write ? "write" : "read", j,
+ chan->msi.address_hi, chan->msi.address_lo,
+ chan->msi.data);
+
+ chan->vc.desc_free = vchan_free_desc;
+ vchan_init(&chan->vc, dma);
+
+ dt_region->paddr = dw->dt_region.paddr + chan->dt_off;
+ dt_region->vaddr = dw->dt_region.vaddr + chan->dt_off;
+ dt_region->sz = dt_chunk;
+
+ dev_vdbg(dev, "Data:\tChannel %s[%u] off=0x%.8lx\n",
+ write ? "write" : "read", j, chan->dt_off);
+
+ dw_edma_v0_core_device_config(chan);
+ }
+
+ /* Set DMA channel capabilities */
+ dma_cap_zero(dma->cap_mask);
+ dma_cap_set(DMA_SLAVE, dma->cap_mask);
+ dma_cap_set(DMA_CYCLIC, dma->cap_mask);
+ dma_cap_set(DMA_PRIVATE, dma->cap_mask);
+ dma->directions = BIT(write ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV);
+ dma->src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
+ dma->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
+ dma->residue_granularity = DMA_RESIDUE_GRANULARITY_DESCRIPTOR;
+ dma->chancnt = cnt;
+
+ /* Set DMA channel callbacks */
+ dma->dev = chip->dev;
+ dma->device_alloc_chan_resources = dw_edma_alloc_chan_resources;
+ dma->device_free_chan_resources = dw_edma_free_chan_resources;
+ dma->device_config = dw_edma_device_config;
+ dma->device_pause = dw_edma_device_pause;
+ dma->device_resume = dw_edma_device_resume;
+ dma->device_terminate_all = dw_edma_device_terminate_all;
+ dma->device_issue_pending = dw_edma_device_issue_pending;
+ dma->device_tx_status = dw_edma_device_tx_status;
+ dma->device_prep_slave_sg = dw_edma_device_prep_slave_sg;
+ dma->device_prep_dma_cyclic = dw_edma_device_prep_dma_cyclic;
+
+ dma_set_max_seg_size(dma->dev, U32_MAX);
+
+ /* Register DMA device */
+ err = dma_async_device_register(dma);
+
+ return err;
+}
+
+static inline void dw_edma_dec_irq_alloc(int *nr_irqs, u32 *alloc, u16 cnt)
+{
+ if (*nr_irqs && *alloc < cnt) {
+ (*alloc)++;
+ (*nr_irqs)--;
+ }
+}
+
+static inline void dw_edma_add_irq_mask(u32 *mask, u32 alloc, u16 cnt)
+{
+ while (*mask * alloc < cnt)
+ (*mask)++;
+}
+
+static int dw_edma_irq_request(struct dw_edma_chip *chip,
+ u32 *wr_alloc, u32 *rd_alloc)
+{
+ struct device *dev = chip->dev;
+ struct dw_edma *dw = chip->dw;
+ u32 wr_mask = 1;
+ u32 rd_mask = 1;
+ int i, err = 0;
+ u32 ch_cnt;
+
+ ch_cnt = dw->wr_ch_cnt + dw->rd_ch_cnt;
+
+ if (dw->nr_irqs < 1)
+ return -EINVAL;
+
+ if (dw->nr_irqs == 1) {
+ /* Common IRQ shared among all channels */
+ err = request_irq(pci_irq_vector(to_pci_dev(dev), 0),
+ dw_edma_interrupt_common,
+ IRQF_SHARED, dw->name, &dw->irq[0]);
+ if (err) {
+ dw->nr_irqs = 0;
+ return err;
+ }
+
+ get_cached_msi_msg(pci_irq_vector(to_pci_dev(dev), 0),
+ &dw->irq[0].msi);
+ } else {
+ /* Distribute IRQs equally among all channels */
+ int tmp = dw->nr_irqs;
+
+ while (tmp && (*wr_alloc + *rd_alloc) < ch_cnt) {
+ dw_edma_dec_irq_alloc(&tmp, wr_alloc, dw->wr_ch_cnt);
+ dw_edma_dec_irq_alloc(&tmp, rd_alloc, dw->rd_ch_cnt);
+ }
+
+ dw_edma_add_irq_mask(&wr_mask, *wr_alloc, dw->wr_ch_cnt);
+ dw_edma_add_irq_mask(&rd_mask, *rd_alloc, dw->rd_ch_cnt);
+
+ for (i = 0; i < (*wr_alloc + *rd_alloc); i++) {
+ err = request_irq(pci_irq_vector(to_pci_dev(dev), i),
+ i < *wr_alloc ?
+ dw_edma_interrupt_write :
+ dw_edma_interrupt_read,
+ IRQF_SHARED, dw->name,
+ &dw->irq[i]);
+ if (err) {
+ dw->nr_irqs = i;
+ return err;
+ }
+
+ get_cached_msi_msg(pci_irq_vector(to_pci_dev(dev), i),
+ &dw->irq[i].msi);
+ }
+
+ dw->nr_irqs = i;
+ }
+
+ return err;
+}
+
+int dw_edma_probe(struct dw_edma_chip *chip)
+{
+ struct device *dev = chip->dev;
+ struct dw_edma *dw = chip->dw;
+ u32 wr_alloc = 0;
+ u32 rd_alloc = 0;
+ int i, err;
+
+ raw_spin_lock_init(&dw->lock);
+
+ /* Find out how many write channels are supported by hardware */
+ dw->wr_ch_cnt = dw_edma_v0_core_ch_count(dw, EDMA_DIR_WRITE);
+ if (!dw->wr_ch_cnt)
+ return -EINVAL;
+
+ /* Find out how many read channels are supported by hardware */
+ dw->rd_ch_cnt = dw_edma_v0_core_ch_count(dw, EDMA_DIR_READ);
+ if (!dw->rd_ch_cnt)
+ return -EINVAL;
+
+ dev_vdbg(dev, "Channels:\twrite=%d, read=%d\n",
+ dw->wr_ch_cnt, dw->rd_ch_cnt);
+
+ /* Allocate channels */
+ dw->chan = devm_kcalloc(dev, dw->wr_ch_cnt + dw->rd_ch_cnt,
+ sizeof(*dw->chan), GFP_KERNEL);
+ if (!dw->chan)
+ return -ENOMEM;
+
+ snprintf(dw->name, sizeof(dw->name), "dw-edma-core:%d", chip->id);
+
+ /* Disable eDMA, only to establish the ideal initial conditions */
+ dw_edma_v0_core_off(dw);
+
+ /* Request IRQs */
+ err = dw_edma_irq_request(chip, &wr_alloc, &rd_alloc);
+ if (err)
+ return err;
+
+ /* Setup write channels */
+ err = dw_edma_channel_setup(chip, true, wr_alloc, rd_alloc);
+ if (err)
+ goto err_irq_free;
+
+ /* Setup read channels */
+ err = dw_edma_channel_setup(chip, false, wr_alloc, rd_alloc);
+ if (err)
+ goto err_irq_free;
+
+ /* Power management */
+ pm_runtime_enable(dev);
+
+ /* Turn debugfs on */
+ dw_edma_v0_core_debugfs_on(chip);
+
+ return 0;
+
+err_irq_free:
+ for (i = (dw->nr_irqs - 1); i >= 0; i--)
+ free_irq(pci_irq_vector(to_pci_dev(dev), i), &dw->irq[i]);
+
+ dw->nr_irqs = 0;
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(dw_edma_probe);
+
+int dw_edma_remove(struct dw_edma_chip *chip)
+{
+ struct dw_edma_chan *chan, *_chan;
+ struct device *dev = chip->dev;
+ struct dw_edma *dw = chip->dw;
+ int i;
+
+ /* Disable eDMA */
+ dw_edma_v0_core_off(dw);
+
+ /* Free irqs */
+ for (i = (dw->nr_irqs - 1); i >= 0; i--)
+ free_irq(pci_irq_vector(to_pci_dev(dev), i), &dw->irq[i]);
+
+ /* Power management */
+ pm_runtime_disable(dev);
+
+ list_for_each_entry_safe(chan, _chan, &dw->wr_edma.channels,
+ vc.chan.device_node) {
+ list_del(&chan->vc.chan.device_node);
+ tasklet_kill(&chan->vc.task);
+ }
+
+ list_for_each_entry_safe(chan, _chan, &dw->rd_edma.channels,
+ vc.chan.device_node) {
+ list_del(&chan->vc.chan.device_node);
+ tasklet_kill(&chan->vc.task);
+ }
+
+ /* Deregister eDMA device */
+ dma_async_device_unregister(&dw->wr_edma);
+ dma_async_device_unregister(&dw->rd_edma);
+
+ /* Turn debugfs off */
+ dw_edma_v0_core_debugfs_off();
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(dw_edma_remove);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Synopsys DesignWare eDMA controller core driver");
+MODULE_AUTHOR("Gustavo Pimentel <gustavo.pimentel@synopsys.com>");
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA core driver
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#ifndef _DW_EDMA_CORE_H
+#define _DW_EDMA_CORE_H
+
+#include <linux/msi.h>
+#include <linux/dma/edma.h>
+
+#include "../virt-dma.h"
+
+#define EDMA_LL_SZ 24
+
+enum dw_edma_dir {
+ EDMA_DIR_WRITE = 0,
+ EDMA_DIR_READ
+};
+
+enum dw_edma_mode {
+ EDMA_MODE_LEGACY = 0,
+ EDMA_MODE_UNROLL
+};
+
+enum dw_edma_request {
+ EDMA_REQ_NONE = 0,
+ EDMA_REQ_STOP,
+ EDMA_REQ_PAUSE
+};
+
+enum dw_edma_status {
+ EDMA_ST_IDLE = 0,
+ EDMA_ST_PAUSE,
+ EDMA_ST_BUSY
+};
+
+struct dw_edma_chan;
+struct dw_edma_chunk;
+
+struct dw_edma_burst {
+ struct list_head list;
+ u64 sar;
+ u64 dar;
+ u32 sz;
+};
+
+struct dw_edma_region {
+ phys_addr_t paddr;
+ dma_addr_t vaddr;
+ size_t sz;
+};
+
+struct dw_edma_chunk {
+ struct list_head list;
+ struct dw_edma_chan *chan;
+ struct dw_edma_burst *burst;
+
+ u32 bursts_alloc;
+
+ u8 cb;
+ struct dw_edma_region ll_region; /* Linked list */
+};
+
+struct dw_edma_desc {
+ struct virt_dma_desc vd;
+ struct dw_edma_chan *chan;
+ struct dw_edma_chunk *chunk;
+
+ u32 chunks_alloc;
+
+ u32 alloc_sz;
+ u32 xfer_sz;
+};
+
+struct dw_edma_chan {
+ struct virt_dma_chan vc;
+ struct dw_edma_chip *chip;
+ int id;
+ enum dw_edma_dir dir;
+
+ off_t ll_off;
+ u32 ll_max;
+
+ off_t dt_off;
+
+ struct msi_msg msi;
+
+ enum dw_edma_request request;
+ enum dw_edma_status status;
+ u8 configured;
+
+ struct dma_slave_config config;
+};
+
+struct dw_edma_irq {
+ struct msi_msg msi;
+ u32 wr_mask;
+ u32 rd_mask;
+ struct dw_edma *dw;
+};
+
+struct dw_edma {
+ char name[20];
+
+ struct dma_device wr_edma;
+ u16 wr_ch_cnt;
+
+ struct dma_device rd_edma;
+ u16 rd_ch_cnt;
+
+ struct dw_edma_region rg_region; /* Registers */
+ struct dw_edma_region ll_region; /* Linked list */
+ struct dw_edma_region dt_region; /* Data */
+
+ struct dw_edma_irq *irq;
+ int nr_irqs;
+
+ u32 version;
+ enum dw_edma_mode mode;
+
+ struct dw_edma_chan *chan;
+ const struct dw_edma_core_ops *ops;
+
+ raw_spinlock_t lock; /* Only for legacy */
+};
+
+struct dw_edma_sg {
+ struct scatterlist *sgl;
+ unsigned int len;
+};
+
+struct dw_edma_cyclic {
+ dma_addr_t paddr;
+ size_t len;
+ size_t cnt;
+};
+
+struct dw_edma_transfer {
+ struct dma_chan *dchan;
+ union dw_edma_xfer {
+ struct dw_edma_sg sg;
+ struct dw_edma_cyclic cyclic;
+ } xfer;
+ enum dma_transfer_direction direction;
+ unsigned long flags;
+ bool cyclic;
+};
+
+static inline
+struct dw_edma_chan *vc2dw_edma_chan(struct virt_dma_chan *vc)
+{
+ return container_of(vc, struct dw_edma_chan, vc);
+}
+
+static inline
+struct dw_edma_chan *dchan2dw_edma_chan(struct dma_chan *dchan)
+{
+ return vc2dw_edma_chan(to_virt_chan(dchan));
+}
+
+#endif /* _DW_EDMA_CORE_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA PCIe driver
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/device.h>
+#include <linux/dma/edma.h>
+#include <linux/pci-epf.h>
+#include <linux/msi.h>
+
+#include "dw-edma-core.h"
+
+struct dw_edma_pcie_data {
+ /* eDMA registers location */
+ enum pci_barno rg_bar;
+ off_t rg_off;
+ size_t rg_sz;
+ /* eDMA memory linked list location */
+ enum pci_barno ll_bar;
+ off_t ll_off;
+ size_t ll_sz;
+ /* eDMA memory data location */
+ enum pci_barno dt_bar;
+ off_t dt_off;
+ size_t dt_sz;
+ /* Other */
+ u32 version;
+ enum dw_edma_mode mode;
+ u8 irqs;
+};
+
+static const struct dw_edma_pcie_data snps_edda_data = {
+ /* eDMA registers location */
+ .rg_bar = BAR_0,
+ .rg_off = 0x00001000, /* 4 Kbytes */
+ .rg_sz = 0x00002000, /* 8 Kbytes */
+ /* eDMA memory linked list location */
+ .ll_bar = BAR_2,
+ .ll_off = 0x00000000, /* 0 Kbytes */
+ .ll_sz = 0x00800000, /* 8 Mbytes */
+ /* eDMA memory data location */
+ .dt_bar = BAR_2,
+ .dt_off = 0x00800000, /* 8 Mbytes */
+ .dt_sz = 0x03800000, /* 56 Mbytes */
+ /* Other */
+ .version = 0,
+ .mode = EDMA_MODE_UNROLL,
+ .irqs = 1,
+};
+
+static int dw_edma_pcie_probe(struct pci_dev *pdev,
+ const struct pci_device_id *pid)
+{
+ const struct dw_edma_pcie_data *pdata = (void *)pid->driver_data;
+ struct device *dev = &pdev->dev;
+ struct dw_edma_chip *chip;
+ int err, nr_irqs;
+ struct dw_edma *dw;
+
+ /* Enable PCI device */
+ err = pcim_enable_device(pdev);
+ if (err) {
+ pci_err(pdev, "enabling device failed\n");
+ return err;
+ }
+
+ /* Mapping PCI BAR regions */
+ err = pcim_iomap_regions(pdev, BIT(pdata->rg_bar) |
+ BIT(pdata->ll_bar) |
+ BIT(pdata->dt_bar),
+ pci_name(pdev));
+ if (err) {
+ pci_err(pdev, "eDMA BAR I/O remapping failed\n");
+ return err;
+ }
+
+ pci_set_master(pdev);
+
+ /* DMA configuration */
+ err = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (!err) {
+ err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (err) {
+ pci_err(pdev, "consistent DMA mask 64 set failed\n");
+ return err;
+ }
+ } else {
+ pci_err(pdev, "DMA mask 64 set failed\n");
+
+ err = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (err) {
+ pci_err(pdev, "DMA mask 32 set failed\n");
+ return err;
+ }
+
+ err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (err) {
+ pci_err(pdev, "consistent DMA mask 32 set failed\n");
+ return err;
+ }
+ }
+
+ /* Data structure allocation */
+ chip = devm_kzalloc(dev, sizeof(*chip), GFP_KERNEL);
+ if (!chip)
+ return -ENOMEM;
+
+ dw = devm_kzalloc(dev, sizeof(*dw), GFP_KERNEL);
+ if (!dw)
+ return -ENOMEM;
+
+ /* IRQs allocation */
+ nr_irqs = pci_alloc_irq_vectors(pdev, 1, pdata->irqs,
+ PCI_IRQ_MSI | PCI_IRQ_MSIX);
+ if (nr_irqs < 1) {
+ pci_err(pdev, "fail to alloc IRQ vector (number of IRQs=%u)\n",
+ nr_irqs);
+ return -EPERM;
+ }
+
+ /* Data structure initialization */
+ chip->dw = dw;
+ chip->dev = dev;
+ chip->id = pdev->devfn;
+ chip->irq = pdev->irq;
+
+ dw->rg_region.vaddr = (dma_addr_t)pcim_iomap_table(pdev)[pdata->rg_bar];
+ dw->rg_region.vaddr += pdata->rg_off;
+ dw->rg_region.paddr = pdev->resource[pdata->rg_bar].start;
+ dw->rg_region.paddr += pdata->rg_off;
+ dw->rg_region.sz = pdata->rg_sz;
+
+ dw->ll_region.vaddr = (dma_addr_t)pcim_iomap_table(pdev)[pdata->ll_bar];
+ dw->ll_region.vaddr += pdata->ll_off;
+ dw->ll_region.paddr = pdev->resource[pdata->ll_bar].start;
+ dw->ll_region.paddr += pdata->ll_off;
+ dw->ll_region.sz = pdata->ll_sz;
+
+ dw->dt_region.vaddr = (dma_addr_t)pcim_iomap_table(pdev)[pdata->dt_bar];
+ dw->dt_region.vaddr += pdata->dt_off;
+ dw->dt_region.paddr = pdev->resource[pdata->dt_bar].start;
+ dw->dt_region.paddr += pdata->dt_off;
+ dw->dt_region.sz = pdata->dt_sz;
+
+ dw->version = pdata->version;
+ dw->mode = pdata->mode;
+ dw->nr_irqs = nr_irqs;
+
+ /* Debug info */
+ pci_dbg(pdev, "Version:\t%u\n", dw->version);
+
+ pci_dbg(pdev, "Mode:\t%s\n",
+ dw->mode == EDMA_MODE_LEGACY ? "Legacy" : "Unroll");
+
+ pci_dbg(pdev, "Registers:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%pa, p=%pa)\n",
+ pdata->rg_bar, pdata->rg_off, pdata->rg_sz,
+ &dw->rg_region.vaddr, &dw->rg_region.paddr);
+
+ pci_dbg(pdev, "L. List:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%pa, p=%pa)\n",
+ pdata->ll_bar, pdata->ll_off, pdata->ll_sz,
+ &dw->ll_region.vaddr, &dw->ll_region.paddr);
+
+ pci_dbg(pdev, "Data:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%pa, p=%pa)\n",
+ pdata->dt_bar, pdata->dt_off, pdata->dt_sz,
+ &dw->dt_region.vaddr, &dw->dt_region.paddr);
+
+ pci_dbg(pdev, "Nr. IRQs:\t%u\n", dw->nr_irqs);
+
+ /* Validating if PCI interrupts were enabled */
+ if (!pci_dev_msi_enabled(pdev)) {
+ pci_err(pdev, "enable interrupt failed\n");
+ return -EPERM;
+ }
+
+ dw->irq = devm_kcalloc(dev, nr_irqs, sizeof(*dw->irq), GFP_KERNEL);
+ if (!dw->irq)
+ return -ENOMEM;
+
+ /* Starting eDMA driver */
+ err = dw_edma_probe(chip);
+ if (err) {
+ pci_err(pdev, "eDMA probe failed\n");
+ return err;
+ }
+
+ /* Saving data structure reference */
+ pci_set_drvdata(pdev, chip);
+
+ return 0;
+}
+
+static void dw_edma_pcie_remove(struct pci_dev *pdev)
+{
+ struct dw_edma_chip *chip = pci_get_drvdata(pdev);
+ int err;
+
+ /* Stopping eDMA driver */
+ err = dw_edma_remove(chip);
+ if (err)
+ pci_warn(pdev, "can't remove device properly: %d\n", err);
+
+ /* Freeing IRQs */
+ pci_free_irq_vectors(pdev);
+}
+
+static const struct pci_device_id dw_edma_pcie_id_table[] = {
+ { PCI_DEVICE_DATA(SYNOPSYS, EDDA, &snps_edda_data) },
+ { }
+};
+MODULE_DEVICE_TABLE(pci, dw_edma_pcie_id_table);
+
+static struct pci_driver dw_edma_pcie_driver = {
+ .name = "dw-edma-pcie",
+ .id_table = dw_edma_pcie_id_table,
+ .probe = dw_edma_pcie_probe,
+ .remove = dw_edma_pcie_remove,
+};
+
+module_pci_driver(dw_edma_pcie_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("Synopsys DesignWare eDMA PCIe driver");
+MODULE_AUTHOR("Gustavo Pimentel <gustavo.pimentel@synopsys.com>");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA v0 core
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#include <linux/bitfield.h>
+
+#include "dw-edma-core.h"
+#include "dw-edma-v0-core.h"
+#include "dw-edma-v0-regs.h"
+#include "dw-edma-v0-debugfs.h"
+
+enum dw_edma_control {
+ DW_EDMA_V0_CB = BIT(0),
+ DW_EDMA_V0_TCB = BIT(1),
+ DW_EDMA_V0_LLP = BIT(2),
+ DW_EDMA_V0_LIE = BIT(3),
+ DW_EDMA_V0_RIE = BIT(4),
+ DW_EDMA_V0_CCS = BIT(8),
+ DW_EDMA_V0_LLE = BIT(9),
+};
+
+static inline struct dw_edma_v0_regs __iomem *__dw_regs(struct dw_edma *dw)
+{
+ return (struct dw_edma_v0_regs __iomem *)dw->rg_region.vaddr;
+}
+
+#define SET(dw, name, value) \
+ writel(value, &(__dw_regs(dw)->name))
+
+#define GET(dw, name) \
+ readl(&(__dw_regs(dw)->name))
+
+#define SET_RW(dw, dir, name, value) \
+ do { \
+ if ((dir) == EDMA_DIR_WRITE) \
+ SET(dw, wr_##name, value); \
+ else \
+ SET(dw, rd_##name, value); \
+ } while (0)
+
+#define GET_RW(dw, dir, name) \
+ ((dir) == EDMA_DIR_WRITE \
+ ? GET(dw, wr_##name) \
+ : GET(dw, rd_##name))
+
+#define SET_BOTH(dw, name, value) \
+ do { \
+ SET(dw, wr_##name, value); \
+ SET(dw, rd_##name, value); \
+ } while (0)
+
+static inline struct dw_edma_v0_ch_regs __iomem *
+__dw_ch_regs(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch)
+{
+ if (dw->mode == EDMA_MODE_LEGACY)
+ return &(__dw_regs(dw)->type.legacy.ch);
+
+ if (dir == EDMA_DIR_WRITE)
+ return &__dw_regs(dw)->type.unroll.ch[ch].wr;
+
+ return &__dw_regs(dw)->type.unroll.ch[ch].rd;
+}
+
+static inline void writel_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
+ u32 value, void __iomem *addr)
+{
+ if (dw->mode == EDMA_MODE_LEGACY) {
+ u32 viewport_sel;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
+ viewport_sel = FIELD_PREP(EDMA_V0_VIEWPORT_MASK, ch);
+ if (dir == EDMA_DIR_READ)
+ viewport_sel |= BIT(31);
+
+ writel(viewport_sel,
+ &(__dw_regs(dw)->type.legacy.viewport_sel));
+ writel(value, addr);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+ } else {
+ writel(value, addr);
+ }
+}
+
+static inline u32 readl_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
+ const void __iomem *addr)
+{
+ u32 value;
+
+ if (dw->mode == EDMA_MODE_LEGACY) {
+ u32 viewport_sel;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
+ viewport_sel = FIELD_PREP(EDMA_V0_VIEWPORT_MASK, ch);
+ if (dir == EDMA_DIR_READ)
+ viewport_sel |= BIT(31);
+
+ writel(viewport_sel,
+ &(__dw_regs(dw)->type.legacy.viewport_sel));
+ value = readl(addr);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+ } else {
+ value = readl(addr);
+ }
+
+ return value;
+}
+
+#define SET_CH(dw, dir, ch, name, value) \
+ writel_ch(dw, dir, ch, value, &(__dw_ch_regs(dw, dir, ch)->name))
+
+#define GET_CH(dw, dir, ch, name) \
+ readl_ch(dw, dir, ch, &(__dw_ch_regs(dw, dir, ch)->name))
+
+#define SET_LL(ll, value) \
+ writel(value, ll)
+
+/* eDMA management callbacks */
+void dw_edma_v0_core_off(struct dw_edma *dw)
+{
+ SET_BOTH(dw, int_mask, EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
+ SET_BOTH(dw, int_clear, EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
+ SET_BOTH(dw, engine_en, 0);
+}
+
+u16 dw_edma_v0_core_ch_count(struct dw_edma *dw, enum dw_edma_dir dir)
+{
+ u32 num_ch;
+
+ if (dir == EDMA_DIR_WRITE)
+ num_ch = FIELD_GET(EDMA_V0_WRITE_CH_COUNT_MASK, GET(dw, ctrl));
+ else
+ num_ch = FIELD_GET(EDMA_V0_READ_CH_COUNT_MASK, GET(dw, ctrl));
+
+ if (num_ch > EDMA_V0_MAX_NR_CH)
+ num_ch = EDMA_V0_MAX_NR_CH;
+
+ return (u16)num_ch;
+}
+
+enum dma_status dw_edma_v0_core_ch_status(struct dw_edma_chan *chan)
+{
+ struct dw_edma *dw = chan->chip->dw;
+ u32 tmp;
+
+ tmp = FIELD_GET(EDMA_V0_CH_STATUS_MASK,
+ GET_CH(dw, chan->dir, chan->id, ch_control1));
+
+ if (tmp == 1)
+ return DMA_IN_PROGRESS;
+ else if (tmp == 3)
+ return DMA_COMPLETE;
+ else
+ return DMA_ERROR;
+}
+
+void dw_edma_v0_core_clear_done_int(struct dw_edma_chan *chan)
+{
+ struct dw_edma *dw = chan->chip->dw;
+
+ SET_RW(dw, chan->dir, int_clear,
+ FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id)));
+}
+
+void dw_edma_v0_core_clear_abort_int(struct dw_edma_chan *chan)
+{
+ struct dw_edma *dw = chan->chip->dw;
+
+ SET_RW(dw, chan->dir, int_clear,
+ FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id)));
+}
+
+u32 dw_edma_v0_core_status_done_int(struct dw_edma *dw, enum dw_edma_dir dir)
+{
+ return FIELD_GET(EDMA_V0_DONE_INT_MASK, GET_RW(dw, dir, int_status));
+}
+
+u32 dw_edma_v0_core_status_abort_int(struct dw_edma *dw, enum dw_edma_dir dir)
+{
+ return FIELD_GET(EDMA_V0_ABORT_INT_MASK, GET_RW(dw, dir, int_status));
+}
+
+static void dw_edma_v0_core_write_chunk(struct dw_edma_chunk *chunk)
+{
+ struct dw_edma_burst *child;
+ struct dw_edma_v0_lli *lli;
+ struct dw_edma_v0_llp *llp;
+ u32 control = 0, i = 0;
+ u64 sar, dar, addr;
+ int j;
+
+ lli = (struct dw_edma_v0_lli *)chunk->ll_region.vaddr;
+
+ if (chunk->cb)
+ control = DW_EDMA_V0_CB;
+
+ j = chunk->bursts_alloc;
+ list_for_each_entry(child, &chunk->burst->list, list) {
+ j--;
+ if (!j)
+ control |= (DW_EDMA_V0_LIE | DW_EDMA_V0_RIE);
+
+ /* Channel control */
+ SET_LL(&lli[i].control, control);
+ /* Transfer size */
+ SET_LL(&lli[i].transfer_size, child->sz);
+ /* SAR - low, high */
+ sar = cpu_to_le64(child->sar);
+ SET_LL(&lli[i].sar_low, lower_32_bits(sar));
+ SET_LL(&lli[i].sar_high, upper_32_bits(sar));
+ /* DAR - low, high */
+ dar = cpu_to_le64(child->dar);
+ SET_LL(&lli[i].dar_low, lower_32_bits(dar));
+ SET_LL(&lli[i].dar_high, upper_32_bits(dar));
+ i++;
+ }
+
+ llp = (struct dw_edma_v0_llp *)&lli[i];
+ control = DW_EDMA_V0_LLP | DW_EDMA_V0_TCB;
+ if (!chunk->cb)
+ control |= DW_EDMA_V0_CB;
+
+ /* Channel control */
+ SET_LL(&llp->control, control);
+ /* Linked list - low, high */
+ addr = cpu_to_le64(chunk->ll_region.paddr);
+ SET_LL(&llp->llp_low, lower_32_bits(addr));
+ SET_LL(&llp->llp_high, upper_32_bits(addr));
+}
+
+void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
+{
+ struct dw_edma_chan *chan = chunk->chan;
+ struct dw_edma *dw = chan->chip->dw;
+ u32 tmp;
+ u64 llp;
+
+ dw_edma_v0_core_write_chunk(chunk);
+
+ if (first) {
+ /* Enable engine */
+ SET_RW(dw, chan->dir, engine_en, BIT(0));
+ /* Interrupt unmask - done, abort */
+ tmp = GET_RW(dw, chan->dir, int_mask);
+ tmp &= ~FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id));
+ tmp &= ~FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id));
+ SET_RW(dw, chan->dir, int_mask, tmp);
+ /* Linked list error */
+ tmp = GET_RW(dw, chan->dir, linked_list_err_en);
+ tmp |= FIELD_PREP(EDMA_V0_LINKED_LIST_ERR_MASK, BIT(chan->id));
+ SET_RW(dw, chan->dir, linked_list_err_en, tmp);
+ /* Channel control */
+ SET_CH(dw, chan->dir, chan->id, ch_control1,
+ (DW_EDMA_V0_CCS | DW_EDMA_V0_LLE));
+ /* Linked list - low, high */
+ llp = cpu_to_le64(chunk->ll_region.paddr);
+ SET_CH(dw, chan->dir, chan->id, llp_low, lower_32_bits(llp));
+ SET_CH(dw, chan->dir, chan->id, llp_high, upper_32_bits(llp));
+ }
+ /* Doorbell */
+ SET_RW(dw, chan->dir, doorbell,
+ FIELD_PREP(EDMA_V0_DOORBELL_CH_MASK, chan->id));
+}
+
+int dw_edma_v0_core_device_config(struct dw_edma_chan *chan)
+{
+ struct dw_edma *dw = chan->chip->dw;
+ u32 tmp = 0;
+
+ /* MSI done addr - low, high */
+ SET_RW(dw, chan->dir, done_imwr_low, chan->msi.address_lo);
+ SET_RW(dw, chan->dir, done_imwr_high, chan->msi.address_hi);
+ /* MSI abort addr - low, high */
+ SET_RW(dw, chan->dir, abort_imwr_low, chan->msi.address_lo);
+ SET_RW(dw, chan->dir, abort_imwr_high, chan->msi.address_hi);
+ /* MSI data - low, high */
+ switch (chan->id) {
+ case 0:
+ case 1:
+ tmp = GET_RW(dw, chan->dir, ch01_imwr_data);
+ break;
+
+ case 2:
+ case 3:
+ tmp = GET_RW(dw, chan->dir, ch23_imwr_data);
+ break;
+
+ case 4:
+ case 5:
+ tmp = GET_RW(dw, chan->dir, ch45_imwr_data);
+ break;
+
+ case 6:
+ case 7:
+ tmp = GET_RW(dw, chan->dir, ch67_imwr_data);
+ break;
+ }
+
+ if (chan->id & BIT(0)) {
+ /* Channel odd {1, 3, 5, 7} */
+ tmp &= EDMA_V0_CH_EVEN_MSI_DATA_MASK;
+ tmp |= FIELD_PREP(EDMA_V0_CH_ODD_MSI_DATA_MASK,
+ chan->msi.data);
+ } else {
+ /* Channel even {0, 2, 4, 6} */
+ tmp &= EDMA_V0_CH_ODD_MSI_DATA_MASK;
+ tmp |= FIELD_PREP(EDMA_V0_CH_EVEN_MSI_DATA_MASK,
+ chan->msi.data);
+ }
+
+ switch (chan->id) {
+ case 0:
+ case 1:
+ SET_RW(dw, chan->dir, ch01_imwr_data, tmp);
+ break;
+
+ case 2:
+ case 3:
+ SET_RW(dw, chan->dir, ch23_imwr_data, tmp);
+ break;
+
+ case 4:
+ case 5:
+ SET_RW(dw, chan->dir, ch45_imwr_data, tmp);
+ break;
+
+ case 6:
+ case 7:
+ SET_RW(dw, chan->dir, ch67_imwr_data, tmp);
+ break;
+ }
+
+ return 0;
+}
+
+/* eDMA debugfs callbacks */
+void dw_edma_v0_core_debugfs_on(struct dw_edma_chip *chip)
+{
+ dw_edma_v0_debugfs_on(chip);
+}
+
+void dw_edma_v0_core_debugfs_off(void)
+{
+ dw_edma_v0_debugfs_off();
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA v0 core
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#ifndef _DW_EDMA_V0_CORE_H
+#define _DW_EDMA_V0_CORE_H
+
+#include <linux/dma/edma.h>
+
+/* eDMA management callbacks */
+void dw_edma_v0_core_off(struct dw_edma *chan);
+u16 dw_edma_v0_core_ch_count(struct dw_edma *chan, enum dw_edma_dir dir);
+enum dma_status dw_edma_v0_core_ch_status(struct dw_edma_chan *chan);
+void dw_edma_v0_core_clear_done_int(struct dw_edma_chan *chan);
+void dw_edma_v0_core_clear_abort_int(struct dw_edma_chan *chan);
+u32 dw_edma_v0_core_status_done_int(struct dw_edma *chan, enum dw_edma_dir dir);
+u32 dw_edma_v0_core_status_abort_int(struct dw_edma *chan, enum dw_edma_dir dir);
+void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first);
+int dw_edma_v0_core_device_config(struct dw_edma_chan *chan);
+/* eDMA debug fs callbacks */
+void dw_edma_v0_core_debugfs_on(struct dw_edma_chip *chip);
+void dw_edma_v0_core_debugfs_off(void);
+
+#endif /* _DW_EDMA_V0_CORE_H */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA v0 core
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#include <linux/debugfs.h>
+#include <linux/bitfield.h>
+
+#include "dw-edma-v0-debugfs.h"
+#include "dw-edma-v0-regs.h"
+#include "dw-edma-core.h"
+
+#define REGS_ADDR(name) \
+ ((dma_addr_t *)®s->name)
+#define REGISTER(name) \
+ { #name, REGS_ADDR(name) }
+
+#define WR_REGISTER(name) \
+ { #name, REGS_ADDR(wr_##name) }
+#define RD_REGISTER(name) \
+ { #name, REGS_ADDR(rd_##name) }
+
+#define WR_REGISTER_LEGACY(name) \
+ { #name, REGS_ADDR(type.legacy.wr_##name) }
+#define RD_REGISTER_LEGACY(name) \
+ { #name, REGS_ADDR(type.legacy.rd_##name) }
+
+#define WR_REGISTER_UNROLL(name) \
+ { #name, REGS_ADDR(type.unroll.wr_##name) }
+#define RD_REGISTER_UNROLL(name) \
+ { #name, REGS_ADDR(type.unroll.rd_##name) }
+
+#define WRITE_STR "write"
+#define READ_STR "read"
+#define CHANNEL_STR "channel"
+#define REGISTERS_STR "registers"
+
+static struct dentry *base_dir;
+static struct dw_edma *dw;
+static struct dw_edma_v0_regs *regs;
+
+static struct {
+ void *start;
+ void *end;
+} lim[2][EDMA_V0_MAX_NR_CH];
+
+struct debugfs_entries {
+ char name[24];
+ dma_addr_t *reg;
+};
+
+static int dw_edma_debugfs_u32_get(void *data, u64 *val)
+{
+ if (dw->mode == EDMA_MODE_LEGACY &&
+ data >= (void *)®s->type.legacy.ch) {
+ void *ptr = (void *)®s->type.legacy.ch;
+ u32 viewport_sel = 0;
+ unsigned long flags;
+ u16 ch;
+
+ for (ch = 0; ch < dw->wr_ch_cnt; ch++)
+ if (lim[0][ch].start >= data && data < lim[0][ch].end) {
+ ptr += (data - lim[0][ch].start);
+ goto legacy_sel_wr;
+ }
+
+ for (ch = 0; ch < dw->rd_ch_cnt; ch++)
+ if (lim[1][ch].start >= data && data < lim[1][ch].end) {
+ ptr += (data - lim[1][ch].start);
+ goto legacy_sel_rd;
+ }
+
+ return 0;
+legacy_sel_rd:
+ viewport_sel = BIT(31);
+legacy_sel_wr:
+ viewport_sel |= FIELD_PREP(EDMA_V0_VIEWPORT_MASK, ch);
+
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
+ writel(viewport_sel, ®s->type.legacy.viewport_sel);
+ *val = readl(ptr);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+ } else {
+ *val = readl(data);
+ }
+
+ return 0;
+}
+DEFINE_DEBUGFS_ATTRIBUTE(fops_x32, dw_edma_debugfs_u32_get, NULL, "0x%08llx\n");
+
+static void dw_edma_debugfs_create_x32(const struct debugfs_entries entries[],
+ int nr_entries, struct dentry *dir)
+{
+ int i;
+
+ for (i = 0; i < nr_entries; i++) {
+ if (!debugfs_create_file_unsafe(entries[i].name, 0444, dir,
+ entries[i].reg, &fops_x32))
+ break;
+ }
+}
+
+static void dw_edma_debugfs_regs_ch(struct dw_edma_v0_ch_regs *regs,
+ struct dentry *dir)
+{
+ int nr_entries;
+ const struct debugfs_entries debugfs_regs[] = {
+ REGISTER(ch_control1),
+ REGISTER(ch_control2),
+ REGISTER(transfer_size),
+ REGISTER(sar_low),
+ REGISTER(sar_high),
+ REGISTER(dar_low),
+ REGISTER(dar_high),
+ REGISTER(llp_low),
+ REGISTER(llp_high),
+ };
+
+ nr_entries = ARRAY_SIZE(debugfs_regs);
+ dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, dir);
+}
+
+static void dw_edma_debugfs_regs_wr(struct dentry *dir)
+{
+ const struct debugfs_entries debugfs_regs[] = {
+ /* eDMA global registers */
+ WR_REGISTER(engine_en),
+ WR_REGISTER(doorbell),
+ WR_REGISTER(ch_arb_weight_low),
+ WR_REGISTER(ch_arb_weight_high),
+ /* eDMA interrupts registers */
+ WR_REGISTER(int_status),
+ WR_REGISTER(int_mask),
+ WR_REGISTER(int_clear),
+ WR_REGISTER(err_status),
+ WR_REGISTER(done_imwr_low),
+ WR_REGISTER(done_imwr_high),
+ WR_REGISTER(abort_imwr_low),
+ WR_REGISTER(abort_imwr_high),
+ WR_REGISTER(ch01_imwr_data),
+ WR_REGISTER(ch23_imwr_data),
+ WR_REGISTER(ch45_imwr_data),
+ WR_REGISTER(ch67_imwr_data),
+ WR_REGISTER(linked_list_err_en),
+ };
+ const struct debugfs_entries debugfs_unroll_regs[] = {
+ /* eDMA channel context grouping */
+ WR_REGISTER_UNROLL(engine_chgroup),
+ WR_REGISTER_UNROLL(engine_hshake_cnt_low),
+ WR_REGISTER_UNROLL(engine_hshake_cnt_high),
+ WR_REGISTER_UNROLL(ch0_pwr_en),
+ WR_REGISTER_UNROLL(ch1_pwr_en),
+ WR_REGISTER_UNROLL(ch2_pwr_en),
+ WR_REGISTER_UNROLL(ch3_pwr_en),
+ WR_REGISTER_UNROLL(ch4_pwr_en),
+ WR_REGISTER_UNROLL(ch5_pwr_en),
+ WR_REGISTER_UNROLL(ch6_pwr_en),
+ WR_REGISTER_UNROLL(ch7_pwr_en),
+ };
+ struct dentry *regs_dir, *ch_dir;
+ int nr_entries, i;
+ char name[16];
+
+ regs_dir = debugfs_create_dir(WRITE_STR, dir);
+ if (!regs_dir)
+ return;
+
+ nr_entries = ARRAY_SIZE(debugfs_regs);
+ dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, regs_dir);
+
+ if (dw->mode == EDMA_MODE_UNROLL) {
+ nr_entries = ARRAY_SIZE(debugfs_unroll_regs);
+ dw_edma_debugfs_create_x32(debugfs_unroll_regs, nr_entries,
+ regs_dir);
+ }
+
+ for (i = 0; i < dw->wr_ch_cnt; i++) {
+ snprintf(name, sizeof(name), "%s:%d", CHANNEL_STR, i);
+
+ ch_dir = debugfs_create_dir(name, regs_dir);
+ if (!ch_dir)
+ return;
+
+ dw_edma_debugfs_regs_ch(®s->type.unroll.ch[i].wr, ch_dir);
+
+ lim[0][i].start = ®s->type.unroll.ch[i].wr;
+ lim[0][i].end = ®s->type.unroll.ch[i].padding_1[0];
+ }
+}
+
+static void dw_edma_debugfs_regs_rd(struct dentry *dir)
+{
+ const struct debugfs_entries debugfs_regs[] = {
+ /* eDMA global registers */
+ RD_REGISTER(engine_en),
+ RD_REGISTER(doorbell),
+ RD_REGISTER(ch_arb_weight_low),
+ RD_REGISTER(ch_arb_weight_high),
+ /* eDMA interrupts registers */
+ RD_REGISTER(int_status),
+ RD_REGISTER(int_mask),
+ RD_REGISTER(int_clear),
+ RD_REGISTER(err_status_low),
+ RD_REGISTER(err_status_high),
+ RD_REGISTER(linked_list_err_en),
+ RD_REGISTER(done_imwr_low),
+ RD_REGISTER(done_imwr_high),
+ RD_REGISTER(abort_imwr_low),
+ RD_REGISTER(abort_imwr_high),
+ RD_REGISTER(ch01_imwr_data),
+ RD_REGISTER(ch23_imwr_data),
+ RD_REGISTER(ch45_imwr_data),
+ RD_REGISTER(ch67_imwr_data),
+ };
+ const struct debugfs_entries debugfs_unroll_regs[] = {
+ /* eDMA channel context grouping */
+ RD_REGISTER_UNROLL(engine_chgroup),
+ RD_REGISTER_UNROLL(engine_hshake_cnt_low),
+ RD_REGISTER_UNROLL(engine_hshake_cnt_high),
+ RD_REGISTER_UNROLL(ch0_pwr_en),
+ RD_REGISTER_UNROLL(ch1_pwr_en),
+ RD_REGISTER_UNROLL(ch2_pwr_en),
+ RD_REGISTER_UNROLL(ch3_pwr_en),
+ RD_REGISTER_UNROLL(ch4_pwr_en),
+ RD_REGISTER_UNROLL(ch5_pwr_en),
+ RD_REGISTER_UNROLL(ch6_pwr_en),
+ RD_REGISTER_UNROLL(ch7_pwr_en),
+ };
+ struct dentry *regs_dir, *ch_dir;
+ int nr_entries, i;
+ char name[16];
+
+ regs_dir = debugfs_create_dir(READ_STR, dir);
+ if (!regs_dir)
+ return;
+
+ nr_entries = ARRAY_SIZE(debugfs_regs);
+ dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, regs_dir);
+
+ if (dw->mode == EDMA_MODE_UNROLL) {
+ nr_entries = ARRAY_SIZE(debugfs_unroll_regs);
+ dw_edma_debugfs_create_x32(debugfs_unroll_regs, nr_entries,
+ regs_dir);
+ }
+
+ for (i = 0; i < dw->rd_ch_cnt; i++) {
+ snprintf(name, sizeof(name), "%s:%d", CHANNEL_STR, i);
+
+ ch_dir = debugfs_create_dir(name, regs_dir);
+ if (!ch_dir)
+ return;
+
+ dw_edma_debugfs_regs_ch(®s->type.unroll.ch[i].rd, ch_dir);
+
+ lim[1][i].start = ®s->type.unroll.ch[i].rd;
+ lim[1][i].end = ®s->type.unroll.ch[i].padding_2[0];
+ }
+}
+
+static void dw_edma_debugfs_regs(void)
+{
+ const struct debugfs_entries debugfs_regs[] = {
+ REGISTER(ctrl_data_arb_prior),
+ REGISTER(ctrl),
+ };
+ struct dentry *regs_dir;
+ int nr_entries;
+
+ regs_dir = debugfs_create_dir(REGISTERS_STR, base_dir);
+ if (!regs_dir)
+ return;
+
+ nr_entries = ARRAY_SIZE(debugfs_regs);
+ dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, regs_dir);
+
+ dw_edma_debugfs_regs_wr(regs_dir);
+ dw_edma_debugfs_regs_rd(regs_dir);
+}
+
+void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip)
+{
+ dw = chip->dw;
+ if (!dw)
+ return;
+
+ regs = (struct dw_edma_v0_regs *)dw->rg_region.vaddr;
+ if (!regs)
+ return;
+
+ base_dir = debugfs_create_dir(dw->name, 0);
+ if (!base_dir)
+ return;
+
+ debugfs_create_u32("version", 0444, base_dir, &dw->version);
+ debugfs_create_u32("mode", 0444, base_dir, &dw->mode);
+ debugfs_create_u16("wr_ch_cnt", 0444, base_dir, &dw->wr_ch_cnt);
+ debugfs_create_u16("rd_ch_cnt", 0444, base_dir, &dw->rd_ch_cnt);
+
+ dw_edma_debugfs_regs();
+}
+
+void dw_edma_v0_debugfs_off(void)
+{
+ debugfs_remove_recursive(base_dir);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA v0 core
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#ifndef _DW_EDMA_V0_DEBUG_FS_H
+#define _DW_EDMA_V0_DEBUG_FS_H
+
+#include <linux/dma/edma.h>
+
+#ifdef CONFIG_DEBUG_FS
+void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip);
+void dw_edma_v0_debugfs_off(void);
+#else
+static inline void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip)
+{
+}
+
+static inline void dw_edma_v0_debugfs_off(void)
+{
+}
+#endif /* CONFIG_DEBUG_FS */
+
+#endif /* _DW_EDMA_V0_DEBUG_FS_H */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA v0 core
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#ifndef _DW_EDMA_V0_REGS_H
+#define _DW_EDMA_V0_REGS_H
+
+#include <linux/dmaengine.h>
+
+#define EDMA_V0_MAX_NR_CH 8
+#define EDMA_V0_VIEWPORT_MASK GENMASK(2, 0)
+#define EDMA_V0_DONE_INT_MASK GENMASK(7, 0)
+#define EDMA_V0_ABORT_INT_MASK GENMASK(23, 16)
+#define EDMA_V0_WRITE_CH_COUNT_MASK GENMASK(3, 0)
+#define EDMA_V0_READ_CH_COUNT_MASK GENMASK(19, 16)
+#define EDMA_V0_CH_STATUS_MASK GENMASK(6, 5)
+#define EDMA_V0_DOORBELL_CH_MASK GENMASK(2, 0)
+#define EDMA_V0_LINKED_LIST_ERR_MASK GENMASK(7, 0)
+
+#define EDMA_V0_CH_ODD_MSI_DATA_MASK GENMASK(31, 16)
+#define EDMA_V0_CH_EVEN_MSI_DATA_MASK GENMASK(15, 0)
+
+struct dw_edma_v0_ch_regs {
+ u32 ch_control1; /* 0x000 */
+ u32 ch_control2; /* 0x004 */
+ u32 transfer_size; /* 0x008 */
+ u32 sar_low; /* 0x00c */
+ u32 sar_high; /* 0x010 */
+ u32 dar_low; /* 0x014 */
+ u32 dar_high; /* 0x018 */
+ u32 llp_low; /* 0x01c */
+ u32 llp_high; /* 0x020 */
+};
+
+struct dw_edma_v0_ch {
+ struct dw_edma_v0_ch_regs wr; /* 0x200 */
+ u32 padding_1[55]; /* [0x224..0x2fc] */
+ struct dw_edma_v0_ch_regs rd; /* 0x300 */
+ u32 padding_2[55]; /* [0x224..0x2fc] */
+};
+
+struct dw_edma_v0_unroll {
+ u32 padding_1; /* 0x0f8 */
+ u32 wr_engine_chgroup; /* 0x100 */
+ u32 rd_engine_chgroup; /* 0x104 */
+ u32 wr_engine_hshake_cnt_low; /* 0x108 */
+ u32 wr_engine_hshake_cnt_high; /* 0x10c */
+ u32 padding_2[2]; /* [0x110..0x114] */
+ u32 rd_engine_hshake_cnt_low; /* 0x118 */
+ u32 rd_engine_hshake_cnt_high; /* 0x11c */
+ u32 padding_3[2]; /* [0x120..0x124] */
+ u32 wr_ch0_pwr_en; /* 0x128 */
+ u32 wr_ch1_pwr_en; /* 0x12c */
+ u32 wr_ch2_pwr_en; /* 0x130 */
+ u32 wr_ch3_pwr_en; /* 0x134 */
+ u32 wr_ch4_pwr_en; /* 0x138 */
+ u32 wr_ch5_pwr_en; /* 0x13c */
+ u32 wr_ch6_pwr_en; /* 0x140 */
+ u32 wr_ch7_pwr_en; /* 0x144 */
+ u32 padding_4[8]; /* [0x148..0x164] */
+ u32 rd_ch0_pwr_en; /* 0x168 */
+ u32 rd_ch1_pwr_en; /* 0x16c */
+ u32 rd_ch2_pwr_en; /* 0x170 */
+ u32 rd_ch3_pwr_en; /* 0x174 */
+ u32 rd_ch4_pwr_en; /* 0x178 */
+ u32 rd_ch5_pwr_en; /* 0x18c */
+ u32 rd_ch6_pwr_en; /* 0x180 */
+ u32 rd_ch7_pwr_en; /* 0x184 */
+ u32 padding_5[30]; /* [0x188..0x1fc] */
+ struct dw_edma_v0_ch ch[EDMA_V0_MAX_NR_CH]; /* [0x200..0x1120] */
+};
+
+struct dw_edma_v0_legacy {
+ u32 viewport_sel; /* 0x0f8 */
+ struct dw_edma_v0_ch_regs ch; /* [0x100..0x120] */
+};
+
+struct dw_edma_v0_regs {
+ /* eDMA global registers */
+ u32 ctrl_data_arb_prior; /* 0x000 */
+ u32 padding_1; /* 0x004 */
+ u32 ctrl; /* 0x008 */
+ u32 wr_engine_en; /* 0x00c */
+ u32 wr_doorbell; /* 0x010 */
+ u32 padding_2; /* 0x014 */
+ u32 wr_ch_arb_weight_low; /* 0x018 */
+ u32 wr_ch_arb_weight_high; /* 0x01c */
+ u32 padding_3[3]; /* [0x020..0x028] */
+ u32 rd_engine_en; /* 0x02c */
+ u32 rd_doorbell; /* 0x030 */
+ u32 padding_4; /* 0x034 */
+ u32 rd_ch_arb_weight_low; /* 0x038 */
+ u32 rd_ch_arb_weight_high; /* 0x03c */
+ u32 padding_5[3]; /* [0x040..0x048] */
+ /* eDMA interrupts registers */
+ u32 wr_int_status; /* 0x04c */
+ u32 padding_6; /* 0x050 */
+ u32 wr_int_mask; /* 0x054 */
+ u32 wr_int_clear; /* 0x058 */
+ u32 wr_err_status; /* 0x05c */
+ u32 wr_done_imwr_low; /* 0x060 */
+ u32 wr_done_imwr_high; /* 0x064 */
+ u32 wr_abort_imwr_low; /* 0x068 */
+ u32 wr_abort_imwr_high; /* 0x06c */
+ u32 wr_ch01_imwr_data; /* 0x070 */
+ u32 wr_ch23_imwr_data; /* 0x074 */
+ u32 wr_ch45_imwr_data; /* 0x078 */
+ u32 wr_ch67_imwr_data; /* 0x07c */
+ u32 padding_7[4]; /* [0x080..0x08c] */
+ u32 wr_linked_list_err_en; /* 0x090 */
+ u32 padding_8[3]; /* [0x094..0x09c] */
+ u32 rd_int_status; /* 0x0a0 */
+ u32 padding_9; /* 0x0a4 */
+ u32 rd_int_mask; /* 0x0a8 */
+ u32 rd_int_clear; /* 0x0ac */
+ u32 padding_10; /* 0x0b0 */
+ u32 rd_err_status_low; /* 0x0b4 */
+ u32 rd_err_status_high; /* 0x0b8 */
+ u32 padding_11[2]; /* [0x0bc..0x0c0] */
+ u32 rd_linked_list_err_en; /* 0x0c4 */
+ u32 padding_12; /* 0x0c8 */
+ u32 rd_done_imwr_low; /* 0x0cc */
+ u32 rd_done_imwr_high; /* 0x0d0 */
+ u32 rd_abort_imwr_low; /* 0x0d4 */
+ u32 rd_abort_imwr_high; /* 0x0d8 */
+ u32 rd_ch01_imwr_data; /* 0x0dc */
+ u32 rd_ch23_imwr_data; /* 0x0e0 */
+ u32 rd_ch45_imwr_data; /* 0x0e4 */
+ u32 rd_ch67_imwr_data; /* 0x0e8 */
+ u32 padding_13[4]; /* [0x0ec..0x0f8] */
+ /* eDMA channel context grouping */
+ union dw_edma_v0_type {
+ struct dw_edma_v0_legacy legacy; /* [0x0f8..0x120] */
+ struct dw_edma_v0_unroll unroll; /* [0x0f8..0x1120] */
+ } type;
+};
+
+struct dw_edma_v0_lli {
+ u32 control;
+ u32 transfer_size;
+ u32 sar_low;
+ u32 sar_high;
+ u32 dar_low;
+ u32 dar_high;
+};
+
+struct dw_edma_v0_llp {
+ u32 control;
+ u32 reserved;
+ u32 llp_low;
+ u32 llp_high;
+};
+
+#endif /* _DW_EDMA_V0_REGS_H */
struct dw_dma_pci_data {
const struct dw_dma_platform_data *pdata;
int (*probe)(struct dw_dma_chip *chip);
+ int (*remove)(struct dw_dma_chip *chip);
+ struct dw_dma_chip *chip;
};
static const struct dw_dma_pci_data dw_pci_data = {
.probe = dw_dma_probe,
+ .remove = dw_dma_remove,
};
static const struct dw_dma_platform_data idma32_pdata = {
static const struct dw_dma_pci_data idma32_pci_data = {
.pdata = &idma32_pdata,
.probe = idma32_dma_probe,
+ .remove = idma32_dma_remove,
};
static int dw_pci_probe(struct pci_dev *pdev, const struct pci_device_id *pid)
{
- const struct dw_dma_pci_data *data = (void *)pid->driver_data;
+ const struct dw_dma_pci_data *drv_data = (void *)pid->driver_data;
+ struct dw_dma_pci_data *data;
struct dw_dma_chip *chip;
int ret;
if (ret)
return ret;
+ data = devm_kmemdup(&pdev->dev, drv_data, sizeof(*drv_data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL);
if (!chip)
return -ENOMEM;
chip->irq = pdev->irq;
chip->pdata = data->pdata;
+ data->chip = chip;
+
ret = data->probe(chip);
if (ret)
return ret;
- pci_set_drvdata(pdev, chip);
+ pci_set_drvdata(pdev, data);
return 0;
}
static void dw_pci_remove(struct pci_dev *pdev)
{
- struct dw_dma_chip *chip = pci_get_drvdata(pdev);
+ struct dw_dma_pci_data *data = pci_get_drvdata(pdev);
+ struct dw_dma_chip *chip = data->chip;
int ret;
- ret = dw_dma_remove(chip);
+ ret = data->remove(chip);
if (ret)
dev_warn(&pdev->dev, "can't remove device properly: %d\n", ret);
}
static int dw_pci_suspend_late(struct device *dev)
{
- struct pci_dev *pci = to_pci_dev(dev);
- struct dw_dma_chip *chip = pci_get_drvdata(pci);
+ struct dw_dma_pci_data *data = dev_get_drvdata(dev);
+ struct dw_dma_chip *chip = data->chip;
return do_dw_dma_disable(chip);
};
static int dw_pci_resume_early(struct device *dev)
{
- struct pci_dev *pci = to_pci_dev(dev);
- struct dw_dma_chip *chip = pci_get_drvdata(pci);
+ struct dw_dma_pci_data *data = dev_get_drvdata(dev);
+ struct dw_dma_chip *chip = data->chip;
return do_dw_dma_enable(chip);
};
{ PCI_VDEVICE(INTEL, 0x2286), (kernel_ulong_t)&dw_pci_data },
{ PCI_VDEVICE(INTEL, 0x22c0), (kernel_ulong_t)&dw_pci_data },
+ /* Elkhart Lake iDMA 32-bit (OSE DMA) */
+ { PCI_VDEVICE(INTEL, 0x4bb4), (kernel_ulong_t)&idma32_pci_data },
+ { PCI_VDEVICE(INTEL, 0x4bb5), (kernel_ulong_t)&idma32_pci_data },
+ { PCI_VDEVICE(INTEL, 0x4bb6), (kernel_ulong_t)&idma32_pci_data },
+
/* Haswell */
{ PCI_VDEVICE(INTEL, 0x9c60), (kernel_ulong_t)&dw_pci_data },
struct edma_regs *regs = &fsl_chan->edma->regs;
u32 ch = fsl_chan->vchan.chan.chan_id;
- if (fsl_chan->edma->version == v1) {
+ if (fsl_chan->edma->drvdata->version == v1) {
edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), regs->seei);
edma_writeb(fsl_chan->edma, ch, regs->serq);
} else {
struct edma_regs *regs = &fsl_chan->edma->regs;
u32 ch = fsl_chan->vchan.chan.chan_id;
- if (fsl_chan->edma->version == v1) {
+ if (fsl_chan->edma->drvdata->version == v1) {
edma_writeb(fsl_chan->edma, ch, regs->cerq);
edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), regs->ceei);
} else {
}
EXPORT_SYMBOL_GPL(fsl_edma_disable_request);
+static void mux_configure8(struct fsl_edma_chan *fsl_chan, void __iomem *addr,
+ u32 off, u32 slot, bool enable)
+{
+ u8 val8;
+
+ if (enable)
+ val8 = EDMAMUX_CHCFG_ENBL | slot;
+ else
+ val8 = EDMAMUX_CHCFG_DIS;
+
+ iowrite8(val8, addr + off);
+}
+
void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan,
unsigned int slot, bool enable)
{
u32 ch = fsl_chan->vchan.chan.chan_id;
void __iomem *muxaddr;
unsigned int chans_per_mux, ch_off;
+ u32 dmamux_nr = fsl_chan->edma->drvdata->dmamuxs;
- chans_per_mux = fsl_chan->edma->n_chans / DMAMUX_NR;
+ chans_per_mux = fsl_chan->edma->n_chans / dmamux_nr;
ch_off = fsl_chan->vchan.chan.chan_id % chans_per_mux;
muxaddr = fsl_chan->edma->muxbase[ch / chans_per_mux];
slot = EDMAMUX_CHCFG_SOURCE(slot);
- if (enable)
- iowrite8(EDMAMUX_CHCFG_ENBL | slot, muxaddr + ch_off);
- else
- iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off);
+ mux_configure8(fsl_chan, muxaddr, ch_off, slot, enable);
}
EXPORT_SYMBOL_GPL(fsl_edma_chan_mux);
edma->regs.erql = edma->membase + EDMA_ERQ;
edma->regs.eeil = edma->membase + EDMA_EEI;
- edma->regs.serq = edma->membase + ((edma->version == v1) ?
- EDMA_SERQ : EDMA64_SERQ);
- edma->regs.cerq = edma->membase + ((edma->version == v1) ?
- EDMA_CERQ : EDMA64_CERQ);
- edma->regs.seei = edma->membase + ((edma->version == v1) ?
- EDMA_SEEI : EDMA64_SEEI);
- edma->regs.ceei = edma->membase + ((edma->version == v1) ?
- EDMA_CEEI : EDMA64_CEEI);
- edma->regs.cint = edma->membase + ((edma->version == v1) ?
- EDMA_CINT : EDMA64_CINT);
- edma->regs.cerr = edma->membase + ((edma->version == v1) ?
- EDMA_CERR : EDMA64_CERR);
- edma->regs.ssrt = edma->membase + ((edma->version == v1) ?
- EDMA_SSRT : EDMA64_SSRT);
- edma->regs.cdne = edma->membase + ((edma->version == v1) ?
- EDMA_CDNE : EDMA64_CDNE);
- edma->regs.intl = edma->membase + ((edma->version == v1) ?
- EDMA_INTR : EDMA64_INTL);
- edma->regs.errl = edma->membase + ((edma->version == v1) ?
- EDMA_ERR : EDMA64_ERRL);
-
- if (edma->version == v2) {
+ edma->regs.serq = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_SERQ : EDMA_SERQ);
+ edma->regs.cerq = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_CERQ : EDMA_CERQ);
+ edma->regs.seei = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_SEEI : EDMA_SEEI);
+ edma->regs.ceei = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_CEEI : EDMA_CEEI);
+ edma->regs.cint = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_CINT : EDMA_CINT);
+ edma->regs.cerr = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_CERR : EDMA_CERR);
+ edma->regs.ssrt = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_SSRT : EDMA_SSRT);
+ edma->regs.cdne = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_CDNE : EDMA_CDNE);
+ edma->regs.intl = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_INTL : EDMA_INTR);
+ edma->regs.errl = edma->membase + ((edma->drvdata->version == v2) ?
+ EDMA64_ERRL : EDMA_ERR);
+
+ if (edma->drvdata->version == v2) {
edma->regs.erqh = edma->membase + EDMA64_ERQH;
edma->regs.eeih = edma->membase + EDMA64_EEIH;
edma->regs.errh = edma->membase + EDMA64_ERRH;
#define _FSL_EDMA_COMMON_H_
#include <linux/dma-direction.h>
+#include <linux/platform_device.h>
#include "virt-dma.h"
#define EDMA_CR_EDBG BIT(1)
v2, /* 64ch Coldfire */
};
+struct fsl_edma_drvdata {
+ enum edma_version version;
+ u32 dmamuxs;
+ int (*setup_irq)(struct platform_device *pdev,
+ struct fsl_edma_engine *fsl_edma);
+};
+
struct fsl_edma_engine {
struct dma_device dma_dev;
void __iomem *membase;
void __iomem *muxbase[DMAMUX_NR];
struct clk *muxclk[DMAMUX_NR];
struct mutex fsl_edma_mutex;
+ const struct fsl_edma_drvdata *drvdata;
u32 n_chans;
int txirq;
int errirq;
bool big_endian;
- enum edma_version version;
struct edma_regs regs;
struct fsl_edma_chan chans[];
};
struct fsl_edma_engine *fsl_edma = ofdma->of_dma_data;
struct dma_chan *chan, *_chan;
struct fsl_edma_chan *fsl_chan;
- unsigned long chans_per_mux = fsl_edma->n_chans / DMAMUX_NR;
+ u32 dmamux_nr = fsl_edma->drvdata->dmamuxs;
+ unsigned long chans_per_mux = fsl_edma->n_chans / dmamux_nr;
if (dma_spec->args_count != 2)
return NULL;
clk_disable_unprepare(fsl_edma->muxclk[i]);
}
+static struct fsl_edma_drvdata vf610_data = {
+ .version = v1,
+ .dmamuxs = DMAMUX_NR,
+ .setup_irq = fsl_edma_irq_init,
+};
+
+static const struct of_device_id fsl_edma_dt_ids[] = {
+ { .compatible = "fsl,vf610-edma", .data = &vf610_data},
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, fsl_edma_dt_ids);
+
static int fsl_edma_probe(struct platform_device *pdev)
{
+ const struct of_device_id *of_id =
+ of_match_device(fsl_edma_dt_ids, &pdev->dev);
struct device_node *np = pdev->dev.of_node;
struct fsl_edma_engine *fsl_edma;
+ const struct fsl_edma_drvdata *drvdata = NULL;
struct fsl_edma_chan *fsl_chan;
struct edma_regs *regs;
struct resource *res;
int len, chans;
int ret, i;
+ if (of_id)
+ drvdata = of_id->data;
+ if (!drvdata) {
+ dev_err(&pdev->dev, "unable to find driver data\n");
+ return -EINVAL;
+ }
+
ret = of_property_read_u32(np, "dma-channels", &chans);
if (ret) {
dev_err(&pdev->dev, "Can't get dma-channels.\n");
if (!fsl_edma)
return -ENOMEM;
- fsl_edma->version = v1;
+ fsl_edma->drvdata = drvdata;
fsl_edma->n_chans = chans;
mutex_init(&fsl_edma->fsl_edma_mutex);
fsl_edma_setup_regs(fsl_edma);
regs = &fsl_edma->regs;
- for (i = 0; i < DMAMUX_NR; i++) {
+ for (i = 0; i < fsl_edma->drvdata->dmamuxs; i++) {
char clkname[32];
res = platform_get_resource(pdev, IORESOURCE_MEM, 1 + i);
}
edma_writel(fsl_edma, ~0, regs->intl);
- ret = fsl_edma_irq_init(pdev, fsl_edma);
+ ret = fsl_edma->drvdata->setup_irq(pdev, fsl_edma);
if (ret)
return ret;
if (ret) {
dev_err(&pdev->dev,
"Can't register Freescale eDMA engine. (%d)\n", ret);
- fsl_disable_clocks(fsl_edma, DMAMUX_NR);
+ fsl_disable_clocks(fsl_edma, fsl_edma->drvdata->dmamuxs);
return ret;
}
dev_err(&pdev->dev,
"Can't register Freescale eDMA of_dma. (%d)\n", ret);
dma_async_device_unregister(&fsl_edma->dma_dev);
- fsl_disable_clocks(fsl_edma, DMAMUX_NR);
+ fsl_disable_clocks(fsl_edma, fsl_edma->drvdata->dmamuxs);
return ret;
}
fsl_edma_cleanup_vchan(&fsl_edma->dma_dev);
of_dma_controller_free(np);
dma_async_device_unregister(&fsl_edma->dma_dev);
- fsl_disable_clocks(fsl_edma, DMAMUX_NR);
+ fsl_disable_clocks(fsl_edma, fsl_edma->drvdata->dmamuxs);
return 0;
}
.resume_early = fsl_edma_resume_early,
};
-static const struct of_device_id fsl_edma_dt_ids[] = {
- { .compatible = "fsl,vf610-edma", },
- { /* sentinel */ }
-};
-MODULE_DEVICE_TABLE(of, fsl_edma_dt_ids);
-
static struct platform_driver fsl_edma_driver = {
.driver = {
.name = "fsl-edma",
/* Field definition for Descriptor offset */
#define QDMA_CCDF_STATUS 20
#define QDMA_CCDF_OFFSET 20
+#define QDMA_SDDF_CMD(x) (((u64)(x)) << 32)
/* Field definition for safe loop count*/
#define FSL_QDMA_HALT_COUNT 1500
static void fsl_qdma_comp_fill_memcpy(struct fsl_qdma_comp *fsl_comp,
dma_addr_t dst, dma_addr_t src, u32 len)
{
+ u32 cmd;
struct fsl_qdma_format *sdf, *ddf;
struct fsl_qdma_format *ccdf, *csgf_desc, *csgf_src, *csgf_dest;
/* This entry is the last entry. */
qdma_csgf_set_f(csgf_dest, len);
/* Descriptor Buffer */
- sdf->data =
- cpu_to_le64(FSL_QDMA_CMD_RWTTYPE <<
- FSL_QDMA_CMD_RWTTYPE_OFFSET);
- ddf->data =
- cpu_to_le64(FSL_QDMA_CMD_RWTTYPE <<
- FSL_QDMA_CMD_RWTTYPE_OFFSET);
- ddf->data |=
- cpu_to_le64(FSL_QDMA_CMD_LWC << FSL_QDMA_CMD_LWC_OFFSET);
+ cmd = cpu_to_le32(FSL_QDMA_CMD_RWTTYPE <<
+ FSL_QDMA_CMD_RWTTYPE_OFFSET);
+ sdf->data = QDMA_SDDF_CMD(cmd);
+
+ cmd = cpu_to_le32(FSL_QDMA_CMD_RWTTYPE <<
+ FSL_QDMA_CMD_RWTTYPE_OFFSET);
+ cmd |= cpu_to_le32(FSL_QDMA_CMD_LWC << FSL_QDMA_CMD_LWC_OFFSET);
+ ddf->data = QDMA_SDDF_CMD(cmd);
}
/*
if (hsuc->direction == DMA_MEM_TO_DEV) {
bsr = config->dst_maxburst;
- mtsr = config->src_addr_width;
+ mtsr = config->dst_addr_width;
} else if (hsuc->direction == DMA_DEV_TO_MEM) {
bsr = config->src_maxburst;
- mtsr = config->dst_addr_width;
+ mtsr = config->src_addr_width;
}
hsu_chan_disable(hsuc);
static bool sdma_filter_fn(struct dma_chan *chan, void *fn_param)
{
struct sdma_channel *sdmac = to_sdma_chan(chan);
- struct sdma_engine *sdma = sdmac->sdma;
struct imx_dma_data *data = fn_param;
if (!imx_dma_is_general_purpose(chan))
return false;
- /* return false if it's not the right device */
- if (sdma->dev->of_node != data->of_node)
- return false;
-
sdmac->data = *data;
chan->private = &sdmac->data;
* be set to sdmac->event_id1.
*/
data.dma_request2 = 0;
- data.of_node = ofdma->of_node;
- return dma_request_channel(mask, sdma_filter_fn, &data);
+ return __dma_request_channel(&mask, sdma_filter_fn, &data,
+ ofdma->of_node);
}
static int sdma_probe(struct platform_device *pdev)
free_irq(irq, mcf_edma);
}
+static struct fsl_edma_drvdata mcf_data = {
+ .version = v2,
+ .setup_irq = mcf_edma_irq_init,
+};
+
static int mcf_edma_probe(struct platform_device *pdev)
{
struct mcf_edma_platform_data *pdata;
mcf_edma->n_chans = chans;
- /* Set up version for ColdFire edma */
- mcf_edma->version = v2;
+ /* Set up drvdata for ColdFire edma */
+ mcf_edma->drvdata = &mcf_data;
mcf_edma->big_endian = 1;
if (!mcf_edma->n_chans) {
iowrite32(~0, regs->inth);
iowrite32(~0, regs->intl);
- ret = mcf_edma_irq_init(pdev, mcf_edma);
+ ret = mcf_edma->drvdata->setup_irq(pdev, mcf_edma);
if (ret)
return ret;
This controller provides the channels which is dedicated to
memory-to-memory transfer to offload from CPU.
+
+config MTK_UART_APDMA
+ tristate "MediaTek SoCs APDMA support for UART"
+ depends on OF && SERIAL_8250_MT6577
+ select DMA_ENGINE
+ select DMA_VIRTUAL_CHANNELS
+ help
+ Support for the UART DMA engine found on MediaTek MTK SoCs.
+ When SERIAL_8250_MT6577 is enabled, and if you want to use DMA,
+ you can enable the config. The DMA engine can only be used
+ with MediaTek SoCs.
# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_MTK_UART_APDMA) += mtk-uart-apdma.o
obj-$(CONFIG_MTK_HSDMA) += mtk-hsdma.o
obj-$(CONFIG_MTK_CQDMA) += mtk-cqdma.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * MediaTek UART APDMA driver.
+ *
+ * Copyright (c) 2019 MediaTek Inc.
+ * Author: Long Cheng <long.cheng@mediatek.com>
+ */
+
+#include <linux/clk.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/iopoll.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of_dma.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "../virt-dma.h"
+
+/* The default number of virtual channel */
+#define MTK_UART_APDMA_NR_VCHANS 8
+
+#define VFF_EN_B BIT(0)
+#define VFF_STOP_B BIT(0)
+#define VFF_FLUSH_B BIT(0)
+#define VFF_4G_EN_B BIT(0)
+/* rx valid size >= vff thre */
+#define VFF_RX_INT_EN_B (BIT(0) | BIT(1))
+/* tx left size >= vff thre */
+#define VFF_TX_INT_EN_B BIT(0)
+#define VFF_WARM_RST_B BIT(0)
+#define VFF_RX_INT_CLR_B (BIT(0) | BIT(1))
+#define VFF_TX_INT_CLR_B 0
+#define VFF_STOP_CLR_B 0
+#define VFF_EN_CLR_B 0
+#define VFF_INT_EN_CLR_B 0
+#define VFF_4G_SUPPORT_CLR_B 0
+
+/*
+ * interrupt trigger level for tx
+ * if threshold is n, no polling is required to start tx.
+ * otherwise need polling VFF_FLUSH.
+ */
+#define VFF_TX_THRE(n) (n)
+/* interrupt trigger level for rx */
+#define VFF_RX_THRE(n) ((n) * 3 / 4)
+
+#define VFF_RING_SIZE 0xffff
+/* invert this bit when wrap ring head again */
+#define VFF_RING_WRAP 0x10000
+
+#define VFF_INT_FLAG 0x00
+#define VFF_INT_EN 0x04
+#define VFF_EN 0x08
+#define VFF_RST 0x0c
+#define VFF_STOP 0x10
+#define VFF_FLUSH 0x14
+#define VFF_ADDR 0x1c
+#define VFF_LEN 0x24
+#define VFF_THRE 0x28
+#define VFF_WPT 0x2c
+#define VFF_RPT 0x30
+/* TX: the buffer size HW can read. RX: the buffer size SW can read. */
+#define VFF_VALID_SIZE 0x3c
+/* TX: the buffer size SW can write. RX: the buffer size HW can write. */
+#define VFF_LEFT_SIZE 0x40
+#define VFF_DEBUG_STATUS 0x50
+#define VFF_4G_SUPPORT 0x54
+
+struct mtk_uart_apdmadev {
+ struct dma_device ddev;
+ struct clk *clk;
+ bool support_33bits;
+ unsigned int dma_requests;
+};
+
+struct mtk_uart_apdma_desc {
+ struct virt_dma_desc vd;
+
+ dma_addr_t addr;
+ unsigned int avail_len;
+};
+
+struct mtk_chan {
+ struct virt_dma_chan vc;
+ struct dma_slave_config cfg;
+ struct mtk_uart_apdma_desc *desc;
+ enum dma_transfer_direction dir;
+
+ void __iomem *base;
+ unsigned int irq;
+
+ unsigned int rx_status;
+};
+
+static inline struct mtk_uart_apdmadev *
+to_mtk_uart_apdma_dev(struct dma_device *d)
+{
+ return container_of(d, struct mtk_uart_apdmadev, ddev);
+}
+
+static inline struct mtk_chan *to_mtk_uart_apdma_chan(struct dma_chan *c)
+{
+ return container_of(c, struct mtk_chan, vc.chan);
+}
+
+static inline struct mtk_uart_apdma_desc *to_mtk_uart_apdma_desc
+ (struct dma_async_tx_descriptor *t)
+{
+ return container_of(t, struct mtk_uart_apdma_desc, vd.tx);
+}
+
+static void mtk_uart_apdma_write(struct mtk_chan *c,
+ unsigned int reg, unsigned int val)
+{
+ writel(val, c->base + reg);
+}
+
+static unsigned int mtk_uart_apdma_read(struct mtk_chan *c, unsigned int reg)
+{
+ return readl(c->base + reg);
+}
+
+static void mtk_uart_apdma_desc_free(struct virt_dma_desc *vd)
+{
+ struct dma_chan *chan = vd->tx.chan;
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+
+ kfree(c->desc);
+}
+
+static void mtk_uart_apdma_start_tx(struct mtk_chan *c)
+{
+ struct mtk_uart_apdmadev *mtkd =
+ to_mtk_uart_apdma_dev(c->vc.chan.device);
+ struct mtk_uart_apdma_desc *d = c->desc;
+ unsigned int wpt, vff_sz;
+
+ vff_sz = c->cfg.dst_port_window_size;
+ if (!mtk_uart_apdma_read(c, VFF_LEN)) {
+ mtk_uart_apdma_write(c, VFF_ADDR, d->addr);
+ mtk_uart_apdma_write(c, VFF_LEN, vff_sz);
+ mtk_uart_apdma_write(c, VFF_THRE, VFF_TX_THRE(vff_sz));
+ mtk_uart_apdma_write(c, VFF_WPT, 0);
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
+
+ if (mtkd->support_33bits)
+ mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_EN_B);
+ }
+
+ mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
+ if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
+ dev_err(c->vc.chan.device->dev, "Enable TX fail\n");
+
+ if (!mtk_uart_apdma_read(c, VFF_LEFT_SIZE)) {
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
+ return;
+ }
+
+ wpt = mtk_uart_apdma_read(c, VFF_WPT);
+
+ wpt += c->desc->avail_len;
+ if ((wpt & VFF_RING_SIZE) == vff_sz)
+ wpt = (wpt & VFF_RING_WRAP) ^ VFF_RING_WRAP;
+
+ /* Let DMA start moving data */
+ mtk_uart_apdma_write(c, VFF_WPT, wpt);
+
+ /* HW auto set to 0 when left size >= threshold */
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_TX_INT_EN_B);
+ if (!mtk_uart_apdma_read(c, VFF_FLUSH))
+ mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
+}
+
+static void mtk_uart_apdma_start_rx(struct mtk_chan *c)
+{
+ struct mtk_uart_apdmadev *mtkd =
+ to_mtk_uart_apdma_dev(c->vc.chan.device);
+ struct mtk_uart_apdma_desc *d = c->desc;
+ unsigned int vff_sz;
+
+ vff_sz = c->cfg.src_port_window_size;
+ if (!mtk_uart_apdma_read(c, VFF_LEN)) {
+ mtk_uart_apdma_write(c, VFF_ADDR, d->addr);
+ mtk_uart_apdma_write(c, VFF_LEN, vff_sz);
+ mtk_uart_apdma_write(c, VFF_THRE, VFF_RX_THRE(vff_sz));
+ mtk_uart_apdma_write(c, VFF_RPT, 0);
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
+
+ if (mtkd->support_33bits)
+ mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_EN_B);
+ }
+
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_RX_INT_EN_B);
+ mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B);
+ if (mtk_uart_apdma_read(c, VFF_EN) != VFF_EN_B)
+ dev_err(c->vc.chan.device->dev, "Enable RX fail\n");
+}
+
+static void mtk_uart_apdma_tx_handler(struct mtk_chan *c)
+{
+ struct mtk_uart_apdma_desc *d = c->desc;
+
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
+ mtk_uart_apdma_write(c, VFF_EN, VFF_EN_CLR_B);
+
+ list_del(&d->vd.node);
+ vchan_cookie_complete(&d->vd);
+}
+
+static void mtk_uart_apdma_rx_handler(struct mtk_chan *c)
+{
+ struct mtk_uart_apdma_desc *d = c->desc;
+ unsigned int len, wg, rg;
+ int cnt;
+
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
+
+ if (!mtk_uart_apdma_read(c, VFF_VALID_SIZE))
+ return;
+
+ mtk_uart_apdma_write(c, VFF_EN, VFF_EN_CLR_B);
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
+
+ len = c->cfg.src_port_window_size;
+ rg = mtk_uart_apdma_read(c, VFF_RPT);
+ wg = mtk_uart_apdma_read(c, VFF_WPT);
+ cnt = (wg & VFF_RING_SIZE) - (rg & VFF_RING_SIZE);
+
+ /*
+ * The buffer is ring buffer. If wrap bit different,
+ * represents the start of the next cycle for WPT
+ */
+ if ((rg ^ wg) & VFF_RING_WRAP)
+ cnt += len;
+
+ c->rx_status = d->avail_len - cnt;
+ mtk_uart_apdma_write(c, VFF_RPT, wg);
+
+ list_del(&d->vd.node);
+ vchan_cookie_complete(&d->vd);
+}
+
+static irqreturn_t mtk_uart_apdma_irq_handler(int irq, void *dev_id)
+{
+ struct dma_chan *chan = (struct dma_chan *)dev_id;
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&c->vc.lock, flags);
+ if (c->dir == DMA_DEV_TO_MEM)
+ mtk_uart_apdma_rx_handler(c);
+ else if (c->dir == DMA_MEM_TO_DEV)
+ mtk_uart_apdma_tx_handler(c);
+ spin_unlock_irqrestore(&c->vc.lock, flags);
+
+ return IRQ_HANDLED;
+}
+
+static int mtk_uart_apdma_alloc_chan_resources(struct dma_chan *chan)
+{
+ struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ unsigned int status;
+ int ret;
+
+ ret = pm_runtime_get_sync(mtkd->ddev.dev);
+ if (ret < 0) {
+ pm_runtime_put_noidle(chan->device->dev);
+ return ret;
+ }
+
+ mtk_uart_apdma_write(c, VFF_ADDR, 0);
+ mtk_uart_apdma_write(c, VFF_THRE, 0);
+ mtk_uart_apdma_write(c, VFF_LEN, 0);
+ mtk_uart_apdma_write(c, VFF_RST, VFF_WARM_RST_B);
+
+ ret = readx_poll_timeout(readl, c->base + VFF_EN,
+ status, !status, 10, 100);
+ if (ret)
+ return ret;
+
+ ret = request_irq(c->irq, mtk_uart_apdma_irq_handler,
+ IRQF_TRIGGER_NONE, KBUILD_MODNAME, chan);
+ if (ret < 0) {
+ dev_err(chan->device->dev, "Can't request dma IRQ\n");
+ return -EINVAL;
+ }
+
+ if (mtkd->support_33bits)
+ mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B);
+
+ return ret;
+}
+
+static void mtk_uart_apdma_free_chan_resources(struct dma_chan *chan)
+{
+ struct mtk_uart_apdmadev *mtkd = to_mtk_uart_apdma_dev(chan->device);
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+
+ free_irq(c->irq, chan);
+
+ tasklet_kill(&c->vc.task);
+
+ vchan_free_chan_resources(&c->vc);
+
+ pm_runtime_put_sync(mtkd->ddev.dev);
+}
+
+static enum dma_status mtk_uart_apdma_tx_status(struct dma_chan *chan,
+ dma_cookie_t cookie,
+ struct dma_tx_state *txstate)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ enum dma_status ret;
+
+ ret = dma_cookie_status(chan, cookie, txstate);
+ if (!txstate)
+ return ret;
+
+ dma_set_residue(txstate, c->rx_status);
+
+ return ret;
+}
+
+/*
+ * dmaengine_prep_slave_single will call the function. and sglen is 1.
+ * 8250 uart using one ring buffer, and deal with one sg.
+ */
+static struct dma_async_tx_descriptor *mtk_uart_apdma_prep_slave_sg
+ (struct dma_chan *chan, struct scatterlist *sgl,
+ unsigned int sglen, enum dma_transfer_direction dir,
+ unsigned long tx_flags, void *context)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ struct mtk_uart_apdma_desc *d;
+
+ if (!is_slave_direction(dir) || sglen != 1)
+ return NULL;
+
+ /* Now allocate and setup the descriptor */
+ d = kzalloc(sizeof(*d), GFP_ATOMIC);
+ if (!d)
+ return NULL;
+
+ d->avail_len = sg_dma_len(sgl);
+ d->addr = sg_dma_address(sgl);
+ c->dir = dir;
+
+ return vchan_tx_prep(&c->vc, &d->vd, tx_flags);
+}
+
+static void mtk_uart_apdma_issue_pending(struct dma_chan *chan)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ struct virt_dma_desc *vd;
+ unsigned long flags;
+
+ spin_lock_irqsave(&c->vc.lock, flags);
+ if (vchan_issue_pending(&c->vc)) {
+ vd = vchan_next_desc(&c->vc);
+ c->desc = to_mtk_uart_apdma_desc(&vd->tx);
+
+ if (c->dir == DMA_DEV_TO_MEM)
+ mtk_uart_apdma_start_rx(c);
+ else if (c->dir == DMA_MEM_TO_DEV)
+ mtk_uart_apdma_start_tx(c);
+ }
+
+ spin_unlock_irqrestore(&c->vc.lock, flags);
+}
+
+static int mtk_uart_apdma_slave_config(struct dma_chan *chan,
+ struct dma_slave_config *config)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+
+ memcpy(&c->cfg, config, sizeof(*config));
+
+ return 0;
+}
+
+static int mtk_uart_apdma_terminate_all(struct dma_chan *chan)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ unsigned long flags;
+ unsigned int status;
+ LIST_HEAD(head);
+ int ret;
+
+ mtk_uart_apdma_write(c, VFF_FLUSH, VFF_FLUSH_B);
+
+ ret = readx_poll_timeout(readl, c->base + VFF_FLUSH,
+ status, status != VFF_FLUSH_B, 10, 100);
+ if (ret)
+ dev_err(c->vc.chan.device->dev, "flush: fail, status=0x%x\n",
+ mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
+
+ /*
+ * Stop need 3 steps.
+ * 1. set stop to 1
+ * 2. wait en to 0
+ * 3. set stop as 0
+ */
+ mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_B);
+ ret = readx_poll_timeout(readl, c->base + VFF_EN,
+ status, !status, 10, 100);
+ if (ret)
+ dev_err(c->vc.chan.device->dev, "stop: fail, status=0x%x\n",
+ mtk_uart_apdma_read(c, VFF_DEBUG_STATUS));
+
+ mtk_uart_apdma_write(c, VFF_STOP, VFF_STOP_CLR_B);
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
+
+ if (c->dir == DMA_DEV_TO_MEM)
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B);
+ else if (c->dir == DMA_MEM_TO_DEV)
+ mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B);
+
+ synchronize_irq(c->irq);
+
+ spin_lock_irqsave(&c->vc.lock, flags);
+ vchan_get_all_descriptors(&c->vc, &head);
+ vchan_dma_desc_free_list(&c->vc, &head);
+ spin_unlock_irqrestore(&c->vc.lock, flags);
+
+ return 0;
+}
+
+static int mtk_uart_apdma_device_pause(struct dma_chan *chan)
+{
+ struct mtk_chan *c = to_mtk_uart_apdma_chan(chan);
+ unsigned long flags;
+
+ spin_lock_irqsave(&c->vc.lock, flags);
+
+ mtk_uart_apdma_write(c, VFF_EN, VFF_EN_CLR_B);
+ mtk_uart_apdma_write(c, VFF_INT_EN, VFF_INT_EN_CLR_B);
+
+ synchronize_irq(c->irq);
+
+ spin_unlock_irqrestore(&c->vc.lock, flags);
+
+ return 0;
+}
+
+static void mtk_uart_apdma_free(struct mtk_uart_apdmadev *mtkd)
+{
+ while (!list_empty(&mtkd->ddev.channels)) {
+ struct mtk_chan *c = list_first_entry(&mtkd->ddev.channels,
+ struct mtk_chan, vc.chan.device_node);
+
+ list_del(&c->vc.chan.device_node);
+ tasklet_kill(&c->vc.task);
+ }
+}
+
+static const struct of_device_id mtk_uart_apdma_match[] = {
+ { .compatible = "mediatek,mt6577-uart-dma", },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match);
+
+static int mtk_uart_apdma_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct mtk_uart_apdmadev *mtkd;
+ int bit_mask = 32, rc;
+ struct resource *res;
+ struct mtk_chan *c;
+ unsigned int i;
+
+ mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL);
+ if (!mtkd)
+ return -ENOMEM;
+
+ mtkd->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(mtkd->clk)) {
+ dev_err(&pdev->dev, "No clock specified\n");
+ rc = PTR_ERR(mtkd->clk);
+ return rc;
+ }
+
+ if (of_property_read_bool(np, "mediatek,dma-33bits"))
+ mtkd->support_33bits = true;
+
+ if (mtkd->support_33bits)
+ bit_mask = 33;
+
+ rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask));
+ if (rc)
+ return rc;
+
+ dma_cap_set(DMA_SLAVE, mtkd->ddev.cap_mask);
+ mtkd->ddev.device_alloc_chan_resources =
+ mtk_uart_apdma_alloc_chan_resources;
+ mtkd->ddev.device_free_chan_resources =
+ mtk_uart_apdma_free_chan_resources;
+ mtkd->ddev.device_tx_status = mtk_uart_apdma_tx_status;
+ mtkd->ddev.device_issue_pending = mtk_uart_apdma_issue_pending;
+ mtkd->ddev.device_prep_slave_sg = mtk_uart_apdma_prep_slave_sg;
+ mtkd->ddev.device_config = mtk_uart_apdma_slave_config;
+ mtkd->ddev.device_pause = mtk_uart_apdma_device_pause;
+ mtkd->ddev.device_terminate_all = mtk_uart_apdma_terminate_all;
+ mtkd->ddev.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
+ mtkd->ddev.dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE);
+ mtkd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ mtkd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
+ mtkd->ddev.dev = &pdev->dev;
+ INIT_LIST_HEAD(&mtkd->ddev.channels);
+
+ mtkd->dma_requests = MTK_UART_APDMA_NR_VCHANS;
+ if (of_property_read_u32(np, "dma-requests", &mtkd->dma_requests)) {
+ dev_info(&pdev->dev,
+ "Using %u as missing dma-requests property\n",
+ MTK_UART_APDMA_NR_VCHANS);
+ }
+
+ for (i = 0; i < mtkd->dma_requests; i++) {
+ c = devm_kzalloc(mtkd->ddev.dev, sizeof(*c), GFP_KERNEL);
+ if (!c) {
+ rc = -ENODEV;
+ goto err_no_dma;
+ }
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+ if (!res) {
+ rc = -ENODEV;
+ goto err_no_dma;
+ }
+
+ c->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(c->base)) {
+ rc = PTR_ERR(c->base);
+ goto err_no_dma;
+ }
+ c->vc.desc_free = mtk_uart_apdma_desc_free;
+ vchan_init(&c->vc, &mtkd->ddev);
+
+ rc = platform_get_irq(pdev, i);
+ if (rc < 0) {
+ dev_err(&pdev->dev, "failed to get IRQ[%d]\n", i);
+ goto err_no_dma;
+ }
+ c->irq = rc;
+ }
+
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_set_active(&pdev->dev);
+
+ rc = dma_async_device_register(&mtkd->ddev);
+ if (rc)
+ goto rpm_disable;
+
+ platform_set_drvdata(pdev, mtkd);
+
+ /* Device-tree DMA controller registration */
+ rc = of_dma_controller_register(np, of_dma_xlate_by_chan_id, mtkd);
+ if (rc)
+ goto dma_remove;
+
+ return rc;
+
+dma_remove:
+ dma_async_device_unregister(&mtkd->ddev);
+rpm_disable:
+ pm_runtime_disable(&pdev->dev);
+err_no_dma:
+ mtk_uart_apdma_free(mtkd);
+ return rc;
+}
+
+static int mtk_uart_apdma_remove(struct platform_device *pdev)
+{
+ struct mtk_uart_apdmadev *mtkd = platform_get_drvdata(pdev);
+
+ of_dma_controller_free(pdev->dev.of_node);
+
+ mtk_uart_apdma_free(mtkd);
+
+ dma_async_device_unregister(&mtkd->ddev);
+
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int mtk_uart_apdma_suspend(struct device *dev)
+{
+ struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
+
+ if (!pm_runtime_suspended(dev))
+ clk_disable_unprepare(mtkd->clk);
+
+ return 0;
+}
+
+static int mtk_uart_apdma_resume(struct device *dev)
+{
+ int ret;
+ struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
+
+ if (!pm_runtime_suspended(dev)) {
+ ret = clk_prepare_enable(mtkd->clk);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
+#ifdef CONFIG_PM
+static int mtk_uart_apdma_runtime_suspend(struct device *dev)
+{
+ struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
+
+ clk_disable_unprepare(mtkd->clk);
+
+ return 0;
+}
+
+static int mtk_uart_apdma_runtime_resume(struct device *dev)
+{
+ int ret;
+ struct mtk_uart_apdmadev *mtkd = dev_get_drvdata(dev);
+
+ ret = clk_prepare_enable(mtkd->clk);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+#endif /* CONFIG_PM */
+
+static const struct dev_pm_ops mtk_uart_apdma_pm_ops = {
+ SET_SYSTEM_SLEEP_PM_OPS(mtk_uart_apdma_suspend, mtk_uart_apdma_resume)
+ SET_RUNTIME_PM_OPS(mtk_uart_apdma_runtime_suspend,
+ mtk_uart_apdma_runtime_resume, NULL)
+};
+
+static struct platform_driver mtk_uart_apdma_driver = {
+ .probe = mtk_uart_apdma_probe,
+ .remove = mtk_uart_apdma_remove,
+ .driver = {
+ .name = KBUILD_MODNAME,
+ .pm = &mtk_uart_apdma_pm_ops,
+ .of_match_table = of_match_ptr(mtk_uart_apdma_match),
+ },
+};
+
+module_platform_driver(mtk_uart_apdma_driver);
+
+MODULE_DESCRIPTION("MediaTek UART APDMA Controller Driver");
+MODULE_AUTHOR("Long Cheng <long.cheng@mediatek.com>");
+MODULE_LICENSE("GPL v2");
if (mic_dma_dbg) {
mic_dma_dev->dbg_dir = debugfs_create_dir(dev_name(&mbdev->dev),
mic_dma_dbg);
- if (mic_dma_dev->dbg_dir)
- debugfs_create_file("mic_dma_reg", 0444,
- mic_dma_dev->dbg_dir, mic_dma_dev,
- &mic_dma_reg_fops);
+ debugfs_create_file("mic_dma_reg", 0444, mic_dma_dev->dbg_dir,
+ mic_dma_dev, &mic_dma_reg_fops);
}
return 0;
}
}
struct mmp_tdma_filter_param {
- struct device_node *of_node;
unsigned int chan_id;
};
static bool mmp_tdma_filter_fn(struct dma_chan *chan, void *fn_param)
{
struct mmp_tdma_filter_param *param = fn_param;
- struct mmp_tdma_chan *tdmac = to_mmp_tdma_chan(chan);
- struct dma_device *pdma_device = tdmac->chan.device;
-
- if (pdma_device->dev->of_node != param->of_node)
- return false;
if (chan->chan_id != param->chan_id)
return false;
if (dma_spec->args_count != 1)
return NULL;
- param.of_node = ofdma->of_node;
param.chan_id = dma_spec->args[0];
if (param.chan_id >= TDMA_CHANNEL_NUM)
return NULL;
- return dma_request_channel(mask, mmp_tdma_filter_fn, ¶m);
+ return __dma_request_channel(&mask, mmp_tdma_filter_fn, ¶m,
+ ofdma->of_node);
}
static const struct of_device_id mmp_tdma_dt_ids[] = {
}
struct mxs_dma_filter_param {
- struct device_node *of_node;
unsigned int chan_id;
};
struct mxs_dma_engine *mxs_dma = mxs_chan->mxs_dma;
int chan_irq;
- if (mxs_dma->dma_device.dev->of_node != param->of_node)
- return false;
-
if (chan->chan_id != param->chan_id)
return false;
if (dma_spec->args_count != 1)
return NULL;
- param.of_node = ofdma->of_node;
param.chan_id = dma_spec->args[0];
if (param.chan_id >= mxs_dma->nr_channels)
return NULL;
- return dma_request_channel(mask, mxs_dma_filter_fn, ¶m);
+ return __dma_request_channel(&mask, mxs_dma_filter_fn, ¶m,
+ ofdma->of_node);
}
static int __init mxs_dma_probe(struct platform_device *pdev)
if (count != 1)
return NULL;
- return dma_request_channel(info->dma_cap, info->filter_fn,
- &dma_spec->args[0]);
+ return __dma_request_channel(&info->dma_cap, info->filter_fn,
+ &dma_spec->args[0], dma_spec->np);
}
EXPORT_SYMBOL_GPL(of_dma_simple_xlate);
#include <linux/err.h>
#include <linux/pm_runtime.h>
#include <linux/bug.h>
+#include <linux/reset.h>
#include "dmaengine.h"
#define PL330_MAX_CHAN 8
unsigned int num_peripherals;
struct dma_pl330_chan *peripherals; /* keep at end */
int quirks;
+
+ struct reset_control *rstc;
+ struct reset_control *rstc_ocp;
};
static struct pl330_of_quirks {
amba_set_drvdata(adev, pl330);
+ pl330->rstc = devm_reset_control_get_optional(&adev->dev, "dma");
+ if (IS_ERR(pl330->rstc)) {
+ if (PTR_ERR(pl330->rstc) != -EPROBE_DEFER)
+ dev_err(&adev->dev, "Failed to get reset!\n");
+ return PTR_ERR(pl330->rstc);
+ } else {
+ ret = reset_control_deassert(pl330->rstc);
+ if (ret) {
+ dev_err(&adev->dev, "Couldn't deassert the device from reset!\n");
+ return ret;
+ }
+ }
+
+ pl330->rstc_ocp = devm_reset_control_get_optional(&adev->dev, "dma-ocp");
+ if (IS_ERR(pl330->rstc_ocp)) {
+ if (PTR_ERR(pl330->rstc_ocp) != -EPROBE_DEFER)
+ dev_err(&adev->dev, "Failed to get OCP reset!\n");
+ return PTR_ERR(pl330->rstc_ocp);
+ } else {
+ ret = reset_control_deassert(pl330->rstc_ocp);
+ if (ret) {
+ dev_err(&adev->dev, "Couldn't deassert the device from OCP reset!\n");
+ return ret;
+ }
+ }
+
for (i = 0; i < AMBA_NR_IRQS; i++) {
irq = adev->irq[i];
if (irq) {
probe_err2:
pl330_del(pl330);
+ if (pl330->rstc_ocp)
+ reset_control_assert(pl330->rstc_ocp);
+
+ if (pl330->rstc)
+ reset_control_assert(pl330->rstc);
return ret;
}
pl330_del(pl330);
+ if (pl330->rstc_ocp)
+ reset_control_assert(pl330->rstc_ocp);
+
+ if (pl330->rstc)
+ reset_control_assert(pl330->rstc);
return 0;
}
spinlock_t phy_lock; /* Phy association */
#ifdef CONFIG_DEBUG_FS
struct dentry *dbgfs_root;
- struct dentry *dbgfs_state;
struct dentry **dbgfs_chan;
#endif
};
int ch, struct dentry *chandir)
{
char chan_name[11];
- struct dentry *chan, *chan_state = NULL, *chan_descr = NULL;
- struct dentry *chan_reqs = NULL;
+ struct dentry *chan;
void *dt;
scnprintf(chan_name, sizeof(chan_name), "%d", ch);
chan = debugfs_create_dir(chan_name, chandir);
dt = (void *)&pdev->phys[ch];
- if (chan)
- chan_state = debugfs_create_file("state", 0400, chan, dt,
- &chan_state_fops);
- if (chan_state)
- chan_descr = debugfs_create_file("descriptors", 0400, chan, dt,
- &descriptors_fops);
- if (chan_descr)
- chan_reqs = debugfs_create_file("requesters", 0400, chan, dt,
- &requester_chan_fops);
- if (!chan_reqs)
- goto err_state;
+ debugfs_create_file("state", 0400, chan, dt, &chan_state_fops);
+ debugfs_create_file("descriptors", 0400, chan, dt, &descriptors_fops);
+ debugfs_create_file("requesters", 0400, chan, dt, &requester_chan_fops);
return chan;
-
-err_state:
- debugfs_remove_recursive(chan);
- return NULL;
}
static void pxad_init_debugfs(struct pxad_device *pdev)
int i;
struct dentry *chandir;
- pdev->dbgfs_root = debugfs_create_dir(dev_name(pdev->slave.dev), NULL);
- if (IS_ERR(pdev->dbgfs_root) || !pdev->dbgfs_root)
- goto err_root;
-
- pdev->dbgfs_state = debugfs_create_file("state", 0400, pdev->dbgfs_root,
- pdev, &state_fops);
- if (!pdev->dbgfs_state)
- goto err_state;
-
pdev->dbgfs_chan =
- kmalloc_array(pdev->nr_chans, sizeof(*pdev->dbgfs_state),
+ kmalloc_array(pdev->nr_chans, sizeof(struct dentry *),
GFP_KERNEL);
if (!pdev->dbgfs_chan)
- goto err_alloc;
+ return;
+
+ pdev->dbgfs_root = debugfs_create_dir(dev_name(pdev->slave.dev), NULL);
+
+ debugfs_create_file("state", 0400, pdev->dbgfs_root, pdev, &state_fops);
chandir = debugfs_create_dir("channels", pdev->dbgfs_root);
- if (!chandir)
- goto err_chandir;
- for (i = 0; i < pdev->nr_chans; i++) {
+ for (i = 0; i < pdev->nr_chans; i++)
pdev->dbgfs_chan[i] = pxad_dbg_alloc_chan(pdev, i, chandir);
- if (!pdev->dbgfs_chan[i])
- goto err_chans;
- }
-
- return;
-err_chans:
-err_chandir:
- kfree(pdev->dbgfs_chan);
-err_alloc:
-err_state:
- debugfs_remove_recursive(pdev->dbgfs_root);
-err_root:
- pr_err("pxad: debugfs is not available\n");
}
static void pxad_cleanup_debugfs(struct pxad_device *pdev)
* It is used by the DMA complete notification to
* locate the descriptor that initiated the transfer.
*/
- struct dentry *debugfs;
- struct dentry *stats;
struct hidma_dev *dmadev;
struct hidma_desc *running;
struct dma_device ddev;
struct dentry *debugfs;
- struct dentry *stats;
/* sysfs entry for the channel id */
struct device_attribute *chid_attrs;
irqreturn_t hidma_ll_inthandler_msi(int irq, void *arg, int cause);
void hidma_cleanup_pending_tre(struct hidma_lldev *llhndl, u8 err_info,
u8 err_code);
-int hidma_debug_init(struct hidma_dev *dmadev);
+void hidma_debug_init(struct hidma_dev *dmadev);
void hidma_debug_uninit(struct hidma_dev *dmadev);
#endif
debugfs_remove_recursive(dmadev->debugfs);
}
-int hidma_debug_init(struct hidma_dev *dmadev)
+void hidma_debug_init(struct hidma_dev *dmadev)
{
- int rc = 0;
int chidx = 0;
struct list_head *position = NULL;
+ struct dentry *dir;
dmadev->debugfs = debugfs_create_dir(dev_name(dmadev->ddev.dev), NULL);
- if (!dmadev->debugfs) {
- rc = -ENODEV;
- return rc;
- }
/* walk through the virtual channel list */
list_for_each(position, &dmadev->ddev.channels) {
chan = list_entry(position, struct hidma_chan,
chan.device_node);
sprintf(chan->dbg_name, "chan%d", chidx);
- chan->debugfs = debugfs_create_dir(chan->dbg_name,
+ dir = debugfs_create_dir(chan->dbg_name,
dmadev->debugfs);
- if (!chan->debugfs) {
- rc = -ENOMEM;
- goto cleanup;
- }
- chan->stats = debugfs_create_file("stats", S_IRUGO,
- chan->debugfs, chan,
- &hidma_chan_fops);
- if (!chan->stats) {
- rc = -ENOMEM;
- goto cleanup;
- }
+ debugfs_create_file("stats", S_IRUGO, dir, chan,
+ &hidma_chan_fops);
chidx++;
}
- dmadev->stats = debugfs_create_file("stats", S_IRUGO,
- dmadev->debugfs, dmadev,
- &hidma_dma_fops);
- if (!dmadev->stats) {
- rc = -ENOMEM;
- goto cleanup;
- }
-
- return 0;
-cleanup:
- hidma_debug_uninit(dmadev);
- return rc;
+ debugfs_create_file("stats", S_IRUGO, dmadev->debugfs, dmadev,
+ &hidma_dma_fops);
}
help
This driver supports the USB-DMA controller found in the Renesas
SoCs.
-
-config SUDMAC
- tristate "Renesas SUDMAC support"
- depends on SH_DMAE_BASE
- help
- Enable support for the Renesas SUDMAC controllers.
obj-$(CONFIG_RCAR_DMAC) += rcar-dmac.o
obj-$(CONFIG_RENESAS_USB_DMAC) += usb-dmac.o
-obj-$(CONFIG_SUDMAC) += sudmac.o
struct rcar_dmac_chan *rchan = to_rcar_dmac_chan(chan);
/* Someone calling slave DMA on a generic channel? */
- if (rchan->mid_rid < 0 || !sg_len) {
+ if (rchan->mid_rid < 0 || !sg_len || !sg_dma_len(sgl)) {
dev_warn(chan->device->dev,
"%s: bad parameter: len=%d, id=%d\n",
__func__, sg_len, rchan->mid_rid);
* Forcing it to call dma_request_channel() and iterate through all
* channels from all controllers is just pointless.
*/
- if (chan->device->device_config != rcar_dmac_device_config ||
- dma_spec->np != chan->device->dev->of_node)
+ if (chan->device->device_config != rcar_dmac_device_config)
return false;
return !test_and_set_bit(dma_spec->args[0], dmac->modules);
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
- chan = dma_request_channel(mask, rcar_dmac_chan_filter, dma_spec);
+ chan = __dma_request_channel(&mask, rcar_dmac_chan_filter, dma_spec,
+ ofdma->of_node);
if (!chan)
return NULL;
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Renesas SUDMAC support
- *
- * Copyright (C) 2013 Renesas Solutions Corp.
- *
- * based on drivers/dma/sh/shdma.c:
- * Copyright (C) 2011-2012 Guennadi Liakhovetski <g.liakhovetski@gmx.de>
- * Copyright (C) 2009 Nobuhiro Iwamatsu <iwamatsu.nobuhiro@renesas.com>
- * Copyright (C) 2009 Renesas Solutions, Inc. All rights reserved.
- * Copyright (C) 2007 Freescale Semiconductor, Inc. All rights reserved.
- */
-
-#include <linux/dmaengine.h>
-#include <linux/err.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/module.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-#include <linux/sudmac.h>
-
-struct sudmac_chan {
- struct shdma_chan shdma_chan;
- void __iomem *base;
- char dev_id[16]; /* unique name per DMAC of channel */
-
- u32 offset; /* for CFG, BA, BBC, CA, CBC, DEN */
- u32 cfg;
- u32 dint_end_bit;
-};
-
-struct sudmac_device {
- struct shdma_dev shdma_dev;
- struct sudmac_pdata *pdata;
- void __iomem *chan_reg;
-};
-
-struct sudmac_regs {
- u32 base_addr;
- u32 base_byte_count;
-};
-
-struct sudmac_desc {
- struct sudmac_regs hw;
- struct shdma_desc shdma_desc;
-};
-
-#define to_chan(schan) container_of(schan, struct sudmac_chan, shdma_chan)
-#define to_desc(sdesc) container_of(sdesc, struct sudmac_desc, shdma_desc)
-#define to_sdev(sc) container_of(sc->shdma_chan.dma_chan.device, \
- struct sudmac_device, shdma_dev.dma_dev)
-
-/* SUDMAC register */
-#define SUDMAC_CH0CFG 0x00
-#define SUDMAC_CH0BA 0x10
-#define SUDMAC_CH0BBC 0x18
-#define SUDMAC_CH0CA 0x20
-#define SUDMAC_CH0CBC 0x28
-#define SUDMAC_CH0DEN 0x30
-#define SUDMAC_DSTSCLR 0x38
-#define SUDMAC_DBUFCTRL 0x3C
-#define SUDMAC_DINTCTRL 0x40
-#define SUDMAC_DINTSTS 0x44
-#define SUDMAC_DINTSTSCLR 0x48
-#define SUDMAC_CH0SHCTRL 0x50
-
-/* Definitions for the sudmac_channel.config */
-#define SUDMAC_SENDBUFM 0x1000 /* b12: Transmit Buffer Mode */
-#define SUDMAC_RCVENDM 0x0100 /* b8: Receive Data Transfer End Mode */
-#define SUDMAC_LBA_WAIT 0x0030 /* b5-4: Local Bus Access Wait */
-
-/* Definitions for the sudmac_channel.dint_end_bit */
-#define SUDMAC_CH1ENDE 0x0002 /* b1: Ch1 DMA Transfer End Int Enable */
-#define SUDMAC_CH0ENDE 0x0001 /* b0: Ch0 DMA Transfer End Int Enable */
-
-#define SUDMAC_DRV_NAME "sudmac"
-
-static void sudmac_writel(struct sudmac_chan *sc, u32 data, u32 reg)
-{
- iowrite32(data, sc->base + reg);
-}
-
-static u32 sudmac_readl(struct sudmac_chan *sc, u32 reg)
-{
- return ioread32(sc->base + reg);
-}
-
-static bool sudmac_is_busy(struct sudmac_chan *sc)
-{
- u32 den = sudmac_readl(sc, SUDMAC_CH0DEN + sc->offset);
-
- if (den)
- return true; /* working */
-
- return false; /* waiting */
-}
-
-static void sudmac_set_reg(struct sudmac_chan *sc, struct sudmac_regs *hw,
- struct shdma_desc *sdesc)
-{
- sudmac_writel(sc, sc->cfg, SUDMAC_CH0CFG + sc->offset);
- sudmac_writel(sc, hw->base_addr, SUDMAC_CH0BA + sc->offset);
- sudmac_writel(sc, hw->base_byte_count, SUDMAC_CH0BBC + sc->offset);
-}
-
-static void sudmac_start(struct sudmac_chan *sc)
-{
- u32 dintctrl = sudmac_readl(sc, SUDMAC_DINTCTRL);
-
- sudmac_writel(sc, dintctrl | sc->dint_end_bit, SUDMAC_DINTCTRL);
- sudmac_writel(sc, 1, SUDMAC_CH0DEN + sc->offset);
-}
-
-static void sudmac_start_xfer(struct shdma_chan *schan,
- struct shdma_desc *sdesc)
-{
- struct sudmac_chan *sc = to_chan(schan);
- struct sudmac_desc *sd = to_desc(sdesc);
-
- sudmac_set_reg(sc, &sd->hw, sdesc);
- sudmac_start(sc);
-}
-
-static bool sudmac_channel_busy(struct shdma_chan *schan)
-{
- struct sudmac_chan *sc = to_chan(schan);
-
- return sudmac_is_busy(sc);
-}
-
-static void sudmac_setup_xfer(struct shdma_chan *schan, int slave_id)
-{
-}
-
-static const struct sudmac_slave_config *sudmac_find_slave(
- struct sudmac_chan *sc, int slave_id)
-{
- struct sudmac_device *sdev = to_sdev(sc);
- struct sudmac_pdata *pdata = sdev->pdata;
- const struct sudmac_slave_config *cfg;
- int i;
-
- for (i = 0, cfg = pdata->slave; i < pdata->slave_num; i++, cfg++)
- if (cfg->slave_id == slave_id)
- return cfg;
-
- return NULL;
-}
-
-static int sudmac_set_slave(struct shdma_chan *schan, int slave_id,
- dma_addr_t slave_addr, bool try)
-{
- struct sudmac_chan *sc = to_chan(schan);
- const struct sudmac_slave_config *cfg = sudmac_find_slave(sc, slave_id);
-
- if (!cfg)
- return -ENODEV;
-
- return 0;
-}
-
-static inline void sudmac_dma_halt(struct sudmac_chan *sc)
-{
- u32 dintctrl = sudmac_readl(sc, SUDMAC_DINTCTRL);
-
- sudmac_writel(sc, 0, SUDMAC_CH0DEN + sc->offset);
- sudmac_writel(sc, dintctrl & ~sc->dint_end_bit, SUDMAC_DINTCTRL);
- sudmac_writel(sc, sc->dint_end_bit, SUDMAC_DINTSTSCLR);
-}
-
-static int sudmac_desc_setup(struct shdma_chan *schan,
- struct shdma_desc *sdesc,
- dma_addr_t src, dma_addr_t dst, size_t *len)
-{
- struct sudmac_chan *sc = to_chan(schan);
- struct sudmac_desc *sd = to_desc(sdesc);
-
- dev_dbg(sc->shdma_chan.dev, "%s: src=%pad, dst=%pad, len=%zu\n",
- __func__, &src, &dst, *len);
-
- if (*len > schan->max_xfer_len)
- *len = schan->max_xfer_len;
-
- if (dst)
- sd->hw.base_addr = dst;
- else if (src)
- sd->hw.base_addr = src;
- sd->hw.base_byte_count = *len;
-
- return 0;
-}
-
-static void sudmac_halt(struct shdma_chan *schan)
-{
- struct sudmac_chan *sc = to_chan(schan);
-
- sudmac_dma_halt(sc);
-}
-
-static bool sudmac_chan_irq(struct shdma_chan *schan, int irq)
-{
- struct sudmac_chan *sc = to_chan(schan);
- u32 dintsts = sudmac_readl(sc, SUDMAC_DINTSTS);
-
- if (!(dintsts & sc->dint_end_bit))
- return false;
-
- /* DMA stop */
- sudmac_dma_halt(sc);
-
- return true;
-}
-
-static size_t sudmac_get_partial(struct shdma_chan *schan,
- struct shdma_desc *sdesc)
-{
- struct sudmac_chan *sc = to_chan(schan);
- struct sudmac_desc *sd = to_desc(sdesc);
- u32 current_byte_count = sudmac_readl(sc, SUDMAC_CH0CBC + sc->offset);
-
- return sd->hw.base_byte_count - current_byte_count;
-}
-
-static bool sudmac_desc_completed(struct shdma_chan *schan,
- struct shdma_desc *sdesc)
-{
- struct sudmac_chan *sc = to_chan(schan);
- struct sudmac_desc *sd = to_desc(sdesc);
- u32 current_addr = sudmac_readl(sc, SUDMAC_CH0CA + sc->offset);
-
- return sd->hw.base_addr + sd->hw.base_byte_count == current_addr;
-}
-
-static int sudmac_chan_probe(struct sudmac_device *su_dev, int id, int irq,
- unsigned long flags)
-{
- struct shdma_dev *sdev = &su_dev->shdma_dev;
- struct platform_device *pdev = to_platform_device(sdev->dma_dev.dev);
- struct sudmac_chan *sc;
- struct shdma_chan *schan;
- int err;
-
- sc = devm_kzalloc(&pdev->dev, sizeof(struct sudmac_chan), GFP_KERNEL);
- if (!sc)
- return -ENOMEM;
-
- schan = &sc->shdma_chan;
- schan->max_xfer_len = 64 * 1024 * 1024 - 1;
-
- shdma_chan_probe(sdev, schan, id);
-
- sc->base = su_dev->chan_reg;
-
- /* get platform_data */
- sc->offset = su_dev->pdata->channel->offset;
- if (su_dev->pdata->channel->config & SUDMAC_TX_BUFFER_MODE)
- sc->cfg |= SUDMAC_SENDBUFM;
- if (su_dev->pdata->channel->config & SUDMAC_RX_END_MODE)
- sc->cfg |= SUDMAC_RCVENDM;
- sc->cfg |= (su_dev->pdata->channel->wait << 4) & SUDMAC_LBA_WAIT;
-
- if (su_dev->pdata->channel->dint_end_bit & SUDMAC_DMA_BIT_CH0)
- sc->dint_end_bit |= SUDMAC_CH0ENDE;
- if (su_dev->pdata->channel->dint_end_bit & SUDMAC_DMA_BIT_CH1)
- sc->dint_end_bit |= SUDMAC_CH1ENDE;
-
- /* set up channel irq */
- if (pdev->id >= 0)
- snprintf(sc->dev_id, sizeof(sc->dev_id), "sudmac%d.%d",
- pdev->id, id);
- else
- snprintf(sc->dev_id, sizeof(sc->dev_id), "sudmac%d", id);
-
- err = shdma_request_irq(schan, irq, flags, sc->dev_id);
- if (err) {
- dev_err(sdev->dma_dev.dev,
- "DMA channel %d request_irq failed %d\n", id, err);
- goto err_no_irq;
- }
-
- return 0;
-
-err_no_irq:
- /* remove from dmaengine device node */
- shdma_chan_remove(schan);
- return err;
-}
-
-static void sudmac_chan_remove(struct sudmac_device *su_dev)
-{
- struct shdma_chan *schan;
- int i;
-
- shdma_for_each_chan(schan, &su_dev->shdma_dev, i) {
- BUG_ON(!schan);
-
- shdma_chan_remove(schan);
- }
-}
-
-static dma_addr_t sudmac_slave_addr(struct shdma_chan *schan)
-{
- /* SUDMAC doesn't need the address */
- return 0;
-}
-
-static struct shdma_desc *sudmac_embedded_desc(void *buf, int i)
-{
- return &((struct sudmac_desc *)buf)[i].shdma_desc;
-}
-
-static const struct shdma_ops sudmac_shdma_ops = {
- .desc_completed = sudmac_desc_completed,
- .halt_channel = sudmac_halt,
- .channel_busy = sudmac_channel_busy,
- .slave_addr = sudmac_slave_addr,
- .desc_setup = sudmac_desc_setup,
- .set_slave = sudmac_set_slave,
- .setup_xfer = sudmac_setup_xfer,
- .start_xfer = sudmac_start_xfer,
- .embedded_desc = sudmac_embedded_desc,
- .chan_irq = sudmac_chan_irq,
- .get_partial = sudmac_get_partial,
-};
-
-static int sudmac_probe(struct platform_device *pdev)
-{
- struct sudmac_pdata *pdata = dev_get_platdata(&pdev->dev);
- int err, i;
- struct sudmac_device *su_dev;
- struct dma_device *dma_dev;
- struct resource *chan, *irq_res;
-
- /* get platform data */
- if (!pdata)
- return -ENODEV;
-
- irq_res = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
- if (!irq_res)
- return -ENODEV;
-
- err = -ENOMEM;
- su_dev = devm_kzalloc(&pdev->dev, sizeof(struct sudmac_device),
- GFP_KERNEL);
- if (!su_dev)
- return err;
-
- dma_dev = &su_dev->shdma_dev.dma_dev;
-
- chan = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- su_dev->chan_reg = devm_ioremap_resource(&pdev->dev, chan);
- if (IS_ERR(su_dev->chan_reg))
- return PTR_ERR(su_dev->chan_reg);
-
- dma_cap_set(DMA_SLAVE, dma_dev->cap_mask);
-
- su_dev->shdma_dev.ops = &sudmac_shdma_ops;
- su_dev->shdma_dev.desc_size = sizeof(struct sudmac_desc);
- err = shdma_init(&pdev->dev, &su_dev->shdma_dev, pdata->channel_num);
- if (err < 0)
- return err;
-
- /* platform data */
- su_dev->pdata = dev_get_platdata(&pdev->dev);
-
- platform_set_drvdata(pdev, su_dev);
-
- /* Create DMA Channel */
- for (i = 0; i < pdata->channel_num; i++) {
- err = sudmac_chan_probe(su_dev, i, irq_res->start, IRQF_SHARED);
- if (err)
- goto chan_probe_err;
- }
-
- err = dma_async_device_register(&su_dev->shdma_dev.dma_dev);
- if (err < 0)
- goto chan_probe_err;
-
- return err;
-
-chan_probe_err:
- sudmac_chan_remove(su_dev);
-
- shdma_cleanup(&su_dev->shdma_dev);
-
- return err;
-}
-
-static int sudmac_remove(struct platform_device *pdev)
-{
- struct sudmac_device *su_dev = platform_get_drvdata(pdev);
- struct dma_device *dma_dev = &su_dev->shdma_dev.dma_dev;
-
- dma_async_device_unregister(dma_dev);
- sudmac_chan_remove(su_dev);
- shdma_cleanup(&su_dev->shdma_dev);
-
- return 0;
-}
-
-static struct platform_driver sudmac_driver = {
- .driver = {
- .name = SUDMAC_DRV_NAME,
- },
- .probe = sudmac_probe,
- .remove = sudmac_remove,
-};
-module_platform_driver(sudmac_driver);
-
-MODULE_AUTHOR("Yoshihiro Shimoda");
-MODULE_DESCRIPTION("Renesas SUDMAC driver");
-MODULE_LICENSE("GPL v2");
-MODULE_ALIAS("platform:" SUDMAC_DRV_NAME);
u32 residue;
struct list_head node;
dma_cookie_t done_cookie;
- struct usb_dmac_sg sg[0];
+ struct usb_dmac_sg sg[];
};
#define to_usb_dmac_desc(vd) container_of(vd, struct usb_dmac_desc, vd)
struct usb_dmac_chan *uchan = to_usb_dmac_chan(chan);
struct of_phandle_args *dma_spec = arg;
- if (dma_spec->np != chan->device->dev->of_node)
- return false;
-
/* USB-DMAC should be used with fixed usb controller's FIFO */
if (uchan->index != dma_spec->args[0])
return false;
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
- chan = dma_request_channel(mask, usb_dmac_chan_filter, dma_spec);
+ chan = __dma_request_channel(&mask, usb_dmac_chan_filter, dma_spec,
+ ofdma->of_node);
if (!chan)
return NULL;
for (i = 0; i < STM32_DMA_MAX_CHANNELS; i++) {
chan = &dmadev->chan[i];
- chan->irq = platform_get_irq(pdev, i);
ret = platform_get_irq(pdev, i);
if (ret < 0) {
if (ret != -EPROBE_DEFER)
#ifdef CONFIG_PM
static int stm32_dmamux_runtime_suspend(struct device *dev)
{
- struct platform_device *pdev =
- container_of(dev, struct platform_device, dev);
+ struct platform_device *pdev = to_platform_device(dev);
struct stm32_dmamux_data *stm32_dmamux = platform_get_drvdata(pdev);
clk_disable_unprepare(stm32_dmamux->clk);
static int stm32_dmamux_runtime_resume(struct device *dev)
{
- struct platform_device *pdev =
- container_of(dev, struct platform_device, dev);
+ struct platform_device *pdev = to_platform_device(dev);
struct stm32_dmamux_data *stm32_dmamux = platform_get_drvdata(pdev);
int ret;
#define DMA_CHAN_LLI_ADDR 0x08
#define DMA_CHAN_CUR_CFG 0x0c
-#define DMA_CHAN_MAX_DRQ 0x1f
-#define DMA_CHAN_CFG_SRC_DRQ(x) ((x) & DMA_CHAN_MAX_DRQ)
-#define DMA_CHAN_CFG_SRC_IO_MODE BIT(5)
-#define DMA_CHAN_CFG_SRC_LINEAR_MODE (0 << 5)
+#define DMA_CHAN_MAX_DRQ_A31 0x1f
+#define DMA_CHAN_MAX_DRQ_H6 0x3f
+#define DMA_CHAN_CFG_SRC_DRQ_A31(x) ((x) & DMA_CHAN_MAX_DRQ_A31)
+#define DMA_CHAN_CFG_SRC_DRQ_H6(x) ((x) & DMA_CHAN_MAX_DRQ_H6)
+#define DMA_CHAN_CFG_SRC_MODE_A31(x) (((x) & 0x1) << 5)
+#define DMA_CHAN_CFG_SRC_MODE_H6(x) (((x) & 0x1) << 8)
#define DMA_CHAN_CFG_SRC_BURST_A31(x) (((x) & 0x3) << 7)
#define DMA_CHAN_CFG_SRC_BURST_H3(x) (((x) & 0x3) << 6)
#define DMA_CHAN_CFG_SRC_WIDTH(x) (((x) & 0x3) << 9)
-#define DMA_CHAN_CFG_DST_DRQ(x) (DMA_CHAN_CFG_SRC_DRQ(x) << 16)
-#define DMA_CHAN_CFG_DST_IO_MODE (DMA_CHAN_CFG_SRC_IO_MODE << 16)
-#define DMA_CHAN_CFG_DST_LINEAR_MODE (DMA_CHAN_CFG_SRC_LINEAR_MODE << 16)
+#define DMA_CHAN_CFG_DST_DRQ_A31(x) (DMA_CHAN_CFG_SRC_DRQ_A31(x) << 16)
+#define DMA_CHAN_CFG_DST_DRQ_H6(x) (DMA_CHAN_CFG_SRC_DRQ_H6(x) << 16)
+#define DMA_CHAN_CFG_DST_MODE_A31(x) (DMA_CHAN_CFG_SRC_MODE_A31(x) << 16)
+#define DMA_CHAN_CFG_DST_MODE_H6(x) (DMA_CHAN_CFG_SRC_MODE_H6(x) << 16)
#define DMA_CHAN_CFG_DST_BURST_A31(x) (DMA_CHAN_CFG_SRC_BURST_A31(x) << 16)
#define DMA_CHAN_CFG_DST_BURST_H3(x) (DMA_CHAN_CFG_SRC_BURST_H3(x) << 16)
#define DMA_CHAN_CFG_DST_WIDTH(x) (DMA_CHAN_CFG_SRC_WIDTH(x) << 16)
#define LLI_LAST_ITEM 0xfffff800
#define NORMAL_WAIT 8
#define DRQ_SDRAM 1
+#define LINEAR_MODE 0
+#define IO_MODE 1
/* forward declaration */
struct sun6i_dma_dev;
*/
void (*clock_autogate_enable)(struct sun6i_dma_dev *);
void (*set_burst_length)(u32 *p_cfg, s8 src_burst, s8 dst_burst);
+ void (*set_drq)(u32 *p_cfg, s8 src_drq, s8 dst_drq);
+ void (*set_mode)(u32 *p_cfg, s8 src_mode, s8 dst_mode);
u32 src_burst_lengths;
u32 dst_burst_lengths;
u32 src_addr_widths;
u32 dst_addr_widths;
+ bool has_mbus_clk;
};
/*
struct dma_device slave;
void __iomem *base;
struct clk *clk;
+ struct clk *clk_mbus;
int irq;
spinlock_t lock;
struct reset_control *rstc;
DMA_CHAN_CFG_DST_BURST_H3(dst_burst);
}
+static void sun6i_set_drq_a31(u32 *p_cfg, s8 src_drq, s8 dst_drq)
+{
+ *p_cfg |= DMA_CHAN_CFG_SRC_DRQ_A31(src_drq) |
+ DMA_CHAN_CFG_DST_DRQ_A31(dst_drq);
+}
+
+static void sun6i_set_drq_h6(u32 *p_cfg, s8 src_drq, s8 dst_drq)
+{
+ *p_cfg |= DMA_CHAN_CFG_SRC_DRQ_H6(src_drq) |
+ DMA_CHAN_CFG_DST_DRQ_H6(dst_drq);
+}
+
+static void sun6i_set_mode_a31(u32 *p_cfg, s8 src_mode, s8 dst_mode)
+{
+ *p_cfg |= DMA_CHAN_CFG_SRC_MODE_A31(src_mode) |
+ DMA_CHAN_CFG_DST_MODE_A31(dst_mode);
+}
+
+static void sun6i_set_mode_h6(u32 *p_cfg, s8 src_mode, s8 dst_mode)
+{
+ *p_cfg |= DMA_CHAN_CFG_SRC_MODE_H6(src_mode) |
+ DMA_CHAN_CFG_DST_MODE_H6(dst_mode);
+}
+
static size_t sun6i_get_chan_size(struct sun6i_pchan *pchan)
{
struct sun6i_desc *txd = pchan->desc;
burst = convert_burst(8);
width = convert_buswidth(DMA_SLAVE_BUSWIDTH_4_BYTES);
- v_lli->cfg = DMA_CHAN_CFG_SRC_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_DST_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_DST_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_WIDTH(width) |
+ v_lli->cfg = DMA_CHAN_CFG_SRC_WIDTH(width) |
DMA_CHAN_CFG_DST_WIDTH(width);
sdev->cfg->set_burst_length(&v_lli->cfg, burst, burst);
+ sdev->cfg->set_drq(&v_lli->cfg, DRQ_SDRAM, DRQ_SDRAM);
+ sdev->cfg->set_mode(&v_lli->cfg, LINEAR_MODE, LINEAR_MODE);
sun6i_dma_lli_add(NULL, v_lli, p_lli, txd);
if (dir == DMA_MEM_TO_DEV) {
v_lli->src = sg_dma_address(sg);
v_lli->dst = sconfig->dst_addr;
- v_lli->cfg = lli_cfg |
- DMA_CHAN_CFG_DST_IO_MODE |
- DMA_CHAN_CFG_SRC_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_DST_DRQ(vchan->port);
+ v_lli->cfg = lli_cfg;
+ sdev->cfg->set_drq(&v_lli->cfg, DRQ_SDRAM, vchan->port);
+ sdev->cfg->set_mode(&v_lli->cfg, LINEAR_MODE, IO_MODE);
dev_dbg(chan2dev(chan),
"%s; chan: %d, dest: %pad, src: %pad, len: %u. flags: 0x%08lx\n",
} else {
v_lli->src = sconfig->src_addr;
v_lli->dst = sg_dma_address(sg);
- v_lli->cfg = lli_cfg |
- DMA_CHAN_CFG_DST_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_IO_MODE |
- DMA_CHAN_CFG_DST_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_SRC_DRQ(vchan->port);
+ v_lli->cfg = lli_cfg;
+ sdev->cfg->set_drq(&v_lli->cfg, vchan->port, DRQ_SDRAM);
+ sdev->cfg->set_mode(&v_lli->cfg, IO_MODE, LINEAR_MODE);
dev_dbg(chan2dev(chan),
"%s; chan: %d, dest: %pad, src: %pad, len: %u. flags: 0x%08lx\n",
if (dir == DMA_MEM_TO_DEV) {
v_lli->src = buf_addr + period_len * i;
v_lli->dst = sconfig->dst_addr;
- v_lli->cfg = lli_cfg |
- DMA_CHAN_CFG_DST_IO_MODE |
- DMA_CHAN_CFG_SRC_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_DST_DRQ(vchan->port);
+ v_lli->cfg = lli_cfg;
+ sdev->cfg->set_drq(&v_lli->cfg, DRQ_SDRAM, vchan->port);
+ sdev->cfg->set_mode(&v_lli->cfg, LINEAR_MODE, IO_MODE);
} else {
v_lli->src = sconfig->src_addr;
v_lli->dst = buf_addr + period_len * i;
- v_lli->cfg = lli_cfg |
- DMA_CHAN_CFG_DST_LINEAR_MODE |
- DMA_CHAN_CFG_SRC_IO_MODE |
- DMA_CHAN_CFG_DST_DRQ(DRQ_SDRAM) |
- DMA_CHAN_CFG_SRC_DRQ(vchan->port);
+ v_lli->cfg = lli_cfg;
+ sdev->cfg->set_drq(&v_lli->cfg, vchan->port, DRQ_SDRAM);
+ sdev->cfg->set_mode(&v_lli->cfg, IO_MODE, LINEAR_MODE);
}
prev = sun6i_dma_lli_add(prev, v_lli, p_lli, txd);
.nr_max_requests = 30,
.nr_max_vchans = 53,
.set_burst_length = sun6i_set_burst_length_a31,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(8),
.dst_burst_lengths = BIT(1) | BIT(8),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
.nr_max_vchans = 37,
.clock_autogate_enable = sun6i_enable_clock_autogate_a23,
.set_burst_length = sun6i_set_burst_length_a31,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(8),
.dst_burst_lengths = BIT(1) | BIT(8),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
.nr_max_vchans = 39,
.clock_autogate_enable = sun6i_enable_clock_autogate_a23,
.set_burst_length = sun6i_set_burst_length_a31,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(8),
.dst_burst_lengths = BIT(1) | BIT(8),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
.nr_max_vchans = 34,
.clock_autogate_enable = sun6i_enable_clock_autogate_h3,
.set_burst_length = sun6i_set_burst_length_h3,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
.dst_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
static struct sun6i_dma_config sun50i_a64_dma_cfg = {
.clock_autogate_enable = sun6i_enable_clock_autogate_h3,
.set_burst_length = sun6i_set_burst_length_h3,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
.dst_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
};
/*
+ * The H6 binding uses the number of dma channels from the
+ * device tree node.
+ */
+static struct sun6i_dma_config sun50i_h6_dma_cfg = {
+ .clock_autogate_enable = sun6i_enable_clock_autogate_h3,
+ .set_burst_length = sun6i_set_burst_length_h3,
+ .set_drq = sun6i_set_drq_h6,
+ .set_mode = sun6i_set_mode_h6,
+ .src_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
+ .dst_burst_lengths = BIT(1) | BIT(4) | BIT(8) | BIT(16),
+ .src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) |
+ BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) |
+ BIT(DMA_SLAVE_BUSWIDTH_8_BYTES),
+ .dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) |
+ BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) |
+ BIT(DMA_SLAVE_BUSWIDTH_8_BYTES),
+ .has_mbus_clk = true,
+};
+
+/*
* The V3s have only 8 physical channels, a maximum DRQ port id of 23,
* and a total of 24 usable source and destination endpoints.
*/
.nr_max_vchans = 24,
.clock_autogate_enable = sun6i_enable_clock_autogate_a23,
.set_burst_length = sun6i_set_burst_length_a31,
+ .set_drq = sun6i_set_drq_a31,
+ .set_mode = sun6i_set_mode_a31,
.src_burst_lengths = BIT(1) | BIT(8),
.dst_burst_lengths = BIT(1) | BIT(8),
.src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) |
{ .compatible = "allwinner,sun8i-h3-dma", .data = &sun8i_h3_dma_cfg },
{ .compatible = "allwinner,sun8i-v3s-dma", .data = &sun8i_v3s_dma_cfg },
{ .compatible = "allwinner,sun50i-a64-dma", .data = &sun50i_a64_dma_cfg },
+ { .compatible = "allwinner,sun50i-h6-dma", .data = &sun50i_h6_dma_cfg },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, sun6i_dma_match);
return PTR_ERR(sdc->clk);
}
+ if (sdc->cfg->has_mbus_clk) {
+ sdc->clk_mbus = devm_clk_get(&pdev->dev, "mbus");
+ if (IS_ERR(sdc->clk_mbus)) {
+ dev_err(&pdev->dev, "No mbus clock specified\n");
+ return PTR_ERR(sdc->clk_mbus);
+ }
+ }
+
sdc->rstc = devm_reset_control_get(&pdev->dev, NULL);
if (IS_ERR(sdc->rstc)) {
dev_err(&pdev->dev, "No reset controller specified\n");
ret = of_property_read_u32(np, "dma-requests", &sdc->max_request);
if (ret && !sdc->max_request) {
dev_info(&pdev->dev, "Missing dma-requests, using %u.\n",
- DMA_CHAN_MAX_DRQ);
- sdc->max_request = DMA_CHAN_MAX_DRQ;
+ DMA_CHAN_MAX_DRQ_A31);
+ sdc->max_request = DMA_CHAN_MAX_DRQ_A31;
}
/*
goto err_reset_assert;
}
+ if (sdc->cfg->has_mbus_clk) {
+ ret = clk_prepare_enable(sdc->clk_mbus);
+ if (ret) {
+ dev_err(&pdev->dev, "Couldn't enable mbus clock\n");
+ goto err_clk_disable;
+ }
+ }
+
ret = devm_request_irq(&pdev->dev, sdc->irq, sun6i_dma_interrupt, 0,
dev_name(&pdev->dev), sdc);
if (ret) {
dev_err(&pdev->dev, "Cannot request IRQ\n");
- goto err_clk_disable;
+ goto err_mbus_clk_disable;
}
ret = dma_async_device_register(&sdc->slave);
dma_async_device_unregister(&sdc->slave);
err_irq_disable:
sun6i_kill_tasklet(sdc);
+err_mbus_clk_disable:
+ clk_disable_unprepare(sdc->clk_mbus);
err_clk_disable:
clk_disable_unprepare(sdc->clk);
err_reset_assert:
sun6i_kill_tasklet(sdc);
+ clk_disable_unprepare(sdc->clk_mbus);
clk_disable_unprepare(sdc->clk);
reset_control_assert(sdc->rstc);
csr |= tdc->slave_id << TEGRA_APBDMA_CSR_REQ_SEL_SHIFT;
}
- if (flags & DMA_PREP_INTERRUPT)
+ if (flags & DMA_PREP_INTERRUPT) {
csr |= TEGRA_APBDMA_CSR_IE_EOC;
+ } else {
+ WARN_ON_ONCE(1);
+ return NULL;
+ }
apb_seq |= TEGRA_APBDMA_APBSEQ_WRAP_WORD_1;
csr |= tdc->slave_id << TEGRA_APBDMA_CSR_REQ_SEL_SHIFT;
}
- if (flags & DMA_PREP_INTERRUPT)
+ if (flags & DMA_PREP_INTERRUPT) {
csr |= TEGRA_APBDMA_CSR_IE_EOC;
+ } else {
+ WARN_ON_ONCE(1);
+ return NULL;
+ }
apb_seq |= TEGRA_APBDMA_APBSEQ_WRAP_WORD_1;
}
spin_unlock_irq(&vc->lock);
- dmaengine_desc_callback_invoke(&cb, NULL);
+ dmaengine_desc_callback_invoke(&cb, &vd->tx_result);
list_for_each_entry_safe(vd, _vd, &head, node) {
dmaengine_desc_get_callback(&vd->tx, &cb);
list_del(&vd->node);
vchan_vdesc_fini(vd);
- dmaengine_desc_callback_invoke(&cb, NULL);
+ dmaengine_desc_callback_invoke(&cb, &vd->tx_result);
}
}
struct virt_dma_desc {
struct dma_async_tx_descriptor tx;
+ struct dmaengine_result tx_result;
/* protected by vc.lock */
struct list_head node;
};
vd->tx.tx_submit = vchan_tx_submit;
vd->tx.desc_free = vchan_tx_desc_free;
+ vd->tx_result.result = DMA_TRANS_NOERROR;
+ vd->tx_result.residue = 0;
+
spin_lock_irqsave(&vc->lock, flags);
list_add_tail(&vd->node, &vc->desc_allocated);
spin_unlock_irqrestore(&vc->lock, flags);
static void xilinx_vdma_start_transfer(struct xilinx_dma_chan *chan)
{
struct xilinx_vdma_config *config = &chan->config;
- struct xilinx_dma_tx_descriptor *desc, *tail_desc;
+ struct xilinx_dma_tx_descriptor *desc;
u32 reg, j;
struct xilinx_vdma_tx_segment *segment, *last = NULL;
int i = 0;
desc = list_first_entry(&chan->pending_list,
struct xilinx_dma_tx_descriptor, node);
- tail_desc = list_last_entry(&chan->pending_list,
- struct xilinx_dma_tx_descriptor, node);
/* Configure the hardware using info in the config structure */
if (chan->has_vflip) {
#define MBOX_CHAN_PROPERTY 8
static struct platform_device *rpi_hwmon;
+static struct platform_device *rpi_clk;
struct rpi_firmware {
struct mbox_client cl;
-1, NULL, 0);
}
+static void rpi_register_clk_driver(struct device *dev)
+{
+ rpi_clk = platform_device_register_data(dev, "raspberrypi-clk",
+ -1, NULL, 0);
+}
+
static int rpi_firmware_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
rpi_firmware_print_firmware_revision(fw);
rpi_register_hwmon_driver(dev, fw);
+ rpi_register_clk_driver(dev);
return 0;
}
platform_device_unregister(rpi_hwmon);
rpi_hwmon = NULL;
+ platform_device_unregister(rpi_clk);
+ rpi_clk = NULL;
mbox_free_channel(fw->chan);
return 0;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_set_clock_state(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id,
+ u32 dev_id, u32 clk_id,
u32 flags, u8 state)
{
struct ti_sci_info *info;
}
req = (struct ti_sci_msg_req_set_clock_state *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
req->request_state = state;
ret = ti_sci_do_xfer(info, xfer);
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_get_clock_state(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id,
+ u32 dev_id, u32 clk_id,
u8 *programmed_state, u8 *current_state)
{
struct ti_sci_info *info;
}
req = (struct ti_sci_msg_req_get_clock_state *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
ret = ti_sci_do_xfer(info, xfer);
if (ret) {
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_get_clock(const struct ti_sci_handle *handle, u32 dev_id,
- u8 clk_id, bool needs_ssc, bool can_change_freq,
- bool enable_input_term)
+ u32 clk_id, bool needs_ssc,
+ bool can_change_freq, bool enable_input_term)
{
u32 flags = 0;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_idle_clock(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id)
+ u32 dev_id, u32 clk_id)
{
return ti_sci_set_clock_state(handle, dev_id, clk_id, 0,
MSG_CLOCK_SW_STATE_UNREQ);
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_put_clock(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id)
+ u32 dev_id, u32 clk_id)
{
return ti_sci_set_clock_state(handle, dev_id, clk_id, 0,
MSG_CLOCK_SW_STATE_AUTO);
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_is_auto(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, bool *req_state)
+ u32 dev_id, u32 clk_id, bool *req_state)
{
u8 state = 0;
int ret;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_is_on(const struct ti_sci_handle *handle, u32 dev_id,
- u8 clk_id, bool *req_state, bool *curr_state)
+ u32 clk_id, bool *req_state, bool *curr_state)
{
u8 c_state = 0, r_state = 0;
int ret;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_is_off(const struct ti_sci_handle *handle, u32 dev_id,
- u8 clk_id, bool *req_state, bool *curr_state)
+ u32 clk_id, bool *req_state, bool *curr_state)
{
u8 c_state = 0, r_state = 0;
int ret;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_set_parent(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, u8 parent_id)
+ u32 dev_id, u32 clk_id, u32 parent_id)
{
struct ti_sci_info *info;
struct ti_sci_msg_req_set_clock_parent *req;
}
req = (struct ti_sci_msg_req_set_clock_parent *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
- req->parent_id = parent_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
+ if (parent_id < 255) {
+ req->parent_id = parent_id;
+ } else {
+ req->parent_id = 255;
+ req->parent_id_32 = parent_id;
+ }
ret = ti_sci_do_xfer(info, xfer);
if (ret) {
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_get_parent(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, u8 *parent_id)
+ u32 dev_id, u32 clk_id, u32 *parent_id)
{
struct ti_sci_info *info;
struct ti_sci_msg_req_get_clock_parent *req;
}
req = (struct ti_sci_msg_req_get_clock_parent *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
ret = ti_sci_do_xfer(info, xfer);
if (ret) {
resp = (struct ti_sci_msg_resp_get_clock_parent *)xfer->xfer_buf;
- if (!ti_sci_is_response_ack(resp))
+ if (!ti_sci_is_response_ack(resp)) {
ret = -ENODEV;
- else
- *parent_id = resp->parent_id;
+ } else {
+ if (resp->parent_id < 255)
+ *parent_id = resp->parent_id;
+ else
+ *parent_id = resp->parent_id_32;
+ }
fail:
ti_sci_put_one_xfer(&info->minfo, xfer);
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_get_num_parents(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id,
- u8 *num_parents)
+ u32 dev_id, u32 clk_id,
+ u32 *num_parents)
{
struct ti_sci_info *info;
struct ti_sci_msg_req_get_clock_num_parents *req;
}
req = (struct ti_sci_msg_req_get_clock_num_parents *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
ret = ti_sci_do_xfer(info, xfer);
if (ret) {
resp = (struct ti_sci_msg_resp_get_clock_num_parents *)xfer->xfer_buf;
- if (!ti_sci_is_response_ack(resp))
+ if (!ti_sci_is_response_ack(resp)) {
ret = -ENODEV;
- else
- *num_parents = resp->num_parents;
+ } else {
+ if (resp->num_parents < 255)
+ *num_parents = resp->num_parents;
+ else
+ *num_parents = resp->num_parents_32;
+ }
fail:
ti_sci_put_one_xfer(&info->minfo, xfer);
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_get_match_freq(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, u64 min_freq,
+ u32 dev_id, u32 clk_id, u64 min_freq,
u64 target_freq, u64 max_freq,
u64 *match_freq)
{
}
req = (struct ti_sci_msg_req_query_clock_freq *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
req->min_freq_hz = min_freq;
req->target_freq_hz = target_freq;
req->max_freq_hz = max_freq;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_set_freq(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, u64 min_freq,
+ u32 dev_id, u32 clk_id, u64 min_freq,
u64 target_freq, u64 max_freq)
{
struct ti_sci_info *info;
}
req = (struct ti_sci_msg_req_set_clock_freq *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
req->min_freq_hz = min_freq;
req->target_freq_hz = target_freq;
req->max_freq_hz = max_freq;
* Return: 0 if all went well, else returns appropriate error value.
*/
static int ti_sci_cmd_clk_get_freq(const struct ti_sci_handle *handle,
- u32 dev_id, u8 clk_id, u64 *freq)
+ u32 dev_id, u32 clk_id, u64 *freq)
{
struct ti_sci_info *info;
struct ti_sci_msg_req_get_clock_freq *req;
}
req = (struct ti_sci_msg_req_get_clock_freq *)xfer->xfer_buf;
req->dev_id = dev_id;
- req->clk_id = clk_id;
+ if (clk_id < 255) {
+ req->clk_id = clk_id;
+ } else {
+ req->clk_id = 255;
+ req->clk_id_32 = clk_id;
+ }
ret = ti_sci_do_xfer(info, xfer);
if (ret) {
if (!res)
return ERR_PTR(-ENOMEM);
- res->sets = of_property_count_elems_of_size(dev_of_node(dev), of_prop,
- sizeof(u32));
- if (res->sets < 0) {
+ ret = of_property_count_elems_of_size(dev_of_node(dev), of_prop,
+ sizeof(u32));
+ if (ret < 0) {
dev_err(dev, "%s resource type ids not available\n", of_prop);
- return ERR_PTR(res->sets);
+ return ERR_PTR(ret);
}
+ res->sets = ret;
res->desc = devm_kcalloc(dev, res->sets, sizeof(*res->desc),
GFP_KERNEL);
* @dev_id: Device identifier this request is for
* @clk_id: Clock identifier for the device for this request.
* Each device has it's own set of clock inputs. This indexes
- * which clock input to modify.
+ * which clock input to modify. Set to 255 if clock ID is
+ * greater than or equal to 255.
* @request_state: Request the state for the clock to be set to.
* MSG_CLOCK_SW_STATE_UNREQ: The IP does not require this clock,
* it can be disabled, regardless of the state of the device
* being required by the device.(default)
* MSG_CLOCK_SW_STATE_REQ: Configure the clock to be enabled,
* regardless of the state of the device.
+ * @clk_id_32: Clock identifier for the device for this request.
+ * Only to be used if the clock ID is greater than or equal to
+ * 255.
*
* Normally, all required clocks are managed by TISCI entity, this is used
* only for specific control *IF* required. Auto managed state is
#define MSG_CLOCK_SW_STATE_AUTO 1
#define MSG_CLOCK_SW_STATE_REQ 2
u8 request_state;
+ u32 clk_id_32;
} __packed;
/**
* @dev_id: Device identifier this request is for
* @clk_id: Clock identifier for the device for this request.
* Each device has it's own set of clock inputs. This indexes
- * which clock input to get state of.
+ * which clock input to get state of. Set to 255 if the clock
+ * ID is greater than or equal to 255.
+ * @clk_id_32: Clock identifier for the device for the request.
+ * Only to be used if the clock ID is greater than or equal to
+ * 255.
*
* Request type is TI_SCI_MSG_GET_CLOCK_STATE, response is state
* of the clock
struct ti_sci_msg_hdr hdr;
u32 dev_id;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
* @dev_id: Device identifier this request is for
* @clk_id: Clock identifier for the device for this request.
* Each device has it's own set of clock inputs. This indexes
- * which clock input to modify.
+ * which clock input to modify. Set to 255 if clock ID is
+ * greater than or equal to 255.
* @parent_id: The new clock parent is selectable by an index via this
- * parameter.
+ * parameter. Set to 255 if clock ID is greater than or
+ * equal to 255.
+ * @clk_id_32: Clock identifier if @clk_id field is 255.
+ * @parent_id_32: Parent identifier if @parent_id is 255.
*
* Request type is TI_SCI_MSG_SET_CLOCK_PARENT, response is generic
* ACK / NACK message.
u32 dev_id;
u8 clk_id;
u8 parent_id;
+ u32 clk_id_32;
+ u32 parent_id_32;
} __packed;
/**
* @dev_id: Device identifier this request is for
* @clk_id: Clock identifier for the device for this request.
* Each device has it's own set of clock inputs. This indexes
- * which clock input to get the parent for.
+ * which clock input to get the parent for. If this field
+ * contains 255, the actual clock identifier is stored in
+ * @clk_id_32.
+ * @clk_id_32: Clock identifier if the @clk_id field contains 255.
*
* Request type is TI_SCI_MSG_GET_CLOCK_PARENT, response is parent information
*/
struct ti_sci_msg_hdr hdr;
u32 dev_id;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
* struct ti_sci_msg_resp_get_clock_parent - Response with clock parent
* @hdr: Generic Header
- * @parent_id: The current clock parent
+ * @parent_id: The current clock parent. If set to 255, the current parent
+ * ID can be found from the @parent_id_32 field.
+ * @parent_id_32: Current clock parent if @parent_id field is set to
+ * 255.
*
* Response to TI_SCI_MSG_GET_CLOCK_PARENT.
*/
struct ti_sci_msg_resp_get_clock_parent {
struct ti_sci_msg_hdr hdr;
u8 parent_id;
+ u32 parent_id_32;
} __packed;
/**
* struct ti_sci_msg_req_get_clock_num_parents - Request to get clock parents
* @hdr: Generic header
* @dev_id: Device identifier this request is for
- * @clk_id: Clock identifier for the device for this request.
+ * @clk_id: Clock identifier for the device for this request. Set to
+ * 255 if clock ID is greater than or equal to 255.
+ * @clk_id_32: Clock identifier if the @clk_id field contains 255.
*
* This request provides information about how many clock parent options
* are available for a given clock to a device. This is typically used
struct ti_sci_msg_hdr hdr;
u32 dev_id;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
* struct ti_sci_msg_resp_get_clock_num_parents - Response for get clk parents
* @hdr: Generic header
- * @num_parents: Number of clock parents
+ * @num_parents: Number of clock parents. If set to 255, the actual
+ * number of parents is stored into @num_parents_32
+ * field instead.
+ * @num_parents_32: Number of clock parents if @num_parents field is
+ * set to 255.
*
* Response to TI_SCI_MSG_GET_NUM_CLOCK_PARENTS
*/
struct ti_sci_msg_resp_get_clock_num_parents {
struct ti_sci_msg_hdr hdr;
u8 num_parents;
+ u32 num_parents_32;
} __packed;
/**
* @max_freq_hz: The maximum allowable frequency in Hz. This is the maximum
* allowable programmed frequency and does not account for clock
* tolerances and jitter.
- * @clk_id: Clock identifier for the device for this request.
+ * @clk_id: Clock identifier for the device for this request. Set to
+ * 255 if clock identifier is greater than or equal to 255.
+ * @clk_id_32: Clock identifier if @clk_id is set to 255.
*
* NOTE: Normally clock frequency management is automatically done by TISCI
* entity. In case of specific requests, TISCI evaluates capability to achieve
u64 target_freq_hz;
u64 max_freq_hz;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
* @max_freq_hz: The maximum allowable frequency in Hz. This is the maximum
* allowable programmed frequency and does not account for clock
* tolerances and jitter.
- * @clk_id: Clock identifier for the device for this request.
+ * @clk_id: Clock identifier for the device for this request. Set to
+ * 255 if clock ID is greater than or equal to 255.
+ * @clk_id_32: Clock identifier if @clk_id field is set to 255.
*
* NOTE: Normally clock frequency management is automatically done by TISCI
* entity. In case of specific requests, TISCI evaluates capability to achieve
u64 target_freq_hz;
u64 max_freq_hz;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
* struct ti_sci_msg_req_get_clock_freq - Request to get the clock frequency
* @hdr: Generic Header
* @dev_id: Device identifier this request is for
- * @clk_id: Clock identifier for the device for this request.
+ * @clk_id: Clock identifier for the device for this request. Set to
+ * 255 if clock ID is greater than or equal to 255.
+ * @clk_id_32: Clock identifier if @clk_id field is set to 255.
*
* NOTE: Normally clock frequency management is automatically done by TISCI
* entity. In some cases, clock frequencies are configured by host.
struct ti_sci_msg_hdr hdr;
u32 dev_id;
u8 clk_id;
+ u32 clk_id_32;
} __packed;
/**
#include <linux/dma-mapping.h>
#include <linux/sched/signal.h>
#include <linux/uaccess.h>
+#include <linux/mm.h>
#include "dfl-afu.h"
}
/**
- * afu_dma_adjust_locked_vm - adjust locked memory
- * @dev: port device
- * @npages: number of pages
- * @incr: increase or decrease locked memory
- *
- * Increase or decrease the locked memory size with npages input.
- *
- * Return 0 on success.
- * Return -ENOMEM if locked memory size is over the limit and no CAP_IPC_LOCK.
- */
-static int afu_dma_adjust_locked_vm(struct device *dev, long npages, bool incr)
-{
- unsigned long locked, lock_limit;
- int ret = 0;
-
- /* the task is exiting. */
- if (!current->mm)
- return 0;
-
- down_write(¤t->mm->mmap_sem);
-
- if (incr) {
- locked = current->mm->locked_vm + npages;
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
-
- if (locked > lock_limit && !capable(CAP_IPC_LOCK))
- ret = -ENOMEM;
- else
- current->mm->locked_vm += npages;
- } else {
- if (WARN_ON_ONCE(npages > current->mm->locked_vm))
- npages = current->mm->locked_vm;
- current->mm->locked_vm -= npages;
- }
-
- dev_dbg(dev, "[%d] RLIMIT_MEMLOCK %c%ld %ld/%ld%s\n", current->pid,
- incr ? '+' : '-', npages << PAGE_SHIFT,
- current->mm->locked_vm << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK),
- ret ? "- exceeded" : "");
-
- up_write(¤t->mm->mmap_sem);
-
- return ret;
-}
-
-/**
* afu_dma_pin_pages - pin pages of given dma memory region
* @pdata: feature device platform data
* @region: dma memory region to be pinned
struct device *dev = &pdata->dev->dev;
int ret, pinned;
- ret = afu_dma_adjust_locked_vm(dev, npages, true);
+ ret = account_locked_vm(current->mm, npages, true);
if (ret)
return ret;
free_pages:
kfree(region->pages);
unlock_vm:
- afu_dma_adjust_locked_vm(dev, npages, false);
+ account_locked_vm(current->mm, npages, false);
return ret;
}
put_all_pages(region->pages, npages);
kfree(region->pages);
- afu_dma_adjust_locked_vm(dev, npages, false);
+ account_locked_vm(current->mm, npages, false);
dev_dbg(dev, "%ld pages unpinned\n", npages);
}
The card needs to be physically altered for using it as a
GPIO card. For more information on how to build a GPIO card
from a BT8xx TV card, see the documentation file at
- Documentation/bt8xxgpio.txt
+ Documentation/driver-api/bt8xxgpio.rst
If unsure, say N.
/*
* FIXME: convert this singleton driver to use the state container
- * design pattern, see Documentation/driver-model/design-patterns.rst
+ * design pattern, see Documentation/driver-api/driver-model/design-patterns.rst
*/
static struct cs5535_gpio_chip {
struct gpio_chip chip;
for (i = 0; i < nirq; i++) {
chips->irqs[i] = platform_get_irq(pdev, i);
if (chips->irqs[i] < 0) {
- dev_info(dev, "IRQ not populated, err = %d\n",
- chips->irqs[i]);
+ if (chips->irqs[i] != -EPROBE_DEFER)
+ dev_info(dev, "IRQ not populated, err = %d\n",
+ chips->irqs[i]);
return chips->irqs[i];
}
}
.xlate = irq_domain_xlate_twocell,
};
+static void em_gio_irq_domain_remove(void *data)
+{
+ struct irq_domain *domain = data;
+
+ irq_domain_remove(domain);
+}
+
static int em_gio_probe(struct platform_device *pdev)
{
struct em_gio_priv *p;
return -ENXIO;
}
+ ret = devm_add_action_or_reset(&pdev->dev, em_gio_irq_domain_remove,
+ p->irq_domain);
+ if (ret)
+ return ret;
+
if (devm_request_irq(&pdev->dev, irq[0]->start,
em_gio_irq_handler, 0, name, p)) {
dev_err(&pdev->dev, "failed to request low IRQ\n");
- ret = -ENOENT;
- goto err1;
+ return -ENOENT;
}
if (devm_request_irq(&pdev->dev, irq[1]->start,
em_gio_irq_handler, 0, name, p)) {
dev_err(&pdev->dev, "failed to request high IRQ\n");
- ret = -ENOENT;
- goto err1;
+ return -ENOENT;
}
ret = devm_gpiochip_add_data(&pdev->dev, gpio_chip, p);
if (ret) {
dev_err(&pdev->dev, "failed to add GPIO controller\n");
- goto err1;
+ return ret;
}
return 0;
-
-err1:
- irq_domain_remove(p->irq_domain);
- return ret;
-}
-
-static int em_gio_remove(struct platform_device *pdev)
-{
- struct em_gio_priv *p = platform_get_drvdata(pdev);
-
- irq_domain_remove(p->irq_domain);
- return 0;
}
static const struct of_device_id em_gio_dt_ids[] = {
static struct platform_driver em_gio_device_driver = {
.probe = em_gio_probe,
- .remove = em_gio_remove,
.driver = {
.name = "em_gio",
.of_match_table = em_gio_dt_ids,
* Legacy handling of SPI active high chip select. If we have a
* property named "cs-gpios" we need to inspect the child node
* to determine if the flags should have inverted semantics.
- *
- * This does not apply to an SPI device named "spi-gpio", because
- * these have traditionally obtained their own GPIOs by parsing
- * the device tree directly and did not respect any "spi-cs-high"
- * property on the SPI bus children.
*/
- if (IS_ENABLED(CONFIG_SPI_MASTER) &&
- !strcmp(propname, "cs-gpios") &&
- !of_device_is_compatible(np, "spi-gpio") &&
+ if (IS_ENABLED(CONFIG_SPI_MASTER) && !strcmp(propname, "cs-gpios") &&
of_property_read_bool(np, "cs-gpios")) {
struct device_node *child;
u32 cs;
of_node_full_name(child));
*flags |= OF_GPIO_ACTIVE_LOW;
}
+ of_node_put(child);
break;
}
}
monitor are unable to provide appropriate EDID data. Since this
feature is provided as a workaround for broken hardware, the
default case is N. Details and instructions how to build your own
- EDID data are given in Documentation/EDID/howto.rst.
+ EDID data are given in Documentation/driver-api/edid.rst.
config DRM_DP_CEC
bool "Enable DisplayPort CEC-Tunneling-over-AUX HDMI support"
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/slab.h>
#include <linux/srcu.h>
static int drm_fs_cnt;
static struct vfsmount *drm_fs_mnt;
-static const struct dentry_operations drm_fs_dops = {
- .d_dname = simple_dname,
-};
-
-static const struct super_operations drm_fs_sops = {
- .statfs = simple_statfs,
-};
-
-static struct dentry *drm_fs_mount(struct file_system_type *fs_type, int flags,
- const char *dev_name, void *data)
+static int drm_fs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type,
- "drm:",
- &drm_fs_sops,
- &drm_fs_dops,
- 0x010203ff);
+ return init_pseudo(fc, 0x010203ff) ? 0 : -ENOMEM;
}
static struct file_system_type drm_fs_type = {
.name = "drm",
.owner = THIS_MODULE,
- .mount = drm_fs_mount,
+ .init_fs_context = drm_fs_init_fs_context,
.kill_sb = kill_anon_super,
};
* };
*
* Please make sure that you follow all the best practices from
- * ``Documentation/ioctl/botching-up-ioctls.txt``. Note that drm_ioctl()
+ * ``Documentation/ioctl/botching-up-ioctls.rst``. Note that drm_ioctl()
* automatically zero-extends structures, hence make sure you can add more stuff
* at the end, i.e. don't put a variable sized array there.
*
* of extra utility/tracking out of our acquire-ctx. This is provided
* by &struct drm_modeset_lock and &struct drm_modeset_acquire_ctx.
*
- * For basic principles of &ww_mutex, see: Documentation/locking/ww-mutex-design.txt
+ * For basic principles of &ww_mutex, see: Documentation/locking/ww-mutex-design.rst
*
* The basic usage pattern is to::
*
#define POLL_PERIOD (NSEC_PER_SEC / POLL_FREQUENCY)
/* for sysctl proc_dointvec_minmax of dev.i915.perf_stream_paranoid */
-static int zero;
-static int one = 1;
static u32 i915_perf_stream_paranoid = true;
/* The maximum exponent the hardware accepts is 63 (essentially it selects one
.maxlen = sizeof(i915_perf_stream_paranoid),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "oa_max_sample_rate",
.maxlen = sizeof(i915_oa_max_sample_rate),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &oa_sample_rate_hard_limit,
},
{}
};
static struct ctl_table_header *hv_ctl_table_hdr;
-static int zero;
-static int one = 1;
/*
* sysctl option to allow the user to control whether kmsg data should be
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE
},
{}
};
config HWSPINLOCK_OMAP
tristate "OMAP Hardware Spinlock device"
depends on HWSPINLOCK
- depends on ARCH_OMAP4 || SOC_OMAP5 || SOC_DRA7XX || SOC_AM33XX || SOC_AM43XX
+ depends on ARCH_OMAP4 || SOC_OMAP5 || SOC_DRA7XX || SOC_AM33XX || SOC_AM43XX || ARCH_K3
help
Say y here to support the OMAP Hardware Spinlock device (firstly
introduced in OMAP4).
#define pr_fmt(fmt) "%s: " fmt, __func__
+#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include "hwspinlock_internal.h"
+/* retry delay used in atomic context */
+#define HWSPINLOCK_RETRY_DELAY_US 100
+
/* radix tree tags */
#define HWSPINLOCK_UNUSED (0) /* tags an hwspinlock as unused */
* user need some time-consuming or sleepable operations under the hardware
* lock, they need one sleepable lock (like mutex) to protect the operations.
*
- * If the mode is not HWLOCK_RAW, upon a successful return from this function,
- * preemption (and possibly interrupts) is disabled, so the caller must not
- * sleep, and is advised to release the hwspinlock as soon as possible. This is
- * required in order to minimize remote cores polling on the hardware
- * interconnect.
+ * If the mode is neither HWLOCK_IN_ATOMIC nor HWLOCK_RAW, upon a successful
+ * return from this function, preemption (and possibly interrupts) is disabled,
+ * so the caller must not sleep, and is advised to release the hwspinlock as
+ * soon as possible. This is required in order to minimize remote cores polling
+ * on the hardware interconnect.
*
* The user decides whether local interrupts are disabled or not, and if yes,
* whether he wants their previous state to be saved. It is up to the user
ret = spin_trylock_irq(&hwlock->lock);
break;
case HWLOCK_RAW:
+ case HWLOCK_IN_ATOMIC:
ret = 1;
break;
default:
spin_unlock_irq(&hwlock->lock);
break;
case HWLOCK_RAW:
+ case HWLOCK_IN_ATOMIC:
/* Nothing to do */
break;
default:
* user need some time-consuming or sleepable operations under the hardware
* lock, they need one sleepable lock (like mutex) to protect the operations.
*
- * If the mode is not HWLOCK_RAW, upon a successful return from this function,
- * preemption is disabled (and possibly local interrupts, too), so the caller
- * must not sleep, and is advised to release the hwspinlock as soon as possible.
- * This is required in order to minimize remote cores polling on the
- * hardware interconnect.
+ * If the mode is HWLOCK_IN_ATOMIC (called from an atomic context) the timeout
+ * is handled with busy-waiting delays, hence shall not exceed few msecs.
+ *
+ * If the mode is neither HWLOCK_IN_ATOMIC nor HWLOCK_RAW, upon a successful
+ * return from this function, preemption (and possibly interrupts) is disabled,
+ * so the caller must not sleep, and is advised to release the hwspinlock as
+ * soon as possible. This is required in order to minimize remote cores polling
+ * on the hardware interconnect.
*
* The user decides whether local interrupts are disabled or not, and if yes,
* whether he wants their previous state to be saved. It is up to the user
int mode, unsigned long *flags)
{
int ret;
- unsigned long expire;
+ unsigned long expire, atomic_delay = 0;
expire = msecs_to_jiffies(to) + jiffies;
* The lock is already taken, let's check if the user wants
* us to try again
*/
- if (time_is_before_eq_jiffies(expire))
- return -ETIMEDOUT;
+ if (mode == HWLOCK_IN_ATOMIC) {
+ udelay(HWSPINLOCK_RETRY_DELAY_US);
+ atomic_delay += HWSPINLOCK_RETRY_DELAY_US;
+ if (atomic_delay > to * 1000)
+ return -ETIMEDOUT;
+ } else {
+ if (time_is_before_eq_jiffies(expire))
+ return -ETIMEDOUT;
+ }
/*
* Allow platform-specific relax handlers to prevent
spin_unlock_irq(&hwlock->lock);
break;
case HWLOCK_RAW:
+ case HWLOCK_IN_ATOMIC:
/* Nothing to do */
break;
default:
if (ret)
return ret;
+ if (!of_device_is_available(args.np)) {
+ ret = -ENOENT;
+ goto out;
+ }
+
/* Find the hwspinlock device: we need its base_id */
ret = -EPROBE_DEFER;
rcu_read_lock();
if (ret)
goto reg_fail;
+ dev_dbg(&pdev->dev, "Registered %d locks with HwSpinlock core\n",
+ num_locks);
+
return 0;
reg_fail:
static const struct of_device_id omap_hwspinlock_of_match[] = {
{ .compatible = "ti,omap4-hwspinlock", },
+ { .compatible = "ti,am654-hwspinlock", },
{ /* end */ },
};
MODULE_DEVICE_TABLE(of, omap_hwspinlock_of_match);
*/
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/hwspinlock.h>
#include <linux/io.h>
#include <linux/kernel.h>
writel(STM32_MUTEX_COREID, lock_addr);
}
+static void stm32_hwspinlock_relax(struct hwspinlock *lock)
+{
+ ndelay(50);
+}
+
static const struct hwspinlock_ops stm32_hwspinlock_ops = {
.trylock = stm32_hwspinlock_trylock,
.unlock = stm32_hwspinlock_unlock,
+ .relax = stm32_hwspinlock_relax,
};
static int stm32_hwspinlock_probe(struct platform_device *pdev)
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/mount.h>
#include <linux/pagemap.h>
#include <linux/init.h>
* after device init. The direct add_cntr_files() call handles adding
* them from the init code, when the fs is already mounted.
*/
-static int qibfs_fill_super(struct super_block *sb, void *data, int silent)
+static int qibfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct qib_devdata *dd;
unsigned long index;
return ret;
}
-static struct dentry *qibfs_mount(struct file_system_type *fs_type, int flags,
- const char *dev_name, void *data)
+static int qibfs_get_tree(struct fs_context *fc)
{
- struct dentry *ret;
-
- ret = mount_single(fs_type, flags, data, qibfs_fill_super);
- if (!IS_ERR(ret))
- qib_super = ret->d_sb;
+ int ret = get_tree_single(fc, qibfs_fill_super);
+ if (ret == 0)
+ qib_super = fc->root->d_sb;
return ret;
}
+static const struct fs_context_operations qibfs_context_ops = {
+ .get_tree = qibfs_get_tree,
+};
+
+static int qibfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &qibfs_context_ops;
+ return 0;
+}
+
static void qibfs_kill_super(struct super_block *s)
{
kill_litter_super(s);
static struct file_system_type qibfs_fs_type = {
.owner = THIS_MODULE,
.name = "ipathfs",
- .mount = qibfs_mount,
+ .init_fs_context = qibfs_init_fs_context,
.kill_sb = qibfs_kill_super,
};
MODULE_ALIAS_FS("ipathfs");
* in the kernel). So this driver offers straight forward, reliable single
* touch functionality only.
*
- * s.a. A20 User Manual "1.15 TP" (Documentation/arm/sunxi/README)
+ * s.a. A20 User Manual "1.15 TP" (Documentation/arm/sunxi.rst)
* (looks like the description in the A20 User Manual v1.3 is better
* than the one in the A10 User Manual v.1.5)
*/
Stub IOMMU driver to handle IRQs as to allow Hyper-V Linux
guests to run with x2APIC mode enabled.
+config VIRTIO_IOMMU
+ bool "Virtio IOMMU driver"
+ depends on VIRTIO=y
+ depends on ARM64
+ select IOMMU_API
+ select INTERVAL_TREE
+ help
+ Para-virtualised IOMMU driver with virtio.
+
+ Say Y here if you intend to run this kernel as a guest.
+
endif # IOMMU_SUPPORT
obj-$(CONFIG_S390_IOMMU) += s390-iommu.o
obj-$(CONFIG_QCOM_IOMMU) += qcom_iommu.o
obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o
+obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Virtio driver for the paravirtualized IOMMU
+ *
+ * Copyright (C) 2018 Arm Limited
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/amba/bus.h>
+#include <linux/delay.h>
+#include <linux/dma-iommu.h>
+#include <linux/freezer.h>
+#include <linux/interval_tree.h>
+#include <linux/iommu.h>
+#include <linux/module.h>
+#include <linux/of_iommu.h>
+#include <linux/of_platform.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/virtio_ids.h>
+#include <linux/wait.h>
+
+#include <uapi/linux/virtio_iommu.h>
+
+#define MSI_IOVA_BASE 0x8000000
+#define MSI_IOVA_LENGTH 0x100000
+
+#define VIOMMU_REQUEST_VQ 0
+#define VIOMMU_EVENT_VQ 1
+#define VIOMMU_NR_VQS 2
+
+struct viommu_dev {
+ struct iommu_device iommu;
+ struct device *dev;
+ struct virtio_device *vdev;
+
+ struct ida domain_ids;
+
+ struct virtqueue *vqs[VIOMMU_NR_VQS];
+ spinlock_t request_lock;
+ struct list_head requests;
+ void *evts;
+
+ /* Device configuration */
+ struct iommu_domain_geometry geometry;
+ u64 pgsize_bitmap;
+ u8 domain_bits;
+ u32 probe_size;
+};
+
+struct viommu_mapping {
+ phys_addr_t paddr;
+ struct interval_tree_node iova;
+ u32 flags;
+};
+
+struct viommu_domain {
+ struct iommu_domain domain;
+ struct viommu_dev *viommu;
+ struct mutex mutex; /* protects viommu pointer */
+ unsigned int id;
+
+ spinlock_t mappings_lock;
+ struct rb_root_cached mappings;
+
+ unsigned long nr_endpoints;
+};
+
+struct viommu_endpoint {
+ struct device *dev;
+ struct viommu_dev *viommu;
+ struct viommu_domain *vdomain;
+ struct list_head resv_regions;
+};
+
+struct viommu_request {
+ struct list_head list;
+ void *writeback;
+ unsigned int write_offset;
+ unsigned int len;
+ char buf[];
+};
+
+#define VIOMMU_FAULT_RESV_MASK 0xffffff00
+
+struct viommu_event {
+ union {
+ u32 head;
+ struct virtio_iommu_fault fault;
+ };
+};
+
+#define to_viommu_domain(domain) \
+ container_of(domain, struct viommu_domain, domain)
+
+static int viommu_get_req_errno(void *buf, size_t len)
+{
+ struct virtio_iommu_req_tail *tail = buf + len - sizeof(*tail);
+
+ switch (tail->status) {
+ case VIRTIO_IOMMU_S_OK:
+ return 0;
+ case VIRTIO_IOMMU_S_UNSUPP:
+ return -ENOSYS;
+ case VIRTIO_IOMMU_S_INVAL:
+ return -EINVAL;
+ case VIRTIO_IOMMU_S_RANGE:
+ return -ERANGE;
+ case VIRTIO_IOMMU_S_NOENT:
+ return -ENOENT;
+ case VIRTIO_IOMMU_S_FAULT:
+ return -EFAULT;
+ case VIRTIO_IOMMU_S_IOERR:
+ case VIRTIO_IOMMU_S_DEVERR:
+ default:
+ return -EIO;
+ }
+}
+
+static void viommu_set_req_status(void *buf, size_t len, int status)
+{
+ struct virtio_iommu_req_tail *tail = buf + len - sizeof(*tail);
+
+ tail->status = status;
+}
+
+static off_t viommu_get_write_desc_offset(struct viommu_dev *viommu,
+ struct virtio_iommu_req_head *req,
+ size_t len)
+{
+ size_t tail_size = sizeof(struct virtio_iommu_req_tail);
+
+ if (req->type == VIRTIO_IOMMU_T_PROBE)
+ return len - viommu->probe_size - tail_size;
+
+ return len - tail_size;
+}
+
+/*
+ * __viommu_sync_req - Complete all in-flight requests
+ *
+ * Wait for all added requests to complete. When this function returns, all
+ * requests that were in-flight at the time of the call have completed.
+ */
+static int __viommu_sync_req(struct viommu_dev *viommu)
+{
+ int ret = 0;
+ unsigned int len;
+ size_t write_len;
+ struct viommu_request *req;
+ struct virtqueue *vq = viommu->vqs[VIOMMU_REQUEST_VQ];
+
+ assert_spin_locked(&viommu->request_lock);
+
+ virtqueue_kick(vq);
+
+ while (!list_empty(&viommu->requests)) {
+ len = 0;
+ req = virtqueue_get_buf(vq, &len);
+ if (!req)
+ continue;
+
+ if (!len)
+ viommu_set_req_status(req->buf, req->len,
+ VIRTIO_IOMMU_S_IOERR);
+
+ write_len = req->len - req->write_offset;
+ if (req->writeback && len == write_len)
+ memcpy(req->writeback, req->buf + req->write_offset,
+ write_len);
+
+ list_del(&req->list);
+ kfree(req);
+ }
+
+ return ret;
+}
+
+static int viommu_sync_req(struct viommu_dev *viommu)
+{
+ int ret;
+ unsigned long flags;
+
+ spin_lock_irqsave(&viommu->request_lock, flags);
+ ret = __viommu_sync_req(viommu);
+ if (ret)
+ dev_dbg(viommu->dev, "could not sync requests (%d)\n", ret);
+ spin_unlock_irqrestore(&viommu->request_lock, flags);
+
+ return ret;
+}
+
+/*
+ * __viommu_add_request - Add one request to the queue
+ * @buf: pointer to the request buffer
+ * @len: length of the request buffer
+ * @writeback: copy data back to the buffer when the request completes.
+ *
+ * Add a request to the queue. Only synchronize the queue if it's already full.
+ * Otherwise don't kick the queue nor wait for requests to complete.
+ *
+ * When @writeback is true, data written by the device, including the request
+ * status, is copied into @buf after the request completes. This is unsafe if
+ * the caller allocates @buf on stack and drops the lock between add_req() and
+ * sync_req().
+ *
+ * Return 0 if the request was successfully added to the queue.
+ */
+static int __viommu_add_req(struct viommu_dev *viommu, void *buf, size_t len,
+ bool writeback)
+{
+ int ret;
+ off_t write_offset;
+ struct viommu_request *req;
+ struct scatterlist top_sg, bottom_sg;
+ struct scatterlist *sg[2] = { &top_sg, &bottom_sg };
+ struct virtqueue *vq = viommu->vqs[VIOMMU_REQUEST_VQ];
+
+ assert_spin_locked(&viommu->request_lock);
+
+ write_offset = viommu_get_write_desc_offset(viommu, buf, len);
+ if (write_offset <= 0)
+ return -EINVAL;
+
+ req = kzalloc(sizeof(*req) + len, GFP_ATOMIC);
+ if (!req)
+ return -ENOMEM;
+
+ req->len = len;
+ if (writeback) {
+ req->writeback = buf + write_offset;
+ req->write_offset = write_offset;
+ }
+ memcpy(&req->buf, buf, write_offset);
+
+ sg_init_one(&top_sg, req->buf, write_offset);
+ sg_init_one(&bottom_sg, req->buf + write_offset, len - write_offset);
+
+ ret = virtqueue_add_sgs(vq, sg, 1, 1, req, GFP_ATOMIC);
+ if (ret == -ENOSPC) {
+ /* If the queue is full, sync and retry */
+ if (!__viommu_sync_req(viommu))
+ ret = virtqueue_add_sgs(vq, sg, 1, 1, req, GFP_ATOMIC);
+ }
+ if (ret)
+ goto err_free;
+
+ list_add_tail(&req->list, &viommu->requests);
+ return 0;
+
+err_free:
+ kfree(req);
+ return ret;
+}
+
+static int viommu_add_req(struct viommu_dev *viommu, void *buf, size_t len)
+{
+ int ret;
+ unsigned long flags;
+
+ spin_lock_irqsave(&viommu->request_lock, flags);
+ ret = __viommu_add_req(viommu, buf, len, false);
+ if (ret)
+ dev_dbg(viommu->dev, "could not add request: %d\n", ret);
+ spin_unlock_irqrestore(&viommu->request_lock, flags);
+
+ return ret;
+}
+
+/*
+ * Send a request and wait for it to complete. Return the request status (as an
+ * errno)
+ */
+static int viommu_send_req_sync(struct viommu_dev *viommu, void *buf,
+ size_t len)
+{
+ int ret;
+ unsigned long flags;
+
+ spin_lock_irqsave(&viommu->request_lock, flags);
+
+ ret = __viommu_add_req(viommu, buf, len, true);
+ if (ret) {
+ dev_dbg(viommu->dev, "could not add request (%d)\n", ret);
+ goto out_unlock;
+ }
+
+ ret = __viommu_sync_req(viommu);
+ if (ret) {
+ dev_dbg(viommu->dev, "could not sync requests (%d)\n", ret);
+ /* Fall-through (get the actual request status) */
+ }
+
+ ret = viommu_get_req_errno(buf, len);
+out_unlock:
+ spin_unlock_irqrestore(&viommu->request_lock, flags);
+ return ret;
+}
+
+/*
+ * viommu_add_mapping - add a mapping to the internal tree
+ *
+ * On success, return the new mapping. Otherwise return NULL.
+ */
+static int viommu_add_mapping(struct viommu_domain *vdomain, unsigned long iova,
+ phys_addr_t paddr, size_t size, u32 flags)
+{
+ unsigned long irqflags;
+ struct viommu_mapping *mapping;
+
+ mapping = kzalloc(sizeof(*mapping), GFP_ATOMIC);
+ if (!mapping)
+ return -ENOMEM;
+
+ mapping->paddr = paddr;
+ mapping->iova.start = iova;
+ mapping->iova.last = iova + size - 1;
+ mapping->flags = flags;
+
+ spin_lock_irqsave(&vdomain->mappings_lock, irqflags);
+ interval_tree_insert(&mapping->iova, &vdomain->mappings);
+ spin_unlock_irqrestore(&vdomain->mappings_lock, irqflags);
+
+ return 0;
+}
+
+/*
+ * viommu_del_mappings - remove mappings from the internal tree
+ *
+ * @vdomain: the domain
+ * @iova: start of the range
+ * @size: size of the range. A size of 0 corresponds to the entire address
+ * space.
+ *
+ * On success, returns the number of unmapped bytes (>= size)
+ */
+static size_t viommu_del_mappings(struct viommu_domain *vdomain,
+ unsigned long iova, size_t size)
+{
+ size_t unmapped = 0;
+ unsigned long flags;
+ unsigned long last = iova + size - 1;
+ struct viommu_mapping *mapping = NULL;
+ struct interval_tree_node *node, *next;
+
+ spin_lock_irqsave(&vdomain->mappings_lock, flags);
+ next = interval_tree_iter_first(&vdomain->mappings, iova, last);
+ while (next) {
+ node = next;
+ mapping = container_of(node, struct viommu_mapping, iova);
+ next = interval_tree_iter_next(node, iova, last);
+
+ /* Trying to split a mapping? */
+ if (mapping->iova.start < iova)
+ break;
+
+ /*
+ * Virtio-iommu doesn't allow UNMAP to split a mapping created
+ * with a single MAP request, so remove the full mapping.
+ */
+ unmapped += mapping->iova.last - mapping->iova.start + 1;
+
+ interval_tree_remove(node, &vdomain->mappings);
+ kfree(mapping);
+ }
+ spin_unlock_irqrestore(&vdomain->mappings_lock, flags);
+
+ return unmapped;
+}
+
+/*
+ * viommu_replay_mappings - re-send MAP requests
+ *
+ * When reattaching a domain that was previously detached from all endpoints,
+ * mappings were deleted from the device. Re-create the mappings available in
+ * the internal tree.
+ */
+static int viommu_replay_mappings(struct viommu_domain *vdomain)
+{
+ int ret = 0;
+ unsigned long flags;
+ struct viommu_mapping *mapping;
+ struct interval_tree_node *node;
+ struct virtio_iommu_req_map map;
+
+ spin_lock_irqsave(&vdomain->mappings_lock, flags);
+ node = interval_tree_iter_first(&vdomain->mappings, 0, -1UL);
+ while (node) {
+ mapping = container_of(node, struct viommu_mapping, iova);
+ map = (struct virtio_iommu_req_map) {
+ .head.type = VIRTIO_IOMMU_T_MAP,
+ .domain = cpu_to_le32(vdomain->id),
+ .virt_start = cpu_to_le64(mapping->iova.start),
+ .virt_end = cpu_to_le64(mapping->iova.last),
+ .phys_start = cpu_to_le64(mapping->paddr),
+ .flags = cpu_to_le32(mapping->flags),
+ };
+
+ ret = viommu_send_req_sync(vdomain->viommu, &map, sizeof(map));
+ if (ret)
+ break;
+
+ node = interval_tree_iter_next(node, 0, -1UL);
+ }
+ spin_unlock_irqrestore(&vdomain->mappings_lock, flags);
+
+ return ret;
+}
+
+static int viommu_add_resv_mem(struct viommu_endpoint *vdev,
+ struct virtio_iommu_probe_resv_mem *mem,
+ size_t len)
+{
+ size_t size;
+ u64 start64, end64;
+ phys_addr_t start, end;
+ struct iommu_resv_region *region = NULL;
+ unsigned long prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+
+ start = start64 = le64_to_cpu(mem->start);
+ end = end64 = le64_to_cpu(mem->end);
+ size = end64 - start64 + 1;
+
+ /* Catch any overflow, including the unlikely end64 - start64 + 1 = 0 */
+ if (start != start64 || end != end64 || size < end64 - start64)
+ return -EOVERFLOW;
+
+ if (len < sizeof(*mem))
+ return -EINVAL;
+
+ switch (mem->subtype) {
+ default:
+ dev_warn(vdev->dev, "unknown resv mem subtype 0x%x\n",
+ mem->subtype);
+ /* Fall-through */
+ case VIRTIO_IOMMU_RESV_MEM_T_RESERVED:
+ region = iommu_alloc_resv_region(start, size, 0,
+ IOMMU_RESV_RESERVED);
+ break;
+ case VIRTIO_IOMMU_RESV_MEM_T_MSI:
+ region = iommu_alloc_resv_region(start, size, prot,
+ IOMMU_RESV_MSI);
+ break;
+ }
+ if (!region)
+ return -ENOMEM;
+
+ list_add(&vdev->resv_regions, ®ion->list);
+ return 0;
+}
+
+static int viommu_probe_endpoint(struct viommu_dev *viommu, struct device *dev)
+{
+ int ret;
+ u16 type, len;
+ size_t cur = 0;
+ size_t probe_len;
+ struct virtio_iommu_req_probe *probe;
+ struct virtio_iommu_probe_property *prop;
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+ struct viommu_endpoint *vdev = fwspec->iommu_priv;
+
+ if (!fwspec->num_ids)
+ return -EINVAL;
+
+ probe_len = sizeof(*probe) + viommu->probe_size +
+ sizeof(struct virtio_iommu_req_tail);
+ probe = kzalloc(probe_len, GFP_KERNEL);
+ if (!probe)
+ return -ENOMEM;
+
+ probe->head.type = VIRTIO_IOMMU_T_PROBE;
+ /*
+ * For now, assume that properties of an endpoint that outputs multiple
+ * IDs are consistent. Only probe the first one.
+ */
+ probe->endpoint = cpu_to_le32(fwspec->ids[0]);
+
+ ret = viommu_send_req_sync(viommu, probe, probe_len);
+ if (ret)
+ goto out_free;
+
+ prop = (void *)probe->properties;
+ type = le16_to_cpu(prop->type) & VIRTIO_IOMMU_PROBE_T_MASK;
+
+ while (type != VIRTIO_IOMMU_PROBE_T_NONE &&
+ cur < viommu->probe_size) {
+ len = le16_to_cpu(prop->length) + sizeof(*prop);
+
+ switch (type) {
+ case VIRTIO_IOMMU_PROBE_T_RESV_MEM:
+ ret = viommu_add_resv_mem(vdev, (void *)prop, len);
+ break;
+ default:
+ dev_err(dev, "unknown viommu prop 0x%x\n", type);
+ }
+
+ if (ret)
+ dev_err(dev, "failed to parse viommu prop 0x%x\n", type);
+
+ cur += len;
+ if (cur >= viommu->probe_size)
+ break;
+
+ prop = (void *)probe->properties + cur;
+ type = le16_to_cpu(prop->type) & VIRTIO_IOMMU_PROBE_T_MASK;
+ }
+
+out_free:
+ kfree(probe);
+ return ret;
+}
+
+static int viommu_fault_handler(struct viommu_dev *viommu,
+ struct virtio_iommu_fault *fault)
+{
+ char *reason_str;
+
+ u8 reason = fault->reason;
+ u32 flags = le32_to_cpu(fault->flags);
+ u32 endpoint = le32_to_cpu(fault->endpoint);
+ u64 address = le64_to_cpu(fault->address);
+
+ switch (reason) {
+ case VIRTIO_IOMMU_FAULT_R_DOMAIN:
+ reason_str = "domain";
+ break;
+ case VIRTIO_IOMMU_FAULT_R_MAPPING:
+ reason_str = "page";
+ break;
+ case VIRTIO_IOMMU_FAULT_R_UNKNOWN:
+ default:
+ reason_str = "unknown";
+ break;
+ }
+
+ /* TODO: find EP by ID and report_iommu_fault */
+ if (flags & VIRTIO_IOMMU_FAULT_F_ADDRESS)
+ dev_err_ratelimited(viommu->dev, "%s fault from EP %u at %#llx [%s%s%s]\n",
+ reason_str, endpoint, address,
+ flags & VIRTIO_IOMMU_FAULT_F_READ ? "R" : "",
+ flags & VIRTIO_IOMMU_FAULT_F_WRITE ? "W" : "",
+ flags & VIRTIO_IOMMU_FAULT_F_EXEC ? "X" : "");
+ else
+ dev_err_ratelimited(viommu->dev, "%s fault from EP %u\n",
+ reason_str, endpoint);
+ return 0;
+}
+
+static void viommu_event_handler(struct virtqueue *vq)
+{
+ int ret;
+ unsigned int len;
+ struct scatterlist sg[1];
+ struct viommu_event *evt;
+ struct viommu_dev *viommu = vq->vdev->priv;
+
+ while ((evt = virtqueue_get_buf(vq, &len)) != NULL) {
+ if (len > sizeof(*evt)) {
+ dev_err(viommu->dev,
+ "invalid event buffer (len %u != %zu)\n",
+ len, sizeof(*evt));
+ } else if (!(evt->head & VIOMMU_FAULT_RESV_MASK)) {
+ viommu_fault_handler(viommu, &evt->fault);
+ }
+
+ sg_init_one(sg, evt, sizeof(*evt));
+ ret = virtqueue_add_inbuf(vq, sg, 1, evt, GFP_ATOMIC);
+ if (ret)
+ dev_err(viommu->dev, "could not add event buffer\n");
+ }
+
+ virtqueue_kick(vq);
+}
+
+/* IOMMU API */
+
+static struct iommu_domain *viommu_domain_alloc(unsigned type)
+{
+ struct viommu_domain *vdomain;
+
+ if (type != IOMMU_DOMAIN_UNMANAGED && type != IOMMU_DOMAIN_DMA)
+ return NULL;
+
+ vdomain = kzalloc(sizeof(*vdomain), GFP_KERNEL);
+ if (!vdomain)
+ return NULL;
+
+ mutex_init(&vdomain->mutex);
+ spin_lock_init(&vdomain->mappings_lock);
+ vdomain->mappings = RB_ROOT_CACHED;
+
+ if (type == IOMMU_DOMAIN_DMA &&
+ iommu_get_dma_cookie(&vdomain->domain)) {
+ kfree(vdomain);
+ return NULL;
+ }
+
+ return &vdomain->domain;
+}
+
+static int viommu_domain_finalise(struct viommu_dev *viommu,
+ struct iommu_domain *domain)
+{
+ int ret;
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+ unsigned int max_domain = viommu->domain_bits > 31 ? ~0 :
+ (1U << viommu->domain_bits) - 1;
+
+ vdomain->viommu = viommu;
+
+ domain->pgsize_bitmap = viommu->pgsize_bitmap;
+ domain->geometry = viommu->geometry;
+
+ ret = ida_alloc_max(&viommu->domain_ids, max_domain, GFP_KERNEL);
+ if (ret >= 0)
+ vdomain->id = (unsigned int)ret;
+
+ return ret > 0 ? 0 : ret;
+}
+
+static void viommu_domain_free(struct iommu_domain *domain)
+{
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ iommu_put_dma_cookie(domain);
+
+ /* Free all remaining mappings (size 2^64) */
+ viommu_del_mappings(vdomain, 0, 0);
+
+ if (vdomain->viommu)
+ ida_free(&vdomain->viommu->domain_ids, vdomain->id);
+
+ kfree(vdomain);
+}
+
+static int viommu_attach_dev(struct iommu_domain *domain, struct device *dev)
+{
+ int i;
+ int ret = 0;
+ struct virtio_iommu_req_attach req;
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+ struct viommu_endpoint *vdev = fwspec->iommu_priv;
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ mutex_lock(&vdomain->mutex);
+ if (!vdomain->viommu) {
+ /*
+ * Properly initialize the domain now that we know which viommu
+ * owns it.
+ */
+ ret = viommu_domain_finalise(vdev->viommu, domain);
+ } else if (vdomain->viommu != vdev->viommu) {
+ dev_err(dev, "cannot attach to foreign vIOMMU\n");
+ ret = -EXDEV;
+ }
+ mutex_unlock(&vdomain->mutex);
+
+ if (ret)
+ return ret;
+
+ /*
+ * In the virtio-iommu device, when attaching the endpoint to a new
+ * domain, it is detached from the old one and, if as as a result the
+ * old domain isn't attached to any endpoint, all mappings are removed
+ * from the old domain and it is freed.
+ *
+ * In the driver the old domain still exists, and its mappings will be
+ * recreated if it gets reattached to an endpoint. Otherwise it will be
+ * freed explicitly.
+ *
+ * vdev->vdomain is protected by group->mutex
+ */
+ if (vdev->vdomain)
+ vdev->vdomain->nr_endpoints--;
+
+ req = (struct virtio_iommu_req_attach) {
+ .head.type = VIRTIO_IOMMU_T_ATTACH,
+ .domain = cpu_to_le32(vdomain->id),
+ };
+
+ for (i = 0; i < fwspec->num_ids; i++) {
+ req.endpoint = cpu_to_le32(fwspec->ids[i]);
+
+ ret = viommu_send_req_sync(vdomain->viommu, &req, sizeof(req));
+ if (ret)
+ return ret;
+ }
+
+ if (!vdomain->nr_endpoints) {
+ /*
+ * This endpoint is the first to be attached to the domain.
+ * Replay existing mappings (e.g. SW MSI).
+ */
+ ret = viommu_replay_mappings(vdomain);
+ if (ret)
+ return ret;
+ }
+
+ vdomain->nr_endpoints++;
+ vdev->vdomain = vdomain;
+
+ return 0;
+}
+
+static int viommu_map(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot)
+{
+ int ret;
+ int flags;
+ struct virtio_iommu_req_map map;
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ flags = (prot & IOMMU_READ ? VIRTIO_IOMMU_MAP_F_READ : 0) |
+ (prot & IOMMU_WRITE ? VIRTIO_IOMMU_MAP_F_WRITE : 0) |
+ (prot & IOMMU_MMIO ? VIRTIO_IOMMU_MAP_F_MMIO : 0);
+
+ ret = viommu_add_mapping(vdomain, iova, paddr, size, flags);
+ if (ret)
+ return ret;
+
+ map = (struct virtio_iommu_req_map) {
+ .head.type = VIRTIO_IOMMU_T_MAP,
+ .domain = cpu_to_le32(vdomain->id),
+ .virt_start = cpu_to_le64(iova),
+ .phys_start = cpu_to_le64(paddr),
+ .virt_end = cpu_to_le64(iova + size - 1),
+ .flags = cpu_to_le32(flags),
+ };
+
+ if (!vdomain->nr_endpoints)
+ return 0;
+
+ ret = viommu_send_req_sync(vdomain->viommu, &map, sizeof(map));
+ if (ret)
+ viommu_del_mappings(vdomain, iova, size);
+
+ return ret;
+}
+
+static size_t viommu_unmap(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ int ret = 0;
+ size_t unmapped;
+ struct virtio_iommu_req_unmap unmap;
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ unmapped = viommu_del_mappings(vdomain, iova, size);
+ if (unmapped < size)
+ return 0;
+
+ /* Device already removed all mappings after detach. */
+ if (!vdomain->nr_endpoints)
+ return unmapped;
+
+ unmap = (struct virtio_iommu_req_unmap) {
+ .head.type = VIRTIO_IOMMU_T_UNMAP,
+ .domain = cpu_to_le32(vdomain->id),
+ .virt_start = cpu_to_le64(iova),
+ .virt_end = cpu_to_le64(iova + unmapped - 1),
+ };
+
+ ret = viommu_add_req(vdomain->viommu, &unmap, sizeof(unmap));
+ return ret ? 0 : unmapped;
+}
+
+static phys_addr_t viommu_iova_to_phys(struct iommu_domain *domain,
+ dma_addr_t iova)
+{
+ u64 paddr = 0;
+ unsigned long flags;
+ struct viommu_mapping *mapping;
+ struct interval_tree_node *node;
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ spin_lock_irqsave(&vdomain->mappings_lock, flags);
+ node = interval_tree_iter_first(&vdomain->mappings, iova, iova);
+ if (node) {
+ mapping = container_of(node, struct viommu_mapping, iova);
+ paddr = mapping->paddr + (iova - mapping->iova.start);
+ }
+ spin_unlock_irqrestore(&vdomain->mappings_lock, flags);
+
+ return paddr;
+}
+
+static void viommu_iotlb_sync(struct iommu_domain *domain)
+{
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ viommu_sync_req(vdomain->viommu);
+}
+
+static void viommu_get_resv_regions(struct device *dev, struct list_head *head)
+{
+ struct iommu_resv_region *entry, *new_entry, *msi = NULL;
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+ struct viommu_endpoint *vdev = fwspec->iommu_priv;
+ int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+
+ list_for_each_entry(entry, &vdev->resv_regions, list) {
+ if (entry->type == IOMMU_RESV_MSI)
+ msi = entry;
+
+ new_entry = kmemdup(entry, sizeof(*entry), GFP_KERNEL);
+ if (!new_entry)
+ return;
+ list_add_tail(&new_entry->list, head);
+ }
+
+ /*
+ * If the device didn't register any bypass MSI window, add a
+ * software-mapped region.
+ */
+ if (!msi) {
+ msi = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
+ prot, IOMMU_RESV_SW_MSI);
+ if (!msi)
+ return;
+
+ list_add_tail(&msi->list, head);
+ }
+
+ iommu_dma_get_resv_regions(dev, head);
+}
+
+static void viommu_put_resv_regions(struct device *dev, struct list_head *head)
+{
+ struct iommu_resv_region *entry, *next;
+
+ list_for_each_entry_safe(entry, next, head, list)
+ kfree(entry);
+}
+
+static struct iommu_ops viommu_ops;
+static struct virtio_driver virtio_iommu_drv;
+
+static int viommu_match_node(struct device *dev, const void *data)
+{
+ return dev->parent->fwnode == data;
+}
+
+static struct viommu_dev *viommu_get_by_fwnode(struct fwnode_handle *fwnode)
+{
+ struct device *dev = driver_find_device(&virtio_iommu_drv.driver, NULL,
+ fwnode, viommu_match_node);
+ put_device(dev);
+
+ return dev ? dev_to_virtio(dev)->priv : NULL;
+}
+
+static int viommu_add_device(struct device *dev)
+{
+ int ret;
+ struct iommu_group *group;
+ struct viommu_endpoint *vdev;
+ struct viommu_dev *viommu = NULL;
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+
+ if (!fwspec || fwspec->ops != &viommu_ops)
+ return -ENODEV;
+
+ viommu = viommu_get_by_fwnode(fwspec->iommu_fwnode);
+ if (!viommu)
+ return -ENODEV;
+
+ vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
+ if (!vdev)
+ return -ENOMEM;
+
+ vdev->dev = dev;
+ vdev->viommu = viommu;
+ INIT_LIST_HEAD(&vdev->resv_regions);
+ fwspec->iommu_priv = vdev;
+
+ if (viommu->probe_size) {
+ /* Get additional information for this endpoint */
+ ret = viommu_probe_endpoint(viommu, dev);
+ if (ret)
+ goto err_free_dev;
+ }
+
+ ret = iommu_device_link(&viommu->iommu, dev);
+ if (ret)
+ goto err_free_dev;
+
+ /*
+ * Last step creates a default domain and attaches to it. Everything
+ * must be ready.
+ */
+ group = iommu_group_get_for_dev(dev);
+ if (IS_ERR(group)) {
+ ret = PTR_ERR(group);
+ goto err_unlink_dev;
+ }
+
+ iommu_group_put(group);
+
+ return PTR_ERR_OR_ZERO(group);
+
+err_unlink_dev:
+ iommu_device_unlink(&viommu->iommu, dev);
+err_free_dev:
+ viommu_put_resv_regions(dev, &vdev->resv_regions);
+ kfree(vdev);
+
+ return ret;
+}
+
+static void viommu_remove_device(struct device *dev)
+{
+ struct viommu_endpoint *vdev;
+ struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+
+ if (!fwspec || fwspec->ops != &viommu_ops)
+ return;
+
+ vdev = fwspec->iommu_priv;
+
+ iommu_group_remove_device(dev);
+ iommu_device_unlink(&vdev->viommu->iommu, dev);
+ viommu_put_resv_regions(dev, &vdev->resv_regions);
+ kfree(vdev);
+}
+
+static struct iommu_group *viommu_device_group(struct device *dev)
+{
+ if (dev_is_pci(dev))
+ return pci_device_group(dev);
+ else
+ return generic_device_group(dev);
+}
+
+static int viommu_of_xlate(struct device *dev, struct of_phandle_args *args)
+{
+ return iommu_fwspec_add_ids(dev, args->args, 1);
+}
+
+static struct iommu_ops viommu_ops = {
+ .domain_alloc = viommu_domain_alloc,
+ .domain_free = viommu_domain_free,
+ .attach_dev = viommu_attach_dev,
+ .map = viommu_map,
+ .unmap = viommu_unmap,
+ .iova_to_phys = viommu_iova_to_phys,
+ .iotlb_sync = viommu_iotlb_sync,
+ .add_device = viommu_add_device,
+ .remove_device = viommu_remove_device,
+ .device_group = viommu_device_group,
+ .get_resv_regions = viommu_get_resv_regions,
+ .put_resv_regions = viommu_put_resv_regions,
+ .of_xlate = viommu_of_xlate,
+};
+
+static int viommu_init_vqs(struct viommu_dev *viommu)
+{
+ struct virtio_device *vdev = dev_to_virtio(viommu->dev);
+ const char *names[] = { "request", "event" };
+ vq_callback_t *callbacks[] = {
+ NULL, /* No async requests */
+ viommu_event_handler,
+ };
+
+ return virtio_find_vqs(vdev, VIOMMU_NR_VQS, viommu->vqs, callbacks,
+ names, NULL);
+}
+
+static int viommu_fill_evtq(struct viommu_dev *viommu)
+{
+ int i, ret;
+ struct scatterlist sg[1];
+ struct viommu_event *evts;
+ struct virtqueue *vq = viommu->vqs[VIOMMU_EVENT_VQ];
+ size_t nr_evts = vq->num_free;
+
+ viommu->evts = evts = devm_kmalloc_array(viommu->dev, nr_evts,
+ sizeof(*evts), GFP_KERNEL);
+ if (!evts)
+ return -ENOMEM;
+
+ for (i = 0; i < nr_evts; i++) {
+ sg_init_one(sg, &evts[i], sizeof(*evts));
+ ret = virtqueue_add_inbuf(vq, sg, 1, &evts[i], GFP_KERNEL);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int viommu_probe(struct virtio_device *vdev)
+{
+ struct device *parent_dev = vdev->dev.parent;
+ struct viommu_dev *viommu = NULL;
+ struct device *dev = &vdev->dev;
+ u64 input_start = 0;
+ u64 input_end = -1UL;
+ int ret;
+
+ if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1) ||
+ !virtio_has_feature(vdev, VIRTIO_IOMMU_F_MAP_UNMAP))
+ return -ENODEV;
+
+ viommu = devm_kzalloc(dev, sizeof(*viommu), GFP_KERNEL);
+ if (!viommu)
+ return -ENOMEM;
+
+ spin_lock_init(&viommu->request_lock);
+ ida_init(&viommu->domain_ids);
+ viommu->dev = dev;
+ viommu->vdev = vdev;
+ INIT_LIST_HEAD(&viommu->requests);
+
+ ret = viommu_init_vqs(viommu);
+ if (ret)
+ return ret;
+
+ virtio_cread(vdev, struct virtio_iommu_config, page_size_mask,
+ &viommu->pgsize_bitmap);
+
+ if (!viommu->pgsize_bitmap) {
+ ret = -EINVAL;
+ goto err_free_vqs;
+ }
+
+ viommu->domain_bits = 32;
+
+ /* Optional features */
+ virtio_cread_feature(vdev, VIRTIO_IOMMU_F_INPUT_RANGE,
+ struct virtio_iommu_config, input_range.start,
+ &input_start);
+
+ virtio_cread_feature(vdev, VIRTIO_IOMMU_F_INPUT_RANGE,
+ struct virtio_iommu_config, input_range.end,
+ &input_end);
+
+ virtio_cread_feature(vdev, VIRTIO_IOMMU_F_DOMAIN_BITS,
+ struct virtio_iommu_config, domain_bits,
+ &viommu->domain_bits);
+
+ virtio_cread_feature(vdev, VIRTIO_IOMMU_F_PROBE,
+ struct virtio_iommu_config, probe_size,
+ &viommu->probe_size);
+
+ viommu->geometry = (struct iommu_domain_geometry) {
+ .aperture_start = input_start,
+ .aperture_end = input_end,
+ .force_aperture = true,
+ };
+
+ viommu_ops.pgsize_bitmap = viommu->pgsize_bitmap;
+
+ virtio_device_ready(vdev);
+
+ /* Populate the event queue with buffers */
+ ret = viommu_fill_evtq(viommu);
+ if (ret)
+ goto err_free_vqs;
+
+ ret = iommu_device_sysfs_add(&viommu->iommu, dev, NULL, "%s",
+ virtio_bus_name(vdev));
+ if (ret)
+ goto err_free_vqs;
+
+ iommu_device_set_ops(&viommu->iommu, &viommu_ops);
+ iommu_device_set_fwnode(&viommu->iommu, parent_dev->fwnode);
+
+ iommu_device_register(&viommu->iommu);
+
+#ifdef CONFIG_PCI
+ if (pci_bus_type.iommu_ops != &viommu_ops) {
+ pci_request_acs();
+ ret = bus_set_iommu(&pci_bus_type, &viommu_ops);
+ if (ret)
+ goto err_unregister;
+ }
+#endif
+#ifdef CONFIG_ARM_AMBA
+ if (amba_bustype.iommu_ops != &viommu_ops) {
+ ret = bus_set_iommu(&amba_bustype, &viommu_ops);
+ if (ret)
+ goto err_unregister;
+ }
+#endif
+ if (platform_bus_type.iommu_ops != &viommu_ops) {
+ ret = bus_set_iommu(&platform_bus_type, &viommu_ops);
+ if (ret)
+ goto err_unregister;
+ }
+
+ vdev->priv = viommu;
+
+ dev_info(dev, "input address: %u bits\n",
+ order_base_2(viommu->geometry.aperture_end));
+ dev_info(dev, "page mask: %#llx\n", viommu->pgsize_bitmap);
+
+ return 0;
+
+err_unregister:
+ iommu_device_sysfs_remove(&viommu->iommu);
+ iommu_device_unregister(&viommu->iommu);
+err_free_vqs:
+ vdev->config->del_vqs(vdev);
+
+ return ret;
+}
+
+static void viommu_remove(struct virtio_device *vdev)
+{
+ struct viommu_dev *viommu = vdev->priv;
+
+ iommu_device_sysfs_remove(&viommu->iommu);
+ iommu_device_unregister(&viommu->iommu);
+
+ /* Stop all virtqueues */
+ vdev->config->reset(vdev);
+ vdev->config->del_vqs(vdev);
+
+ dev_info(&vdev->dev, "device removed\n");
+}
+
+static void viommu_config_changed(struct virtio_device *vdev)
+{
+ dev_warn(&vdev->dev, "config changed\n");
+}
+
+static unsigned int features[] = {
+ VIRTIO_IOMMU_F_MAP_UNMAP,
+ VIRTIO_IOMMU_F_DOMAIN_BITS,
+ VIRTIO_IOMMU_F_INPUT_RANGE,
+ VIRTIO_IOMMU_F_PROBE,
+};
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_IOMMU, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+static struct virtio_driver virtio_iommu_drv = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .feature_table = features,
+ .feature_table_size = ARRAY_SIZE(features),
+ .probe = viommu_probe,
+ .remove = viommu_remove,
+ .config_changed = viommu_config_changed,
+};
+
+module_virtio_driver(virtio_iommu_drv);
+
+MODULE_DESCRIPTION("Virtio IOMMU driver");
+MODULE_AUTHOR("Jean-Philippe Brucker <jean-philippe.brucker@arm.com>");
+MODULE_LICENSE("GPL v2");
/* get endpoint base */
idx = ((ep_addr & 0x7f) - 1) * 2;
+ if (idx > 15)
+ return -EIO;
+
if (ep_addr & 0x80)
idx++;
attr = ep->desc.bmAttributes;
Enable "dm-mod.create=" parameter to create mapped devices at init time.
This option is useful to allow mounting rootfs without requiring an
initramfs.
- See Documentation/device-mapper/dm-init.rst for dm-mod.create="..."
+ See Documentation/admin-guide/device-mapper/dm-init.rst for dm-mod.create="..."
format.
If unsure, say N.
* Format: dm-mod.create=<name>,<uuid>,<minor>,<flags>,<table>[,<table>+][;<name>,<uuid>,<minor>,<flags>,<table>[,<table>+]+]
* Table format: <start_sector> <num_sectors> <target_type> <target_args>
*
- * See Documentation/device-mapper/dm-init.rst for dm-mod.create="..." format
+ * See Documentation/admin-guide/device-mapper/dm-init.rst for dm-mod.create="..." format
* details.
*/
#include "dm-core.h"
-#define SUB_JOB_SIZE 128
#define SPLIT_COUNT 8
#define MIN_JOBS 8
-#define RESERVE_PAGES (DIV_ROUND_UP(SUB_JOB_SIZE << SECTOR_SHIFT, PAGE_SIZE))
+
+#define DEFAULT_SUB_JOB_SIZE_KB 512
+#define MAX_SUB_JOB_SIZE_KB 1024
+
+static unsigned kcopyd_subjob_size_kb = DEFAULT_SUB_JOB_SIZE_KB;
+
+module_param(kcopyd_subjob_size_kb, uint, S_IRUGO | S_IWUSR);
+MODULE_PARM_DESC(kcopyd_subjob_size_kb, "Sub-job size for dm-kcopyd clients");
+
+static unsigned dm_get_kcopyd_subjob_size(void)
+{
+ unsigned sub_job_size_kb;
+
+ sub_job_size_kb = __dm_get_module_param(&kcopyd_subjob_size_kb,
+ DEFAULT_SUB_JOB_SIZE_KB,
+ MAX_SUB_JOB_SIZE_KB);
+
+ return sub_job_size_kb << 1;
+}
/*-----------------------------------------------------------------
* Each kcopyd client has its own little pool of preallocated
struct page_list *pages;
unsigned nr_reserved_pages;
unsigned nr_free_pages;
+ unsigned sub_job_size;
struct dm_io_client *io_client;
progress = job->progress;
count = job->source.count - progress;
if (count) {
- if (count > SUB_JOB_SIZE)
- count = SUB_JOB_SIZE;
+ if (count > kc->sub_job_size)
+ count = kc->sub_job_size;
job->progress += count;
}
job->master_job = job;
job->write_offset = 0;
- if (job->source.count <= SUB_JOB_SIZE)
+ if (job->source.count <= kc->sub_job_size)
dispatch_job(job);
else {
job->progress = 0;
struct dm_kcopyd_client *dm_kcopyd_client_create(struct dm_kcopyd_throttle *throttle)
{
int r;
+ unsigned reserve_pages;
struct dm_kcopyd_client *kc;
kc = kzalloc(sizeof(*kc), GFP_KERNEL);
goto bad_workqueue;
}
+ kc->sub_job_size = dm_get_kcopyd_subjob_size();
+ reserve_pages = DIV_ROUND_UP(kc->sub_job_size << SECTOR_SHIFT, PAGE_SIZE);
+
kc->pages = NULL;
kc->nr_reserved_pages = kc->nr_free_pages = 0;
- r = client_reserve_pages(kc, RESERVE_PAGES);
+ r = client_reserve_pages(kc, reserve_pages);
if (r)
goto bad_client_pages;
* v1.5.0+:
*
* Sync action:
- * See Documentation/device-mapper/dm-raid.rst for
+ * See Documentation/admin-guide/device-mapper/dm-raid.rst for
* information on each of these states.
*/
DMEMIT(" %s", sync_action);
return DM_MAPIO_REMAPPED;
}
+ if (unlikely(bio_op(bio) == REQ_OP_DISCARD)) {
+ /* Once merging, discards no longer effect change */
+ bio_endio(bio);
+ return DM_MAPIO_SUBMITTED;
+ }
+
chunk = sector_to_chunk(s->store, bio->bi_iter.bi_sector);
down_write(&s->lock);
if (snap->discard_zeroes_cow) {
struct dm_snapshot *snap_src = NULL, *snap_dest = NULL;
+ down_read(&_origins_lock);
+
(void) __find_snapshots_sharing_cow(snap, &snap_src, &snap_dest, NULL);
if (snap_src && snap_dest)
snap = snap_src;
/* All discards are split on chunk_size boundary */
limits->discard_granularity = snap->store->chunk_size;
limits->max_discard_sectors = snap->store->chunk_size;
+
+ up_read(&_origins_lock);
}
}
EXPORT_SYMBOL_GPL(dm_table_set_type);
/* validate the dax capability of the target device span */
-static int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
+int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
sector_t start, sector_t len, void *data)
{
int blocksize = *(int *) data;
start, len);
}
-bool dm_table_supports_dax(struct dm_table *t, int blocksize)
+/* Check devices support synchronous DAX */
+static int device_synchronous(struct dm_target *ti, struct dm_dev *dev,
+ sector_t start, sector_t len, void *data)
+{
+ return dax_synchronous(dev->dax_dev);
+}
+
+bool dm_table_supports_dax(struct dm_table *t,
+ iterate_devices_callout_fn iterate_fn, int *blocksize)
{
struct dm_target *ti;
unsigned i;
return false;
if (!ti->type->iterate_devices ||
- !ti->type->iterate_devices(ti, device_supports_dax,
- &blocksize))
+ !ti->type->iterate_devices(ti, iterate_fn, blocksize))
return false;
}
struct dm_target *tgt;
struct list_head *devices = dm_table_get_devices(t);
enum dm_queue_mode live_md_type = dm_get_md_type(t->md);
+ int page_size = PAGE_SIZE;
if (t->type != DM_TYPE_NONE) {
/* target already set the table's type */
verify_bio_based:
/* We must use this table as bio-based */
t->type = DM_TYPE_BIO_BASED;
- if (dm_table_supports_dax(t, PAGE_SIZE) ||
+ if (dm_table_supports_dax(t, device_supports_dax, &page_size) ||
(list_empty(devices) && live_md_type == DM_TYPE_DAX_BIO_BASED)) {
t->type = DM_TYPE_DAX_BIO_BASED;
} else {
struct queue_limits *limits)
{
bool wc = false, fua = false;
+ int page_size = PAGE_SIZE;
/*
* Copy table's limits to the DM device's request_queue
}
blk_queue_write_cache(q, wc, fua);
- if (dm_table_supports_dax(t, PAGE_SIZE))
+ if (dm_table_supports_dax(t, device_supports_dax, &page_size)) {
blk_queue_flag_set(QUEUE_FLAG_DAX, q);
+ if (dm_table_supports_dax(t, device_synchronous, NULL))
+ set_dax_synchronous(t->md->dax_dev);
+ }
else
blk_queue_flag_clear(QUEUE_FLAG_DAX, q);
}
/*
- * Activate a zone (increment its reference count).
- */
-void dmz_activate_zone(struct dm_zone *zone)
-{
- set_bit(DMZ_ACTIVE, &zone->flags);
- atomic_inc(&zone->refcount);
-}
-
-/*
- * Deactivate a zone. This decrement the zone reference counter
- * and clears the active state of the zone once the count reaches 0,
- * indicating that all BIOs to the zone have completed. Returns
- * true if the zone was deactivated.
- */
-void dmz_deactivate_zone(struct dm_zone *zone)
-{
- if (atomic_dec_and_test(&zone->refcount)) {
- WARN_ON(!test_bit(DMZ_ACTIVE, &zone->flags));
- clear_bit_unlock(DMZ_ACTIVE, &zone->flags);
- smp_mb__after_atomic();
- }
-}
-
-/*
* Get the zone mapping a chunk, if the chunk is mapped already.
* If no mapping exist and the operation is WRITE, a zone is
* allocated and used to map the chunk.
DMZ_BUF,
/* Zone internal state */
- DMZ_ACTIVE,
DMZ_RECLAIM,
DMZ_SEQ_WRITE_ERR,
};
#define dmz_is_empty(z) ((z)->wp_block == 0)
#define dmz_is_offline(z) test_bit(DMZ_OFFLINE, &(z)->flags)
#define dmz_is_readonly(z) test_bit(DMZ_READ_ONLY, &(z)->flags)
-#define dmz_is_active(z) test_bit(DMZ_ACTIVE, &(z)->flags)
#define dmz_in_reclaim(z) test_bit(DMZ_RECLAIM, &(z)->flags)
#define dmz_seq_write_err(z) test_bit(DMZ_SEQ_WRITE_ERR, &(z)->flags)
unsigned int dmz_nr_rnd_zones(struct dmz_metadata *zmd);
unsigned int dmz_nr_unmap_rnd_zones(struct dmz_metadata *zmd);
-void dmz_activate_zone(struct dm_zone *zone);
-void dmz_deactivate_zone(struct dm_zone *zone);
+/*
+ * Activate a zone (increment its reference count).
+ */
+static inline void dmz_activate_zone(struct dm_zone *zone)
+{
+ atomic_inc(&zone->refcount);
+}
+
+/*
+ * Deactivate a zone. This decrement the zone reference counter
+ * indicating that all BIOs to the zone have completed when the count is 0.
+ */
+static inline void dmz_deactivate_zone(struct dm_zone *zone)
+{
+ atomic_dec(&zone->refcount);
+}
+
+/*
+ * Test if a zone is active, that is, has a refcount > 0.
+ */
+static inline bool dmz_is_active(struct dm_zone *zone)
+{
+ return atomic_read(&zone->refcount);
+}
int dmz_lock_zone_reclaim(struct dm_zone *zone);
void dmz_unlock_zone_reclaim(struct dm_zone *zone);
if (!map)
return false;
- ret = dm_table_supports_dax(map, blocksize);
+ ret = dm_table_supports_dax(map, device_supports_dax, &blocksize);
dm_put_live_table(md, srcu_idx);
sprintf(md->disk->disk_name, "dm-%d", minor);
if (IS_ENABLED(CONFIG_DAX_DRIVER)) {
- md->dax_dev = alloc_dax(md, md->disk->disk_name, &dm_dax_ops);
+ md->dax_dev = alloc_dax(md, md->disk->disk_name,
+ &dm_dax_ops, 0);
if (!md->dax_dev)
goto bad;
}
bool dm_table_request_based(struct dm_table *t);
void dm_table_free_md_mempools(struct dm_table *t);
struct dm_md_mempools *dm_table_get_md_mempools(struct dm_table *t);
-bool dm_table_supports_dax(struct dm_table *t, int blocksize);
+bool dm_table_supports_dax(struct dm_table *t, iterate_devices_callout_fn fn,
+ int *blocksize);
+int device_supports_dax(struct dm_target *ti, struct dm_dev *dev,
+ sector_t start, sector_t len, void *data);
void dm_lock_md_type(struct mapped_device *md);
void dm_unlock_md_type(struct mapped_device *md);
*
* Return: The number of unique NEMC banks referred to by the specified NEMC
* child device. Unique here means that a device that references the same bank
- * multiple times in the its "reg" property will only count once.
+ * multiple times in its "reg" property will only count once.
*/
unsigned int jz4780_nemc_num_banks(struct device *dev)
{
#include <misc/cxl.h>
#include <linux/module.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/sched/mm.h>
#include <linux/mmu_context.h>
static int cxl_fs_cnt;
static struct vfsmount *cxl_vfs_mount;
-static const struct dentry_operations cxl_fs_dops = {
- .d_dname = simple_dname,
-};
-
-static struct dentry *cxl_fs_mount(struct file_system_type *fs_type, int flags,
- const char *dev_name, void *data)
+static int cxl_fs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "cxl:", NULL, &cxl_fs_dops,
- CXL_PSEUDO_FS_MAGIC);
+ return init_pseudo(fc, CXL_PSEUDO_FS_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type cxl_fs_type = {
.name = "cxl",
.owner = THIS_MODULE,
- .mount = cxl_fs_mount,
+ .init_fs_context = cxl_fs_init_fs_context,
.kill_sb = kill_anon_super,
};
*/
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/pagemap.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
static struct inode *ibmasmfs_make_inode(struct super_block *sb, int mode);
static void ibmasmfs_create_files (struct super_block *sb);
-static int ibmasmfs_fill_super (struct super_block *sb, void *data, int silent);
+static int ibmasmfs_fill_super(struct super_block *sb, struct fs_context *fc);
+static int ibmasmfs_get_tree(struct fs_context *fc)
+{
+ return get_tree_single(fc, ibmasmfs_fill_super);
+}
-static struct dentry *ibmasmfs_mount(struct file_system_type *fst,
- int flags, const char *name, void *data)
+static const struct fs_context_operations ibmasmfs_context_ops = {
+ .get_tree = ibmasmfs_get_tree,
+};
+
+static int ibmasmfs_init_fs_context(struct fs_context *fc)
{
- return mount_single(fst, flags, data, ibmasmfs_fill_super);
+ fc->ops = &ibmasmfs_context_ops;
+ return 0;
}
static const struct super_operations ibmasmfs_s_ops = {
static struct file_system_type ibmasmfs_type = {
.owner = THIS_MODULE,
.name = "ibmasmfs",
- .mount = ibmasmfs_mount,
+ .init_fs_context = ibmasmfs_init_fs_context,
.kill_sb = kill_litter_super,
};
MODULE_ALIAS_FS("ibmasmfs");
-static int ibmasmfs_fill_super (struct super_block *sb, void *data, int silent)
+static int ibmasmfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *root;
{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA74x) },
{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA72x) },
{ PCI_DEVICE(PCI_VENDOR_ID_FREESCALE, 0x81c0) },
- { PCI_DEVICE(PCI_VENDOR_ID_SYNOPSYS, 0xedda) },
+ { PCI_DEVICE_DATA(SYNOPSYS, EDDA, NULL) },
{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_AM654),
.driver_data = (kernel_ulong_t)&am654_data
},
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/balloon_compaction.h>
#include <linux/vmw_vmci_defs.h>
#include <linux/vmw_vmci_api.h>
#ifdef CONFIG_BALLOON_COMPACTION
-static struct dentry *vmballoon_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name,
- void *data)
+static int vmballoon_init_fs_context(struct fs_context *fc)
{
- static const struct dentry_operations ops = {
- .d_dname = simple_dname,
- };
-
- return mount_pseudo(fs_type, "balloon-vmware:", NULL, &ops,
- BALLOON_VMW_MAGIC);
+ return init_pseudo(fc, BALLOON_VMW_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type vmballoon_fs = {
- .name = "balloon-vmware",
- .mount = vmballoon_mount,
- .kill_sb = kill_anon_super,
+ .name = "balloon-vmware",
+ .init_fs_context = vmballoon_init_fs_context,
+ .kill_sb = kill_anon_super,
};
static struct vfsmount *vmballoon_mnt;
* Thomas Gleixner (tglx@linutronix.de)
*
* Information on how this algorithm works and how it was developed
- * can be found in Documentation/mtd/nand_ecc.txt
+ * can be found in Documentation/driver-api/mtd/nand_ecc.rst
*/
#include <linux/types.h>
rtnl_lock();
list_for_each_safe(list_node, n, &cfhsi_list) {
cfhsi = list_entry(list_node, struct cfhsi, list);
- unregister_netdev(cfhsi->ndev);
+ unregister_netdevice(cfhsi->ndev);
}
rtnl_unlock();
}
#include <linux/regmap.h>
#include <linux/reset.h>
#include <linux/clk.h>
+#include <linux/io.h>
/* For our NAPI weight bigger does *NOT* mean better - it means more
* D-cache misses and lots more wasted cycles than we'll ever
ag->stop_desc = dmam_alloc_coherent(&pdev->dev,
sizeof(struct ag71xx_desc),
&ag->stop_desc_dma, GFP_KERNEL);
- if (!ag->stop_desc)
+ if (!ag->stop_desc) {
+ err = -ENOMEM;
goto err_free;
+ }
ag->stop_desc->data = 0;
ag->stop_desc->ctrl = 0;
ag->stop_desc->next = (u32)ag->stop_desc_dma;
mac_addr = of_get_mac_address(np);
- if (mac_addr)
+ if (!IS_ERR(mac_addr))
memcpy(ndev->dev_addr, mac_addr, ETH_ALEN);
- if (!mac_addr || !is_valid_ether_addr(ndev->dev_addr)) {
+ if (IS_ERR(mac_addr) || !is_valid_ether_addr(ndev->dev_addr)) {
netif_err(ag, probe, ndev, "invalid MAC address, using random address\n");
eth_random_addr(ndev->dev_addr);
}
goto err_nomem;
}
- memset(ring_header->desc, 0, ring_header->size);
-
/* init TPD ring */
tpd_ring->dma = ring_header->dma;
offset = (tpd_ring->dma & 0x7) ? (8 - (ring_header->dma & 0x7)) : 0;
&adapter->ring_dma);
if (!adapter->ring_vir_addr)
return -ENOMEM;
- memset(adapter->ring_vir_addr, 0, adapter->ring_size);
/* Init TXD Ring */
adapter->txd_dma = adapter->ring_dma ;
mapping = txr->tx_push_mapping +
sizeof(struct tx_push_bd);
txr->data_mapping = cpu_to_le64(mapping);
-
- memset(txr->tx_push, 0, sizeof(struct tx_push_bd));
}
qidx = bp->tc_to_qidx[j];
ring->queue_id = bp->q_info[qidx].queue_id;
int num_vnics = 1;
#ifdef CONFIG_RFS_ACCEL
- if (bp->flags & BNXT_FLAG_RFS)
+ if ((bp->flags & (BNXT_FLAG_RFS | BNXT_FLAG_CHIP_P5)) == BNXT_FLAG_RFS)
num_vnics += bp->rx_nr_rings;
#endif
#ifdef CONFIG_RFS_ACCEL
int i, rc = 0;
+ if (bp->flags & BNXT_FLAG_CHIP_P5)
+ return 0;
+
for (i = 0; i < bp->rx_nr_rings; i++) {
struct bnxt_vnic_info *vnic;
u16 vnic_id = i + 1;
return -ENOMEM;
vnics = 1;
- if (bp->flags & BNXT_FLAG_RFS)
+ if ((bp->flags & (BNXT_FLAG_RFS | BNXT_FLAG_CHIP_P5)) == BNXT_FLAG_RFS)
vnics += rx_rings;
if (bp->flags & BNXT_FLAG_AGG_RINGS)
netif_tx_wake_all_queues(dev);
}
-#define MAX_MC_COUNT 16
+#define MAX_MDF_FILTER 17
static inline void bcmgenet_set_mdf_addr(struct bcmgenet_priv *priv,
unsigned char *addr,
- int *i,
- int *mc)
+ int *i)
{
- u32 reg;
-
bcmgenet_umac_writel(priv, addr[0] << 8 | addr[1],
UMAC_MDF_ADDR + (*i * 4));
bcmgenet_umac_writel(priv, addr[2] << 24 | addr[3] << 16 |
addr[4] << 8 | addr[5],
UMAC_MDF_ADDR + ((*i + 1) * 4));
- reg = bcmgenet_umac_readl(priv, UMAC_MDF_CTRL);
- reg |= (1 << (MAX_MC_COUNT - *mc));
- bcmgenet_umac_writel(priv, reg, UMAC_MDF_CTRL);
*i += 2;
- (*mc)++;
}
static void bcmgenet_set_rx_mode(struct net_device *dev)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
struct netdev_hw_addr *ha;
- int i, mc;
+ int i, nfilter;
u32 reg;
netif_dbg(priv, hw, dev, "%s: %08X\n", __func__, dev->flags);
- /* Promiscuous mode */
+ /* Number of filters needed */
+ nfilter = netdev_uc_count(dev) + netdev_mc_count(dev) + 2;
+
+ /*
+ * Turn on promicuous mode for three scenarios
+ * 1. IFF_PROMISC flag is set
+ * 2. IFF_ALLMULTI flag is set
+ * 3. The number of filters needed exceeds the number filters
+ * supported by the hardware.
+ */
reg = bcmgenet_umac_readl(priv, UMAC_CMD);
- if (dev->flags & IFF_PROMISC) {
+ if ((dev->flags & (IFF_PROMISC | IFF_ALLMULTI)) ||
+ (nfilter > MAX_MDF_FILTER)) {
reg |= CMD_PROMISC;
bcmgenet_umac_writel(priv, reg, UMAC_CMD);
bcmgenet_umac_writel(priv, 0, UMAC_MDF_CTRL);
bcmgenet_umac_writel(priv, reg, UMAC_CMD);
}
- /* UniMac doesn't support ALLMULTI */
- if (dev->flags & IFF_ALLMULTI) {
- netdev_warn(dev, "ALLMULTI is not supported\n");
- return;
- }
-
/* update MDF filter */
i = 0;
- mc = 0;
/* Broadcast */
- bcmgenet_set_mdf_addr(priv, dev->broadcast, &i, &mc);
+ bcmgenet_set_mdf_addr(priv, dev->broadcast, &i);
/* my own address.*/
- bcmgenet_set_mdf_addr(priv, dev->dev_addr, &i, &mc);
- /* Unicast list*/
- if (netdev_uc_count(dev) > (MAX_MC_COUNT - mc))
- return;
+ bcmgenet_set_mdf_addr(priv, dev->dev_addr, &i);
- if (!netdev_uc_empty(dev))
- netdev_for_each_uc_addr(ha, dev)
- bcmgenet_set_mdf_addr(priv, ha->addr, &i, &mc);
- /* Multicast */
- if (netdev_mc_empty(dev) || netdev_mc_count(dev) >= (MAX_MC_COUNT - mc))
- return;
+ /* Unicast */
+ netdev_for_each_uc_addr(ha, dev)
+ bcmgenet_set_mdf_addr(priv, ha->addr, &i);
+ /* Multicast */
netdev_for_each_mc_addr(ha, dev)
- bcmgenet_set_mdf_addr(priv, ha->addr, &i, &mc);
+ bcmgenet_set_mdf_addr(priv, ha->addr, &i);
+
+ /* Enable filters */
+ reg = GENMASK(MAX_MDF_FILTER - 1, MAX_MDF_FILTER - nfilter);
+ bcmgenet_umac_writel(priv, reg, UMAC_MDF_CTRL);
}
/* Set the hardware MAC address. */
return 0;
}
oct->instr_queue[iq_no] =
- vmalloc_node(sizeof(struct octeon_instr_queue), numa_node);
+ vzalloc_node(sizeof(struct octeon_instr_queue), numa_node);
if (!oct->instr_queue[iq_no])
oct->instr_queue[iq_no] =
- vmalloc(sizeof(struct octeon_instr_queue));
+ vzalloc(sizeof(struct octeon_instr_queue));
if (!oct->instr_queue[iq_no])
return 1;
- memset(oct->instr_queue[iq_no], 0,
- sizeof(struct octeon_instr_queue));
oct->instr_queue[iq_no]->q_index = q_index;
oct->instr_queue[iq_no]->app_ctx = app_ctx;
goto out_err;
/* Bind queue to specified class */
- memset(qe, 0, sizeof(*qe));
qe->cntxt_id = qid;
memcpy(&qe->param, p, sizeof(qe->param));
struct net_device *netdev = adapter->netdev;
int status;
- if (netif_running(netdev))
+ if (netif_running(netdev)) {
+ /* device cannot transmit now, avoid dev_watchdog timeouts */
+ netif_carrier_off(netdev);
+
be_close(netdev);
+ }
be_cancel_worker(adapter);
return -ENOMEM;
}
- memset(cbd_base, 0, bd_size);
-
/* Get the Ethernet address */
fec_get_mac(ndev);
/* make sure MAC we just acquired is programmed into the hw */
abort_with_msix_enabled:
pci_disable_msix(priv->pdev);
abort_with_msix_vectors:
- kfree(priv->msix_vectors);
+ kvfree(priv->msix_vectors);
priv->msix_vectors = NULL;
return err;
}
priv->ntfy_blocks = NULL;
free_irq(priv->msix_vectors[priv->mgmt_msix_idx].vector, priv);
pci_disable_msix(priv->pdev);
- kfree(priv->msix_vectors);
+ kvfree(priv->msix_vectors);
priv->msix_vectors = NULL;
}
return 0;
free_rx:
- kfree(priv->rx);
+ kvfree(priv->rx);
priv->rx = NULL;
free_tx_queue:
gve_tx_free_rings(priv);
free_tx:
- kfree(priv->tx);
+ kvfree(priv->tx);
priv->tx = NULL;
return err;
}
gve_remove_napi(priv, ntfy_idx);
}
gve_tx_free_rings(priv);
- kfree(priv->tx);
+ kvfree(priv->tx);
priv->tx = NULL;
}
if (priv->rx) {
gve_remove_napi(priv, ntfy_idx);
}
gve_rx_free_rings(priv);
- kfree(priv->rx);
+ kvfree(priv->rx);
priv->rx = NULL;
}
}
gve_free_page(&priv->pdev->dev, qpl->pages[i],
qpl->page_buses[i], gve_qpl_dma_dir(priv, id));
- kfree(qpl->page_buses);
+ kvfree(qpl->page_buses);
free_pages:
- kfree(qpl->pages);
+ kvfree(qpl->pages);
priv->num_registered_pages -= qpl->num_entries;
}
free_qpls:
for (j = 0; j <= i; j++)
gve_free_queue_page_list(priv, j);
- kfree(priv->qpls);
+ kvfree(priv->qpls);
return err;
}
int num_qpls = gve_num_tx_qpls(priv) + gve_num_rx_qpls(priv);
int i;
- kfree(priv->qpl_cfg.qpl_id_map);
+ kvfree(priv->qpl_cfg.qpl_id_map);
for (i = 0; i < num_qpls; i++)
gve_free_queue_page_list(priv, i);
- kfree(priv->qpls);
+ kvfree(priv->qpls);
}
/* Use this to schedule a reset when the device is capable of continuing
pci_disable_device(pdev);
return -ENXIO;
}
-EXPORT_SYMBOL(gve_probe);
static void gve_remove(struct pci_dev *pdev)
{
gve_unassign_qpl(priv, rx->data.qpl->id);
rx->data.qpl = NULL;
- kfree(rx->data.page_info);
+ kvfree(rx->data.page_info);
slots = rx->data.mask + 1;
bytes = sizeof(*rx->data.data_ring) * slots;
rx->q_resources, rx->q_resources_bus);
rx->q_resources = NULL;
abort_filled:
- kfree(rx->data.page_info);
+ kvfree(rx->data.page_info);
abort_with_slots:
bytes = sizeof(*rx->data.data_ring) * slots;
dma_free_coherent(hdev, bytes, rx->data.data_ring, rx->data.data_bus);
struct ice_hw *hw;
int err;
- /* this driver uses devres, see Documentation/driver-model/devres.rst */
+ /* this driver uses devres, see Documentation/driver-api/driver-model/devres.rst */
err = pcim_enable_device(pdev);
if (err)
return err;
if (unlikely(!(txring->bufinf)))
goto err_free_txring;
- /*
- * Initialize Transmit Descriptors
- */
- memset(txring->alloc, 0, TX_RING_ALLOC_SIZE(jme->tx_ring_size));
-
return 0;
err_free_txring:
goto free_pci_mem;
}
- memset(skge->mem, 0, skge->mem_size);
-
err = skge_ring_alloc(&skge->rx_ring, skge->mem, skge->dma);
if (err)
goto free_pci_mem;
DMI_MATCH(DMI_PRODUCT_NAME, "P-79"),
},
},
+ {
+ .ident = "ASUS P5W DH Deluxe",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTEK COMPUTER INC"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "P5W DH Deluxe"),
+ },
+ },
{}
};
continue;
err = mtk_add_mac(eth, mac_np);
- if (err)
+ if (err) {
+ of_node_put(mac_np);
goto err_deinit_hw;
+ }
}
if (MTK_HAS_CAPS(eth->soc->caps, MTK_SHARED_INT)) {
dma_list[i] = t;
eq->page_list[i].map = t;
-
- memset(eq->page_list[i].buf, 0, PAGE_SIZE);
}
eq->eqn = mlx4_bitmap_alloc(&priv->eq_table.bitmap);
*match_level = MLX5_MATCH_NONE;
if (dissector->used_keys &
- ~(BIT(FLOW_DISSECTOR_KEY_CONTROL) |
+ ~(BIT(FLOW_DISSECTOR_KEY_META) |
+ BIT(FLOW_DISSECTOR_KEY_CONTROL) |
BIT(FLOW_DISSECTOR_KEY_BASIC) |
BIT(FLOW_DISSECTOR_KEY_ETH_ADDRS) |
BIT(FLOW_DISSECTOR_KEY_VLAN) |
return -EOPNOTSUPP;
}
- if (flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_IPV4_ADDRS) ||
- flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_IPV6_ADDRS) ||
- flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_KEYID) ||
- flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_PORTS) ||
- flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_ENC_OPTS)) {
+ if (mlx5e_get_tc_tun(filter_dev)) {
if (parse_tunnel_attr(priv, spec, f, filter_dev, tunnel_match_level))
return -EOPNOTSUPP;
family = ip_tunnel_info_af(tun_info);
key.ip_tun_key = &tun_info->key;
key.tc_tunnel = mlx5e_get_tc_tun(mirred_dev);
+ if (!key.tc_tunnel) {
+ NL_SET_ERR_MSG_MOD(extack, "Unsupported tunnel");
+ return -EOPNOTSUPP;
+ }
hash_key = hash_encap_info(&key);
MLX5_SET(query_vport_counter_in, in, vport_number, vport->vport);
MLX5_SET(query_vport_counter_in, in, other_vport, 1);
- memset(out, 0, outlen);
err = mlx5_cmd_exec(esw->dev, in, sizeof(in), out, outlen);
if (err)
goto free_out;
}
/* create send-to-vport group */
- memset(flow_group_in, 0, inlen);
MLX5_SET(create_flow_group_in, flow_group_in, match_criteria_enable,
MLX5_MATCH_MISC_PARAMETERS);
return -ENOMEM;
/* create vport rx group */
- memset(flow_group_in, 0, inlen);
-
esw_set_flow_group_source_port(esw, flow_group_in);
MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, 0);
err = devlink_fmsg_arr_pair_nest_end(fmsg);
free_data:
- kfree(cr_data);
+ kvfree(cr_data);
return err;
}
&mem_item->mapaddr);
if (!mem_item->buf)
return -ENOMEM;
- memset(mem_item->buf, 0, mem_item->size);
q->elem_info = kcalloc(q->count, sizeof(*q->elem_info), GFP_KERNEL);
if (!q->elem_info) {
struct tc_prio_qopt_offload *p);
/* spectrum_fid.c */
+bool mlxsw_sp_fid_is_dummy(struct mlxsw_sp *mlxsw_sp, u16 fid_index);
bool mlxsw_sp_fid_lag_vid_valid(const struct mlxsw_sp_fid *fid);
struct mlxsw_sp_fid *mlxsw_sp_fid_lookup_by_index(struct mlxsw_sp *mlxsw_sp,
u16 fid_index);
have_dscp = mlxsw_sp_port_dcb_app_prio_dscp_map(mlxsw_sp_port,
&prio_map);
- if (!have_dscp) {
- err = mlxsw_sp_port_dcb_toggle_trust(mlxsw_sp_port,
- MLXSW_REG_QPTS_TRUST_STATE_PCP);
- if (err)
- netdev_err(mlxsw_sp_port->dev, "Couldn't switch to trust L2\n");
- return err;
- }
-
mlxsw_sp_port_dcb_app_dscp_prio_map(mlxsw_sp_port, default_prio,
&dscp_map);
err = mlxsw_sp_port_dcb_app_update_qpdpm(mlxsw_sp_port,
return err;
}
+ if (!have_dscp) {
+ err = mlxsw_sp_port_dcb_toggle_trust(mlxsw_sp_port,
+ MLXSW_REG_QPTS_TRUST_STATE_PCP);
+ if (err)
+ netdev_err(mlxsw_sp_port->dev, "Couldn't switch to trust L2\n");
+ return err;
+ }
+
err = mlxsw_sp_port_dcb_toggle_trust(mlxsw_sp_port,
MLXSW_REG_QPTS_TRUST_STATE_DSCP);
if (err) {
[MLXSW_SP_FLOOD_TYPE_MC] = mlxsw_sp_sfgc_mc_packet_types,
};
+bool mlxsw_sp_fid_is_dummy(struct mlxsw_sp *mlxsw_sp, u16 fid_index)
+{
+ enum mlxsw_sp_fid_type fid_type = MLXSW_SP_FID_TYPE_DUMMY;
+ struct mlxsw_sp_fid_family *fid_family;
+
+ fid_family = mlxsw_sp->fid_core->fid_family_arr[fid_type];
+
+ return fid_family->start_index == fid_index;
+}
+
bool mlxsw_sp_fid_lag_vid_valid(const struct mlxsw_sp_fid *fid)
{
return fid->fid_family->lag_vid_valid;
goto just_remove;
}
+ if (mlxsw_sp_fid_is_dummy(mlxsw_sp, fid))
+ goto just_remove;
+
mlxsw_sp_port_vlan = mlxsw_sp_port_vlan_find_by_fid(mlxsw_sp_port, fid);
if (!mlxsw_sp_port_vlan) {
netdev_err(mlxsw_sp_port->dev, "Failed to find a matching {Port, VID} following FDB notification\n");
goto just_remove;
}
+ if (mlxsw_sp_fid_is_dummy(mlxsw_sp, fid))
+ goto just_remove;
+
mlxsw_sp_port_vlan = mlxsw_sp_port_vlan_find_by_fid(mlxsw_sp_port, fid);
if (!mlxsw_sp_port_vlan) {
netdev_err(mlxsw_sp_port->dev, "Failed to find a matching {Port, VID} following FDB notification\n");
continue;
err = ocelot_probe_port(ocelot, port, regs, phy);
- if (err)
+ if (err) {
+ of_node_put(portnp);
return err;
+ }
phy_mode = of_get_phy_mode(portnp);
if (phy_mode < 0)
dev_err(ocelot->dev,
"invalid phy mode for port%d, (Q)SGMII only\n",
port);
+ of_node_put(portnp);
return -EINVAL;
}
return -ENOMEM;
}
mem_allocated += size;
- memset(tmp_v_addr, 0, size);
size = sizeof(struct rxd_info) *
rxd_count[nic->rxd_mode];
goto out_free_rq;
}
- memset(rq_addr, 0, rq_size);
prq = rq_addr;
- memset(rsp_addr, 0, rsp_size);
prsp = rsp_addr;
prq->host_rsp_dma_addr = cpu_to_le64(rsp_phys_addr);
return -ENOMEM;
}
- memset(addr, 0, sizeof(struct netxen_ring_ctx));
recv_ctx->hwctx = addr;
recv_ctx->hwctx->ctx_id = cpu_to_le32(port);
recv_ctx->hwctx->cmd_consumer_offset =
/* disable aspm and clock request before access ephy */
rtl_hw_aspm_clkreq_enable(tp, false);
rtl_ephy_init(tp, e_info_8411_2);
+
+ /* The following Realtek-provided magic fixes an issue with the RX unit
+ * getting confused after the PHY having been powered-down.
+ */
+ r8168_mac_ocp_write(tp, 0xFC28, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC2A, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC2C, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC2E, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC30, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC32, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC34, 0x0000);
+ r8168_mac_ocp_write(tp, 0xFC36, 0x0000);
+ mdelay(3);
+ r8168_mac_ocp_write(tp, 0xFC26, 0x0000);
+
+ r8168_mac_ocp_write(tp, 0xF800, 0xE008);
+ r8168_mac_ocp_write(tp, 0xF802, 0xE00A);
+ r8168_mac_ocp_write(tp, 0xF804, 0xE00C);
+ r8168_mac_ocp_write(tp, 0xF806, 0xE00E);
+ r8168_mac_ocp_write(tp, 0xF808, 0xE027);
+ r8168_mac_ocp_write(tp, 0xF80A, 0xE04F);
+ r8168_mac_ocp_write(tp, 0xF80C, 0xE05E);
+ r8168_mac_ocp_write(tp, 0xF80E, 0xE065);
+ r8168_mac_ocp_write(tp, 0xF810, 0xC602);
+ r8168_mac_ocp_write(tp, 0xF812, 0xBE00);
+ r8168_mac_ocp_write(tp, 0xF814, 0x0000);
+ r8168_mac_ocp_write(tp, 0xF816, 0xC502);
+ r8168_mac_ocp_write(tp, 0xF818, 0xBD00);
+ r8168_mac_ocp_write(tp, 0xF81A, 0x074C);
+ r8168_mac_ocp_write(tp, 0xF81C, 0xC302);
+ r8168_mac_ocp_write(tp, 0xF81E, 0xBB00);
+ r8168_mac_ocp_write(tp, 0xF820, 0x080A);
+ r8168_mac_ocp_write(tp, 0xF822, 0x6420);
+ r8168_mac_ocp_write(tp, 0xF824, 0x48C2);
+ r8168_mac_ocp_write(tp, 0xF826, 0x8C20);
+ r8168_mac_ocp_write(tp, 0xF828, 0xC516);
+ r8168_mac_ocp_write(tp, 0xF82A, 0x64A4);
+ r8168_mac_ocp_write(tp, 0xF82C, 0x49C0);
+ r8168_mac_ocp_write(tp, 0xF82E, 0xF009);
+ r8168_mac_ocp_write(tp, 0xF830, 0x74A2);
+ r8168_mac_ocp_write(tp, 0xF832, 0x8CA5);
+ r8168_mac_ocp_write(tp, 0xF834, 0x74A0);
+ r8168_mac_ocp_write(tp, 0xF836, 0xC50E);
+ r8168_mac_ocp_write(tp, 0xF838, 0x9CA2);
+ r8168_mac_ocp_write(tp, 0xF83A, 0x1C11);
+ r8168_mac_ocp_write(tp, 0xF83C, 0x9CA0);
+ r8168_mac_ocp_write(tp, 0xF83E, 0xE006);
+ r8168_mac_ocp_write(tp, 0xF840, 0x74F8);
+ r8168_mac_ocp_write(tp, 0xF842, 0x48C4);
+ r8168_mac_ocp_write(tp, 0xF844, 0x8CF8);
+ r8168_mac_ocp_write(tp, 0xF846, 0xC404);
+ r8168_mac_ocp_write(tp, 0xF848, 0xBC00);
+ r8168_mac_ocp_write(tp, 0xF84A, 0xC403);
+ r8168_mac_ocp_write(tp, 0xF84C, 0xBC00);
+ r8168_mac_ocp_write(tp, 0xF84E, 0x0BF2);
+ r8168_mac_ocp_write(tp, 0xF850, 0x0C0A);
+ r8168_mac_ocp_write(tp, 0xF852, 0xE434);
+ r8168_mac_ocp_write(tp, 0xF854, 0xD3C0);
+ r8168_mac_ocp_write(tp, 0xF856, 0x49D9);
+ r8168_mac_ocp_write(tp, 0xF858, 0xF01F);
+ r8168_mac_ocp_write(tp, 0xF85A, 0xC526);
+ r8168_mac_ocp_write(tp, 0xF85C, 0x64A5);
+ r8168_mac_ocp_write(tp, 0xF85E, 0x1400);
+ r8168_mac_ocp_write(tp, 0xF860, 0xF007);
+ r8168_mac_ocp_write(tp, 0xF862, 0x0C01);
+ r8168_mac_ocp_write(tp, 0xF864, 0x8CA5);
+ r8168_mac_ocp_write(tp, 0xF866, 0x1C15);
+ r8168_mac_ocp_write(tp, 0xF868, 0xC51B);
+ r8168_mac_ocp_write(tp, 0xF86A, 0x9CA0);
+ r8168_mac_ocp_write(tp, 0xF86C, 0xE013);
+ r8168_mac_ocp_write(tp, 0xF86E, 0xC519);
+ r8168_mac_ocp_write(tp, 0xF870, 0x74A0);
+ r8168_mac_ocp_write(tp, 0xF872, 0x48C4);
+ r8168_mac_ocp_write(tp, 0xF874, 0x8CA0);
+ r8168_mac_ocp_write(tp, 0xF876, 0xC516);
+ r8168_mac_ocp_write(tp, 0xF878, 0x74A4);
+ r8168_mac_ocp_write(tp, 0xF87A, 0x48C8);
+ r8168_mac_ocp_write(tp, 0xF87C, 0x48CA);
+ r8168_mac_ocp_write(tp, 0xF87E, 0x9CA4);
+ r8168_mac_ocp_write(tp, 0xF880, 0xC512);
+ r8168_mac_ocp_write(tp, 0xF882, 0x1B00);
+ r8168_mac_ocp_write(tp, 0xF884, 0x9BA0);
+ r8168_mac_ocp_write(tp, 0xF886, 0x1B1C);
+ r8168_mac_ocp_write(tp, 0xF888, 0x483F);
+ r8168_mac_ocp_write(tp, 0xF88A, 0x9BA2);
+ r8168_mac_ocp_write(tp, 0xF88C, 0x1B04);
+ r8168_mac_ocp_write(tp, 0xF88E, 0xC508);
+ r8168_mac_ocp_write(tp, 0xF890, 0x9BA0);
+ r8168_mac_ocp_write(tp, 0xF892, 0xC505);
+ r8168_mac_ocp_write(tp, 0xF894, 0xBD00);
+ r8168_mac_ocp_write(tp, 0xF896, 0xC502);
+ r8168_mac_ocp_write(tp, 0xF898, 0xBD00);
+ r8168_mac_ocp_write(tp, 0xF89A, 0x0300);
+ r8168_mac_ocp_write(tp, 0xF89C, 0x051E);
+ r8168_mac_ocp_write(tp, 0xF89E, 0xE434);
+ r8168_mac_ocp_write(tp, 0xF8A0, 0xE018);
+ r8168_mac_ocp_write(tp, 0xF8A2, 0xE092);
+ r8168_mac_ocp_write(tp, 0xF8A4, 0xDE20);
+ r8168_mac_ocp_write(tp, 0xF8A6, 0xD3C0);
+ r8168_mac_ocp_write(tp, 0xF8A8, 0xC50F);
+ r8168_mac_ocp_write(tp, 0xF8AA, 0x76A4);
+ r8168_mac_ocp_write(tp, 0xF8AC, 0x49E3);
+ r8168_mac_ocp_write(tp, 0xF8AE, 0xF007);
+ r8168_mac_ocp_write(tp, 0xF8B0, 0x49C0);
+ r8168_mac_ocp_write(tp, 0xF8B2, 0xF103);
+ r8168_mac_ocp_write(tp, 0xF8B4, 0xC607);
+ r8168_mac_ocp_write(tp, 0xF8B6, 0xBE00);
+ r8168_mac_ocp_write(tp, 0xF8B8, 0xC606);
+ r8168_mac_ocp_write(tp, 0xF8BA, 0xBE00);
+ r8168_mac_ocp_write(tp, 0xF8BC, 0xC602);
+ r8168_mac_ocp_write(tp, 0xF8BE, 0xBE00);
+ r8168_mac_ocp_write(tp, 0xF8C0, 0x0C4C);
+ r8168_mac_ocp_write(tp, 0xF8C2, 0x0C28);
+ r8168_mac_ocp_write(tp, 0xF8C4, 0x0C2C);
+ r8168_mac_ocp_write(tp, 0xF8C6, 0xDC00);
+ r8168_mac_ocp_write(tp, 0xF8C8, 0xC707);
+ r8168_mac_ocp_write(tp, 0xF8CA, 0x1D00);
+ r8168_mac_ocp_write(tp, 0xF8CC, 0x8DE2);
+ r8168_mac_ocp_write(tp, 0xF8CE, 0x48C1);
+ r8168_mac_ocp_write(tp, 0xF8D0, 0xC502);
+ r8168_mac_ocp_write(tp, 0xF8D2, 0xBD00);
+ r8168_mac_ocp_write(tp, 0xF8D4, 0x00AA);
+ r8168_mac_ocp_write(tp, 0xF8D6, 0xE0C0);
+ r8168_mac_ocp_write(tp, 0xF8D8, 0xC502);
+ r8168_mac_ocp_write(tp, 0xF8DA, 0xBD00);
+ r8168_mac_ocp_write(tp, 0xF8DC, 0x0132);
+
+ r8168_mac_ocp_write(tp, 0xFC26, 0x8000);
+
+ r8168_mac_ocp_write(tp, 0xFC2A, 0x0743);
+ r8168_mac_ocp_write(tp, 0xFC2C, 0x0801);
+ r8168_mac_ocp_write(tp, 0xFC2E, 0x0BE9);
+ r8168_mac_ocp_write(tp, 0xFC30, 0x02FD);
+ r8168_mac_ocp_write(tp, 0xFC32, 0x0C25);
+ r8168_mac_ocp_write(tp, 0xFC34, 0x00A9);
+ r8168_mac_ocp_write(tp, 0xFC36, 0x012D);
+
rtl_hw_aspm_clkreq_enable(tp, true);
}
/* check to see if we have sane EEPROM */
signature = (u16) read_eeprom(ioaddr, EEPROMSignature);
if (signature == 0xffff || signature == 0x0000) {
- printk (KERN_WARNING "%s: Error EERPOM read %x\n",
+ printk (KERN_WARNING "%s: Error EEPROM read %x\n",
pci_name(pci_dev), signature);
return 0;
}
*
* SiS962 or SiS963 model, use EEPROM to store MAC address. And EEPROM
* is shared by
- * LAN and 1394. When access EEPROM, send EEREQ signal to hardware first
+ * LAN and 1394. When accessing EEPROM, send EEREQ signal to hardware first
* and wait for EEGNT. If EEGNT is ON, EEPROM is permitted to be accessed
- * by LAN, otherwise is not. After MAC address is read from EEPROM, send
+ * by LAN, otherwise it is not. After MAC address is read from EEPROM, send
* EEDONE signal to refuse EEPROM access by LAN.
* The EEPROM map of SiS962 or SiS963 is different to SiS900.
* The signature field in SiS962 or SiS963 spec is meaningless.
ret = PTR_ERR(slave_data->ifphy);
dev_err(&pdev->dev,
"%d: Error retrieving port phy: %d\n", i, ret);
- return ret;
+ goto err_node_put;
}
slave_data->slave_node = slave_node;
if (ret) {
if (ret != -EPROBE_DEFER)
dev_err(&pdev->dev, "failed to register fixed-link phy: %d\n", ret);
- return ret;
+ goto err_node_put;
}
slave_data->phy_node = of_node_get(slave_node);
} else if (parp) {
of_node_put(mdio_node);
if (!mdio) {
dev_err(&pdev->dev, "Missing mdio platform device\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto err_node_put;
}
snprintf(slave_data->phy_id, sizeof(slave_data->phy_id),
PHY_ID_FMT, mdio->name, phyid);
if (slave_data->phy_if < 0) {
dev_err(&pdev->dev, "Missing or malformed slave[%d] phy-mode property\n",
i);
- return slave_data->phy_if;
+ ret = slave_data->phy_if;
+ goto err_node_put;
}
no_phy_slave:
ret = ti_cm_get_macid(&pdev->dev, i,
slave_data->mac_addr);
if (ret)
- return ret;
+ goto err_node_put;
}
if (data->dual_emac) {
if (of_property_read_u32(slave_node, "dual_emac_res_vlan",
}
i++;
- if (i == data->slaves)
- break;
+ if (i == data->slaves) {
+ ret = 0;
+ goto err_node_put;
+ }
}
return 0;
+
+err_node_put:
+ of_node_put(slave_node);
+ return ret;
}
static void cpsw_remove_dt(struct platform_device *pdev)
of_node_put(slave_data->phy_node);
i++;
- if (i == data->slaves)
+ if (i == data->slaves) {
+ of_node_put(slave_node);
break;
+ }
}
of_platform_depopulate(&pdev->dev);
dev->name);
return -ENOMEM;
}
- memset(priv->dma_storage, 0, dma_size);
priv->rx_list = (struct tlan_list *)
ALIGN((unsigned long)priv->dma_storage, 8);
priv->rx_list_dma = ALIGN(priv->dma_storage_dma, 8);
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/ioport.h>
#include <linux/kernel.h>
#include <linux/list.h>
goto error;
}
rrpriv->rx_ctrl_dma = dma_addr;
- memset(rrpriv->rx_ctrl, 0, 256*sizeof(struct ring_ctrl));
rrpriv->info = pci_alloc_consistent(pdev, sizeof(struct rr_info),
&dma_addr);
goto error;
}
rrpriv->info_dma = dma_addr;
- memset(rrpriv->info, 0, sizeof(struct rr_info));
wmb();
spin_lock_irqsave(&rrpriv->lock, flags);
{QMI_FIXED_INTF(0x2001, 0x7e16, 3)}, /* D-Link DWM-221 */
{QMI_FIXED_INTF(0x2001, 0x7e19, 4)}, /* D-Link DWM-221 B1 */
{QMI_FIXED_INTF(0x2001, 0x7e35, 4)}, /* D-Link DWM-222 */
+ {QMI_FIXED_INTF(0x2001, 0x7e3d, 4)}, /* D-Link DWM-222 A2 */
{QMI_FIXED_INTF(0x2020, 0x2031, 4)}, /* Olicard 600 */
{QMI_FIXED_INTF(0x2020, 0x2033, 4)}, /* BroadMobi BM806U */
{QMI_FIXED_INTF(0x0f3d, 0x68a2, 8)}, /* Sierra Wireless MC7700 */
err = -ENOMEM;
goto err_ver;
}
- memset(adapter->coal_conf, 0, sizeof(*adapter->coal_conf));
adapter->coal_conf->coalMode = VMXNET3_COALESCE_DISABLED;
adapter->default_coal_mode = true;
}
&vht_nss,
true);
update_bitrate_mask = false;
+ } else {
+ vht_pfr = 0;
}
mutex_lock(&ar->conf_mutex);
#define IWL_22000_QU_B_HR_B_FW_PRE "iwlwifi-Qu-b0-hr-b0-"
#define IWL_22000_HR_B_FW_PRE "iwlwifi-QuQnj-b0-hr-b0-"
#define IWL_22000_HR_A0_FW_PRE "iwlwifi-QuQnj-a0-hr-a0-"
+#define IWL_QU_C_HR_B_FW_PRE "iwlwifi-Qu-c0-hr-b0-"
#define IWL_QU_B_JF_B_FW_PRE "iwlwifi-Qu-b0-jf-b0-"
+#define IWL_QU_C_JF_B_FW_PRE "iwlwifi-Qu-c0-jf-b0-"
#define IWL_QUZ_A_HR_B_FW_PRE "iwlwifi-QuZ-a0-hr-b0-"
#define IWL_QUZ_A_JF_B_FW_PRE "iwlwifi-QuZ-a0-jf-b0-"
#define IWL_QNJ_B_JF_B_FW_PRE "iwlwifi-QuQnj-b0-jf-b0-"
IWL_QUZ_A_HR_B_FW_PRE __stringify(api) ".ucode"
#define IWL_QUZ_A_JF_B_MODULE_FIRMWARE(api) \
IWL_QUZ_A_JF_B_FW_PRE __stringify(api) ".ucode"
+#define IWL_QU_C_HR_B_MODULE_FIRMWARE(api) \
+ IWL_QU_C_HR_B_FW_PRE __stringify(api) ".ucode"
#define IWL_QU_B_JF_B_MODULE_FIRMWARE(api) \
IWL_QU_B_JF_B_FW_PRE __stringify(api) ".ucode"
#define IWL_QNJ_B_JF_B_MODULE_FIRMWARE(api) \
.max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
};
+const struct iwl_cfg iwl_ax101_cfg_qu_c0_hr_b0 = {
+ .name = "Intel(R) Wi-Fi 6 AX101",
+ .fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
+ IWL_DEVICE_22500,
+ /*
+ * This device doesn't support receiving BlockAck with a large bitmap
+ * so we need to restrict the size of transmitted aggregation to the
+ * HT size; mac80211 would otherwise pick the HE max (256) by default.
+ */
+ .max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
+};
+
+const struct iwl_cfg iwl_ax201_cfg_qu_c0_hr_b0 = {
+ .name = "Intel(R) Wi-Fi 6 AX201 160MHz",
+ .fw_name_pre = IWL_QU_C_HR_B_FW_PRE,
+ IWL_DEVICE_22500,
+ /*
+ * This device doesn't support receiving BlockAck with a large bitmap
+ * so we need to restrict the size of transmitted aggregation to the
+ * HT size; mac80211 would otherwise pick the HE max (256) by default.
+ */
+ .max_tx_agg_size = IEEE80211_MAX_AMPDU_BUF_HT,
+};
+
const struct iwl_cfg iwl_ax101_cfg_quz_hr = {
.name = "Intel(R) Wi-Fi 6 AX101",
.fw_name_pre = IWL_QUZ_A_HR_B_FW_PRE,
IWL_DEVICE_22500,
};
+const struct iwl_cfg iwl9461_2ac_cfg_qu_c0_jf_b0 = {
+ .name = "Intel(R) Wireless-AC 9461",
+ .fw_name_pre = IWL_QU_C_JF_B_FW_PRE,
+ IWL_DEVICE_22500,
+};
+
+const struct iwl_cfg iwl9462_2ac_cfg_qu_c0_jf_b0 = {
+ .name = "Intel(R) Wireless-AC 9462",
+ .fw_name_pre = IWL_QU_C_JF_B_FW_PRE,
+ IWL_DEVICE_22500,
+};
+
+const struct iwl_cfg iwl9560_2ac_cfg_qu_c0_jf_b0 = {
+ .name = "Intel(R) Wireless-AC 9560",
+ .fw_name_pre = IWL_QU_C_JF_B_FW_PRE,
+ IWL_DEVICE_22500,
+};
+
+const struct iwl_cfg iwl9560_2ac_160_cfg_qu_c0_jf_b0 = {
+ .name = "Intel(R) Wireless-AC 9560 160MHz",
+ .fw_name_pre = IWL_QU_C_JF_B_FW_PRE,
+ IWL_DEVICE_22500,
+};
+
const struct iwl_cfg iwl9560_2ac_cfg_qnj_jf_b0 = {
.name = "Intel(R) Wireless-AC 9560 160MHz",
.fw_name_pre = IWL_QNJ_B_JF_B_FW_PRE,
MODULE_FIRMWARE(IWL_22000_HR_B_F0_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_HR_B_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_22000_HR_A0_QNJ_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
+MODULE_FIRMWARE(IWL_QU_C_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QU_B_JF_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QUZ_A_HR_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
MODULE_FIRMWARE(IWL_QUZ_A_JF_B_MODULE_FIRMWARE(IWL_22000_UCODE_API_MAX));
extern const struct iwl_cfg iwl22000_2ac_cfg_hr_cdb;
extern const struct iwl_cfg iwl22000_2ac_cfg_jf;
extern const struct iwl_cfg iwl_ax101_cfg_qu_hr;
+extern const struct iwl_cfg iwl_ax101_cfg_qu_c0_hr_b0;
extern const struct iwl_cfg iwl_ax101_cfg_quz_hr;
extern const struct iwl_cfg iwl22000_2ax_cfg_hr;
extern const struct iwl_cfg iwl_ax200_cfg_cc;
extern const struct iwl_cfg iwl_ax201_cfg_qu_hr;
+extern const struct iwl_cfg iwl_ax201_cfg_qu_hr;
+extern const struct iwl_cfg iwl_ax201_cfg_qu_c0_hr_b0;
extern const struct iwl_cfg iwl_ax201_cfg_quz_hr;
extern const struct iwl_cfg iwl_ax1650i_cfg_quz_hr;
extern const struct iwl_cfg iwl_ax1650s_cfg_quz_hr;
extern const struct iwl_cfg iwl9462_2ac_cfg_qu_b0_jf_b0;
extern const struct iwl_cfg iwl9560_2ac_cfg_qu_b0_jf_b0;
extern const struct iwl_cfg iwl9560_2ac_160_cfg_qu_b0_jf_b0;
+extern const struct iwl_cfg iwl9461_2ac_cfg_qu_c0_jf_b0;
+extern const struct iwl_cfg iwl9462_2ac_cfg_qu_c0_jf_b0;
+extern const struct iwl_cfg iwl9560_2ac_cfg_qu_c0_jf_b0;
+extern const struct iwl_cfg iwl9560_2ac_160_cfg_qu_c0_jf_b0;
extern const struct iwl_cfg killer1550i_2ac_cfg_qu_b0_jf_b0;
extern const struct iwl_cfg killer1550s_2ac_cfg_qu_b0_jf_b0;
extern const struct iwl_cfg iwl22000_2ax_cfg_jf;
#define CSR_HW_REV_TYPE_NONE (0x00001F0)
#define CSR_HW_REV_TYPE_QNJ (0x0000360)
#define CSR_HW_REV_TYPE_QNJ_B0 (0x0000364)
+#define CSR_HW_REV_TYPE_QU_B0 (0x0000334)
+#define CSR_HW_REV_TYPE_QU_C0 (0x0000338)
#define CSR_HW_REV_TYPE_QUZ (0x0000354)
#define CSR_HW_REV_TYPE_HR_CDB (0x0000340)
#define CSR_HW_REV_TYPE_SO (0x0000370)
{IWL_PCI_DEVICE(0x2526, 0x40A4, iwl9460_2ac_cfg)},
{IWL_PCI_DEVICE(0x2526, 0x4234, iwl9560_2ac_cfg_soc)},
{IWL_PCI_DEVICE(0x2526, 0x42A4, iwl9462_2ac_cfg_soc)},
+ {IWL_PCI_DEVICE(0x2526, 0x6014, iwl9260_2ac_160_cfg)},
{IWL_PCI_DEVICE(0x2526, 0x8014, iwl9260_2ac_160_cfg)},
{IWL_PCI_DEVICE(0x2526, 0x8010, iwl9260_2ac_160_cfg)},
{IWL_PCI_DEVICE(0x2526, 0xA014, iwl9260_2ac_160_cfg)},
{IWL_PCI_DEVICE(0x7A70, 0x0310, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x7A70, 0x0510, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x7A70, 0x0A10, iwlax211_2ax_cfg_so_gf_a0)},
+ {IWL_PCI_DEVICE(0x7AF0, 0x0090, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x7AF0, 0x0310, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x7AF0, 0x0510, iwlax211_2ax_cfg_so_gf_a0)},
{IWL_PCI_DEVICE(0x7AF0, 0x0A10, iwlax211_2ax_cfg_so_gf_a0)},
}
iwl_trans->cfg = cfg;
}
+
+ /*
+ * This is a hack to switch from Qu B0 to Qu C0. We need to
+ * do this for all cfgs that use Qu B0. All this code is in
+ * urgent need for a refactor, but for now this is the easiest
+ * thing to do to support Qu C-step.
+ */
+ if (iwl_trans->hw_rev == CSR_HW_REV_TYPE_QU_C0) {
+ if (iwl_trans->cfg == &iwl_ax101_cfg_qu_hr)
+ iwl_trans->cfg = &iwl_ax101_cfg_qu_c0_hr_b0;
+ else if (iwl_trans->cfg == &iwl_ax201_cfg_qu_hr)
+ iwl_trans->cfg = &iwl_ax201_cfg_qu_c0_hr_b0;
+ else if (iwl_trans->cfg == &iwl9461_2ac_cfg_qu_b0_jf_b0)
+ iwl_trans->cfg = &iwl9461_2ac_cfg_qu_c0_jf_b0;
+ else if (iwl_trans->cfg == &iwl9462_2ac_cfg_qu_b0_jf_b0)
+ iwl_trans->cfg = &iwl9462_2ac_cfg_qu_c0_jf_b0;
+ else if (iwl_trans->cfg == &iwl9560_2ac_cfg_qu_b0_jf_b0)
+ iwl_trans->cfg = &iwl9560_2ac_cfg_qu_c0_jf_b0;
+ else if (iwl_trans->cfg == &iwl9560_2ac_160_cfg_qu_b0_jf_b0)
+ iwl_trans->cfg = &iwl9560_2ac_160_cfg_qu_c0_jf_b0;
+ }
#endif
pci_set_drvdata(pdev, iwl_trans);
pad = round_up(skb->len, 4) + 4 - skb->len;
/* First packet of a A-MSDU burst keeps track of the whole burst
- * length, need to update lenght of it and the last packet.
+ * length, need to update length of it and the last packet.
*/
skb_walk_frags(skb, iter) {
last = iter;
struct queue_entry *entry = (struct queue_entry *)urb->context;
struct rt2x00_dev *rt2x00dev = entry->queue->rt2x00dev;
- if (!test_and_clear_bit(ENTRY_OWNER_DEVICE_DATA, &entry->flags))
+ if (!test_bit(ENTRY_OWNER_DEVICE_DATA, &entry->flags))
return;
/*
- * Report the frame as DMA done
- */
- rt2x00lib_dmadone(entry);
-
- /*
* Check if the received data is simply too small
* to be actually valid, or if the urb is signaling
* a problem.
set_bit(ENTRY_DATA_IO_FAILED, &entry->flags);
/*
+ * Report the frame as DMA done
+ */
+ rt2x00lib_dmadone(entry);
+
+ /*
* Schedule the delayed work for reading the RX status
* from the device.
*/
Documentation/admin-guide/kernel-parameters.rst). This driver converts
these persistent memory ranges into block devices that are
capable of DAX (direct-access) file system mappings. See
- Documentation/nvdimm/nvdimm.txt for more details.
+ Documentation/driver-api/nvdimm/nvdimm.rst for more details.
Say Y if you want to use an NVDIMM
obj-$(CONFIG_ND_BLK) += nd_blk.o
obj-$(CONFIG_X86_PMEM_LEGACY) += nd_e820.o
obj-$(CONFIG_OF_PMEM) += of_pmem.o
+obj-$(CONFIG_VIRTIO_PMEM) += virtio_pmem.o nd_virtio.o
nd_pmem-y := pmem.o
struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
unsigned int sz_align = ALIGN(size + (offset & (512 - 1)), 512);
sector_t sector = offset >> 9;
- int rc = 0;
+ int rc = 0, ret = 0;
if (unlikely(!size))
return 0;
}
memcpy_flushcache(nsio->addr + offset, buf, size);
- nvdimm_flush(to_nd_region(ndns->dev.parent));
+ ret = nvdimm_flush(to_nd_region(ndns->dev.parent), NULL);
+ if (ret)
+ rc = ret;
return rc;
}
nvdimm_bus_unlock(&ndns->dev);
if (!dax_dev)
return -ENOMEM;
- pfn_sb = devm_kzalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
+ pfn_sb = devm_kmalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
nd_pfn->pfn_sb = pfn_sb;
rc = nd_pfn_validate(nd_pfn, DAX_SIG);
dev_dbg(dev, "dax: %s\n", rc == 0 ? dev_name(dax_dev) : "<none>");
&& !guid_equal(&nd_set->type_guid,
&nd_label->type_guid)) {
dev_dbg(ndd->dev, "expect type_guid %pUb got %pUb\n",
- nd_set->type_guid.b,
- nd_label->type_guid.b);
+ &nd_set->type_guid,
+ &nd_label->type_guid);
continue;
}
if (namespace_label_has(ndd, type_guid)) {
if (!guid_equal(&nd_set->type_guid, &nd_label->type_guid)) {
dev_dbg(ndd->dev, "expect type_guid %pUb got %pUb\n",
- nd_set->type_guid.b,
- nd_label->type_guid.b);
+ &nd_set->type_guid,
+ &nd_label->type_guid);
return ERR_PTR(-EAGAIN);
}
struct badblocks bb;
struct nd_interleave_set *nd_set;
struct nd_percpu_lane __percpu *lane;
+ int (*flush)(struct nd_region *nd_region, struct bio *bio);
struct nd_mapping mapping[0];
};
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * virtio_pmem.c: Virtio pmem Driver
+ *
+ * Discovers persistent memory range information
+ * from host and provides a virtio based flushing
+ * interface.
+ */
+#include "virtio_pmem.h"
+#include "nd.h"
+
+ /* The interrupt handler */
+void virtio_pmem_host_ack(struct virtqueue *vq)
+{
+ struct virtio_pmem *vpmem = vq->vdev->priv;
+ struct virtio_pmem_request *req_data, *req_buf;
+ unsigned long flags;
+ unsigned int len;
+
+ spin_lock_irqsave(&vpmem->pmem_lock, flags);
+ while ((req_data = virtqueue_get_buf(vq, &len)) != NULL) {
+ req_data->done = true;
+ wake_up(&req_data->host_acked);
+
+ if (!list_empty(&vpmem->req_list)) {
+ req_buf = list_first_entry(&vpmem->req_list,
+ struct virtio_pmem_request, list);
+ req_buf->wq_buf_avail = true;
+ wake_up(&req_buf->wq_buf);
+ list_del(&req_buf->list);
+ }
+ }
+ spin_unlock_irqrestore(&vpmem->pmem_lock, flags);
+}
+EXPORT_SYMBOL_GPL(virtio_pmem_host_ack);
+
+ /* The request submission function */
+static int virtio_pmem_flush(struct nd_region *nd_region)
+{
+ struct virtio_device *vdev = nd_region->provider_data;
+ struct virtio_pmem *vpmem = vdev->priv;
+ struct virtio_pmem_request *req_data;
+ struct scatterlist *sgs[2], sg, ret;
+ unsigned long flags;
+ int err, err1;
+
+ might_sleep();
+ req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
+ if (!req_data)
+ return -ENOMEM;
+
+ req_data->done = false;
+ init_waitqueue_head(&req_data->host_acked);
+ init_waitqueue_head(&req_data->wq_buf);
+ INIT_LIST_HEAD(&req_data->list);
+ req_data->req.type = cpu_to_le32(VIRTIO_PMEM_REQ_TYPE_FLUSH);
+ sg_init_one(&sg, &req_data->req, sizeof(req_data->req));
+ sgs[0] = &sg;
+ sg_init_one(&ret, &req_data->resp.ret, sizeof(req_data->resp));
+ sgs[1] = &ret;
+
+ spin_lock_irqsave(&vpmem->pmem_lock, flags);
+ /*
+ * If virtqueue_add_sgs returns -ENOSPC then req_vq virtual
+ * queue does not have free descriptor. We add the request
+ * to req_list and wait for host_ack to wake us up when free
+ * slots are available.
+ */
+ while ((err = virtqueue_add_sgs(vpmem->req_vq, sgs, 1, 1, req_data,
+ GFP_ATOMIC)) == -ENOSPC) {
+
+ dev_info(&vdev->dev, "failed to send command to virtio pmem device, no free slots in the virtqueue\n");
+ req_data->wq_buf_avail = false;
+ list_add_tail(&req_data->list, &vpmem->req_list);
+ spin_unlock_irqrestore(&vpmem->pmem_lock, flags);
+
+ /* A host response results in "host_ack" getting called */
+ wait_event(req_data->wq_buf, req_data->wq_buf_avail);
+ spin_lock_irqsave(&vpmem->pmem_lock, flags);
+ }
+ err1 = virtqueue_kick(vpmem->req_vq);
+ spin_unlock_irqrestore(&vpmem->pmem_lock, flags);
+ /*
+ * virtqueue_add_sgs failed with error different than -ENOSPC, we can't
+ * do anything about that.
+ */
+ if (err || !err1) {
+ dev_info(&vdev->dev, "failed to send command to virtio pmem device\n");
+ err = -EIO;
+ } else {
+ /* A host repsonse results in "host_ack" getting called */
+ wait_event(req_data->host_acked, req_data->done);
+ err = le32_to_cpu(req_data->resp.ret);
+ }
+
+ kfree(req_data);
+ return err;
+};
+
+/* The asynchronous flush callback function */
+int async_pmem_flush(struct nd_region *nd_region, struct bio *bio)
+{
+ /*
+ * Create child bio for asynchronous flush and chain with
+ * parent bio. Otherwise directly call nd_region flush.
+ */
+ if (bio && bio->bi_iter.bi_sector != -1) {
+ struct bio *child = bio_alloc(GFP_ATOMIC, 0);
+
+ if (!child)
+ return -ENOMEM;
+ bio_copy_dev(child, bio);
+ child->bi_opf = REQ_PREFLUSH;
+ child->bi_iter.bi_sector = -1;
+ bio_chain(child, bio);
+ submit_bio(child);
+ return 0;
+ }
+ if (virtio_pmem_flush(nd_region))
+ return -EIO;
+
+ return 0;
+};
+EXPORT_SYMBOL_GPL(async_pmem_flush);
+MODULE_LICENSE("GPL");
__le32 end_trunc;
/* minor-version-2 record the base alignment of the mapping */
__le32 align;
+ /* minor-version-3 guarantee the padding and flags are zero */
u8 padding[4000];
__le64 checksum;
};
-#ifdef CONFIG_SPARSEMEM
-#define PFN_SECTION_ALIGN_DOWN(x) SECTION_ALIGN_DOWN(x)
-#define PFN_SECTION_ALIGN_UP(x) SECTION_ALIGN_UP(x)
-#else
-/*
- * In this case ZONE_DEVICE=n and we will disable 'pfn' device support,
- * but we still want pmem to compile.
- */
-#define PFN_SECTION_ALIGN_DOWN(x) (x)
-#define PFN_SECTION_ALIGN_UP(x) (x)
-#endif
-
-#define PHYS_SECTION_ALIGN_DOWN(x) PFN_PHYS(PFN_SECTION_ALIGN_DOWN(PHYS_PFN(x)))
-#define PHYS_SECTION_ALIGN_UP(x) PFN_PHYS(PFN_SECTION_ALIGN_UP(PHYS_PFN(x)))
#endif /* __NVDIMM_PFN_H */
return 0;
}
+/**
+ * nd_pfn_validate - read and validate info-block
+ * @nd_pfn: fsdax namespace runtime state / properties
+ * @sig: 'devdax' or 'fsdax' signature
+ *
+ * Upon return the info-block buffer contents (->pfn_sb) are
+ * indeterminate when validation fails, and a coherent info-block
+ * otherwise.
+ */
int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig)
{
u64 checksum, offset;
nvdimm_bus_unlock(&ndns->dev);
if (!pfn_dev)
return -ENOMEM;
- pfn_sb = devm_kzalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
+ pfn_sb = devm_kmalloc(dev, sizeof(*pfn_sb), GFP_KERNEL);
nd_pfn = to_nd_pfn(pfn_dev);
nd_pfn->pfn_sb = pfn_sb;
rc = nd_pfn_validate(nd_pfn, PFN_SIG);
}
/*
- * We hotplug memory at section granularity, pad the reserved area from
- * the previous section base to the namespace base address.
+ * We hotplug memory at sub-section granularity, pad the reserved area
+ * from the previous section base to the namespace base address.
*/
static unsigned long init_altmap_base(resource_size_t base)
{
unsigned long base_pfn = PHYS_PFN(base);
- return PFN_SECTION_ALIGN_DOWN(base_pfn);
+ return SUBSECTION_ALIGN_DOWN(base_pfn);
}
static unsigned long init_altmap_reserve(resource_size_t base)
unsigned long reserve = info_block_reserve() >> PAGE_SHIFT;
unsigned long base_pfn = PHYS_PFN(base);
- reserve += base_pfn - PFN_SECTION_ALIGN_DOWN(base_pfn);
+ reserve += base_pfn - SUBSECTION_ALIGN_DOWN(base_pfn);
return reserve;
}
return -EINVAL;
nd_pfn->npfns = le64_to_cpu(pfn_sb->npfns);
} else if (nd_pfn->mode == PFN_MODE_PMEM) {
- nd_pfn->npfns = PFN_SECTION_ALIGN_UP((resource_size(res)
- - offset) / PAGE_SIZE);
+ nd_pfn->npfns = PHYS_PFN((resource_size(res) - offset));
if (le64_to_cpu(nd_pfn->pfn_sb->npfns) > nd_pfn->npfns)
dev_info(&nd_pfn->dev,
"number of pfns truncated from %lld to %ld\n",
return 0;
}
-static u64 phys_pmem_align_down(struct nd_pfn *nd_pfn, u64 phys)
-{
- return min_t(u64, PHYS_SECTION_ALIGN_DOWN(phys),
- ALIGN_DOWN(phys, nd_pfn->align));
-}
-
-/*
- * Check if pmem collides with 'System RAM', or other regions when
- * section aligned. Trim it accordingly.
- */
-static void trim_pfn_device(struct nd_pfn *nd_pfn, u32 *start_pad, u32 *end_trunc)
-{
- struct nd_namespace_common *ndns = nd_pfn->ndns;
- struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
- struct nd_region *nd_region = to_nd_region(nd_pfn->dev.parent);
- const resource_size_t start = nsio->res.start;
- const resource_size_t end = start + resource_size(&nsio->res);
- resource_size_t adjust, size;
-
- *start_pad = 0;
- *end_trunc = 0;
-
- adjust = start - PHYS_SECTION_ALIGN_DOWN(start);
- size = resource_size(&nsio->res) + adjust;
- if (region_intersects(start - adjust, size, IORESOURCE_SYSTEM_RAM,
- IORES_DESC_NONE) == REGION_MIXED
- || nd_region_conflict(nd_region, start - adjust, size))
- *start_pad = PHYS_SECTION_ALIGN_UP(start) - start;
-
- /* Now check that end of the range does not collide. */
- adjust = PHYS_SECTION_ALIGN_UP(end) - end;
- size = resource_size(&nsio->res) + adjust;
- if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM,
- IORES_DESC_NONE) == REGION_MIXED
- || !IS_ALIGNED(end, nd_pfn->align)
- || nd_region_conflict(nd_region, start, size))
- *end_trunc = end - phys_pmem_align_down(nd_pfn, end);
-}
-
static int nd_pfn_init(struct nd_pfn *nd_pfn)
{
struct nd_namespace_common *ndns = nd_pfn->ndns;
struct nd_namespace_io *nsio = to_nd_namespace_io(&ndns->dev);
- u32 start_pad, end_trunc, reserve = info_block_reserve();
resource_size_t start, size;
struct nd_region *nd_region;
+ unsigned long npfns, align;
struct nd_pfn_sb *pfn_sb;
- unsigned long npfns;
phys_addr_t offset;
const char *sig;
u64 checksum;
int rc;
- pfn_sb = devm_kzalloc(&nd_pfn->dev, sizeof(*pfn_sb), GFP_KERNEL);
+ pfn_sb = devm_kmalloc(&nd_pfn->dev, sizeof(*pfn_sb), GFP_KERNEL);
if (!pfn_sb)
return -ENOMEM;
sig = DAX_SIG;
else
sig = PFN_SIG;
+
rc = nd_pfn_validate(nd_pfn, sig);
if (rc != -ENODEV)
return rc;
/* no info block, do init */;
+ memset(pfn_sb, 0, sizeof(*pfn_sb));
+
nd_region = to_nd_region(nd_pfn->dev.parent);
if (nd_region->ro) {
dev_info(&nd_pfn->dev,
return -ENXIO;
}
- memset(pfn_sb, 0, sizeof(*pfn_sb));
-
- trim_pfn_device(nd_pfn, &start_pad, &end_trunc);
- if (start_pad + end_trunc)
- dev_info(&nd_pfn->dev, "%s alignment collision, truncate %d bytes\n",
- dev_name(&ndns->dev), start_pad + end_trunc);
-
/*
* Note, we use 64 here for the standard size of struct page,
* debugging options may cause it to be larger in which case the
* implementation will limit the pfns advertised through
* ->direct_access() to those that are included in the memmap.
*/
- start = nsio->res.start + start_pad;
+ start = nsio->res.start;
size = resource_size(&nsio->res);
- npfns = PFN_SECTION_ALIGN_UP((size - start_pad - end_trunc - reserve)
- / PAGE_SIZE);
+ npfns = PHYS_PFN(size - SZ_8K);
+ align = max(nd_pfn->align, (1UL << SUBSECTION_SHIFT));
if (nd_pfn->mode == PFN_MODE_PMEM) {
/*
* The altmap should be padded out to the block size used
* when populating the vmemmap. This *should* be equal to
* PMD_SIZE for most architectures.
*/
- offset = ALIGN(start + reserve + 64 * npfns,
- max(nd_pfn->align, PMD_SIZE)) - start;
+ offset = ALIGN(start + SZ_8K + 64 * npfns, align) - start;
} else if (nd_pfn->mode == PFN_MODE_RAM)
- offset = ALIGN(start + reserve, nd_pfn->align) - start;
+ offset = ALIGN(start + SZ_8K, align) - start;
else
return -ENXIO;
- if (offset + start_pad + end_trunc >= size) {
+ if (offset >= size) {
dev_err(&nd_pfn->dev, "%s unable to satisfy requested alignment\n",
dev_name(&ndns->dev));
return -ENXIO;
}
- npfns = (size - offset - start_pad - end_trunc) / SZ_4K;
+ npfns = PHYS_PFN(size - offset);
pfn_sb->mode = cpu_to_le32(nd_pfn->mode);
pfn_sb->dataoff = cpu_to_le64(offset);
pfn_sb->npfns = cpu_to_le64(npfns);
memcpy(pfn_sb->uuid, nd_pfn->uuid, 16);
memcpy(pfn_sb->parent_uuid, nd_dev_to_uuid(&ndns->dev), 16);
pfn_sb->version_major = cpu_to_le16(1);
- pfn_sb->version_minor = cpu_to_le16(2);
- pfn_sb->start_pad = cpu_to_le32(start_pad);
- pfn_sb->end_trunc = cpu_to_le32(end_trunc);
+ pfn_sb->version_minor = cpu_to_le16(3);
pfn_sb->align = cpu_to_le32(nd_pfn->align);
checksum = nd_sb_checksum((struct nd_gen_sb *) pfn_sb);
pfn_sb->checksum = cpu_to_le64(checksum);
static blk_qc_t pmem_make_request(struct request_queue *q, struct bio *bio)
{
+ int ret = 0;
blk_status_t rc = 0;
bool do_acct;
unsigned long start;
struct nd_region *nd_region = to_region(pmem);
if (bio->bi_opf & REQ_PREFLUSH)
- nvdimm_flush(nd_region);
+ ret = nvdimm_flush(nd_region, bio);
do_acct = nd_iostat_start(bio, &start);
bio_for_each_segment(bvec, bio, iter) {
nd_iostat_end(bio, start);
if (bio->bi_opf & REQ_FUA)
- nvdimm_flush(nd_region);
+ ret = nvdimm_flush(nd_region, bio);
+
+ if (ret)
+ bio->bi_status = errno_to_blk_status(ret);
bio_endio(bio);
return BLK_QC_T_NONE;
struct gendisk *disk;
void *addr;
int rc;
+ unsigned long flags = 0UL;
pmem = devm_kzalloc(dev, sizeof(*pmem), GFP_KERNEL);
if (!pmem)
nvdimm_badblocks_populate(nd_region, &pmem->bb, &bb_res);
disk->bb = &pmem->bb;
- dax_dev = alloc_dax(pmem, disk->disk_name, &pmem_dax_ops);
+ if (is_nvdimm_sync(nd_region))
+ flags = DAXDEV_F_SYNC;
+ dax_dev = alloc_dax(pmem, disk->disk_name, &pmem_dax_ops, flags);
if (!dax_dev) {
put_disk(disk);
return -ENOMEM;
}
dax_write_cache(dax_dev, nvdimm_has_cache(nd_region));
pmem->dax_dev = dax_dev;
-
gendev = disk_to_dev(disk);
gendev->groups = pmem_attribute_groups;
sysfs_put(pmem->bb_state);
pmem->bb_state = NULL;
}
- nvdimm_flush(to_nd_region(dev->parent));
+ nvdimm_flush(to_nd_region(dev->parent), NULL);
return 0;
}
static void nd_pmem_shutdown(struct device *dev)
{
- nvdimm_flush(to_nd_region(dev->parent));
+ nvdimm_flush(to_nd_region(dev->parent), NULL);
}
static void nd_pmem_notify(struct device *dev, enum nvdimm_event event)
return rc;
if (!flush)
return -EINVAL;
- nvdimm_flush(nd_region);
+ rc = nvdimm_flush(nd_region, NULL);
+ if (rc)
+ return rc;
return len;
}
dev->of_node = ndr_desc->of_node;
nd_region->ndr_size = resource_size(ndr_desc->res);
nd_region->ndr_start = ndr_desc->res->start;
+ if (ndr_desc->flush)
+ nd_region->flush = ndr_desc->flush;
+ else
+ nd_region->flush = NULL;
+
nd_device_register(dev);
return nd_region;
}
EXPORT_SYMBOL_GPL(nvdimm_volatile_region_create);
+int nvdimm_flush(struct nd_region *nd_region, struct bio *bio)
+{
+ int rc = 0;
+
+ if (!nd_region->flush)
+ rc = generic_nvdimm_flush(nd_region);
+ else {
+ if (nd_region->flush(nd_region, bio))
+ rc = -EIO;
+ }
+
+ return rc;
+}
/**
* nvdimm_flush - flush any posted write queues between the cpu and pmem media
* @nd_region: blk or interleaved pmem region
*/
-void nvdimm_flush(struct nd_region *nd_region)
+int generic_nvdimm_flush(struct nd_region *nd_region)
{
struct nd_region_data *ndrd = dev_get_drvdata(&nd_region->dev);
int i, idx;
if (ndrd_get_flush_wpq(ndrd, i, 0))
writeq(1, ndrd_get_flush_wpq(ndrd, i, idx));
wmb();
+
+ return 0;
}
EXPORT_SYMBOL_GPL(nvdimm_flush);
}
EXPORT_SYMBOL_GPL(nvdimm_has_cache);
+bool is_nvdimm_sync(struct nd_region *nd_region)
+{
+ return is_nd_pmem(&nd_region->dev) &&
+ !test_bit(ND_REGION_ASYNC, &nd_region->flags);
+}
+EXPORT_SYMBOL_GPL(is_nvdimm_sync);
+
struct conflict_context {
struct nd_region *nd_region;
resource_size_t start, size;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * virtio_pmem.c: Virtio pmem Driver
+ *
+ * Discovers persistent memory range information
+ * from host and registers the virtual pmem device
+ * with libnvdimm core.
+ */
+#include "virtio_pmem.h"
+#include "nd.h"
+
+static struct virtio_device_id id_table[] = {
+ { VIRTIO_ID_PMEM, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+ /* Initialize virt queue */
+static int init_vq(struct virtio_pmem *vpmem)
+{
+ /* single vq */
+ vpmem->req_vq = virtio_find_single_vq(vpmem->vdev,
+ virtio_pmem_host_ack, "flush_queue");
+ if (IS_ERR(vpmem->req_vq))
+ return PTR_ERR(vpmem->req_vq);
+
+ spin_lock_init(&vpmem->pmem_lock);
+ INIT_LIST_HEAD(&vpmem->req_list);
+
+ return 0;
+};
+
+static int virtio_pmem_probe(struct virtio_device *vdev)
+{
+ struct nd_region_desc ndr_desc = {};
+ int nid = dev_to_node(&vdev->dev);
+ struct nd_region *nd_region;
+ struct virtio_pmem *vpmem;
+ struct resource res;
+ int err = 0;
+
+ if (!vdev->config->get) {
+ dev_err(&vdev->dev, "%s failure: config access disabled\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ vpmem = devm_kzalloc(&vdev->dev, sizeof(*vpmem), GFP_KERNEL);
+ if (!vpmem) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+
+ vpmem->vdev = vdev;
+ vdev->priv = vpmem;
+ err = init_vq(vpmem);
+ if (err) {
+ dev_err(&vdev->dev, "failed to initialize virtio pmem vq's\n");
+ goto out_err;
+ }
+
+ virtio_cread(vpmem->vdev, struct virtio_pmem_config,
+ start, &vpmem->start);
+ virtio_cread(vpmem->vdev, struct virtio_pmem_config,
+ size, &vpmem->size);
+
+ res.start = vpmem->start;
+ res.end = vpmem->start + vpmem->size - 1;
+ vpmem->nd_desc.provider_name = "virtio-pmem";
+ vpmem->nd_desc.module = THIS_MODULE;
+
+ vpmem->nvdimm_bus = nvdimm_bus_register(&vdev->dev,
+ &vpmem->nd_desc);
+ if (!vpmem->nvdimm_bus) {
+ dev_err(&vdev->dev, "failed to register device with nvdimm_bus\n");
+ err = -ENXIO;
+ goto out_vq;
+ }
+
+ dev_set_drvdata(&vdev->dev, vpmem->nvdimm_bus);
+
+ ndr_desc.res = &res;
+ ndr_desc.numa_node = nid;
+ ndr_desc.flush = async_pmem_flush;
+ set_bit(ND_REGION_PAGEMAP, &ndr_desc.flags);
+ set_bit(ND_REGION_ASYNC, &ndr_desc.flags);
+ nd_region = nvdimm_pmem_region_create(vpmem->nvdimm_bus, &ndr_desc);
+ if (!nd_region) {
+ dev_err(&vdev->dev, "failed to create nvdimm region\n");
+ err = -ENXIO;
+ goto out_nd;
+ }
+ nd_region->provider_data = dev_to_virtio(nd_region->dev.parent->parent);
+ return 0;
+out_nd:
+ nvdimm_bus_unregister(vpmem->nvdimm_bus);
+out_vq:
+ vdev->config->del_vqs(vdev);
+out_err:
+ return err;
+}
+
+static void virtio_pmem_remove(struct virtio_device *vdev)
+{
+ struct nvdimm_bus *nvdimm_bus = dev_get_drvdata(&vdev->dev);
+
+ nvdimm_bus_unregister(nvdimm_bus);
+ vdev->config->del_vqs(vdev);
+ vdev->config->reset(vdev);
+}
+
+static struct virtio_driver virtio_pmem_driver = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .id_table = id_table,
+ .probe = virtio_pmem_probe,
+ .remove = virtio_pmem_remove,
+};
+
+module_virtio_driver(virtio_pmem_driver);
+MODULE_DEVICE_TABLE(virtio, id_table);
+MODULE_DESCRIPTION("Virtio pmem driver");
+MODULE_LICENSE("GPL");
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * virtio_pmem.h: virtio pmem Driver
+ *
+ * Discovers persistent memory range information
+ * from host and provides a virtio based flushing
+ * interface.
+ **/
+
+#ifndef _LINUX_VIRTIO_PMEM_H
+#define _LINUX_VIRTIO_PMEM_H
+
+#include <linux/module.h>
+#include <uapi/linux/virtio_pmem.h>
+#include <linux/libnvdimm.h>
+#include <linux/spinlock.h>
+
+struct virtio_pmem_request {
+ struct virtio_pmem_req req;
+ struct virtio_pmem_resp resp;
+
+ /* Wait queue to process deferred work after ack from host */
+ wait_queue_head_t host_acked;
+ bool done;
+
+ /* Wait queue to process deferred work after virt queue buffer avail */
+ wait_queue_head_t wq_buf;
+ bool wq_buf_avail;
+ struct list_head list;
+};
+
+struct virtio_pmem {
+ struct virtio_device *vdev;
+
+ /* Virtio pmem request queue */
+ struct virtqueue *req_vq;
+
+ /* nvdimm bus registers virtio pmem device */
+ struct nvdimm_bus *nvdimm_bus;
+ struct nvdimm_bus_descriptor nd_desc;
+
+ /* List to store deferred work if virtqueue is full */
+ struct list_head req_list;
+
+ /* Synchronize virtqueue data */
+ spinlock_t pmem_lock;
+
+ /* Memory region information */
+ __u64 start;
+ __u64 size;
+};
+
+void virtio_pmem_host_ack(struct virtqueue *vq);
+int async_pmem_flush(struct nd_region *nd_region, struct bio *bio);
+#endif
return 0;
}
- pr_err("%pOF: Invalid %s translation - no match for rid 0x%x on %pOF\n",
- np, map_name, rid, target && *target ? *target : NULL);
- return -EFAULT;
+ pr_info("%pOF: no %s translation for rid 0x%x on %pOF\n", np, map_name,
+ rid, target && *target ? *target : NULL);
+
+ /* Bypasses translation */
+ if (id_out)
+ *id_out = rid;
+ return 0;
}
EXPORT_SYMBOL_GPL(of_map_rid);
#include <linux/module.h>
#include <linux/oprofile.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/pagemap.h>
#include <linux/uaccess.h>
}
-static int oprofilefs_fill_super(struct super_block *sb, void *data, int silent)
+static int oprofilefs_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *root_inode;
return 0;
}
-
-static struct dentry *oprofilefs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int oprofilefs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, oprofilefs_fill_super);
+ return get_tree_single(fc, oprofilefs_fill_super);
}
+static const struct fs_context_operations oprofilefs_context_ops = {
+ .get_tree = oprofilefs_get_tree,
+};
+
+static int oprofilefs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &oprofilefs_context_ops;
+ return 0;
+}
static struct file_system_type oprofilefs_type = {
.owner = THIS_MODULE,
.name = "oprofilefs",
- .mount = oprofilefs_mount,
+ .init_fs_context = oprofilefs_init_fs_context,
.kill_sb = kill_litter_super,
};
MODULE_ALIAS_FS("oprofilefs");
return;
dev->dev.of_node = of_pci_find_child_device(dev->bus->dev.of_node,
dev->devfn);
+ if (dev->dev.of_node)
+ dev->dev.fwnode = &dev->dev.of_node->fwnode;
}
void pci_release_of_node(struct pci_dev *dev)
{
of_node_put(dev->dev.of_node);
dev->dev.of_node = NULL;
+ dev->dev.fwnode = NULL;
}
void pci_set_bus_of_node(struct pci_bus *bus)
if (node && of_property_read_bool(node, "external-facing"))
bus->self->untrusted = true;
}
+
bus->dev.of_node = node;
+
+ if (bus->dev.of_node)
+ bus->dev.fwnode = &bus->dev.of_node->fwnode;
}
void pci_release_bus_of_node(struct pci_bus *bus)
{
of_node_put(bus->dev.of_node);
bus->dev.of_node = NULL;
+ bus->dev.fwnode = NULL;
}
struct device_node * __weak pcibios_get_phb_of_node(struct pci_bus *bus)
Enables support for the management interface for the MicroSemi
Switchtec series of PCIe switches. Supports userspace access
to submit MRPC commands to the switch via /dev/switchtecX
- devices. See <file:Documentation/switchtec.txt> for more
+ devices. See <file:Documentation/driver-api/switchtec.rst> for more
information.
endmenu
* the slices. User space needs to aggregate to individual counts to provide
* a global picture.
*
- * See Documentation/perf/qcom_l3_pmu.txt for more details.
+ * See Documentation/admin-guide/perf/qcom_l3_pmu.rst for more details.
*
* Copyright (c) 2015-2017, The Linux Foundation. All rights reserved.
*/
Interrupts (SMIs) and Host Control Actions (system power cycle or
power off after OS shutdown) on certain Dell systems.
- See <file:Documentation/dcdbas.txt> for more details on the driver
+ See <file:Documentation/driver-api/dcdbas.rst> for more details on the driver
and the Dell systems on which Dell systems management software makes
use of this driver.
DELL system. Note you need a Dell OpenManage or Dell Update package (DUP)
supporting application to communicate with the BIOS regarding the new
image for the image update to take effect.
- See <file:Documentation/dell_rbu.txt> for more details on the driver.
+ See <file:Documentation/driver-api/dell_rbu.rst> for more details on the driver.
config FUJITSU_LAPTOP
screen brightness control, Fn keys and allows powering on/off some
devices.
- Read <file:Documentation/laptops/sony-laptop.txt> for more information.
+ Read <file:Documentation/admin-guide/laptops/sony-laptop.rst> for more information.
config SONYPI_COMPAT
bool "Sonypi compatibility"
support for Fn-Fx key combinations, Bluetooth control, video
output switching, ThinkLight control, UltraBay eject and more.
For more information about this driver see
- <file:Documentation/laptops/thinkpad-acpi.txt> and
+ <file:Documentation/admin-guide/laptops/thinkpad-acpi.rst> and
<http://ibm-acpi.sf.net/> .
This driver was formerly known as ibm-acpi.
#define ASUS_FAN_CTRL_MANUAL 1
#define ASUS_FAN_CTRL_AUTO 2
-#define ASUS_FAN_MODE_NORMAL 0
-#define ASUS_FAN_MODE_OVERBOOST 1
-#define ASUS_FAN_MODE_OVERBOOST_MASK 0x01
-#define ASUS_FAN_MODE_SILENT 2
-#define ASUS_FAN_MODE_SILENT_MASK 0x02
-#define ASUS_FAN_MODES_MASK 0x03
+#define ASUS_FAN_BOOST_MODE_NORMAL 0
+#define ASUS_FAN_BOOST_MODE_OVERBOOST 1
+#define ASUS_FAN_BOOST_MODE_OVERBOOST_MASK 0x01
+#define ASUS_FAN_BOOST_MODE_SILENT 2
+#define ASUS_FAN_BOOST_MODE_SILENT_MASK 0x02
+#define ASUS_FAN_BOOST_MODES_MASK 0x03
#define USB_INTEL_XUSB2PR 0xD0
#define PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_XHCI 0x9c31
int asus_hwmon_num_fans;
int asus_hwmon_pwm;
- bool fan_mode_available;
- u8 fan_mode_mask;
- u8 fan_mode;
+ bool fan_boost_mode_available;
+ u8 fan_boost_mode_mask;
+ u8 fan_boost_mode;
struct hotplug_slot hotplug_slot;
struct mutex hotplug_lock;
/* Fan mode *******************************************************************/
-static int fan_mode_check_present(struct asus_wmi *asus)
+static int fan_boost_mode_check_present(struct asus_wmi *asus)
{
u32 result;
int err;
- asus->fan_mode_available = false;
+ asus->fan_boost_mode_available = false;
- err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_FAN_MODE, &result);
+ err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_FAN_BOOST_MODE,
+ &result);
if (err) {
if (err == -ENODEV)
return 0;
}
if ((result & ASUS_WMI_DSTS_PRESENCE_BIT) &&
- (result & ASUS_FAN_MODES_MASK)) {
- asus->fan_mode_available = true;
- asus->fan_mode_mask = result & ASUS_FAN_MODES_MASK;
+ (result & ASUS_FAN_BOOST_MODES_MASK)) {
+ asus->fan_boost_mode_available = true;
+ asus->fan_boost_mode_mask = result & ASUS_FAN_BOOST_MODES_MASK;
}
return 0;
}
-static int fan_mode_write(struct asus_wmi *asus)
+static int fan_boost_mode_write(struct asus_wmi *asus)
{
int err;
u8 value;
u32 retval;
- value = asus->fan_mode;
+ value = asus->fan_boost_mode;
- pr_info("Set fan mode: %u\n", value);
- err = asus_wmi_set_devstate(ASUS_WMI_DEVID_FAN_MODE, value, &retval);
+ pr_info("Set fan boost mode: %u\n", value);
+ err = asus_wmi_set_devstate(ASUS_WMI_DEVID_FAN_BOOST_MODE, value,
+ &retval);
if (err) {
- pr_warn("Failed to set fan mode: %d\n", err);
+ pr_warn("Failed to set fan boost mode: %d\n", err);
return err;
}
if (retval != 1) {
- pr_warn("Failed to set fan mode (retval): 0x%x\n", retval);
+ pr_warn("Failed to set fan boost mode (retval): 0x%x\n",
+ retval);
return -EIO;
}
return 0;
}
-static int fan_mode_switch_next(struct asus_wmi *asus)
+static int fan_boost_mode_switch_next(struct asus_wmi *asus)
{
- if (asus->fan_mode == ASUS_FAN_MODE_NORMAL) {
- if (asus->fan_mode_mask & ASUS_FAN_MODE_OVERBOOST_MASK)
- asus->fan_mode = ASUS_FAN_MODE_OVERBOOST;
- else if (asus->fan_mode_mask & ASUS_FAN_MODE_SILENT_MASK)
- asus->fan_mode = ASUS_FAN_MODE_SILENT;
- } else if (asus->fan_mode == ASUS_FAN_MODE_OVERBOOST) {
- if (asus->fan_mode_mask & ASUS_FAN_MODE_SILENT_MASK)
- asus->fan_mode = ASUS_FAN_MODE_SILENT;
+ u8 mask = asus->fan_boost_mode_mask;
+
+ if (asus->fan_boost_mode == ASUS_FAN_BOOST_MODE_NORMAL) {
+ if (mask & ASUS_FAN_BOOST_MODE_OVERBOOST_MASK)
+ asus->fan_boost_mode = ASUS_FAN_BOOST_MODE_OVERBOOST;
+ else if (mask & ASUS_FAN_BOOST_MODE_SILENT_MASK)
+ asus->fan_boost_mode = ASUS_FAN_BOOST_MODE_SILENT;
+ } else if (asus->fan_boost_mode == ASUS_FAN_BOOST_MODE_OVERBOOST) {
+ if (mask & ASUS_FAN_BOOST_MODE_SILENT_MASK)
+ asus->fan_boost_mode = ASUS_FAN_BOOST_MODE_SILENT;
else
- asus->fan_mode = ASUS_FAN_MODE_NORMAL;
+ asus->fan_boost_mode = ASUS_FAN_BOOST_MODE_NORMAL;
} else {
- asus->fan_mode = ASUS_FAN_MODE_NORMAL;
+ asus->fan_boost_mode = ASUS_FAN_BOOST_MODE_NORMAL;
}
- return fan_mode_write(asus);
+ return fan_boost_mode_write(asus);
}
-static ssize_t fan_mode_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t fan_boost_mode_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
struct asus_wmi *asus = dev_get_drvdata(dev);
- return scnprintf(buf, PAGE_SIZE, "%d\n", asus->fan_mode);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", asus->fan_boost_mode);
}
-static ssize_t fan_mode_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
+static ssize_t fan_boost_mode_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
{
int result;
u8 new_mode;
-
struct asus_wmi *asus = dev_get_drvdata(dev);
+ u8 mask = asus->fan_boost_mode_mask;
result = kstrtou8(buf, 10, &new_mode);
if (result < 0) {
return result;
}
- if (new_mode == ASUS_FAN_MODE_OVERBOOST) {
- if (!(asus->fan_mode_mask & ASUS_FAN_MODE_OVERBOOST_MASK))
+ if (new_mode == ASUS_FAN_BOOST_MODE_OVERBOOST) {
+ if (!(mask & ASUS_FAN_BOOST_MODE_OVERBOOST_MASK))
return -EINVAL;
- } else if (new_mode == ASUS_FAN_MODE_SILENT) {
- if (!(asus->fan_mode_mask & ASUS_FAN_MODE_SILENT_MASK))
+ } else if (new_mode == ASUS_FAN_BOOST_MODE_SILENT) {
+ if (!(mask & ASUS_FAN_BOOST_MODE_SILENT_MASK))
return -EINVAL;
- } else if (new_mode != ASUS_FAN_MODE_NORMAL) {
+ } else if (new_mode != ASUS_FAN_BOOST_MODE_NORMAL) {
return -EINVAL;
}
- asus->fan_mode = new_mode;
- fan_mode_write(asus);
+ asus->fan_boost_mode = new_mode;
+ fan_boost_mode_write(asus);
return result;
}
-// Fan mode: 0 - normal, 1 - overboost, 2 - silent
-static DEVICE_ATTR_RW(fan_mode);
+// Fan boost mode: 0 - normal, 1 - overboost, 2 - silent
+static DEVICE_ATTR_RW(fan_boost_mode);
/* Backlight ******************************************************************/
return;
}
- if (asus->fan_mode_available && code == NOTIFY_KBD_FBM) {
- fan_mode_switch_next(asus);
+ if (asus->fan_boost_mode_available && code == NOTIFY_KBD_FBM) {
+ fan_boost_mode_switch_next(asus);
return;
}
&dev_attr_touchpad.attr,
&dev_attr_lid_resume.attr,
&dev_attr_als_enable.attr,
- &dev_attr_fan_mode.attr,
+ &dev_attr_fan_boost_mode.attr,
NULL
};
devid = ASUS_WMI_DEVID_LID_RESUME;
else if (attr == &dev_attr_als_enable.attr)
devid = ASUS_WMI_DEVID_ALS_ENABLE;
- else if (attr == &dev_attr_fan_mode.attr)
- ok = asus->fan_mode_available;
+ else if (attr == &dev_attr_fan_boost_mode.attr)
+ ok = asus->fan_boost_mode_available;
if (devid != -1)
ok = !(asus_wmi_get_devstate_simple(asus, devid) < 0);
if (err)
goto fail_platform;
- err = fan_mode_check_present(asus);
+ err = fan_boost_mode_check_present(asus);
if (err)
- goto fail_fan_mode;
+ goto fail_fan_boost_mode;
err = asus_wmi_sysfs_init(asus->platform_device);
if (err)
fail_input:
asus_wmi_sysfs_exit(asus->platform_device);
fail_sysfs:
-fail_fan_mode:
+fail_fan_boost_mode:
fail_platform:
kfree(asus);
return err;
* and Host Control Actions (power cycle or power off after OS shutdown) on
* Dell systems.
*
- * See Documentation/dcdbas.txt for more information.
+ * See Documentation/driver-api/dcdbas.rst for more information.
*
* Copyright (C) 1995-2006 Dell Inc.
*/
* on every time the packet data is written. This driver requires an
* application to break the BIOS image in to fixed sized packet chunks.
*
- * See Documentation/dell_rbu.txt for more info.
+ * See Documentation/driver-api/dell_rbu.rst for more info.
*/
#include <linux/init.h>
#include <linux/module.h>
depends on ISA || COMPILE_TEST
help
Say Y here if you would like support for ISA Plug and Play devices.
- Some information is in <file:Documentation/isapnp.txt>.
+ Some information is in <file:Documentation/driver-api/isapnp.rst>.
If unsure, say Y.
if POWERCAP
# Client driver configurations go here.
+config INTEL_RAPL_CORE
+ tristate
+
config INTEL_RAPL
- tristate "Intel RAPL Support"
+ tristate "Intel RAPL Support via MSR Interface"
depends on X86 && IOSF_MBI
- default n
+ select INTEL_RAPL_CORE
---help---
This enables support for the Intel Running Average Power Limit (RAPL)
- technology which allows power limits to be enforced and monitored on
- modern Intel processors (Sandy Bridge and later).
+ technology via MSR interface, which allows power limits to be enforced
+ and monitored on modern Intel processors (Sandy Bridge and later).
In RAPL, the platform level settings are divided into domains for
fine grained control. These domains include processor package, DRAM
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_POWERCAP) += powercap_sys.o
-obj-$(CONFIG_INTEL_RAPL) += intel_rapl.o
+obj-$(CONFIG_INTEL_RAPL_CORE) += intel_rapl_common.o
+obj-$(CONFIG_INTEL_RAPL) += intel_rapl_msr.o
obj-$(CONFIG_IDLE_INJECT) += idle_inject.o
// SPDX-License-Identifier: GPL-2.0-only
/*
- * Intel Running Average Power Limit (RAPL) Driver
- * Copyright (c) 2013, Intel Corporation.
+ * Common code for Intel Running Average Power Limit (RAPL) support.
+ * Copyright (c) 2019, Intel Corporation.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/cpu.h>
#include <linux/powercap.h>
#include <linux/suspend.h>
-#include <asm/iosf_mbi.h>
+#include <linux/intel_rapl.h>
+#include <linux/processor.h>
+#include <linux/platform_device.h>
-#include <asm/processor.h>
+#include <asm/iosf_mbi.h>
#include <asm/cpu_device_id.h>
#include <asm/intel-family.h>
#define POWER_LIMIT2_MASK (0x7FFFULL<<32)
#define POWER_LIMIT2_ENABLE BIT_ULL(47)
#define POWER_LIMIT2_CLAMP BIT_ULL(48)
-#define POWER_PACKAGE_LOCK BIT_ULL(63)
-#define POWER_PP_LOCK BIT(31)
+#define POWER_HIGH_LOCK BIT_ULL(63)
+#define POWER_LOW_LOCK BIT(31)
#define TIME_WINDOW1_MASK (0x7FULL<<17)
#define TIME_WINDOW2_MASK (0x7FULL<<49)
#define PP_POLICY_MASK 0x1F
/* Non HW constants */
-#define RAPL_PRIMITIVE_DERIVED BIT(1) /* not from raw data */
+#define RAPL_PRIMITIVE_DERIVED BIT(1) /* not from raw data */
#define RAPL_PRIMITIVE_DUMMY BIT(2)
#define TIME_WINDOW_MAX_MSEC 40000
#define TIME_WINDOW_MIN_MSEC 250
-#define ENERGY_UNIT_SCALE 1000 /* scale from driver unit to powercap unit */
+#define ENERGY_UNIT_SCALE 1000 /* scale from driver unit to powercap unit */
enum unit_type {
- ARBITRARY_UNIT, /* no translation */
+ ARBITRARY_UNIT, /* no translation */
POWER_UNIT,
ENERGY_UNIT,
TIME_UNIT,
};
-enum rapl_domain_type {
- RAPL_DOMAIN_PACKAGE, /* entire package/socket */
- RAPL_DOMAIN_PP0, /* core power plane */
- RAPL_DOMAIN_PP1, /* graphics uncore */
- RAPL_DOMAIN_DRAM,/* DRAM control_type */
- RAPL_DOMAIN_PLATFORM, /* PSys control_type */
- RAPL_DOMAIN_MAX,
-};
-
-enum rapl_domain_msr_id {
- RAPL_DOMAIN_MSR_LIMIT,
- RAPL_DOMAIN_MSR_STATUS,
- RAPL_DOMAIN_MSR_PERF,
- RAPL_DOMAIN_MSR_POLICY,
- RAPL_DOMAIN_MSR_INFO,
- RAPL_DOMAIN_MSR_MAX,
-};
-
/* per domain data, some are optional */
-enum rapl_primitives {
- ENERGY_COUNTER,
- POWER_LIMIT1,
- POWER_LIMIT2,
- FW_LOCK,
-
- PL1_ENABLE, /* power limit 1, aka long term */
- PL1_CLAMP, /* allow frequency to go below OS request */
- PL2_ENABLE, /* power limit 2, aka short term, instantaneous */
- PL2_CLAMP,
-
- TIME_WINDOW1, /* long term */
- TIME_WINDOW2, /* short term */
- THERMAL_SPEC_POWER,
- MAX_POWER,
-
- MIN_POWER,
- MAX_TIME_WINDOW,
- THROTTLED_TIME,
- PRIORITY_LEVEL,
-
- /* below are not raw primitive data */
- AVERAGE_POWER,
- NR_RAPL_PRIMITIVES,
-};
-
#define NR_RAW_PRIMITIVES (NR_RAPL_PRIMITIVES - 2)
-/* Can be expanded to include events, etc.*/
-struct rapl_domain_data {
- u64 primitives[NR_RAPL_PRIMITIVES];
- unsigned long timestamp;
-};
-
-struct msrl_action {
- u32 msr_no;
- u64 clear_mask;
- u64 set_mask;
- int err;
-};
-
#define DOMAIN_STATE_INACTIVE BIT(0)
#define DOMAIN_STATE_POWER_LIMIT_SET BIT(1)
#define DOMAIN_STATE_BIOS_LOCKED BIT(2)
-#define NR_POWER_LIMITS (2)
-struct rapl_power_limit {
- struct powercap_zone_constraint *constraint;
- int prim_id; /* primitive ID used to enable */
- struct rapl_domain *domain;
- const char *name;
- u64 last_power_limit;
-};
-
static const char pl1_name[] = "long_term";
static const char pl2_name[] = "short_term";
-struct rapl_package;
-struct rapl_domain {
- const char *name;
- enum rapl_domain_type id;
- int msrs[RAPL_DOMAIN_MSR_MAX];
- struct powercap_zone power_zone;
- struct rapl_domain_data rdd;
- struct rapl_power_limit rpl[NR_POWER_LIMITS];
- u64 attr_map; /* track capabilities */
- unsigned int state;
- unsigned int domain_energy_unit;
- struct rapl_package *rp;
-};
#define power_zone_to_rapl_domain(_zone) \
container_of(_zone, struct rapl_domain, power_zone)
-/* maximum rapl package domain name: package-%d-die-%d */
-#define PACKAGE_DOMAIN_NAME_LENGTH 30
-
-
-/* Each rapl package contains multiple domains, these are the common
- * data across RAPL domains within a package.
- */
-struct rapl_package {
- unsigned int id; /* logical die id, equals physical 1-die systems */
- unsigned int nr_domains;
- unsigned long domain_map; /* bit map of active domains */
- unsigned int power_unit;
- unsigned int energy_unit;
- unsigned int time_unit;
- struct rapl_domain *domains; /* array of domains, sized at runtime */
- struct powercap_zone *power_zone; /* keep track of parent zone */
- unsigned long power_limit_irq; /* keep track of package power limit
- * notify interrupt enable status.
- */
- struct list_head plist;
- int lead_cpu; /* one active cpu per package for access */
- /* Track active cpus */
- struct cpumask cpumask;
- char name[PACKAGE_DOMAIN_NAME_LENGTH];
-};
-
struct rapl_defaults {
u8 floor_freq_reg_addr;
int (*check_unit)(struct rapl_package *rp, int cpu);
void (*set_floor_freq)(struct rapl_domain *rd, bool mode);
u64 (*compute_time_window)(struct rapl_package *rp, u64 val,
- bool to_raw);
+ bool to_raw);
unsigned int dram_domain_energy_unit;
};
static struct rapl_defaults *rapl_defaults;
const char *name;
u64 mask;
int shift;
- enum rapl_domain_msr_id id;
+ enum rapl_domain_reg_id id;
enum unit_type unit;
u32 flag;
};
static void rapl_init_domains(struct rapl_package *rp);
static int rapl_read_data_raw(struct rapl_domain *rd,
- enum rapl_primitives prim,
- bool xlate, u64 *data);
+ enum rapl_primitives prim,
+ bool xlate, u64 *data);
static int rapl_write_data_raw(struct rapl_domain *rd,
- enum rapl_primitives prim,
- unsigned long long value);
+ enum rapl_primitives prim,
+ unsigned long long value);
static u64 rapl_unit_xlate(struct rapl_domain *rd,
- enum unit_type type, u64 value,
- int to_raw);
+ enum unit_type type, u64 value, int to_raw);
static void package_power_limit_irq_save(struct rapl_package *rp);
-static LIST_HEAD(rapl_packages); /* guarded by CPU hotplug lock */
+static LIST_HEAD(rapl_packages); /* guarded by CPU hotplug lock */
-static const char * const rapl_domain_names[] = {
+static const char *const rapl_domain_names[] = {
"package",
"core",
"uncore",
"psys",
};
-static struct powercap_control_type *control_type; /* PowerCap Controller */
-static struct rapl_domain *platform_rapl_domain; /* Platform (PSys) domain */
-
-/* caller to ensure CPU hotplug lock is held */
-static struct rapl_package *rapl_find_package_domain(int cpu)
-{
- int id = topology_logical_die_id(cpu);
- struct rapl_package *rp;
-
- list_for_each_entry(rp, &rapl_packages, plist) {
- if (rp->id == id)
- return rp;
- }
-
- return NULL;
-}
-
-static int get_energy_counter(struct powercap_zone *power_zone, u64 *energy_raw)
+static int get_energy_counter(struct powercap_zone *power_zone,
+ u64 *energy_raw)
{
struct rapl_domain *rd;
u64 energy_now;
static const struct powercap_zone_ops zone_ops[] = {
/* RAPL_DOMAIN_PACKAGE */
{
- .get_energy_uj = get_energy_counter,
- .get_max_energy_range_uj = get_max_energy_counter,
- .release = release_zone,
- .set_enable = set_domain_enable,
- .get_enable = get_domain_enable,
- },
+ .get_energy_uj = get_energy_counter,
+ .get_max_energy_range_uj = get_max_energy_counter,
+ .release = release_zone,
+ .set_enable = set_domain_enable,
+ .get_enable = get_domain_enable,
+ },
/* RAPL_DOMAIN_PP0 */
{
- .get_energy_uj = get_energy_counter,
- .get_max_energy_range_uj = get_max_energy_counter,
- .release = release_zone,
- .set_enable = set_domain_enable,
- .get_enable = get_domain_enable,
- },
+ .get_energy_uj = get_energy_counter,
+ .get_max_energy_range_uj = get_max_energy_counter,
+ .release = release_zone,
+ .set_enable = set_domain_enable,
+ .get_enable = get_domain_enable,
+ },
/* RAPL_DOMAIN_PP1 */
{
- .get_energy_uj = get_energy_counter,
- .get_max_energy_range_uj = get_max_energy_counter,
- .release = release_zone,
- .set_enable = set_domain_enable,
- .get_enable = get_domain_enable,
- },
+ .get_energy_uj = get_energy_counter,
+ .get_max_energy_range_uj = get_max_energy_counter,
+ .release = release_zone,
+ .set_enable = set_domain_enable,
+ .get_enable = get_domain_enable,
+ },
/* RAPL_DOMAIN_DRAM */
{
- .get_energy_uj = get_energy_counter,
- .get_max_energy_range_uj = get_max_energy_counter,
- .release = release_zone,
- .set_enable = set_domain_enable,
- .get_enable = get_domain_enable,
- },
+ .get_energy_uj = get_energy_counter,
+ .get_max_energy_range_uj = get_max_energy_counter,
+ .release = release_zone,
+ .set_enable = set_domain_enable,
+ .get_enable = get_domain_enable,
+ },
/* RAPL_DOMAIN_PLATFORM */
{
- .get_energy_uj = get_energy_counter,
- .get_max_energy_range_uj = get_max_energy_counter,
- .release = release_zone,
- .set_enable = set_domain_enable,
- .get_enable = get_domain_enable,
- },
+ .get_energy_uj = get_energy_counter,
+ .get_max_energy_range_uj = get_max_energy_counter,
+ .release = release_zone,
+ .set_enable = set_domain_enable,
+ .get_enable = get_domain_enable,
+ },
};
-
/*
* Constraint index used by powercap can be different than power limit (PL)
- * index in that some PLs maybe missing due to non-existant MSRs. So we
+ * index in that some PLs maybe missing due to non-existent MSRs. So we
* need to convert here by finding the valid PLs only (name populated).
*/
static int contraint_to_pl(struct rapl_domain *rd, int cid)
}
static int set_power_limit(struct powercap_zone *power_zone, int cid,
- u64 power_limit)
+ u64 power_limit)
{
struct rapl_domain *rd;
struct rapl_package *rp;
rp = rd->rp;
if (rd->state & DOMAIN_STATE_BIOS_LOCKED) {
- dev_warn(&power_zone->dev, "%s locked by BIOS, monitoring only\n",
- rd->name);
+ dev_warn(&power_zone->dev,
+ "%s locked by BIOS, monitoring only\n", rd->name);
ret = -EACCES;
goto set_exit;
}
}
static int get_current_power_limit(struct powercap_zone *power_zone, int cid,
- u64 *data)
+ u64 *data)
{
struct rapl_domain *rd;
u64 val;
}
static int set_time_window(struct powercap_zone *power_zone, int cid,
- u64 window)
+ u64 window)
{
struct rapl_domain *rd;
int ret = 0;
return ret;
}
-static int get_time_window(struct powercap_zone *power_zone, int cid, u64 *data)
+static int get_time_window(struct powercap_zone *power_zone, int cid,
+ u64 *data)
{
struct rapl_domain *rd;
u64 val;
return ret;
}
-static const char *get_constraint_name(struct powercap_zone *power_zone, int cid)
+static const char *get_constraint_name(struct powercap_zone *power_zone,
+ int cid)
{
struct rapl_domain *rd;
int id;
return NULL;
}
-
-static int get_max_power(struct powercap_zone *power_zone, int id,
- u64 *data)
+static int get_max_power(struct powercap_zone *power_zone, int id, u64 *data)
{
struct rapl_domain *rd;
u64 val;
/* called after domain detection and package level data are set */
static void rapl_init_domains(struct rapl_package *rp)
{
- int i;
+ enum rapl_domain_type i;
+ enum rapl_domain_reg_id j;
struct rapl_domain *rd = rp->domains;
for (i = 0; i < RAPL_DOMAIN_MAX; i++) {
unsigned int mask = rp->domain_map & (1 << i);
- switch (mask) {
- case BIT(RAPL_DOMAIN_PACKAGE):
- rd->name = rapl_domain_names[RAPL_DOMAIN_PACKAGE];
- rd->id = RAPL_DOMAIN_PACKAGE;
- rd->msrs[0] = MSR_PKG_POWER_LIMIT;
- rd->msrs[1] = MSR_PKG_ENERGY_STATUS;
- rd->msrs[2] = MSR_PKG_PERF_STATUS;
- rd->msrs[3] = 0;
- rd->msrs[4] = MSR_PKG_POWER_INFO;
- rd->rpl[0].prim_id = PL1_ENABLE;
- rd->rpl[0].name = pl1_name;
+
+ if (!mask)
+ continue;
+
+ rd->rp = rp;
+ rd->name = rapl_domain_names[i];
+ rd->id = i;
+ rd->rpl[0].prim_id = PL1_ENABLE;
+ rd->rpl[0].name = pl1_name;
+ /* some domain may support two power limits */
+ if (rp->priv->limits[i] == 2) {
rd->rpl[1].prim_id = PL2_ENABLE;
rd->rpl[1].name = pl2_name;
- break;
- case BIT(RAPL_DOMAIN_PP0):
- rd->name = rapl_domain_names[RAPL_DOMAIN_PP0];
- rd->id = RAPL_DOMAIN_PP0;
- rd->msrs[0] = MSR_PP0_POWER_LIMIT;
- rd->msrs[1] = MSR_PP0_ENERGY_STATUS;
- rd->msrs[2] = 0;
- rd->msrs[3] = MSR_PP0_POLICY;
- rd->msrs[4] = 0;
- rd->rpl[0].prim_id = PL1_ENABLE;
- rd->rpl[0].name = pl1_name;
- break;
- case BIT(RAPL_DOMAIN_PP1):
- rd->name = rapl_domain_names[RAPL_DOMAIN_PP1];
- rd->id = RAPL_DOMAIN_PP1;
- rd->msrs[0] = MSR_PP1_POWER_LIMIT;
- rd->msrs[1] = MSR_PP1_ENERGY_STATUS;
- rd->msrs[2] = 0;
- rd->msrs[3] = MSR_PP1_POLICY;
- rd->msrs[4] = 0;
- rd->rpl[0].prim_id = PL1_ENABLE;
- rd->rpl[0].name = pl1_name;
- break;
- case BIT(RAPL_DOMAIN_DRAM):
- rd->name = rapl_domain_names[RAPL_DOMAIN_DRAM];
- rd->id = RAPL_DOMAIN_DRAM;
- rd->msrs[0] = MSR_DRAM_POWER_LIMIT;
- rd->msrs[1] = MSR_DRAM_ENERGY_STATUS;
- rd->msrs[2] = MSR_DRAM_PERF_STATUS;
- rd->msrs[3] = 0;
- rd->msrs[4] = MSR_DRAM_POWER_INFO;
- rd->rpl[0].prim_id = PL1_ENABLE;
- rd->rpl[0].name = pl1_name;
+ }
+
+ for (j = 0; j < RAPL_DOMAIN_REG_MAX; j++)
+ rd->regs[j] = rp->priv->regs[i][j];
+
+ if (i == RAPL_DOMAIN_DRAM) {
rd->domain_energy_unit =
- rapl_defaults->dram_domain_energy_unit;
+ rapl_defaults->dram_domain_energy_unit;
if (rd->domain_energy_unit)
pr_info("DRAM domain energy unit %dpj\n",
rd->domain_energy_unit);
- break;
- }
- if (mask) {
- rd->rp = rp;
- rd++;
}
+ rd++;
}
}
static u64 rapl_unit_xlate(struct rapl_domain *rd, enum unit_type type,
- u64 value, int to_raw)
+ u64 value, int to_raw)
{
u64 units = 1;
struct rapl_package *rp = rd->rp;
static struct rapl_primitive_info rpi[] = {
/* name, mask, shift, msr index, unit divisor */
PRIMITIVE_INFO_INIT(ENERGY_COUNTER, ENERGY_STATUS_MASK, 0,
- RAPL_DOMAIN_MSR_STATUS, ENERGY_UNIT, 0),
+ RAPL_DOMAIN_REG_STATUS, ENERGY_UNIT, 0),
PRIMITIVE_INFO_INIT(POWER_LIMIT1, POWER_LIMIT1_MASK, 0,
- RAPL_DOMAIN_MSR_LIMIT, POWER_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0),
PRIMITIVE_INFO_INIT(POWER_LIMIT2, POWER_LIMIT2_MASK, 32,
- RAPL_DOMAIN_MSR_LIMIT, POWER_UNIT, 0),
- PRIMITIVE_INFO_INIT(FW_LOCK, POWER_PP_LOCK, 31,
- RAPL_DOMAIN_MSR_LIMIT, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, POWER_UNIT, 0),
+ PRIMITIVE_INFO_INIT(FW_LOCK, POWER_LOW_LOCK, 31,
+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
PRIMITIVE_INFO_INIT(PL1_ENABLE, POWER_LIMIT1_ENABLE, 15,
- RAPL_DOMAIN_MSR_LIMIT, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
PRIMITIVE_INFO_INIT(PL1_CLAMP, POWER_LIMIT1_CLAMP, 16,
- RAPL_DOMAIN_MSR_LIMIT, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
PRIMITIVE_INFO_INIT(PL2_ENABLE, POWER_LIMIT2_ENABLE, 47,
- RAPL_DOMAIN_MSR_LIMIT, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
PRIMITIVE_INFO_INIT(PL2_CLAMP, POWER_LIMIT2_CLAMP, 48,
- RAPL_DOMAIN_MSR_LIMIT, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, ARBITRARY_UNIT, 0),
PRIMITIVE_INFO_INIT(TIME_WINDOW1, TIME_WINDOW1_MASK, 17,
- RAPL_DOMAIN_MSR_LIMIT, TIME_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0),
PRIMITIVE_INFO_INIT(TIME_WINDOW2, TIME_WINDOW2_MASK, 49,
- RAPL_DOMAIN_MSR_LIMIT, TIME_UNIT, 0),
+ RAPL_DOMAIN_REG_LIMIT, TIME_UNIT, 0),
PRIMITIVE_INFO_INIT(THERMAL_SPEC_POWER, POWER_INFO_THERMAL_SPEC_MASK,
- 0, RAPL_DOMAIN_MSR_INFO, POWER_UNIT, 0),
+ 0, RAPL_DOMAIN_REG_INFO, POWER_UNIT, 0),
PRIMITIVE_INFO_INIT(MAX_POWER, POWER_INFO_MAX_MASK, 32,
- RAPL_DOMAIN_MSR_INFO, POWER_UNIT, 0),
+ RAPL_DOMAIN_REG_INFO, POWER_UNIT, 0),
PRIMITIVE_INFO_INIT(MIN_POWER, POWER_INFO_MIN_MASK, 16,
- RAPL_DOMAIN_MSR_INFO, POWER_UNIT, 0),
+ RAPL_DOMAIN_REG_INFO, POWER_UNIT, 0),
PRIMITIVE_INFO_INIT(MAX_TIME_WINDOW, POWER_INFO_MAX_TIME_WIN_MASK, 48,
- RAPL_DOMAIN_MSR_INFO, TIME_UNIT, 0),
+ RAPL_DOMAIN_REG_INFO, TIME_UNIT, 0),
PRIMITIVE_INFO_INIT(THROTTLED_TIME, PERF_STATUS_THROTTLE_TIME_MASK, 0,
- RAPL_DOMAIN_MSR_PERF, TIME_UNIT, 0),
+ RAPL_DOMAIN_REG_PERF, TIME_UNIT, 0),
PRIMITIVE_INFO_INIT(PRIORITY_LEVEL, PP_POLICY_MASK, 0,
- RAPL_DOMAIN_MSR_POLICY, ARBITRARY_UNIT, 0),
+ RAPL_DOMAIN_REG_POLICY, ARBITRARY_UNIT, 0),
/* non-hardware */
PRIMITIVE_INFO_INIT(AVERAGE_POWER, 0, 0, 0, POWER_UNIT,
- RAPL_PRIMITIVE_DERIVED),
+ RAPL_PRIMITIVE_DERIVED),
{NULL, 0, 0, 0},
};
* 63-------------------------- 31--------------------------- 0
*/
static int rapl_read_data_raw(struct rapl_domain *rd,
- enum rapl_primitives prim,
- bool xlate, u64 *data)
+ enum rapl_primitives prim, bool xlate, u64 *data)
{
- u64 value, final;
- u32 msr;
+ u64 value;
struct rapl_primitive_info *rp = &rpi[prim];
+ struct reg_action ra;
int cpu;
if (!rp->name || rp->flag & RAPL_PRIMITIVE_DUMMY)
return -EINVAL;
- msr = rd->msrs[rp->id];
- if (!msr)
+ ra.reg = rd->regs[rp->id];
+ if (!ra.reg)
return -EINVAL;
cpu = rd->rp->lead_cpu;
- /* special-case package domain, which uses a different bit*/
- if (prim == FW_LOCK && rd->id == RAPL_DOMAIN_PACKAGE) {
- rp->mask = POWER_PACKAGE_LOCK;
+ /* domain with 2 limits has different bit */
+ if (prim == FW_LOCK && rd->rp->priv->limits[rd->id] == 2) {
+ rp->mask = POWER_HIGH_LOCK;
rp->shift = 63;
}
/* non-hardware data are collected by the polling thread */
return 0;
}
- if (rdmsrl_safe_on_cpu(cpu, msr, &value)) {
- pr_debug("failed to read msr 0x%x on cpu %d\n", msr, cpu);
+ ra.mask = rp->mask;
+
+ if (rd->rp->priv->read_raw(cpu, &ra)) {
+ pr_debug("failed to read reg 0x%llx on cpu %d\n", ra.reg, cpu);
return -EIO;
}
- final = value & rp->mask;
- final = final >> rp->shift;
+ value = ra.value >> rp->shift;
+
if (xlate)
- *data = rapl_unit_xlate(rd, rp->unit, final, 0);
+ *data = rapl_unit_xlate(rd, rp->unit, value, 0);
else
- *data = final;
+ *data = value;
return 0;
}
-
-static int msrl_update_safe(u32 msr_no, u64 clear_mask, u64 set_mask)
-{
- int err;
- u64 val;
-
- err = rdmsrl_safe(msr_no, &val);
- if (err)
- goto out;
-
- val &= ~clear_mask;
- val |= set_mask;
-
- err = wrmsrl_safe(msr_no, val);
-
-out:
- return err;
-}
-
-static void msrl_update_func(void *info)
-{
- struct msrl_action *ma = info;
-
- ma->err = msrl_update_safe(ma->msr_no, ma->clear_mask, ma->set_mask);
-}
-
/* Similar use of primitive info in the read counterpart */
static int rapl_write_data_raw(struct rapl_domain *rd,
- enum rapl_primitives prim,
- unsigned long long value)
+ enum rapl_primitives prim,
+ unsigned long long value)
{
struct rapl_primitive_info *rp = &rpi[prim];
int cpu;
u64 bits;
- struct msrl_action ma;
+ struct reg_action ra;
int ret;
cpu = rd->rp->lead_cpu;
bits <<= rp->shift;
bits &= rp->mask;
- memset(&ma, 0, sizeof(ma));
+ memset(&ra, 0, sizeof(ra));
- ma.msr_no = rd->msrs[rp->id];
- ma.clear_mask = rp->mask;
- ma.set_mask = bits;
+ ra.reg = rd->regs[rp->id];
+ ra.mask = rp->mask;
+ ra.value = bits;
- ret = smp_call_function_single(cpu, msrl_update_func, &ma, 1);
- if (ret)
- WARN_ON_ONCE(ret);
- else
- ret = ma.err;
+ ret = rd->rp->priv->write_raw(cpu, &ra);
return ret;
}
*/
static int rapl_check_unit_core(struct rapl_package *rp, int cpu)
{
- u64 msr_val;
+ struct reg_action ra;
u32 value;
- if (rdmsrl_safe_on_cpu(cpu, MSR_RAPL_POWER_UNIT, &msr_val)) {
- pr_err("Failed to read power unit MSR 0x%x on CPU %d, exit.\n",
- MSR_RAPL_POWER_UNIT, cpu);
+ ra.reg = rp->priv->reg_unit;
+ ra.mask = ~0;
+ if (rp->priv->read_raw(cpu, &ra)) {
+ pr_err("Failed to read power unit REG 0x%llx on CPU %d, exit.\n",
+ rp->priv->reg_unit, cpu);
return -ENODEV;
}
- value = (msr_val & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
+ value = (ra.value & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
rp->energy_unit = ENERGY_UNIT_SCALE * 1000000 / (1 << value);
- value = (msr_val & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
+ value = (ra.value & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
rp->power_unit = 1000000 / (1 << value);
- value = (msr_val & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
+ value = (ra.value & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
rp->time_unit = 1000000 / (1 << value);
pr_debug("Core CPU %s energy=%dpJ, time=%dus, power=%duW\n",
- rp->name, rp->energy_unit, rp->time_unit, rp->power_unit);
+ rp->name, rp->energy_unit, rp->time_unit, rp->power_unit);
return 0;
}
static int rapl_check_unit_atom(struct rapl_package *rp, int cpu)
{
- u64 msr_val;
+ struct reg_action ra;
u32 value;
- if (rdmsrl_safe_on_cpu(cpu, MSR_RAPL_POWER_UNIT, &msr_val)) {
- pr_err("Failed to read power unit MSR 0x%x on CPU %d, exit.\n",
- MSR_RAPL_POWER_UNIT, cpu);
+ ra.reg = rp->priv->reg_unit;
+ ra.mask = ~0;
+ if (rp->priv->read_raw(cpu, &ra)) {
+ pr_err("Failed to read power unit REG 0x%llx on CPU %d, exit.\n",
+ rp->priv->reg_unit, cpu);
return -ENODEV;
}
- value = (msr_val & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
+
+ value = (ra.value & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
rp->energy_unit = ENERGY_UNIT_SCALE * 1 << value;
- value = (msr_val & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
+ value = (ra.value & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
rp->power_unit = (1 << value) * 1000;
- value = (msr_val & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
+ value = (ra.value & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
rp->time_unit = 1000000 / (1 << value);
pr_debug("Atom %s energy=%dpJ, time=%dus, power=%duW\n",
- rp->name, rp->energy_unit, rp->time_unit, rp->power_unit);
+ rp->name, rp->energy_unit, rp->time_unit, rp->power_unit);
return 0;
}
wrmsr_safe(MSR_IA32_PACKAGE_THERM_INTERRUPT, l, h);
}
-
/* REVISIT:
* When package power limit is set artificially low by RAPL, LVT
* thermal interrupt for package power limit should be ignored
}
static u64 rapl_compute_time_window_core(struct rapl_package *rp, u64 value,
- bool to_raw)
+ bool to_raw)
{
- u64 f, y; /* fraction and exp. used for time unit */
+ u64 f, y; /* fraction and exp. used for time unit */
/*
* Special processing based on 2^Y*(1+F/4), refer
}
static u64 rapl_compute_time_window_atom(struct rapl_package *rp, u64 value,
- bool to_raw)
+ bool to_raw)
{
/*
* Atom time unit encoding is straight forward val * time_unit,
*/
if (!to_raw)
return (value) ? value *= rp->time_unit : rp->time_unit;
- else
- value = div64_u64(value, rp->time_unit);
+
+ value = div64_u64(value, rp->time_unit);
return value;
}
};
static const struct x86_cpu_id rapl_ids[] __initconst = {
- INTEL_CPU_FAM6(SANDYBRIDGE, rapl_defaults_core),
- INTEL_CPU_FAM6(SANDYBRIDGE_X, rapl_defaults_core),
-
- INTEL_CPU_FAM6(IVYBRIDGE, rapl_defaults_core),
- INTEL_CPU_FAM6(IVYBRIDGE_X, rapl_defaults_core),
-
- INTEL_CPU_FAM6(HASWELL_CORE, rapl_defaults_core),
- INTEL_CPU_FAM6(HASWELL_ULT, rapl_defaults_core),
- INTEL_CPU_FAM6(HASWELL_GT3E, rapl_defaults_core),
- INTEL_CPU_FAM6(HASWELL_X, rapl_defaults_hsw_server),
-
- INTEL_CPU_FAM6(BROADWELL_CORE, rapl_defaults_core),
- INTEL_CPU_FAM6(BROADWELL_GT3E, rapl_defaults_core),
- INTEL_CPU_FAM6(BROADWELL_XEON_D, rapl_defaults_core),
- INTEL_CPU_FAM6(BROADWELL_X, rapl_defaults_hsw_server),
-
- INTEL_CPU_FAM6(SKYLAKE_DESKTOP, rapl_defaults_core),
- INTEL_CPU_FAM6(SKYLAKE_MOBILE, rapl_defaults_core),
- INTEL_CPU_FAM6(SKYLAKE_X, rapl_defaults_hsw_server),
- INTEL_CPU_FAM6(KABYLAKE_MOBILE, rapl_defaults_core),
- INTEL_CPU_FAM6(KABYLAKE_DESKTOP, rapl_defaults_core),
- INTEL_CPU_FAM6(CANNONLAKE_MOBILE, rapl_defaults_core),
- INTEL_CPU_FAM6(ICELAKE_MOBILE, rapl_defaults_core),
-
- INTEL_CPU_FAM6(ATOM_SILVERMONT, rapl_defaults_byt),
- INTEL_CPU_FAM6(ATOM_AIRMONT, rapl_defaults_cht),
- INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, rapl_defaults_tng),
- INTEL_CPU_FAM6(ATOM_AIRMONT_MID, rapl_defaults_ann),
- INTEL_CPU_FAM6(ATOM_GOLDMONT, rapl_defaults_core),
- INTEL_CPU_FAM6(ATOM_GOLDMONT_PLUS, rapl_defaults_core),
- INTEL_CPU_FAM6(ATOM_GOLDMONT_X, rapl_defaults_core),
- INTEL_CPU_FAM6(ATOM_TREMONT_X, rapl_defaults_core),
-
- INTEL_CPU_FAM6(XEON_PHI_KNL, rapl_defaults_hsw_server),
- INTEL_CPU_FAM6(XEON_PHI_KNM, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(SANDYBRIDGE, rapl_defaults_core),
+ INTEL_CPU_FAM6(SANDYBRIDGE_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(IVYBRIDGE, rapl_defaults_core),
+ INTEL_CPU_FAM6(IVYBRIDGE_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(HASWELL_CORE, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_ULT, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_GT3E, rapl_defaults_core),
+ INTEL_CPU_FAM6(HASWELL_X, rapl_defaults_hsw_server),
+
+ INTEL_CPU_FAM6(BROADWELL_CORE, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_GT3E, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_XEON_D, rapl_defaults_core),
+ INTEL_CPU_FAM6(BROADWELL_X, rapl_defaults_hsw_server),
+
+ INTEL_CPU_FAM6(SKYLAKE_DESKTOP, rapl_defaults_core),
+ INTEL_CPU_FAM6(SKYLAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(SKYLAKE_X, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(KABYLAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(KABYLAKE_DESKTOP, rapl_defaults_core),
+ INTEL_CPU_FAM6(CANNONLAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(ICELAKE_MOBILE, rapl_defaults_core),
+ INTEL_CPU_FAM6(ICELAKE_DESKTOP, rapl_defaults_core),
+ INTEL_CPU_FAM6(ICELAKE_NNPI, rapl_defaults_core),
+ INTEL_CPU_FAM6(ICELAKE_X, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(ICELAKE_XEON_D, rapl_defaults_hsw_server),
+
+ INTEL_CPU_FAM6(ATOM_SILVERMONT, rapl_defaults_byt),
+ INTEL_CPU_FAM6(ATOM_AIRMONT, rapl_defaults_cht),
+ INTEL_CPU_FAM6(ATOM_SILVERMONT_MID, rapl_defaults_tng),
+ INTEL_CPU_FAM6(ATOM_AIRMONT_MID, rapl_defaults_ann),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT, rapl_defaults_core),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_PLUS, rapl_defaults_core),
+ INTEL_CPU_FAM6(ATOM_GOLDMONT_X, rapl_defaults_core),
+ INTEL_CPU_FAM6(ATOM_TREMONT_X, rapl_defaults_core),
+
+ INTEL_CPU_FAM6(XEON_PHI_KNL, rapl_defaults_hsw_server),
+ INTEL_CPU_FAM6(XEON_PHI_KNM, rapl_defaults_hsw_server),
{}
};
+
MODULE_DEVICE_TABLE(x86cpu, rapl_ids);
/* Read once for all raw primitive data for domains */
for (prim = 0; prim < NR_RAW_PRIMITIVES; prim++) {
if (!rapl_read_data_raw(&rp->domains[dmn], prim,
rpi[prim].unit, &val))
- rp->domains[dmn].rdd.primitives[prim] = val;
+ rp->domains[dmn].rdd.primitives[prim] = val;
}
}
}
-static void rapl_unregister_powercap(void)
-{
- if (platform_rapl_domain) {
- powercap_unregister_zone(control_type,
- &platform_rapl_domain->power_zone);
- kfree(platform_rapl_domain);
- }
- powercap_unregister_control_type(control_type);
-}
-
static int rapl_package_register_powercap(struct rapl_package *rp)
{
struct rapl_domain *rd;
/* Update the domain data of the new package */
rapl_update_domain_data(rp);
- /* first we register package domain as the parent zone*/
+ /* first we register package domain as the parent zone */
for (rd = rp->domains; rd < rp->domains + rp->nr_domains; rd++) {
if (rd->id == RAPL_DOMAIN_PACKAGE) {
nr_pl = find_nr_power_limit(rd);
pr_debug("register package domain %s\n", rp->name);
power_zone = powercap_register_zone(&rd->power_zone,
- control_type,
- rp->name, NULL,
- &zone_ops[rd->id],
- nr_pl,
- &constraint_ops);
+ rp->priv->control_type, rp->name,
+ NULL, &zone_ops[rd->id], nr_pl,
+ &constraint_ops);
if (IS_ERR(power_zone)) {
pr_debug("failed to register power zone %s\n",
- rp->name);
+ rp->name);
return PTR_ERR(power_zone);
}
/* track parent zone in per package/socket data */
pr_err("no package domain found, unknown topology!\n");
return -ENODEV;
}
- /* now register domains as children of the socket/package*/
+ /* now register domains as children of the socket/package */
for (rd = rp->domains; rd < rp->domains + rp->nr_domains; rd++) {
if (rd->id == RAPL_DOMAIN_PACKAGE)
continue;
/* number of power limits per domain varies */
nr_pl = find_nr_power_limit(rd);
power_zone = powercap_register_zone(&rd->power_zone,
- control_type, rd->name,
- rp->power_zone,
- &zone_ops[rd->id], nr_pl,
- &constraint_ops);
+ rp->priv->control_type,
+ rd->name, rp->power_zone,
+ &zone_ops[rd->id], nr_pl,
+ &constraint_ops);
if (IS_ERR(power_zone)) {
pr_debug("failed to register power_zone, %s:%s\n",
- rp->name, rd->name);
+ rp->name, rd->name);
ret = PTR_ERR(power_zone);
goto err_cleanup;
}
*/
while (--rd >= rp->domains) {
pr_debug("unregister %s domain %s\n", rp->name, rd->name);
- powercap_unregister_zone(control_type, &rd->power_zone);
+ powercap_unregister_zone(rp->priv->control_type,
+ &rd->power_zone);
}
return ret;
}
-static int __init rapl_register_psys(void)
+int rapl_add_platform_domain(struct rapl_if_priv *priv)
{
struct rapl_domain *rd;
struct powercap_zone *power_zone;
- u64 val;
+ struct reg_action ra;
+ int ret;
- if (rdmsrl_safe_on_cpu(0, MSR_PLATFORM_ENERGY_STATUS, &val) || !val)
+ ra.reg = priv->regs[RAPL_DOMAIN_PLATFORM][RAPL_DOMAIN_REG_STATUS];
+ ra.mask = ~0;
+ ret = priv->read_raw(0, &ra);
+ if (ret || !ra.value)
return -ENODEV;
- if (rdmsrl_safe_on_cpu(0, MSR_PLATFORM_POWER_LIMIT, &val) || !val)
+ ra.reg = priv->regs[RAPL_DOMAIN_PLATFORM][RAPL_DOMAIN_REG_LIMIT];
+ ra.mask = ~0;
+ ret = priv->read_raw(0, &ra);
+ if (ret || !ra.value)
return -ENODEV;
rd = kzalloc(sizeof(*rd), GFP_KERNEL);
rd->name = rapl_domain_names[RAPL_DOMAIN_PLATFORM];
rd->id = RAPL_DOMAIN_PLATFORM;
- rd->msrs[0] = MSR_PLATFORM_POWER_LIMIT;
- rd->msrs[1] = MSR_PLATFORM_ENERGY_STATUS;
+ rd->regs[RAPL_DOMAIN_REG_LIMIT] =
+ priv->regs[RAPL_DOMAIN_PLATFORM][RAPL_DOMAIN_REG_LIMIT];
+ rd->regs[RAPL_DOMAIN_REG_STATUS] =
+ priv->regs[RAPL_DOMAIN_PLATFORM][RAPL_DOMAIN_REG_STATUS];
rd->rpl[0].prim_id = PL1_ENABLE;
rd->rpl[0].name = pl1_name;
rd->rpl[1].prim_id = PL2_ENABLE;
rd->rpl[1].name = pl2_name;
- rd->rp = rapl_find_package_domain(0);
+ rd->rp = rapl_find_package_domain(0, priv);
- power_zone = powercap_register_zone(&rd->power_zone, control_type,
+ power_zone = powercap_register_zone(&rd->power_zone, priv->control_type,
"psys", NULL,
&zone_ops[RAPL_DOMAIN_PLATFORM],
2, &constraint_ops);
return PTR_ERR(power_zone);
}
- platform_rapl_domain = rd;
+ priv->platform_rapl_domain = rd;
return 0;
}
+EXPORT_SYMBOL_GPL(rapl_add_platform_domain);
-static int __init rapl_register_powercap(void)
+void rapl_remove_platform_domain(struct rapl_if_priv *priv)
{
- control_type = powercap_register_control_type(NULL, "intel-rapl", NULL);
- if (IS_ERR(control_type)) {
- pr_debug("failed to register powercap control_type.\n");
- return PTR_ERR(control_type);
+ if (priv->platform_rapl_domain) {
+ powercap_unregister_zone(priv->control_type,
+ &priv->platform_rapl_domain->power_zone);
+ kfree(priv->platform_rapl_domain);
}
- return 0;
}
+EXPORT_SYMBOL_GPL(rapl_remove_platform_domain);
-static int rapl_check_domain(int cpu, int domain)
+static int rapl_check_domain(int cpu, int domain, struct rapl_package *rp)
{
- unsigned msr;
- u64 val = 0;
+ struct reg_action ra;
switch (domain) {
case RAPL_DOMAIN_PACKAGE:
- msr = MSR_PKG_ENERGY_STATUS;
- break;
case RAPL_DOMAIN_PP0:
- msr = MSR_PP0_ENERGY_STATUS;
- break;
case RAPL_DOMAIN_PP1:
- msr = MSR_PP1_ENERGY_STATUS;
- break;
case RAPL_DOMAIN_DRAM:
- msr = MSR_DRAM_ENERGY_STATUS;
+ ra.reg = rp->priv->regs[domain][RAPL_DOMAIN_REG_STATUS];
break;
case RAPL_DOMAIN_PLATFORM:
/* PSYS(PLATFORM) is not a CPU domain, so avoid printng error */
/* make sure domain counters are available and contains non-zero
* values, otherwise skip it.
*/
- if (rdmsrl_safe_on_cpu(cpu, msr, &val) || !val)
+
+ ra.mask = ~0;
+ if (rp->priv->read_raw(cpu, &ra) || !ra.value)
return -ENODEV;
return 0;
}
-
/*
* Check if power limits are available. Two cases when they are not available:
* 1. Locked by BIOS, in this case we still provide read-only access so that
* users can see what limit is set by the BIOS.
* 2. Some CPUs make some domains monitoring only which means PLx MSRs may not
- * exist at all. In this case, we do not show the contraints in powercap.
+ * exist at all. In this case, we do not show the constraints in powercap.
*
* Called after domains are detected and initialized.
*/
rd->state |= DOMAIN_STATE_BIOS_LOCKED;
}
}
- /* check if power limit MSRs exists, otherwise domain is monitoring only */
+ /* check if power limit MSR exists, otherwise domain is monitoring only */
for (i = 0; i < NR_POWER_LIMITS; i++) {
int prim = rd->rpl[i].prim_id;
+
if (rapl_read_data_raw(rd, prim, false, &val64))
rd->rpl[i].name = NULL;
}
for (i = 0; i < RAPL_DOMAIN_MAX; i++) {
/* use physical package id to read counters */
- if (!rapl_check_domain(cpu, i)) {
+ if (!rapl_check_domain(cpu, i, rp)) {
rp->domain_map |= 1 << i;
pr_info("Found RAPL domain %s\n", rapl_domain_names[i]);
}
}
- rp->nr_domains = bitmap_weight(&rp->domain_map, RAPL_DOMAIN_MAX);
+ rp->nr_domains = bitmap_weight(&rp->domain_map, RAPL_DOMAIN_MAX);
if (!rp->nr_domains) {
pr_debug("no valid rapl domains found in %s\n", rp->name);
return -ENODEV;
pr_debug("found %d domains on %s\n", rp->nr_domains, rp->name);
rp->domains = kcalloc(rp->nr_domains + 1, sizeof(struct rapl_domain),
- GFP_KERNEL);
+ GFP_KERNEL);
if (!rp->domains)
return -ENOMEM;
}
/* called from CPU hotplug notifier, hotplug lock held */
-static void rapl_remove_package(struct rapl_package *rp)
+void rapl_remove_package(struct rapl_package *rp)
{
struct rapl_domain *rd, *rd_package = NULL;
}
pr_debug("remove package, undo power limit on %s: %s\n",
rp->name, rd->name);
- powercap_unregister_zone(control_type, &rd->power_zone);
+ powercap_unregister_zone(rp->priv->control_type,
+ &rd->power_zone);
}
/* do parent zone last */
- powercap_unregister_zone(control_type, &rd_package->power_zone);
+ powercap_unregister_zone(rp->priv->control_type,
+ &rd_package->power_zone);
list_del(&rp->plist);
kfree(rp);
}
+EXPORT_SYMBOL_GPL(rapl_remove_package);
+
+/* caller to ensure CPU hotplug lock is held */
+struct rapl_package *rapl_find_package_domain(int cpu, struct rapl_if_priv *priv)
+{
+ int id = topology_logical_die_id(cpu);
+ struct rapl_package *rp;
+
+ list_for_each_entry(rp, &rapl_packages, plist) {
+ if (rp->id == id
+ && rp->priv->control_type == priv->control_type)
+ return rp;
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(rapl_find_package_domain);
/* called from CPU hotplug notifier, hotplug lock held */
-static struct rapl_package *rapl_add_package(int cpu)
+struct rapl_package *rapl_add_package(int cpu, struct rapl_if_priv *priv)
{
int id = topology_logical_die_id(cpu);
struct rapl_package *rp;
/* add the new package to the list */
rp->id = id;
rp->lead_cpu = cpu;
+ rp->priv = priv;
if (topology_max_die_per_package() > 1)
snprintf(rp->name, PACKAGE_DOMAIN_NAME_LENGTH,
- "package-%d-die-%d", c->phys_proc_id, c->cpu_die_id);
+ "package-%d-die-%d", c->phys_proc_id, c->cpu_die_id);
else
snprintf(rp->name, PACKAGE_DOMAIN_NAME_LENGTH, "package-%d",
- c->phys_proc_id);
+ c->phys_proc_id);
/* check if the package contains valid domains */
- if (rapl_detect_domains(rp, cpu) ||
- rapl_defaults->check_unit(rp, cpu)) {
+ if (rapl_detect_domains(rp, cpu) || rapl_defaults->check_unit(rp, cpu)) {
ret = -ENODEV;
goto err_free_package;
}
kfree(rp);
return ERR_PTR(ret);
}
-
-/* Handles CPU hotplug on multi-socket systems.
- * If a CPU goes online as the first CPU of the physical package
- * we add the RAPL package to the system. Similarly, when the last
- * CPU of the package is removed, we remove the RAPL package and its
- * associated domains. Cooling devices are handled accordingly at
- * per-domain level.
- */
-static int rapl_cpu_online(unsigned int cpu)
-{
- struct rapl_package *rp;
-
- rp = rapl_find_package_domain(cpu);
- if (!rp) {
- rp = rapl_add_package(cpu);
- if (IS_ERR(rp))
- return PTR_ERR(rp);
- }
- cpumask_set_cpu(cpu, &rp->cpumask);
- return 0;
-}
-
-static int rapl_cpu_down_prep(unsigned int cpu)
-{
- struct rapl_package *rp;
- int lead_cpu;
-
- rp = rapl_find_package_domain(cpu);
- if (!rp)
- return 0;
-
- cpumask_clear_cpu(cpu, &rp->cpumask);
- lead_cpu = cpumask_first(&rp->cpumask);
- if (lead_cpu >= nr_cpu_ids)
- rapl_remove_package(rp);
- else if (rp->lead_cpu == cpu)
- rp->lead_cpu = lead_cpu;
- return 0;
-}
-
-static enum cpuhp_state pcap_rapl_online;
+EXPORT_SYMBOL_GPL(rapl_add_package);
static void power_limit_state_save(void)
{
switch (rd->rpl[i].prim_id) {
case PL1_ENABLE:
ret = rapl_read_data_raw(rd,
- POWER_LIMIT1,
- true,
- &rd->rpl[i].last_power_limit);
+ POWER_LIMIT1, true,
+ &rd->rpl[i].last_power_limit);
if (ret)
rd->rpl[i].last_power_limit = 0;
break;
case PL2_ENABLE:
ret = rapl_read_data_raw(rd,
- POWER_LIMIT2,
- true,
- &rd->rpl[i].last_power_limit);
+ POWER_LIMIT2, true,
+ &rd->rpl[i].last_power_limit);
if (ret)
rd->rpl[i].last_power_limit = 0;
break;
switch (rd->rpl[i].prim_id) {
case PL1_ENABLE:
if (rd->rpl[i].last_power_limit)
- rapl_write_data_raw(rd,
- POWER_LIMIT1,
- rd->rpl[i].last_power_limit);
+ rapl_write_data_raw(rd, POWER_LIMIT1,
+ rd->rpl[i].last_power_limit);
break;
case PL2_ENABLE:
if (rd->rpl[i].last_power_limit)
- rapl_write_data_raw(rd,
- POWER_LIMIT2,
- rd->rpl[i].last_power_limit);
+ rapl_write_data_raw(rd, POWER_LIMIT2,
+ rd->rpl[i].last_power_limit);
break;
}
}
}
static int rapl_pm_callback(struct notifier_block *nb,
- unsigned long mode, void *_unused)
+ unsigned long mode, void *_unused)
{
switch (mode) {
case PM_SUSPEND_PREPARE:
.notifier_call = rapl_pm_callback,
};
+static struct platform_device *rapl_msr_platdev;
+
static int __init rapl_init(void)
{
const struct x86_cpu_id *id;
id = x86_match_cpu(rapl_ids);
if (!id) {
pr_err("driver does not support CPU family %d model %d\n",
- boot_cpu_data.x86, boot_cpu_data.x86_model);
+ boot_cpu_data.x86, boot_cpu_data.x86_model);
return -ENODEV;
}
rapl_defaults = (struct rapl_defaults *)id->driver_data;
- ret = rapl_register_powercap();
+ ret = register_pm_notifier(&rapl_pm_notifier);
if (ret)
return ret;
- ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "powercap/rapl:online",
- rapl_cpu_online, rapl_cpu_down_prep);
- if (ret < 0)
- goto err_unreg;
- pcap_rapl_online = ret;
-
- /* Don't bail out if PSys is not supported */
- rapl_register_psys();
+ rapl_msr_platdev = platform_device_alloc("intel_rapl_msr", 0);
+ if (!rapl_msr_platdev) {
+ ret = -ENOMEM;
+ goto end;
+ }
- ret = register_pm_notifier(&rapl_pm_notifier);
+ ret = platform_device_add(rapl_msr_platdev);
if (ret)
- goto err_unreg_all;
+ platform_device_put(rapl_msr_platdev);
- return 0;
-
-err_unreg_all:
- cpuhp_remove_state(pcap_rapl_online);
+end:
+ if (ret)
+ unregister_pm_notifier(&rapl_pm_notifier);
-err_unreg:
- rapl_unregister_powercap();
return ret;
}
static void __exit rapl_exit(void)
{
+ platform_device_unregister(rapl_msr_platdev);
unregister_pm_notifier(&rapl_pm_notifier);
- cpuhp_remove_state(pcap_rapl_online);
- rapl_unregister_powercap();
}
module_init(rapl_init);
module_exit(rapl_exit);
-MODULE_DESCRIPTION("Driver for Intel RAPL (Running Average Power Limit)");
+MODULE_DESCRIPTION("Intel Runtime Average Power Limit (RAPL) common code");
MODULE_AUTHOR("Jacob Pan <jacob.jun.pan@intel.com>");
MODULE_LICENSE("GPL v2");
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Running Average Power Limit (RAPL) Driver via MSR interface
+ * Copyright (c) 2019, Intel Corporation.
+ */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/list.h>
+#include <linux/types.h>
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/log2.h>
+#include <linux/bitmap.h>
+#include <linux/delay.h>
+#include <linux/sysfs.h>
+#include <linux/cpu.h>
+#include <linux/powercap.h>
+#include <linux/suspend.h>
+#include <linux/intel_rapl.h>
+#include <linux/processor.h>
+#include <linux/platform_device.h>
+
+#include <asm/iosf_mbi.h>
+#include <asm/cpu_device_id.h>
+#include <asm/intel-family.h>
+
+/* Local defines */
+#define MSR_PLATFORM_POWER_LIMIT 0x0000065C
+
+/* private data for RAPL MSR Interface */
+static struct rapl_if_priv rapl_msr_priv = {
+ .reg_unit = MSR_RAPL_POWER_UNIT,
+ .regs[RAPL_DOMAIN_PACKAGE] = {
+ MSR_PKG_POWER_LIMIT, MSR_PKG_ENERGY_STATUS, MSR_PKG_PERF_STATUS, 0, MSR_PKG_POWER_INFO },
+ .regs[RAPL_DOMAIN_PP0] = {
+ MSR_PP0_POWER_LIMIT, MSR_PP0_ENERGY_STATUS, 0, MSR_PP0_POLICY, 0 },
+ .regs[RAPL_DOMAIN_PP1] = {
+ MSR_PP1_POWER_LIMIT, MSR_PP1_ENERGY_STATUS, 0, MSR_PP1_POLICY, 0 },
+ .regs[RAPL_DOMAIN_DRAM] = {
+ MSR_DRAM_POWER_LIMIT, MSR_DRAM_ENERGY_STATUS, MSR_DRAM_PERF_STATUS, 0, MSR_DRAM_POWER_INFO },
+ .regs[RAPL_DOMAIN_PLATFORM] = {
+ MSR_PLATFORM_POWER_LIMIT, MSR_PLATFORM_ENERGY_STATUS, 0, 0, 0},
+ .limits[RAPL_DOMAIN_PACKAGE] = 2,
+};
+
+/* Handles CPU hotplug on multi-socket systems.
+ * If a CPU goes online as the first CPU of the physical package
+ * we add the RAPL package to the system. Similarly, when the last
+ * CPU of the package is removed, we remove the RAPL package and its
+ * associated domains. Cooling devices are handled accordingly at
+ * per-domain level.
+ */
+static int rapl_cpu_online(unsigned int cpu)
+{
+ struct rapl_package *rp;
+
+ rp = rapl_find_package_domain(cpu, &rapl_msr_priv);
+ if (!rp) {
+ rp = rapl_add_package(cpu, &rapl_msr_priv);
+ if (IS_ERR(rp))
+ return PTR_ERR(rp);
+ }
+ cpumask_set_cpu(cpu, &rp->cpumask);
+ return 0;
+}
+
+static int rapl_cpu_down_prep(unsigned int cpu)
+{
+ struct rapl_package *rp;
+ int lead_cpu;
+
+ rp = rapl_find_package_domain(cpu, &rapl_msr_priv);
+ if (!rp)
+ return 0;
+
+ cpumask_clear_cpu(cpu, &rp->cpumask);
+ lead_cpu = cpumask_first(&rp->cpumask);
+ if (lead_cpu >= nr_cpu_ids)
+ rapl_remove_package(rp);
+ else if (rp->lead_cpu == cpu)
+ rp->lead_cpu = lead_cpu;
+ return 0;
+}
+
+static int rapl_msr_read_raw(int cpu, struct reg_action *ra)
+{
+ u32 msr = (u32)ra->reg;
+
+ if (rdmsrl_safe_on_cpu(cpu, msr, &ra->value)) {
+ pr_debug("failed to read msr 0x%x on cpu %d\n", msr, cpu);
+ return -EIO;
+ }
+ ra->value &= ra->mask;
+ return 0;
+}
+
+static void rapl_msr_update_func(void *info)
+{
+ struct reg_action *ra = info;
+ u32 msr = (u32)ra->reg;
+ u64 val;
+
+ ra->err = rdmsrl_safe(msr, &val);
+ if (ra->err)
+ return;
+
+ val &= ~ra->mask;
+ val |= ra->value;
+
+ ra->err = wrmsrl_safe(msr, val);
+}
+
+static int rapl_msr_write_raw(int cpu, struct reg_action *ra)
+{
+ int ret;
+
+ ret = smp_call_function_single(cpu, rapl_msr_update_func, ra, 1);
+ if (WARN_ON_ONCE(ret))
+ return ret;
+
+ return ra->err;
+}
+
+static int rapl_msr_probe(struct platform_device *pdev)
+{
+ int ret;
+
+ rapl_msr_priv.read_raw = rapl_msr_read_raw;
+ rapl_msr_priv.write_raw = rapl_msr_write_raw;
+
+ rapl_msr_priv.control_type = powercap_register_control_type(NULL, "intel-rapl", NULL);
+ if (IS_ERR(rapl_msr_priv.control_type)) {
+ pr_debug("failed to register powercap control_type.\n");
+ return PTR_ERR(rapl_msr_priv.control_type);
+ }
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "powercap/rapl:online",
+ rapl_cpu_online, rapl_cpu_down_prep);
+ if (ret < 0)
+ goto out;
+ rapl_msr_priv.pcap_rapl_online = ret;
+
+ /* Don't bail out if PSys is not supported */
+ rapl_add_platform_domain(&rapl_msr_priv);
+
+ return 0;
+
+out:
+ if (ret)
+ powercap_unregister_control_type(rapl_msr_priv.control_type);
+ return ret;
+}
+
+static int rapl_msr_remove(struct platform_device *pdev)
+{
+ cpuhp_remove_state(rapl_msr_priv.pcap_rapl_online);
+ rapl_remove_platform_domain(&rapl_msr_priv);
+ powercap_unregister_control_type(rapl_msr_priv.control_type);
+ return 0;
+}
+
+static const struct platform_device_id rapl_msr_ids[] = {
+ { .name = "intel_rapl_msr", },
+ {}
+};
+MODULE_DEVICE_TABLE(platform, rapl_msr_ids);
+
+static struct platform_driver intel_rapl_msr_driver = {
+ .probe = rapl_msr_probe,
+ .remove = rapl_msr_remove,
+ .id_table = rapl_msr_ids,
+ .driver = {
+ .name = "intel_rapl_msr",
+ },
+};
+
+module_platform_driver(intel_rapl_msr_driver);
+
+MODULE_DESCRIPTION("Driver for Intel RAPL (Running Average Power Limit) control via MSR interface");
+MODULE_AUTHOR("Zhang Rui <rui.zhang@intel.com>");
+MODULE_LICENSE("GPL v2");
pps->params.mode |= PPS_CANWAIT;
pps->params.api_version = PPS_API_VERS;
+ /*
+ * Clear unused fields of pps_kparams to avoid leaking
+ * uninitialized data of the PPS_SETPARAMS caller via
+ * PPS_GETPARAMS
+ */
+ pps->params.assert_off_tu.flags = 0;
+ pps->params.clear_off_tu.flags = 0;
+
spin_unlock_irq(&pps->lock);
break;
This option includes RapidIO channelized messaging driver which
provides socket-like interface to allow sharing of single RapidIO
messaging mailbox between multiple user-space applications.
- See "Documentation/rapidio/rio_cm.txt" for driver description.
+ See "Documentation/driver-api/rapidio/rio_cm.rst" for driver description.
config RAPIDIO_MPORT_CDEV
tristate "RapidIO /dev mport device driver"
if (copy_from_user(&dev_info, arg, sizeof(dev_info)))
return -EFAULT;
+ dev_info.name[sizeof(dev_info.name) - 1] = '\0';
rmcd_debug(RDEV, "name:%s ct:0x%x did:0x%x hc:0x%x", dev_info.name,
dev_info.comptag, dev_info.destid, dev_info.hopcount);
if (copy_from_user(&dev_info, arg, sizeof(dev_info)))
return -EFAULT;
+ dev_info.name[sizeof(dev_info.name) - 1] = '\0';
mport = priv->md->mport;
config IMX_REMOTEPROC
tristate "IMX6/7 remoteproc support"
- depends on SOC_IMX6SX || SOC_IMX7D
+ depends on ARCH_MXC
help
Say y here to support iMX's remote processors (Cortex M4
on iMX7D) via the remote processor framework.
depends on RPMSG_QCOM_GLINK_SMEM || RPMSG_QCOM_GLINK_SMEM=n
depends on QCOM_SYSMON || QCOM_SYSMON=n
select MFD_SYSCON
+ select QCOM_MDT_LOADER
select QCOM_Q6V5_COMMON
select QCOM_RPROC_COMMON
select QCOM_SCM
config ST_SLIM_REMOTEPROC
tristate
+config STM32_RPROC
+ tristate "STM32 remoteproc support"
+ depends on ARCH_STM32
+ depends on REMOTEPROC
+ select MAILBOX
+ help
+ Say y here to support STM32 MCU processors via the
+ remote processor framework.
+
+ You want to say y here in order to enable AMP
+ use-cases to run on your platform (dedicated firmware could be
+ offloaded to remote MCU processors using this framework).
+
+ This can be either built-in or a loadable module.
+
endif # REMOTEPROC
endmenu
qcom_wcnss_pil-y += qcom_wcnss_iris.o
obj-$(CONFIG_ST_REMOTEPROC) += st_remoteproc.o
obj-$(CONFIG_ST_SLIM_REMOTEPROC) += st_slim_rproc.o
+obj-$(CONFIG_STM32_RPROC) += stm32_rproc.o
ret = regmap_update_bits(priv->regmap, dcfg->src_reg,
dcfg->src_mask, dcfg->src_start);
if (ret)
- dev_err(dev, "Filed to enable M4!\n");
+ dev_err(dev, "Failed to enable M4!\n");
return ret;
}
ret = regmap_update_bits(priv->regmap, dcfg->src_reg,
dcfg->src_mask, dcfg->src_stop);
if (ret)
- dev_err(dev, "Filed to stop M4!\n");
+ dev_err(dev, "Failed to stop M4!\n");
return ret;
}
}
}
- dev_warn(priv->dev, "Translation filed: da = 0x%llx len = 0x%x\n",
+ dev_warn(priv->dev, "Translation failed: da = 0x%llx len = 0x%x\n",
da, len);
return -ENOENT;
}
ret = imx_rproc_addr_init(priv, pdev);
if (ret) {
- dev_err(dev, "filed on imx_rproc_addr_init\n");
+ dev_err(dev, "failed on imx_rproc_addr_init\n");
goto err_put_rproc;
}
#define LPASS_PWR_ON_REG 0x10
#define LPASS_HALTREQ_REG 0x0
-/* list of clocks required by ADSP PIL */
-static const char * const adsp_clk_id[] = {
- "sway_cbcr", "lpass_ahbs_aon_cbcr", "lpass_ahbm_aon_cbcr",
- "qdsp6ss_xo", "qdsp6ss_sleep", "qdsp6ss_core",
-};
+#define QDSP6SS_XO_CBCR 0x38
+#define QDSP6SS_CORE_CBCR 0x20
+#define QDSP6SS_SLEEP_CBCR 0x3c
struct adsp_pil_data {
int crash_reason_smem;
const char *ssr_name;
const char *sysmon_name;
int ssctl_id;
+
+ const char **clk_ids;
+ int num_clks;
};
struct qcom_adsp {
void __iomem *qdsp6ss_base;
struct reset_control *pdc_sync_reset;
- struct reset_control *cc_lpass_restart;
+ struct reset_control *restart;
struct regmap *halt_map;
unsigned int halt_lpass;
/* Assert the LPASS PDC Reset */
reset_control_assert(adsp->pdc_sync_reset);
/* Place the LPASS processor into reset */
- reset_control_assert(adsp->cc_lpass_restart);
+ reset_control_assert(adsp->restart);
/* wait after asserting subsystem restart from AOSS */
usleep_range(200, 300);
/* De-assert the LPASS PDC Reset */
reset_control_deassert(adsp->pdc_sync_reset);
/* Remove the LPASS reset */
- reset_control_deassert(adsp->cc_lpass_restart);
+ reset_control_deassert(adsp->restart);
/* wait after de-asserting subsystem restart from AOSS */
usleep_range(200, 300);
goto disable_power_domain;
}
+ /* Enable the XO clock */
+ writel(1, adsp->qdsp6ss_base + QDSP6SS_XO_CBCR);
+
+ /* Enable the QDSP6SS sleep clock */
+ writel(1, adsp->qdsp6ss_base + QDSP6SS_SLEEP_CBCR);
+
+ /* Enable the QDSP6 core clock */
+ writel(1, adsp->qdsp6ss_base + QDSP6SS_CORE_CBCR);
+
/* Program boot address */
writel(adsp->mem_phys >> 4, adsp->qdsp6ss_base + RST_EVB_REG);
.load = adsp_load,
};
-static int adsp_init_clock(struct qcom_adsp *adsp)
+static int adsp_init_clock(struct qcom_adsp *adsp, const char **clk_ids)
{
+ int num_clks = 0;
int i, ret;
adsp->xo = devm_clk_get(adsp->dev, "xo");
return ret;
}
- adsp->num_clks = ARRAY_SIZE(adsp_clk_id);
+ for (i = 0; clk_ids[i]; i++)
+ num_clks++;
+
+ adsp->num_clks = num_clks;
adsp->clks = devm_kcalloc(adsp->dev, adsp->num_clks,
sizeof(*adsp->clks), GFP_KERNEL);
if (!adsp->clks)
return -ENOMEM;
for (i = 0; i < adsp->num_clks; i++)
- adsp->clks[i].id = adsp_clk_id[i];
+ adsp->clks[i].id = clk_ids[i];
return devm_clk_bulk_get(adsp->dev, adsp->num_clks, adsp->clks);
}
static int adsp_init_reset(struct qcom_adsp *adsp)
{
- adsp->pdc_sync_reset = devm_reset_control_get_exclusive(adsp->dev,
+ adsp->pdc_sync_reset = devm_reset_control_get_optional_exclusive(adsp->dev,
"pdc_sync");
if (IS_ERR(adsp->pdc_sync_reset)) {
dev_err(adsp->dev, "failed to acquire pdc_sync reset\n");
return PTR_ERR(adsp->pdc_sync_reset);
}
- adsp->cc_lpass_restart = devm_reset_control_get_exclusive(adsp->dev,
- "cc_lpass");
- if (IS_ERR(adsp->cc_lpass_restart)) {
- dev_err(adsp->dev, "failed to acquire cc_lpass restart\n");
- return PTR_ERR(adsp->cc_lpass_restart);
+ adsp->restart = devm_reset_control_get_optional_exclusive(adsp->dev, "restart");
+
+ /* Fall back to the old "cc_lpass" if "restart" is absent */
+ if (!adsp->restart)
+ adsp->restart = devm_reset_control_get_exclusive(adsp->dev, "cc_lpass");
+
+ if (IS_ERR(adsp->restart)) {
+ dev_err(adsp->dev, "failed to acquire restart\n");
+ return PTR_ERR(adsp->restart);
}
return 0;
if (ret)
goto free_rproc;
- ret = adsp_init_clock(adsp);
+ ret = adsp_init_clock(adsp, desc->clk_ids);
if (ret)
goto free_rproc;
.ssr_name = "lpass",
.sysmon_name = "adsp",
.ssctl_id = 0x14,
+ .clk_ids = (const char*[]) {
+ "sway_cbcr", "lpass_ahbs_aon_cbcr", "lpass_ahbm_aon_cbcr",
+ "qdsp6ss_xo", "qdsp6ss_sleep", "qdsp6ss_core", NULL
+ },
+ .num_clks = 7,
+};
+
+static const struct adsp_pil_data cdsp_resource_init = {
+ .crash_reason_smem = 601,
+ .firmware_name = "cdsp.mdt",
+ .ssr_name = "cdsp",
+ .sysmon_name = "cdsp",
+ .ssctl_id = 0x17,
+ .clk_ids = (const char*[]) {
+ "sway", "tbu", "bimc", "ahb_aon", "q6ss_slave", "q6ss_master",
+ "q6_axim", NULL
+ },
+ .num_clks = 7,
};
static const struct of_device_id adsp_of_match[] = {
+ { .compatible = "qcom,qcs404-cdsp-pil", .data = &cdsp_resource_init },
{ .compatible = "qcom,sdm845-adsp-pil", .data = &adsp_resource_init },
{ },
};
{
unsigned long dma_attrs = DMA_ATTR_FORCE_CONTIGUOUS;
dma_addr_t phys;
+ void *metadata;
int mdata_perm;
int xferop_ret;
+ size_t size;
void *ptr;
int ret;
- ptr = dma_alloc_attrs(qproc->dev, fw->size, &phys, GFP_KERNEL, dma_attrs);
+ metadata = qcom_mdt_read_metadata(fw, &size);
+ if (IS_ERR(metadata))
+ return PTR_ERR(metadata);
+
+ ptr = dma_alloc_attrs(qproc->dev, size, &phys, GFP_KERNEL, dma_attrs);
if (!ptr) {
+ kfree(metadata);
dev_err(qproc->dev, "failed to allocate mdt buffer\n");
return -ENOMEM;
}
- memcpy(ptr, fw->data, fw->size);
+ memcpy(ptr, metadata, size);
/* Hypervisor mapping to access metadata by modem */
mdata_perm = BIT(QCOM_SCM_VMID_HLOS);
- ret = q6v5_xfer_mem_ownership(qproc, &mdata_perm,
- true, phys, fw->size);
+ ret = q6v5_xfer_mem_ownership(qproc, &mdata_perm, true, phys, size);
if (ret) {
dev_err(qproc->dev,
"assigning Q6 access to metadata failed: %d\n", ret);
dev_err(qproc->dev, "MPSS header authentication failed: %d\n", ret);
/* Metadata authentication done, remove modem access */
- xferop_ret = q6v5_xfer_mem_ownership(qproc, &mdata_perm,
- false, phys, fw->size);
+ xferop_ret = q6v5_xfer_mem_ownership(qproc, &mdata_perm, false, phys, size);
if (xferop_ret)
dev_warn(qproc->dev,
"mdt buffer not reclaimed system may become unstable\n");
free_dma_attrs:
- dma_free_attrs(qproc->dev, fw->size, ptr, phys, dma_attrs);
+ dma_free_attrs(qproc->dev, size, ptr, phys, dma_attrs);
+ kfree(metadata);
return ret < 0 ? ret : 0;
}
ptr = qproc->mpss_region + offset;
- if (phdr->p_filesz) {
+ if (phdr->p_filesz && phdr->p_offset < fw->size) {
+ /* Firmware is large enough to be non-split */
+ if (phdr->p_offset + phdr->p_filesz > fw->size) {
+ dev_err(qproc->dev,
+ "failed to load segment %d from truncated file %s\n",
+ i, fw_name);
+ ret = -EINVAL;
+ goto release_firmware;
+ }
+
+ memcpy(ptr, fw->data + phdr->p_offset, phdr->p_filesz);
+ } else if (phdr->p_filesz) {
/* Replace "xxx.xxx" with "xxx.bxx" */
sprintf(fw_name + fw_name_len - 3, "b%02d", i);
ret = request_firmware(&seg_fw, fw_name, qproc->dev);
/* Initialise vdev subdevice */
snprintf(name, sizeof(name), "vdev%dbuffer", rvdev->index);
rvdev->dev.parent = rproc->dev.parent;
+ rvdev->dev.dma_pfn_offset = rproc->dev.parent->dma_pfn_offset;
rvdev->dev.release = rproc_rvdev_release;
dev_set_name(&rvdev->dev, "%s#%s", dev_name(rvdev->dev.parent), name);
dev_set_drvdata(&rvdev->dev, rvdev);
dev_dbg(dev, "rsc: type %d\n", hdr->type);
+ if (hdr->type >= RSC_VENDOR_START &&
+ hdr->type <= RSC_VENDOR_END) {
+ ret = rproc_handle_rsc(rproc, hdr->type, rsc,
+ offset + sizeof(*hdr), avail);
+ if (ret == RSC_HANDLED)
+ continue;
+ else if (ret < 0)
+ break;
+
+ dev_warn(dev, "unsupported vendor resource %d\n",
+ hdr->type);
+ continue;
+ }
+
if (hdr->type >= RSC_LAST) {
dev_warn(dev, "unsupported resource %d\n", hdr->type);
continue;
}
/* make sure the offsets array isn't truncated */
- if (table->num * sizeof(table->offset[0]) +
- sizeof(struct resource_table) > size) {
+ if (struct_size(table, offset, table->num) > size) {
dev_err(dev, "resource table incomplete\n");
return NULL;
}
}
static inline
+int rproc_handle_rsc(struct rproc *rproc, u32 rsc_type, void *rsc, int offset,
+ int avail)
+{
+ if (rproc->ops->handle_rsc)
+ return rproc->ops->handle_rsc(rproc, rsc_type, rsc, offset,
+ avail);
+
+ return RSC_IGNORED;
+}
+
+static inline
struct resource_table *rproc_find_loaded_rsc_table(struct rproc *rproc,
const struct firmware *fw)
{
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) STMicroelectronics 2018 - All Rights Reserved
+ * Authors: Ludovic Barre <ludovic.barre@st.com> for STMicroelectronics.
+ * Fabien Dessenne <fabien.dessenne@st.com> for STMicroelectronics.
+ */
+
+#include <linux/arm-smccc.h>
+#include <linux/dma-mapping.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/mailbox_client.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/regmap.h>
+#include <linux/remoteproc.h>
+#include <linux/reset.h>
+
+#include "remoteproc_internal.h"
+
+#define HOLD_BOOT 0
+#define RELEASE_BOOT 1
+
+#define MBOX_NB_VQ 2
+#define MBOX_NB_MBX 3
+
+#define STM32_SMC_RCC 0x82001000
+#define STM32_SMC_REG_WRITE 0x1
+
+#define STM32_MBX_VQ0 "vq0"
+#define STM32_MBX_VQ1 "vq1"
+#define STM32_MBX_SHUTDOWN "shutdown"
+
+struct stm32_syscon {
+ struct regmap *map;
+ u32 reg;
+ u32 mask;
+};
+
+struct stm32_rproc_mem {
+ char name[20];
+ void __iomem *cpu_addr;
+ phys_addr_t bus_addr;
+ u32 dev_addr;
+ size_t size;
+};
+
+struct stm32_rproc_mem_ranges {
+ u32 dev_addr;
+ u32 bus_addr;
+ u32 size;
+};
+
+struct stm32_mbox {
+ const unsigned char name[10];
+ struct mbox_chan *chan;
+ struct mbox_client client;
+ int vq_id;
+};
+
+struct stm32_rproc {
+ struct reset_control *rst;
+ struct stm32_syscon hold_boot;
+ struct stm32_syscon pdds;
+ u32 nb_rmems;
+ struct stm32_rproc_mem *rmems;
+ struct stm32_mbox mb[MBOX_NB_MBX];
+ bool secured_soc;
+};
+
+static int stm32_rproc_pa_to_da(struct rproc *rproc, phys_addr_t pa, u64 *da)
+{
+ unsigned int i;
+ struct stm32_rproc *ddata = rproc->priv;
+ struct stm32_rproc_mem *p_mem;
+
+ for (i = 0; i < ddata->nb_rmems; i++) {
+ p_mem = &ddata->rmems[i];
+
+ if (pa < p_mem->bus_addr ||
+ pa >= p_mem->bus_addr + p_mem->size)
+ continue;
+ *da = pa - p_mem->bus_addr + p_mem->dev_addr;
+ dev_dbg(rproc->dev.parent, "pa %pa to da %llx\n", &pa, *da);
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int stm32_rproc_mem_alloc(struct rproc *rproc,
+ struct rproc_mem_entry *mem)
+{
+ struct device *dev = rproc->dev.parent;
+ void *va;
+
+ dev_dbg(dev, "map memory: %pa+%x\n", &mem->dma, mem->len);
+ va = ioremap_wc(mem->dma, mem->len);
+ if (IS_ERR_OR_NULL(va)) {
+ dev_err(dev, "Unable to map memory region: %pa+%x\n",
+ &mem->dma, mem->len);
+ return -ENOMEM;
+ }
+
+ /* Update memory entry va */
+ mem->va = va;
+
+ return 0;
+}
+
+static int stm32_rproc_mem_release(struct rproc *rproc,
+ struct rproc_mem_entry *mem)
+{
+ dev_dbg(rproc->dev.parent, "unmap memory: %pa\n", &mem->dma);
+ iounmap(mem->va);
+
+ return 0;
+}
+
+static int stm32_rproc_of_memory_translations(struct rproc *rproc)
+{
+ struct device *parent, *dev = rproc->dev.parent;
+ struct stm32_rproc *ddata = rproc->priv;
+ struct device_node *np;
+ struct stm32_rproc_mem *p_mems;
+ struct stm32_rproc_mem_ranges *mem_range;
+ int cnt, array_size, i, ret = 0;
+
+ parent = dev->parent;
+ np = parent->of_node;
+
+ cnt = of_property_count_elems_of_size(np, "dma-ranges",
+ sizeof(*mem_range));
+ if (cnt <= 0) {
+ dev_err(dev, "%s: dma-ranges property not defined\n", __func__);
+ return -EINVAL;
+ }
+
+ p_mems = devm_kcalloc(dev, cnt, sizeof(*p_mems), GFP_KERNEL);
+ if (!p_mems)
+ return -ENOMEM;
+ mem_range = kcalloc(cnt, sizeof(*mem_range), GFP_KERNEL);
+ if (!mem_range)
+ return -ENOMEM;
+
+ array_size = cnt * sizeof(struct stm32_rproc_mem_ranges) / sizeof(u32);
+
+ ret = of_property_read_u32_array(np, "dma-ranges",
+ (u32 *)mem_range, array_size);
+ if (ret) {
+ dev_err(dev, "error while get dma-ranges property: %x\n", ret);
+ goto free_mem;
+ }
+
+ for (i = 0; i < cnt; i++) {
+ p_mems[i].bus_addr = mem_range[i].bus_addr;
+ p_mems[i].dev_addr = mem_range[i].dev_addr;
+ p_mems[i].size = mem_range[i].size;
+
+ dev_dbg(dev, "memory range[%i]: da %#x, pa %pa, size %#zx:\n",
+ i, p_mems[i].dev_addr, &p_mems[i].bus_addr,
+ p_mems[i].size);
+ }
+
+ ddata->rmems = p_mems;
+ ddata->nb_rmems = cnt;
+
+free_mem:
+ kfree(mem_range);
+ return ret;
+}
+
+static int stm32_rproc_mbox_idx(struct rproc *rproc, const unsigned char *name)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(ddata->mb); i++) {
+ if (!strncmp(ddata->mb[i].name, name, strlen(name)))
+ return i;
+ }
+ dev_err(&rproc->dev, "mailbox %s not found\n", name);
+
+ return -EINVAL;
+}
+
+static int stm32_rproc_elf_load_rsc_table(struct rproc *rproc,
+ const struct firmware *fw)
+{
+ if (rproc_elf_load_rsc_table(rproc, fw))
+ dev_warn(&rproc->dev, "no resource table found for this firmware\n");
+
+ return 0;
+}
+
+static int stm32_rproc_parse_fw(struct rproc *rproc, const struct firmware *fw)
+{
+ struct device *dev = rproc->dev.parent;
+ struct device_node *np = dev->of_node;
+ struct of_phandle_iterator it;
+ struct rproc_mem_entry *mem;
+ struct reserved_mem *rmem;
+ u64 da;
+ int index = 0;
+
+ /* Register associated reserved memory regions */
+ of_phandle_iterator_init(&it, np, "memory-region", NULL, 0);
+ while (of_phandle_iterator_next(&it) == 0) {
+ rmem = of_reserved_mem_lookup(it.node);
+ if (!rmem) {
+ dev_err(dev, "unable to acquire memory-region\n");
+ return -EINVAL;
+ }
+
+ if (stm32_rproc_pa_to_da(rproc, rmem->base, &da) < 0) {
+ dev_err(dev, "memory region not valid %pa\n",
+ &rmem->base);
+ return -EINVAL;
+ }
+
+ /* No need to map vdev buffer */
+ if (strcmp(it.node->name, "vdev0buffer")) {
+ /* Register memory region */
+ mem = rproc_mem_entry_init(dev, NULL,
+ (dma_addr_t)rmem->base,
+ rmem->size, da,
+ stm32_rproc_mem_alloc,
+ stm32_rproc_mem_release,
+ it.node->name);
+
+ if (mem)
+ rproc_coredump_add_segment(rproc, da,
+ rmem->size);
+ } else {
+ /* Register reserved memory for vdev buffer alloc */
+ mem = rproc_of_resm_mem_entry_init(dev, index,
+ rmem->size,
+ rmem->base,
+ it.node->name);
+ }
+
+ if (!mem)
+ return -ENOMEM;
+
+ rproc_add_carveout(rproc, mem);
+ index++;
+ }
+
+ return stm32_rproc_elf_load_rsc_table(rproc, fw);
+}
+
+static irqreturn_t stm32_rproc_wdg(int irq, void *data)
+{
+ struct rproc *rproc = data;
+
+ rproc_report_crash(rproc, RPROC_WATCHDOG);
+
+ return IRQ_HANDLED;
+}
+
+static void stm32_rproc_mb_callback(struct mbox_client *cl, void *data)
+{
+ struct rproc *rproc = dev_get_drvdata(cl->dev);
+ struct stm32_mbox *mb = container_of(cl, struct stm32_mbox, client);
+
+ if (rproc_vq_interrupt(rproc, mb->vq_id) == IRQ_NONE)
+ dev_dbg(&rproc->dev, "no message found in vq%d\n", mb->vq_id);
+}
+
+static void stm32_rproc_free_mbox(struct rproc *rproc)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(ddata->mb); i++) {
+ if (ddata->mb[i].chan)
+ mbox_free_channel(ddata->mb[i].chan);
+ ddata->mb[i].chan = NULL;
+ }
+}
+
+static const struct stm32_mbox stm32_rproc_mbox[MBOX_NB_MBX] = {
+ {
+ .name = STM32_MBX_VQ0,
+ .vq_id = 0,
+ .client = {
+ .rx_callback = stm32_rproc_mb_callback,
+ .tx_block = false,
+ },
+ },
+ {
+ .name = STM32_MBX_VQ1,
+ .vq_id = 1,
+ .client = {
+ .rx_callback = stm32_rproc_mb_callback,
+ .tx_block = false,
+ },
+ },
+ {
+ .name = STM32_MBX_SHUTDOWN,
+ .vq_id = -1,
+ .client = {
+ .tx_block = true,
+ .tx_done = NULL,
+ .tx_tout = 500, /* 500 ms time out */
+ },
+ }
+};
+
+static void stm32_rproc_request_mbox(struct rproc *rproc)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ struct device *dev = &rproc->dev;
+ unsigned int i;
+ const unsigned char *name;
+ struct mbox_client *cl;
+
+ /* Initialise mailbox structure table */
+ memcpy(ddata->mb, stm32_rproc_mbox, sizeof(stm32_rproc_mbox));
+
+ for (i = 0; i < MBOX_NB_MBX; i++) {
+ name = ddata->mb[i].name;
+
+ cl = &ddata->mb[i].client;
+ cl->dev = dev->parent;
+
+ ddata->mb[i].chan = mbox_request_channel_byname(cl, name);
+ if (IS_ERR(ddata->mb[i].chan)) {
+ dev_warn(dev, "cannot get %s mbox\n", name);
+ ddata->mb[i].chan = NULL;
+ }
+ }
+}
+
+static int stm32_rproc_set_hold_boot(struct rproc *rproc, bool hold)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ struct stm32_syscon hold_boot = ddata->hold_boot;
+ struct arm_smccc_res smc_res;
+ int val, err;
+
+ val = hold ? HOLD_BOOT : RELEASE_BOOT;
+
+ if (IS_ENABLED(CONFIG_HAVE_ARM_SMCCC) && ddata->secured_soc) {
+ arm_smccc_smc(STM32_SMC_RCC, STM32_SMC_REG_WRITE,
+ hold_boot.reg, val, 0, 0, 0, 0, &smc_res);
+ err = smc_res.a0;
+ } else {
+ err = regmap_update_bits(hold_boot.map, hold_boot.reg,
+ hold_boot.mask, val);
+ }
+
+ if (err)
+ dev_err(&rproc->dev, "failed to set hold boot\n");
+
+ return err;
+}
+
+static void stm32_rproc_add_coredump_trace(struct rproc *rproc)
+{
+ struct rproc_debug_trace *trace;
+ struct rproc_dump_segment *segment;
+ bool already_added;
+
+ list_for_each_entry(trace, &rproc->traces, node) {
+ already_added = false;
+
+ list_for_each_entry(segment, &rproc->dump_segments, node) {
+ if (segment->da == trace->trace_mem.da) {
+ already_added = true;
+ break;
+ }
+ }
+
+ if (!already_added)
+ rproc_coredump_add_segment(rproc, trace->trace_mem.da,
+ trace->trace_mem.len);
+ }
+}
+
+static int stm32_rproc_start(struct rproc *rproc)
+{
+ int err;
+
+ stm32_rproc_add_coredump_trace(rproc);
+
+ err = stm32_rproc_set_hold_boot(rproc, false);
+ if (err)
+ return err;
+
+ return stm32_rproc_set_hold_boot(rproc, true);
+}
+
+static int stm32_rproc_stop(struct rproc *rproc)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ int err, dummy_data, idx;
+
+ /* request shutdown of the remote processor */
+ if (rproc->state != RPROC_OFFLINE) {
+ idx = stm32_rproc_mbox_idx(rproc, STM32_MBX_SHUTDOWN);
+ if (idx >= 0 && ddata->mb[idx].chan) {
+ /* a dummy data is sent to allow to block on transmit */
+ err = mbox_send_message(ddata->mb[idx].chan,
+ &dummy_data);
+ if (err < 0)
+ dev_warn(&rproc->dev, "warning: remote FW shutdown without ack\n");
+ }
+ }
+
+ err = stm32_rproc_set_hold_boot(rproc, true);
+ if (err)
+ return err;
+
+ err = reset_control_assert(ddata->rst);
+ if (err) {
+ dev_err(&rproc->dev, "failed to assert the reset\n");
+ return err;
+ }
+
+ /* to allow platform Standby power mode, set remote proc Deep Sleep */
+ if (ddata->pdds.map) {
+ err = regmap_update_bits(ddata->pdds.map, ddata->pdds.reg,
+ ddata->pdds.mask, 1);
+ if (err) {
+ dev_err(&rproc->dev, "failed to set pdds\n");
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static void stm32_rproc_kick(struct rproc *rproc, int vqid)
+{
+ struct stm32_rproc *ddata = rproc->priv;
+ unsigned int i;
+ int err;
+
+ if (WARN_ON(vqid >= MBOX_NB_VQ))
+ return;
+
+ for (i = 0; i < MBOX_NB_MBX; i++) {
+ if (vqid != ddata->mb[i].vq_id)
+ continue;
+ if (!ddata->mb[i].chan)
+ return;
+ err = mbox_send_message(ddata->mb[i].chan, (void *)(long)vqid);
+ if (err < 0)
+ dev_err(&rproc->dev, "%s: failed (%s, err:%d)\n",
+ __func__, ddata->mb[i].name, err);
+ return;
+ }
+}
+
+static struct rproc_ops st_rproc_ops = {
+ .start = stm32_rproc_start,
+ .stop = stm32_rproc_stop,
+ .kick = stm32_rproc_kick,
+ .load = rproc_elf_load_segments,
+ .parse_fw = stm32_rproc_parse_fw,
+ .find_loaded_rsc_table = rproc_elf_find_loaded_rsc_table,
+ .sanity_check = rproc_elf_sanity_check,
+ .get_boot_addr = rproc_elf_get_boot_addr,
+};
+
+static const struct of_device_id stm32_rproc_match[] = {
+ { .compatible = "st,stm32mp1-m4" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, stm32_rproc_match);
+
+static int stm32_rproc_get_syscon(struct device_node *np, const char *prop,
+ struct stm32_syscon *syscon)
+{
+ int err = 0;
+
+ syscon->map = syscon_regmap_lookup_by_phandle(np, prop);
+ if (IS_ERR(syscon->map)) {
+ err = PTR_ERR(syscon->map);
+ syscon->map = NULL;
+ goto out;
+ }
+
+ err = of_property_read_u32_index(np, prop, 1, &syscon->reg);
+ if (err)
+ goto out;
+
+ err = of_property_read_u32_index(np, prop, 2, &syscon->mask);
+
+out:
+ return err;
+}
+
+static int stm32_rproc_parse_dt(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct device_node *np = dev->of_node;
+ struct rproc *rproc = platform_get_drvdata(pdev);
+ struct stm32_rproc *ddata = rproc->priv;
+ struct stm32_syscon tz;
+ unsigned int tzen;
+ int err, irq;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq > 0) {
+ err = devm_request_irq(dev, irq, stm32_rproc_wdg, 0,
+ dev_name(dev), rproc);
+ if (err) {
+ dev_err(dev, "failed to request wdg irq\n");
+ return err;
+ }
+
+ dev_info(dev, "wdg irq registered\n");
+ }
+
+ ddata->rst = devm_reset_control_get_by_index(dev, 0);
+ if (IS_ERR(ddata->rst)) {
+ dev_err(dev, "failed to get mcu reset\n");
+ return PTR_ERR(ddata->rst);
+ }
+
+ /*
+ * if platform is secured the hold boot bit must be written by
+ * smc call and read normally.
+ * if not secure the hold boot bit could be read/write normally
+ */
+ err = stm32_rproc_get_syscon(np, "st,syscfg-tz", &tz);
+ if (err) {
+ dev_err(dev, "failed to get tz syscfg\n");
+ return err;
+ }
+
+ err = regmap_read(tz.map, tz.reg, &tzen);
+ if (err) {
+ dev_err(&rproc->dev, "failed to read tzen\n");
+ return err;
+ }
+ ddata->secured_soc = tzen & tz.mask;
+
+ err = stm32_rproc_get_syscon(np, "st,syscfg-holdboot",
+ &ddata->hold_boot);
+ if (err) {
+ dev_err(dev, "failed to get hold boot\n");
+ return err;
+ }
+
+ err = stm32_rproc_get_syscon(np, "st,syscfg-pdds", &ddata->pdds);
+ if (err)
+ dev_warn(dev, "failed to get pdds\n");
+
+ rproc->auto_boot = of_property_read_bool(np, "st,auto-boot");
+
+ return stm32_rproc_of_memory_translations(rproc);
+}
+
+static int stm32_rproc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct stm32_rproc *ddata;
+ struct device_node *np = dev->of_node;
+ struct rproc *rproc;
+ int ret;
+
+ ret = dma_coerce_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (ret)
+ return ret;
+
+ rproc = rproc_alloc(dev, np->name, &st_rproc_ops, NULL, sizeof(*ddata));
+ if (!rproc)
+ return -ENOMEM;
+
+ rproc->has_iommu = false;
+ ddata = rproc->priv;
+
+ platform_set_drvdata(pdev, rproc);
+
+ ret = stm32_rproc_parse_dt(pdev);
+ if (ret)
+ goto free_rproc;
+
+ stm32_rproc_request_mbox(rproc);
+
+ ret = rproc_add(rproc);
+ if (ret)
+ goto free_mb;
+
+ return 0;
+
+free_mb:
+ stm32_rproc_free_mbox(rproc);
+free_rproc:
+ rproc_free(rproc);
+ return ret;
+}
+
+static int stm32_rproc_remove(struct platform_device *pdev)
+{
+ struct rproc *rproc = platform_get_drvdata(pdev);
+
+ if (atomic_read(&rproc->power) > 0)
+ rproc_shutdown(rproc);
+
+ rproc_del(rproc);
+ stm32_rproc_free_mbox(rproc);
+ rproc_free(rproc);
+
+ return 0;
+}
+
+static struct platform_driver stm32_rproc_driver = {
+ .probe = stm32_rproc_probe,
+ .remove = stm32_rproc_remove,
+ .driver = {
+ .name = "stm32-rproc",
+ .of_match_table = of_match_ptr(stm32_rproc_match),
+ },
+};
+module_platform_driver(stm32_rproc_driver);
+
+MODULE_DESCRIPTION("STM32 Remote Processor Control Driver");
+MODULE_AUTHOR("Ludovic Barre <ludovic.barre@st.com>");
+MODULE_AUTHOR("Fabien Dessenne <fabien.dessenne@st.com>");
+MODULE_LICENSE("GPL v2");
+
if (rpdev->ops->announce_destroy)
err = rpdev->ops->announce_destroy(rpdev);
- rpdrv->remove(rpdev);
+ if (rpdrv->remove)
+ rpdrv->remove(rpdev);
dev_pm_domain_detach(dev, true);
config RTC_DRV_TPS65910
tristate "TI TPS65910 RTC driver"
- depends on RTC_CLASS && MFD_TPS65910
+ depends on MFD_TPS65910
help
If you say yes here you get support for the RTC on the
TPS65910 chips.
config RTC_DRV_PCF2123
tristate "NXP PCF2123"
+ select REGMAP_SPI
help
If you say yes here you get support for the NXP PCF2123
RTC chip.
{
struct rtc_device *rtc;
ktime_t period;
- int count;
+ u64 count;
rtc = container_of(timer, struct rtc_device, pie_timer);
return -EINVAL;
}
+ tmp = regs[DS1307_REG_SECS];
+ switch (ds1307->type) {
+ case ds_1307:
+ case m41t0:
+ case m41t00:
+ case m41t11:
+ if (tmp & DS1307_BIT_CH)
+ return -EINVAL;
+ break;
+ case ds_1308:
+ case ds_1338:
+ if (tmp & DS1307_BIT_CH)
+ return -EINVAL;
+
+ ret = regmap_read(ds1307->regmap, DS1307_REG_CONTROL, &tmp);
+ if (ret)
+ return ret;
+ if (tmp & DS1338_BIT_OSF)
+ return -EINVAL;
+ break;
+ case ds_1340:
+ if (tmp & DS1340_BIT_nEOSC)
+ return -EINVAL;
+
+ ret = regmap_read(ds1307->regmap, DS1340_REG_FLAG, &tmp);
+ if (ret)
+ return ret;
+ if (tmp & DS1340_BIT_OSF)
+ return -EINVAL;
+ break;
+ case mcp794xx:
+ if (!(tmp & MCP794XX_BIT_ST))
+ return -EINVAL;
+
+ break;
+ default:
+ break;
+ }
+
t->tm_sec = bcd2bin(regs[DS1307_REG_SECS] & 0x7f);
t->tm_min = bcd2bin(regs[DS1307_REG_MIN] & 0x7f);
tmp = regs[DS1307_REG_HOUR] & 0x3f;
if (t->tm_year > 199 && chip->century_bit)
regs[chip->century_reg] |= chip->century_bit;
- if (ds1307->type == mcp794xx) {
+ switch (ds1307->type) {
+ case ds_1308:
+ case ds_1338:
+ regmap_update_bits(ds1307->regmap, DS1307_REG_CONTROL,
+ DS1338_BIT_OSF, 0);
+ break;
+ case ds_1340:
+ regmap_update_bits(ds1307->regmap, DS1340_REG_FLAG,
+ DS1340_BIT_OSF, 0);
+ break;
+ case mcp794xx:
/*
* these bits were cleared when preparing the date/time
* values and need to be set again before writing the
*/
regs[DS1307_REG_SECS] |= MCP794XX_BIT_ST;
regs[DS1307_REG_WDAY] |= MCP794XX_BIT_VBATEN;
+ break;
+ default:
+ break;
}
dev_dbg(dev, "%s: %7ph\n", "write", regs);
break;
}
-read_rtc:
/* read RTC registers */
err = regmap_bulk_read(ds1307->regmap, chip->offset, regs,
sizeof(regs));
goto exit;
}
- /*
- * minimal sanity checking; some chips (like DS1340) don't
- * specify the extra bits as must-be-zero, but there are
- * still a few values that are clearly out-of-range.
- */
- tmp = regs[DS1307_REG_SECS];
- switch (ds1307->type) {
- case ds_1307:
- case m41t0:
- case m41t00:
- case m41t11:
- /* clock halted? turn it on, so clock can tick. */
- if (tmp & DS1307_BIT_CH) {
- regmap_write(ds1307->regmap, DS1307_REG_SECS, 0);
- dev_warn(ds1307->dev, "SET TIME!\n");
- goto read_rtc;
- }
- break;
- case ds_1308:
- case ds_1338:
- /* clock halted? turn it on, so clock can tick. */
- if (tmp & DS1307_BIT_CH)
- regmap_write(ds1307->regmap, DS1307_REG_SECS, 0);
-
- /* oscillator fault? clear flag, and warn */
- if (regs[DS1307_REG_CONTROL] & DS1338_BIT_OSF) {
- regmap_write(ds1307->regmap, DS1307_REG_CONTROL,
- regs[DS1307_REG_CONTROL] &
- ~DS1338_BIT_OSF);
- dev_warn(ds1307->dev, "SET TIME!\n");
- goto read_rtc;
- }
- break;
- case ds_1340:
- /* clock halted? turn it on, so clock can tick. */
- if (tmp & DS1340_BIT_nEOSC)
- regmap_write(ds1307->regmap, DS1307_REG_SECS, 0);
-
- err = regmap_read(ds1307->regmap, DS1340_REG_FLAG, &tmp);
- if (err) {
- dev_dbg(ds1307->dev, "read error %d\n", err);
- goto exit;
- }
-
- /* oscillator fault? clear flag, and warn */
- if (tmp & DS1340_BIT_OSF) {
- regmap_write(ds1307->regmap, DS1340_REG_FLAG, 0);
- dev_warn(ds1307->dev, "SET TIME!\n");
- }
- break;
- case mcp794xx:
- /* make sure that the backup battery is enabled */
- if (!(regs[DS1307_REG_WDAY] & MCP794XX_BIT_VBATEN)) {
- regmap_write(ds1307->regmap, DS1307_REG_WDAY,
- regs[DS1307_REG_WDAY] |
- MCP794XX_BIT_VBATEN);
- }
-
- /* clock halted? turn it on, so clock can tick. */
- if (!(tmp & MCP794XX_BIT_ST)) {
- regmap_write(ds1307->regmap, DS1307_REG_SECS,
- MCP794XX_BIT_ST);
- dev_warn(ds1307->dev, "SET TIME!\n");
- goto read_rtc;
- }
-
- break;
- default:
- break;
+ if (ds1307->type == mcp794xx &&
+ !(regs[DS1307_REG_WDAY] & MCP794XX_BIT_VBATEN)) {
+ regmap_write(ds1307->regmap, DS1307_REG_WDAY,
+ regs[DS1307_REG_WDAY] |
+ MCP794XX_BIT_VBATEN);
}
tmp = regs[DS1307_REG_HOUR];
static int ds2404_read_time(struct device *dev, struct rtc_time *dt)
{
unsigned long time = 0;
+ __le32 hw_time = 0;
- ds2404_read_memory(dev, 0x203, 4, (u8 *)&time);
- time = le32_to_cpu(time);
+ ds2404_read_memory(dev, 0x203, 4, (u8 *)&hw_time);
+ time = le32_to_cpu(hw_time);
rtc_time64_to_tm(time, dt);
return 0;
fm3130_rtc_mode(dev, FM3130_MODE_READ);
/* read the RTC date and time registers all at once */
- tmp = i2c_transfer(to_i2c_adapter(fm3130->client->dev.parent),
- fm3130->msg, 2);
+ tmp = i2c_transfer(fm3130->client->adapter, fm3130->msg, 2);
if (tmp != 2) {
dev_err(dev, "%s error %d\n", "read", tmp);
return -EIO;
}
/* read the RTC alarm registers all at once */
- tmp = i2c_transfer(to_i2c_adapter(fm3130->client->dev.parent),
- &fm3130->msg[2], 2);
+ tmp = i2c_transfer(fm3130->client->adapter, &fm3130->msg[2], 2);
if (tmp != 2) {
dev_err(dev, "%s error %d\n", "read", tmp);
return -EIO;
struct fm3130 *fm3130;
int err = -ENODEV;
int tmp;
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
if (!i2c_check_functionality(adapter,
I2C_FUNC_I2C | I2C_FUNC_SMBUS_WRITE_BYTE_DATA))
* Copyright 2018 NXP.
*/
+#include <dt-bindings/firmware/imx/rsrc.h>
#include <linux/arm-smccc.h>
#include <linux/firmware/imx/sci.h>
#include <linux/module.h>
#include <linux/rtc.h>
#define IMX_SC_TIMER_FUNC_GET_RTC_SEC1970 9
+#define IMX_SC_TIMER_FUNC_SET_RTC_ALARM 8
#define IMX_SC_TIMER_FUNC_SET_RTC_TIME 6
#define IMX_SIP_SRTC 0xC2000002
#define IMX_SIP_SRTC_SET_TIME 0x0
+#define SC_IRQ_GROUP_RTC 2
+#define SC_IRQ_RTC 1
+
static struct imx_sc_ipc *rtc_ipc_handle;
static struct rtc_device *imx_sc_rtc;
u32 time;
} __packed;
+struct imx_sc_msg_timer_rtc_set_alarm {
+ struct imx_sc_rpc_msg hdr;
+ u16 year;
+ u8 mon;
+ u8 day;
+ u8 hour;
+ u8 min;
+ u8 sec;
+} __packed;
+
static int imx_sc_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
struct imx_sc_msg_timer_get_rtc_time msg;
return res.a0;
}
+static int imx_sc_rtc_alarm_irq_enable(struct device *dev, unsigned int enable)
+{
+ return imx_scu_irq_group_enable(SC_IRQ_GROUP_RTC, SC_IRQ_RTC, enable);
+}
+
+static int imx_sc_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ /*
+ * SCU firmware does NOT provide read alarm API, but .read_alarm
+ * callback is required by RTC framework to support alarm function,
+ * so just return here.
+ */
+ return 0;
+}
+
+static int imx_sc_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alrm)
+{
+ struct imx_sc_msg_timer_rtc_set_alarm msg;
+ struct imx_sc_rpc_msg *hdr = &msg.hdr;
+ int ret;
+ struct rtc_time *alrm_tm = &alrm->time;
+
+ hdr->ver = IMX_SC_RPC_VERSION;
+ hdr->svc = IMX_SC_RPC_SVC_TIMER;
+ hdr->func = IMX_SC_TIMER_FUNC_SET_RTC_ALARM;
+ hdr->size = 3;
+
+ msg.year = alrm_tm->tm_year + 1900;
+ msg.mon = alrm_tm->tm_mon + 1;
+ msg.day = alrm_tm->tm_mday;
+ msg.hour = alrm_tm->tm_hour;
+ msg.min = alrm_tm->tm_min;
+ msg.sec = alrm_tm->tm_sec;
+
+ ret = imx_scu_call_rpc(rtc_ipc_handle, &msg, true);
+ if (ret) {
+ dev_err(dev, "set rtc alarm failed, ret %d\n", ret);
+ return ret;
+ }
+
+ ret = imx_sc_rtc_alarm_irq_enable(dev, alrm->enabled);
+ if (ret) {
+ dev_err(dev, "enable rtc alarm failed, ret %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
static const struct rtc_class_ops imx_sc_rtc_ops = {
.read_time = imx_sc_rtc_read_time,
.set_time = imx_sc_rtc_set_time,
+ .read_alarm = imx_sc_rtc_read_alarm,
+ .set_alarm = imx_sc_rtc_set_alarm,
+ .alarm_irq_enable = imx_sc_rtc_alarm_irq_enable,
+};
+
+static int imx_sc_rtc_alarm_notify(struct notifier_block *nb,
+ unsigned long event, void *group)
+{
+ /* ignore non-rtc irq */
+ if (!((event & SC_IRQ_RTC) && (*(u8 *)group == SC_IRQ_GROUP_RTC)))
+ return 0;
+
+ rtc_update_irq(imx_sc_rtc, 1, RTC_IRQF | RTC_AF);
+
+ return 0;
+}
+
+static struct notifier_block imx_sc_rtc_alarm_sc_notifier = {
+ .notifier_call = imx_sc_rtc_alarm_notify,
};
static int imx_sc_rtc_probe(struct platform_device *pdev)
if (ret)
return ret;
+ device_init_wakeup(&pdev->dev, true);
+
imx_sc_rtc = devm_rtc_allocate_device(&pdev->dev);
if (IS_ERR(imx_sc_rtc))
return PTR_ERR(imx_sc_rtc);
return ret;
}
+ imx_scu_irq_register_notifier(&imx_sc_rtc_alarm_sc_notifier);
+
return 0;
}
static int m41t80_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
int rc = 0;
struct rtc_time tm;
struct m41t80_data *m41t80_data = NULL;
#include <linux/rtc.h>
#include <linux/spi/spi.h>
#include <linux/module.h>
-#include <linux/sysfs.h>
+#include <linux/regmap.h>
/* REGISTERS */
#define PCF2123_REG_CTRL1 (0x00) /* Control Register 1 */
#define OFFSET_SIGN_BIT 6 /* 2's complement sign bit */
#define OFFSET_COARSE BIT(7) /* Coarse mode offset */
#define OFFSET_STEP (2170) /* Offset step in parts per billion */
+#define OFFSET_MASK GENMASK(6, 0) /* Offset value */
/* READ/WRITE ADDRESS BITS */
#define PCF2123_WRITE BIT(4)
static struct spi_driver pcf2123_driver;
-struct pcf2123_sysfs_reg {
- struct device_attribute attr;
- char name[2];
-};
-
struct pcf2123_plat_data {
struct rtc_device *rtc;
- struct pcf2123_sysfs_reg regs[16];
+ struct regmap *map;
};
-/*
- * Causes a 30 nanosecond delay to ensure that the PCF2123 chip select
- * is released properly after an SPI write. This function should be
- * called after EVERY read/write call over SPI.
- */
-static inline void pcf2123_delay_trec(void)
-{
- ndelay(30);
-}
-
-static int pcf2123_read(struct device *dev, u8 reg, u8 *rxbuf, size_t size)
-{
- struct spi_device *spi = to_spi_device(dev);
- int ret;
-
- reg |= PCF2123_READ;
- ret = spi_write_then_read(spi, ®, 1, rxbuf, size);
- pcf2123_delay_trec();
-
- return ret;
-}
-
-static int pcf2123_write(struct device *dev, u8 *txbuf, size_t size)
-{
- struct spi_device *spi = to_spi_device(dev);
- int ret;
-
- txbuf[0] |= PCF2123_WRITE;
- ret = spi_write(spi, txbuf, size);
- pcf2123_delay_trec();
-
- return ret;
-}
-
-static int pcf2123_write_reg(struct device *dev, u8 reg, u8 val)
-{
- u8 txbuf[2];
-
- txbuf[0] = reg;
- txbuf[1] = val;
- return pcf2123_write(dev, txbuf, sizeof(txbuf));
-}
-
-static ssize_t pcf2123_show(struct device *dev, struct device_attribute *attr,
- char *buffer)
-{
- struct pcf2123_sysfs_reg *r;
- u8 rxbuf[1];
- unsigned long reg;
- int ret;
-
- r = container_of(attr, struct pcf2123_sysfs_reg, attr);
-
- ret = kstrtoul(r->name, 16, ®);
- if (ret)
- return ret;
-
- ret = pcf2123_read(dev, reg, rxbuf, 1);
- if (ret < 0)
- return -EIO;
-
- return sprintf(buffer, "0x%x\n", rxbuf[0]);
-}
+static const struct regmap_config pcf2123_regmap_config = {
+ .reg_bits = 8,
+ .val_bits = 8,
+ .read_flag_mask = PCF2123_READ,
+ .write_flag_mask = PCF2123_WRITE,
+ .max_register = PCF2123_REG_CTDWN_TMR,
+};
-static ssize_t pcf2123_store(struct device *dev, struct device_attribute *attr,
- const char *buffer, size_t count)
+static int pcf2123_read_offset(struct device *dev, long *offset)
{
- struct pcf2123_sysfs_reg *r;
- unsigned long reg;
- unsigned long val;
-
- int ret;
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
+ int ret, val;
+ unsigned int reg;
- r = container_of(attr, struct pcf2123_sysfs_reg, attr);
-
- ret = kstrtoul(r->name, 16, ®);
+ ret = regmap_read(pdata->map, PCF2123_REG_OFFSET, ®);
if (ret)
return ret;
- ret = kstrtoul(buffer, 10, &val);
- if (ret)
- return ret;
-
- ret = pcf2123_write_reg(dev, reg, val);
- if (ret < 0)
- return -EIO;
- return count;
-}
-
-static int pcf2123_read_offset(struct device *dev, long *offset)
-{
- int ret;
- s8 reg;
-
- ret = pcf2123_read(dev, PCF2123_REG_OFFSET, ®, 1);
- if (ret < 0)
- return ret;
+ val = sign_extend32((reg & OFFSET_MASK), OFFSET_SIGN_BIT);
if (reg & OFFSET_COARSE)
- reg <<= 1; /* multiply by 2 and sign extend */
- else
- reg = sign_extend32(reg, OFFSET_SIGN_BIT);
+ val *= 2;
- *offset = ((long)reg) * OFFSET_STEP;
+ *offset = ((long)val) * OFFSET_STEP;
return 0;
}
*/
static int pcf2123_set_offset(struct device *dev, long offset)
{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
s8 reg;
if (offset > OFFSET_STEP * 127)
else if (offset < OFFSET_STEP * -128)
reg = -128;
else
- reg = (s8)((offset + (OFFSET_STEP >> 1)) / OFFSET_STEP);
+ reg = DIV_ROUND_CLOSEST(offset, OFFSET_STEP);
/* choose fine offset only for odd values in the normal range */
if (reg & 1 && reg <= 63 && reg >= -64) {
reg |= OFFSET_COARSE;
}
- return pcf2123_write_reg(dev, PCF2123_REG_OFFSET, reg);
+ return regmap_write(pdata->map, PCF2123_REG_OFFSET, (unsigned int)reg);
}
static int pcf2123_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
u8 rxbuf[7];
int ret;
- ret = pcf2123_read(dev, PCF2123_REG_SC, rxbuf, sizeof(rxbuf));
- if (ret < 0)
+ ret = regmap_bulk_read(pdata->map, PCF2123_REG_SC, rxbuf,
+ sizeof(rxbuf));
+ if (ret)
return ret;
if (rxbuf[0] & OSC_HAS_STOPPED) {
if (tm->tm_year < 70)
tm->tm_year += 100; /* assume we are in 1970...2069 */
- dev_dbg(dev, "%s: tm is secs=%d, mins=%d, hours=%d, "
- "mday=%d, mon=%d, year=%d, wday=%d\n",
- __func__,
- tm->tm_sec, tm->tm_min, tm->tm_hour,
- tm->tm_mday, tm->tm_mon, tm->tm_year, tm->tm_wday);
+ dev_dbg(dev, "%s: tm is %ptR\n", __func__, tm);
return 0;
}
static int pcf2123_rtc_set_time(struct device *dev, struct rtc_time *tm)
{
- u8 txbuf[8];
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
+ u8 txbuf[7];
int ret;
- dev_dbg(dev, "%s: tm is secs=%d, mins=%d, hours=%d, "
- "mday=%d, mon=%d, year=%d, wday=%d\n",
- __func__,
- tm->tm_sec, tm->tm_min, tm->tm_hour,
- tm->tm_mday, tm->tm_mon, tm->tm_year, tm->tm_wday);
+ dev_dbg(dev, "%s: tm is %ptR\n", __func__, tm);
/* Stop the counter first */
- ret = pcf2123_write_reg(dev, PCF2123_REG_CTRL1, CTRL1_STOP);
- if (ret < 0)
+ ret = regmap_write(pdata->map, PCF2123_REG_CTRL1, CTRL1_STOP);
+ if (ret)
return ret;
/* Set the new time */
- txbuf[0] = PCF2123_REG_SC;
- txbuf[1] = bin2bcd(tm->tm_sec & 0x7F);
- txbuf[2] = bin2bcd(tm->tm_min & 0x7F);
- txbuf[3] = bin2bcd(tm->tm_hour & 0x3F);
- txbuf[4] = bin2bcd(tm->tm_mday & 0x3F);
- txbuf[5] = tm->tm_wday & 0x07;
- txbuf[6] = bin2bcd((tm->tm_mon + 1) & 0x1F); /* rtc mn 1-12 */
- txbuf[7] = bin2bcd(tm->tm_year < 100 ? tm->tm_year : tm->tm_year - 100);
-
- ret = pcf2123_write(dev, txbuf, sizeof(txbuf));
- if (ret < 0)
+ txbuf[0] = bin2bcd(tm->tm_sec & 0x7F);
+ txbuf[1] = bin2bcd(tm->tm_min & 0x7F);
+ txbuf[2] = bin2bcd(tm->tm_hour & 0x3F);
+ txbuf[3] = bin2bcd(tm->tm_mday & 0x3F);
+ txbuf[4] = tm->tm_wday & 0x07;
+ txbuf[5] = bin2bcd((tm->tm_mon + 1) & 0x1F); /* rtc mn 1-12 */
+ txbuf[6] = bin2bcd(tm->tm_year < 100 ? tm->tm_year : tm->tm_year - 100);
+
+ ret = regmap_bulk_write(pdata->map, PCF2123_REG_SC, txbuf,
+ sizeof(txbuf));
+ if (ret)
return ret;
/* Start the counter */
- ret = pcf2123_write_reg(dev, PCF2123_REG_CTRL1, CTRL1_CLEAR);
- if (ret < 0)
+ ret = regmap_write(pdata->map, PCF2123_REG_CTRL1, CTRL1_CLEAR);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int pcf2123_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alm)
+{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
+ u8 rxbuf[4];
+ int ret;
+ unsigned int val = 0;
+
+ ret = regmap_bulk_read(pdata->map, PCF2123_REG_ALRM_MN, rxbuf,
+ sizeof(rxbuf));
+ if (ret)
+ return ret;
+
+ alm->time.tm_min = bcd2bin(rxbuf[0] & 0x7F);
+ alm->time.tm_hour = bcd2bin(rxbuf[1] & 0x3F);
+ alm->time.tm_mday = bcd2bin(rxbuf[2] & 0x3F);
+ alm->time.tm_wday = bcd2bin(rxbuf[3] & 0x07);
+
+ dev_dbg(dev, "%s: alm is %ptR\n", __func__, &alm->time);
+
+ ret = regmap_read(pdata->map, PCF2123_REG_CTRL2, &val);
+ if (ret)
+ return ret;
+
+ alm->enabled = !!(val & CTRL2_AIE);
+
+ return 0;
+}
+
+static int pcf2123_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alm)
+{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
+ u8 txbuf[4];
+ int ret;
+
+ dev_dbg(dev, "%s: alm is %ptR\n", __func__, &alm->time);
+
+ /* Ensure alarm flag is clear */
+ ret = regmap_update_bits(pdata->map, PCF2123_REG_CTRL2, CTRL2_AF, 0);
+ if (ret)
return ret;
+ /* Disable alarm interrupt */
+ ret = regmap_update_bits(pdata->map, PCF2123_REG_CTRL2, CTRL2_AIE, 0);
+ if (ret)
+ return ret;
+
+ /* Set new alarm */
+ txbuf[0] = bin2bcd(alm->time.tm_min & 0x7F);
+ txbuf[1] = bin2bcd(alm->time.tm_hour & 0x3F);
+ txbuf[2] = bin2bcd(alm->time.tm_mday & 0x3F);
+ txbuf[3] = bin2bcd(alm->time.tm_wday & 0x07);
+
+ ret = regmap_bulk_write(pdata->map, PCF2123_REG_ALRM_MN, txbuf,
+ sizeof(txbuf));
+ if (ret)
+ return ret;
+
+ /* Enable alarm interrupt */
+ if (alm->enabled) {
+ ret = regmap_update_bits(pdata->map, PCF2123_REG_CTRL2,
+ CTRL2_AIE, CTRL2_AIE);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
+static irqreturn_t pcf2123_rtc_irq(int irq, void *dev)
+{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
+ struct mutex *lock = &pdata->rtc->ops_lock;
+ unsigned int val = 0;
+ int ret = IRQ_NONE;
+
+ mutex_lock(lock);
+ regmap_read(pdata->map, PCF2123_REG_CTRL2, &val);
+
+ /* Alarm? */
+ if (val & CTRL2_AF) {
+ ret = IRQ_HANDLED;
+
+ /* Clear alarm flag */
+ regmap_update_bits(pdata->map, PCF2123_REG_CTRL2, CTRL2_AF, 0);
+
+ rtc_update_irq(pdata->rtc, 1, RTC_IRQF | RTC_AF);
+ }
+
+ mutex_unlock(lock);
+
+ return ret;
+}
+
static int pcf2123_reset(struct device *dev)
{
+ struct pcf2123_plat_data *pdata = dev_get_platdata(dev);
int ret;
- u8 rxbuf[2];
+ unsigned int val = 0;
- ret = pcf2123_write_reg(dev, PCF2123_REG_CTRL1, CTRL1_SW_RESET);
- if (ret < 0)
+ ret = regmap_write(pdata->map, PCF2123_REG_CTRL1, CTRL1_SW_RESET);
+ if (ret)
return ret;
/* Stop the counter */
dev_dbg(dev, "stopping RTC\n");
- ret = pcf2123_write_reg(dev, PCF2123_REG_CTRL1, CTRL1_STOP);
- if (ret < 0)
+ ret = regmap_write(pdata->map, PCF2123_REG_CTRL1, CTRL1_STOP);
+ if (ret)
return ret;
/* See if the counter was actually stopped */
dev_dbg(dev, "checking for presence of RTC\n");
- ret = pcf2123_read(dev, PCF2123_REG_CTRL1, rxbuf, sizeof(rxbuf));
- if (ret < 0)
+ ret = regmap_read(pdata->map, PCF2123_REG_CTRL1, &val);
+ if (ret)
return ret;
- dev_dbg(dev, "received data from RTC (0x%02X 0x%02X)\n",
- rxbuf[0], rxbuf[1]);
- if (!(rxbuf[0] & CTRL1_STOP))
+ dev_dbg(dev, "received data from RTC (0x%08X)\n", val);
+ if (!(val & CTRL1_STOP))
return -ENODEV;
/* Start the counter */
- ret = pcf2123_write_reg(dev, PCF2123_REG_CTRL1, CTRL1_CLEAR);
- if (ret < 0)
+ ret = regmap_write(pdata->map, PCF2123_REG_CTRL1, CTRL1_CLEAR);
+ if (ret)
return ret;
return 0;
.set_time = pcf2123_rtc_set_time,
.read_offset = pcf2123_read_offset,
.set_offset = pcf2123_set_offset,
-
+ .read_alarm = pcf2123_rtc_read_alarm,
+ .set_alarm = pcf2123_rtc_set_alarm,
};
static int pcf2123_probe(struct spi_device *spi)
struct rtc_device *rtc;
struct rtc_time tm;
struct pcf2123_plat_data *pdata;
- int ret, i;
+ int ret = 0;
pdata = devm_kzalloc(&spi->dev, sizeof(struct pcf2123_plat_data),
GFP_KERNEL);
return -ENOMEM;
spi->dev.platform_data = pdata;
+ pdata->map = devm_regmap_init_spi(spi, &pcf2123_regmap_config);
+
+ if (IS_ERR(pdata->map)) {
+ dev_err(&spi->dev, "regmap init failed.\n");
+ goto kfree_exit;
+ }
+
ret = pcf2123_rtc_read_time(&spi->dev, &tm);
if (ret < 0) {
ret = pcf2123_reset(&spi->dev);
pdata->rtc = rtc;
- for (i = 0; i < 16; i++) {
- sysfs_attr_init(&pdata->regs[i].attr.attr);
- sprintf(pdata->regs[i].name, "%1x", i);
- pdata->regs[i].attr.attr.mode = S_IRUGO | S_IWUSR;
- pdata->regs[i].attr.attr.name = pdata->regs[i].name;
- pdata->regs[i].attr.show = pcf2123_show;
- pdata->regs[i].attr.store = pcf2123_store;
- ret = device_create_file(&spi->dev, &pdata->regs[i].attr);
- if (ret) {
- dev_err(&spi->dev, "Unable to create sysfs %s\n",
- pdata->regs[i].name);
- goto sysfs_exit;
- }
+ /* Register alarm irq */
+ if (spi->irq > 0) {
+ ret = devm_request_threaded_irq(&spi->dev, spi->irq, NULL,
+ pcf2123_rtc_irq,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
+ pcf2123_driver.driver.name, &spi->dev);
+ if (!ret)
+ device_init_wakeup(&spi->dev, true);
+ else
+ dev_err(&spi->dev, "could not request irq.\n");
}
- return 0;
+ /* The PCF2123's alarm only has minute accuracy. Must add timer
+ * support to this driver to generate interrupts more than once
+ * per minute.
+ */
+ pdata->rtc->uie_unsupported = 1;
-sysfs_exit:
- for (i--; i >= 0; i--)
- device_remove_file(&spi->dev, &pdata->regs[i].attr);
+ return 0;
kfree_exit:
spi->dev.platform_data = NULL;
return ret;
}
-static int pcf2123_remove(struct spi_device *spi)
-{
- struct pcf2123_plat_data *pdata = dev_get_platdata(&spi->dev);
- int i;
-
- if (pdata) {
- for (i = 0; i < 16; i++)
- if (pdata->regs[i].name[0])
- device_remove_file(&spi->dev,
- &pdata->regs[i].attr);
- }
-
- return 0;
-}
-
#ifdef CONFIG_OF
static const struct of_device_id pcf2123_dt_ids[] = {
{ .compatible = "nxp,rtc-pcf2123", },
.of_match_table = of_match_ptr(pcf2123_dt_ids),
},
.probe = pcf2123_probe,
- .remove = pcf2123_remove,
};
module_spi_driver(pcf2123_driver);
struct pcf8563 *pcf8563;
int err;
unsigned char buf;
- unsigned char alm_pending;
dev_dbg(&client->dev, "%s\n", __func__);
return err;
}
- err = pcf8563_get_alarm_mode(client, NULL, &alm_pending);
- if (err) {
- dev_err(&client->dev, "%s: read error\n", __func__);
+ /* Clear flags and disable interrupts */
+ buf = 0;
+ err = pcf8563_write_block_data(client, PCF8563_REG_ST2, 1, &buf);
+ if (err < 0) {
+ dev_err(&client->dev, "%s: write error\n", __func__);
return err;
}
- if (alm_pending)
- pcf8563_set_alarm_mode(client, 0);
pcf8563->rtc = devm_rtc_device_register(&client->dev,
pcf8563_driver.driver.name,
if (client->irq > 0) {
err = devm_request_threaded_irq(&client->dev, client->irq,
NULL, pcf8563_irq,
- IRQF_SHARED|IRQF_ONESHOT|IRQF_TRIGGER_FALLING,
+ IRQF_SHARED | IRQF_ONESHOT | IRQF_TRIGGER_LOW,
pcf8563_driver.driver.name, client);
if (err) {
dev_err(&client->dev, "unable to request IRQ %d\n",
static int rv8803_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
struct rv8803_data *rv8803;
int err, flags;
struct nvmem_config nvmem_cfg = {
static int rx8010_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
struct rx8010_data *rx8010;
int err = 0;
static int rx8025_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
- struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct i2c_adapter *adapter = client->adapter;
struct rx8025_data *rx8025;
int err = 0;
#define S35390A_ALRM_BYTE_MINS 2
/* flags for STATUS1 */
-#define S35390A_FLAG_POC 0x01
-#define S35390A_FLAG_BLD 0x02
-#define S35390A_FLAG_INT2 0x04
-#define S35390A_FLAG_24H 0x40
-#define S35390A_FLAG_RESET 0x80
+#define S35390A_FLAG_POC BIT(0)
+#define S35390A_FLAG_BLD BIT(1)
+#define S35390A_FLAG_INT2 BIT(2)
+#define S35390A_FLAG_24H BIT(6)
+#define S35390A_FLAG_RESET BIT(7)
/* flag for STATUS2 */
-#define S35390A_FLAG_TEST 0x01
-
-#define S35390A_INT2_MODE_MASK 0xF0
+#define S35390A_FLAG_TEST BIT(0)
+/* INT2 pin output mode */
+#define S35390A_INT2_MODE_MASK 0x0E
#define S35390A_INT2_MODE_NOINTR 0x00
-#define S35390A_INT2_MODE_FREQ 0x10
-#define S35390A_INT2_MODE_ALARM 0x40
-#define S35390A_INT2_MODE_PMIN_EDG 0x20
+#define S35390A_INT2_MODE_ALARM BIT(1) /* INT2AE */
+#define S35390A_INT2_MODE_PMIN_EDG BIT(2) /* INT2ME */
+#define S35390A_INT2_MODE_FREQ BIT(3) /* INT2FE */
+#define S35390A_INT2_MODE_PMIN (BIT(3) | BIT(2)) /* INT2FE | INT2ME */
static const struct i2c_device_id s35390a_id[] = {
{ "s35390a", 0 },
alm->time.tm_min, alm->time.tm_hour, alm->time.tm_mday,
alm->time.tm_mon, alm->time.tm_year, alm->time.tm_wday);
+ if (alm->time.tm_sec != 0)
+ dev_warn(&client->dev, "Alarms are only supported on a per minute basis!\n");
+
/* disable interrupt (which deasserts the irq line) */
err = s35390a_set_reg(s35390a, S35390A_CMD_STATUS2, &sts, sizeof(sts));
if (err < 0)
else
sts = S35390A_INT2_MODE_NOINTR;
- /* This chip expects the bits of each byte to be in reverse order */
- sts = bitrev8(sts);
-
/* set interupt mode*/
err = s35390a_set_reg(s35390a, S35390A_CMD_STATUS2, &sts, sizeof(sts));
if (err < 0)
if (err < 0)
return err;
- if ((bitrev8(sts) & S35390A_INT2_MODE_MASK) != S35390A_INT2_MODE_ALARM) {
+ if ((sts & S35390A_INT2_MODE_MASK) != S35390A_INT2_MODE_ALARM) {
/*
* When the alarm isn't enabled, the register to configure
* the alarm time isn't accessible.
unsigned int i;
struct s35390a *s35390a;
char buf, status1;
+ struct device *dev = &client->dev;
if (!i2c_check_functionality(client->adapter, I2C_FUNC_I2C)) {
err = -ENODEV;
goto exit;
}
- s35390a = devm_kzalloc(&client->dev, sizeof(struct s35390a),
- GFP_KERNEL);
+ s35390a = devm_kzalloc(dev, sizeof(struct s35390a), GFP_KERNEL);
if (!s35390a) {
err = -ENOMEM;
goto exit;
s35390a->client[i] = i2c_new_dummy(client->adapter,
client->addr + i);
if (!s35390a->client[i]) {
- dev_err(&client->dev, "Address %02x unavailable\n",
- client->addr + i);
+ dev_err(dev, "Address %02x unavailable\n",
+ client->addr + i);
err = -EBUSY;
goto exit_dummy;
}
err_read = s35390a_read_status(s35390a, &status1);
if (err_read < 0) {
err = err_read;
- dev_err(&client->dev, "error resetting chip\n");
+ dev_err(dev, "error resetting chip\n");
goto exit_dummy;
}
buf = 0;
err = s35390a_set_reg(s35390a, S35390A_CMD_STATUS2, &buf, 1);
if (err < 0) {
- dev_err(&client->dev, "error disabling alarm");
+ dev_err(dev, "error disabling alarm");
goto exit_dummy;
}
} else {
err = s35390a_disable_test_mode(s35390a);
if (err < 0) {
- dev_err(&client->dev, "error disabling test mode\n");
+ dev_err(dev, "error disabling test mode\n");
goto exit_dummy;
}
}
- device_set_wakeup_capable(&client->dev, 1);
+ device_set_wakeup_capable(dev, 1);
- s35390a->rtc = devm_rtc_device_register(&client->dev,
- s35390a_driver.driver.name,
- &s35390a_rtc_ops, THIS_MODULE);
+ s35390a->rtc = devm_rtc_device_register(dev, s35390a_driver.driver.name,
+ &s35390a_rtc_ops, THIS_MODULE);
if (IS_ERR(s35390a->rtc)) {
err = PTR_ERR(s35390a->rtc);
goto exit_dummy;
}
+ /* supports per-minute alarms only, therefore set uie_unsupported */
+ s35390a->rtc->uie_unsupported = 1;
+
if (status1 & S35390A_FLAG_INT2)
rtc_update_irq(s35390a->rtc, 1, RTC_AF);
now_secs = rtc_tm_to_time64(&now);
alarm_secs = rtc_tm_to_time64(&t->time);
- /* Invalid alarm time */
- if (now_secs > alarm_secs)
- return -EINVAL;
-
memcpy(&rtc->alarm, t, sizeof(struct rtc_wkalrm));
/* Now many secs to fire */
/* Write to Alarm register */
writel_relaxed(alrmar, rtc->base + regs->alrmar);
- if (alrm->enabled)
- stm32_rtc_alarm_irq_enable(dev, 1);
- else
- stm32_rtc_alarm_irq_enable(dev, 0);
-
+ stm32_rtc_alarm_irq_enable(dev, alrm->enabled);
end:
stm32_rtc_wpr_lock(rtc);
{ .compatible = "allwinner,sun6i-a31-rtc" },
{ .compatible = "allwinner,sun8i-a23-rtc" },
{ .compatible = "allwinner,sun8i-h3-rtc" },
+ { .compatible = "allwinner,sun8i-r40-rtc" },
{ .compatible = "allwinner,sun8i-v3-rtc" },
{ .compatible = "allwinner,sun50i-h5-rtc" },
{ /* sentinel */ },
/*
* An RTC driver for the NVIDIA Tegra 200 series internal RTC.
*
- * Copyright (c) 2010, NVIDIA Corporation.
+ * Copyright (c) 2010-2019, NVIDIA Corporation.
*/
#include <linux/clk.h>
#include <linux/rtc.h>
#include <linux/slab.h>
-/* set to 1 = busy every eight 32kHz clocks during copy of sec+msec to AHB */
+/* Set to 1 = busy every eight 32 kHz clocks during copy of sec+msec to AHB. */
#define TEGRA_RTC_REG_BUSY 0x004
#define TEGRA_RTC_REG_SECONDS 0x008
-/* when msec is read, the seconds are buffered into shadow seconds. */
+/* When msec is read, the seconds are buffered into shadow seconds. */
#define TEGRA_RTC_REG_SHADOW_SECONDS 0x00c
#define TEGRA_RTC_REG_MILLI_SECONDS 0x010
#define TEGRA_RTC_REG_SECONDS_ALARM0 0x014
#define TEGRA_RTC_INTR_STATUS_SEC_ALARM0 (1<<0)
struct tegra_rtc_info {
- struct platform_device *pdev;
- struct rtc_device *rtc_dev;
- void __iomem *rtc_base; /* NULL if not initialized. */
- struct clk *clk;
- int tegra_rtc_irq; /* alarm and periodic irq */
- spinlock_t tegra_rtc_lock;
+ struct platform_device *pdev;
+ struct rtc_device *rtc;
+ void __iomem *base; /* NULL if not initialized */
+ struct clk *clk;
+ int irq; /* alarm and periodic IRQ */
+ spinlock_t lock;
};
-/* RTC hardware is busy when it is updating its values over AHB once
- * every eight 32kHz clocks (~250uS).
- * outside of these updates the CPU is free to write.
- * CPU is always free to read.
+/*
+ * RTC hardware is busy when it is updating its values over AHB once every
+ * eight 32 kHz clocks (~250 us). Outside of these updates the CPU is free to
+ * write. CPU is always free to read.
*/
static inline u32 tegra_rtc_check_busy(struct tegra_rtc_info *info)
{
- return readl(info->rtc_base + TEGRA_RTC_REG_BUSY) & 1;
+ return readl(info->base + TEGRA_RTC_REG_BUSY) & 1;
}
-/* Wait for hardware to be ready for writing.
- * This function tries to maximize the amount of time before the next update.
- * It does this by waiting for the RTC to become busy with its periodic update,
- * then returning once the RTC first becomes not busy.
+/*
+ * Wait for hardware to be ready for writing. This function tries to maximize
+ * the amount of time before the next update. It does this by waiting for the
+ * RTC to become busy with its periodic update, then returning once the RTC
+ * first becomes not busy.
+ *
* This periodic update (where the seconds and milliseconds are copied to the
- * AHB side) occurs every eight 32kHz clocks (~250uS).
- * The behavior of this function allows us to make some assumptions without
- * introducing a race, because 250uS is plenty of time to read/write a value.
+ * AHB side) occurs every eight 32 kHz clocks (~250 us). The behavior of this
+ * function allows us to make some assumptions without introducing a race,
+ * because 250 us is plenty of time to read/write a value.
*/
static int tegra_rtc_wait_while_busy(struct device *dev)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
+ int retries = 500; /* ~490 us is the worst case, ~250 us is best */
- int retries = 500; /* ~490 us is the worst case, ~250 us is best. */
-
- /* first wait for the RTC to become busy. this is when it
- * posts its updated seconds+msec registers to AHB side. */
+ /*
+ * First wait for the RTC to become busy. This is when it posts its
+ * updated seconds+msec registers to AHB side.
+ */
while (tegra_rtc_check_busy(info)) {
if (!retries--)
goto retry_failed;
+
udelay(1);
}
return 0;
retry_failed:
- dev_err(dev, "write failed:retry count exceeded.\n");
+ dev_err(dev, "write failed: retry count exceeded\n");
return -ETIMEDOUT;
}
static int tegra_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned long sec, msec;
- unsigned long sl_irq_flags;
+ unsigned long flags;
+ u32 sec, msec;
- /* RTC hardware copies seconds to shadow seconds when a read
- * of milliseconds occurs. use a lock to keep other threads out. */
- spin_lock_irqsave(&info->tegra_rtc_lock, sl_irq_flags);
+ /*
+ * RTC hardware copies seconds to shadow seconds when a read of
+ * milliseconds occurs. use a lock to keep other threads out.
+ */
+ spin_lock_irqsave(&info->lock, flags);
- msec = readl(info->rtc_base + TEGRA_RTC_REG_MILLI_SECONDS);
- sec = readl(info->rtc_base + TEGRA_RTC_REG_SHADOW_SECONDS);
+ msec = readl(info->base + TEGRA_RTC_REG_MILLI_SECONDS);
+ sec = readl(info->base + TEGRA_RTC_REG_SHADOW_SECONDS);
- spin_unlock_irqrestore(&info->tegra_rtc_lock, sl_irq_flags);
+ spin_unlock_irqrestore(&info->lock, flags);
rtc_time64_to_tm(sec, tm);
- dev_vdbg(dev, "time read as %lu. %ptR\n", sec, tm);
+ dev_vdbg(dev, "time read as %u, %ptR\n", sec, tm);
return 0;
}
static int tegra_rtc_set_time(struct device *dev, struct rtc_time *tm)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned long sec;
+ u32 sec;
int ret;
- /* convert tm to seconds. */
+ /* convert tm to seconds */
sec = rtc_tm_to_time64(tm);
- dev_vdbg(dev, "time set to %lu. %ptR\n", sec, tm);
+ dev_vdbg(dev, "time set to %u, %ptR\n", sec, tm);
- /* seconds only written if wait succeeded. */
+ /* seconds only written if wait succeeded */
ret = tegra_rtc_wait_while_busy(dev);
if (!ret)
- writel(sec, info->rtc_base + TEGRA_RTC_REG_SECONDS);
+ writel(sec, info->base + TEGRA_RTC_REG_SECONDS);
dev_vdbg(dev, "time read back as %d\n",
- readl(info->rtc_base + TEGRA_RTC_REG_SECONDS));
+ readl(info->base + TEGRA_RTC_REG_SECONDS));
return ret;
}
static int tegra_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned long sec;
- unsigned tmp;
+ u32 sec, value;
- sec = readl(info->rtc_base + TEGRA_RTC_REG_SECONDS_ALARM0);
+ sec = readl(info->base + TEGRA_RTC_REG_SECONDS_ALARM0);
if (sec == 0) {
- /* alarm is disabled. */
+ /* alarm is disabled */
alarm->enabled = 0;
} else {
- /* alarm is enabled. */
+ /* alarm is enabled */
alarm->enabled = 1;
rtc_time64_to_tm(sec, &alarm->time);
}
- tmp = readl(info->rtc_base + TEGRA_RTC_REG_INTR_STATUS);
- alarm->pending = (tmp & TEGRA_RTC_INTR_STATUS_SEC_ALARM0) != 0;
+ value = readl(info->base + TEGRA_RTC_REG_INTR_STATUS);
+ alarm->pending = (value & TEGRA_RTC_INTR_STATUS_SEC_ALARM0) != 0;
return 0;
}
static int tegra_rtc_alarm_irq_enable(struct device *dev, unsigned int enabled)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned status;
- unsigned long sl_irq_flags;
+ unsigned long flags;
+ u32 status;
tegra_rtc_wait_while_busy(dev);
- spin_lock_irqsave(&info->tegra_rtc_lock, sl_irq_flags);
+ spin_lock_irqsave(&info->lock, flags);
- /* read the original value, and OR in the flag. */
- status = readl(info->rtc_base + TEGRA_RTC_REG_INTR_MASK);
+ /* read the original value, and OR in the flag */
+ status = readl(info->base + TEGRA_RTC_REG_INTR_MASK);
if (enabled)
status |= TEGRA_RTC_INTR_MASK_SEC_ALARM0; /* set it */
else
status &= ~TEGRA_RTC_INTR_MASK_SEC_ALARM0; /* clear it */
- writel(status, info->rtc_base + TEGRA_RTC_REG_INTR_MASK);
+ writel(status, info->base + TEGRA_RTC_REG_INTR_MASK);
- spin_unlock_irqrestore(&info->tegra_rtc_lock, sl_irq_flags);
+ spin_unlock_irqrestore(&info->lock, flags);
return 0;
}
static int tegra_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned long sec;
+ u32 sec;
if (alarm->enabled)
sec = rtc_tm_to_time64(&alarm->time);
sec = 0;
tegra_rtc_wait_while_busy(dev);
- writel(sec, info->rtc_base + TEGRA_RTC_REG_SECONDS_ALARM0);
+ writel(sec, info->base + TEGRA_RTC_REG_SECONDS_ALARM0);
dev_vdbg(dev, "alarm read back as %d\n",
- readl(info->rtc_base + TEGRA_RTC_REG_SECONDS_ALARM0));
+ readl(info->base + TEGRA_RTC_REG_SECONDS_ALARM0));
/* if successfully written and alarm is enabled ... */
if (sec) {
tegra_rtc_alarm_irq_enable(dev, 1);
- dev_vdbg(dev, "alarm set as %lu. %ptR\n", sec, &alarm->time);
+ dev_vdbg(dev, "alarm set as %u, %ptR\n", sec, &alarm->time);
} else {
- /* disable alarm if 0 or write error. */
+ /* disable alarm if 0 or write error */
dev_vdbg(dev, "alarm disabled\n");
tegra_rtc_alarm_irq_enable(dev, 0);
}
{
struct device *dev = data;
struct tegra_rtc_info *info = dev_get_drvdata(dev);
- unsigned long events = 0;
- unsigned status;
- unsigned long sl_irq_flags;
+ unsigned long events = 0, flags;
+ u32 status;
- status = readl(info->rtc_base + TEGRA_RTC_REG_INTR_STATUS);
+ status = readl(info->base + TEGRA_RTC_REG_INTR_STATUS);
if (status) {
- /* clear the interrupt masks and status on any irq. */
+ /* clear the interrupt masks and status on any IRQ */
tegra_rtc_wait_while_busy(dev);
- spin_lock_irqsave(&info->tegra_rtc_lock, sl_irq_flags);
- writel(0, info->rtc_base + TEGRA_RTC_REG_INTR_MASK);
- writel(status, info->rtc_base + TEGRA_RTC_REG_INTR_STATUS);
- spin_unlock_irqrestore(&info->tegra_rtc_lock, sl_irq_flags);
+
+ spin_lock_irqsave(&info->lock, flags);
+ writel(0, info->base + TEGRA_RTC_REG_INTR_MASK);
+ writel(status, info->base + TEGRA_RTC_REG_INTR_STATUS);
+ spin_unlock_irqrestore(&info->lock, flags);
}
- /* check if Alarm */
- if ((status & TEGRA_RTC_INTR_STATUS_SEC_ALARM0))
+ /* check if alarm */
+ if (status & TEGRA_RTC_INTR_STATUS_SEC_ALARM0)
events |= RTC_IRQF | RTC_AF;
- /* check if Periodic */
- if ((status & TEGRA_RTC_INTR_STATUS_SEC_CDN_ALARM))
+ /* check if periodic */
+ if (status & TEGRA_RTC_INTR_STATUS_SEC_CDN_ALARM)
events |= RTC_IRQF | RTC_PF;
- rtc_update_irq(info->rtc_dev, 1, events);
+ rtc_update_irq(info->rtc, 1, events);
return IRQ_HANDLED;
}
static const struct rtc_class_ops tegra_rtc_ops = {
- .read_time = tegra_rtc_read_time,
- .set_time = tegra_rtc_set_time,
- .read_alarm = tegra_rtc_read_alarm,
- .set_alarm = tegra_rtc_set_alarm,
- .proc = tegra_rtc_proc,
+ .read_time = tegra_rtc_read_time,
+ .set_time = tegra_rtc_set_time,
+ .read_alarm = tegra_rtc_read_alarm,
+ .set_alarm = tegra_rtc_set_alarm,
+ .proc = tegra_rtc_proc,
.alarm_irq_enable = tegra_rtc_alarm_irq_enable,
};
};
MODULE_DEVICE_TABLE(of, tegra_rtc_dt_match);
-static int __init tegra_rtc_probe(struct platform_device *pdev)
+static int tegra_rtc_probe(struct platform_device *pdev)
{
struct tegra_rtc_info *info;
struct resource *res;
int ret;
- info = devm_kzalloc(&pdev->dev, sizeof(struct tegra_rtc_info),
- GFP_KERNEL);
+ info = devm_kzalloc(&pdev->dev, sizeof(*info), GFP_KERNEL);
if (!info)
return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- info->rtc_base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(info->rtc_base))
- return PTR_ERR(info->rtc_base);
+ info->base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(info->base))
+ return PTR_ERR(info->base);
ret = platform_get_irq(pdev, 0);
if (ret <= 0) {
return ret;
}
- info->tegra_rtc_irq = ret;
+ info->irq = ret;
- info->rtc_dev = devm_rtc_allocate_device(&pdev->dev);
- if (IS_ERR(info->rtc_dev))
- return PTR_ERR(info->rtc_dev);
+ info->rtc = devm_rtc_allocate_device(&pdev->dev);
+ if (IS_ERR(info->rtc))
+ return PTR_ERR(info->rtc);
- info->rtc_dev->ops = &tegra_rtc_ops;
- info->rtc_dev->range_max = U32_MAX;
+ info->rtc->ops = &tegra_rtc_ops;
+ info->rtc->range_max = U32_MAX;
info->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(info->clk))
if (ret < 0)
return ret;
- /* set context info. */
+ /* set context info */
info->pdev = pdev;
- spin_lock_init(&info->tegra_rtc_lock);
+ spin_lock_init(&info->lock);
platform_set_drvdata(pdev, info);
- /* clear out the hardware. */
- writel(0, info->rtc_base + TEGRA_RTC_REG_SECONDS_ALARM0);
- writel(0xffffffff, info->rtc_base + TEGRA_RTC_REG_INTR_STATUS);
- writel(0, info->rtc_base + TEGRA_RTC_REG_INTR_MASK);
+ /* clear out the hardware */
+ writel(0, info->base + TEGRA_RTC_REG_SECONDS_ALARM0);
+ writel(0xffffffff, info->base + TEGRA_RTC_REG_INTR_STATUS);
+ writel(0, info->base + TEGRA_RTC_REG_INTR_MASK);
device_init_wakeup(&pdev->dev, 1);
- ret = devm_request_irq(&pdev->dev, info->tegra_rtc_irq,
- tegra_rtc_irq_handler, IRQF_TRIGGER_HIGH,
- dev_name(&pdev->dev), &pdev->dev);
+ ret = devm_request_irq(&pdev->dev, info->irq, tegra_rtc_irq_handler,
+ IRQF_TRIGGER_HIGH, dev_name(&pdev->dev),
+ &pdev->dev);
if (ret) {
- dev_err(&pdev->dev,
- "Unable to request interrupt for device (err=%d).\n",
- ret);
+ dev_err(&pdev->dev, "failed to request interrupt: %d\n", ret);
goto disable_clk;
}
- ret = rtc_register_device(info->rtc_dev);
+ ret = rtc_register_device(info->rtc);
if (ret) {
- dev_err(&pdev->dev, "Unable to register device (err=%d).\n",
- ret);
+ dev_err(&pdev->dev, "failed to register device: %d\n", ret);
goto disable_clk;
}
tegra_rtc_wait_while_busy(dev);
- /* only use ALARM0 as a wake source. */
- writel(0xffffffff, info->rtc_base + TEGRA_RTC_REG_INTR_STATUS);
+ /* only use ALARM0 as a wake source */
+ writel(0xffffffff, info->base + TEGRA_RTC_REG_INTR_STATUS);
writel(TEGRA_RTC_INTR_STATUS_SEC_ALARM0,
- info->rtc_base + TEGRA_RTC_REG_INTR_MASK);
+ info->base + TEGRA_RTC_REG_INTR_MASK);
dev_vdbg(dev, "alarm sec = %d\n",
- readl(info->rtc_base + TEGRA_RTC_REG_SECONDS_ALARM0));
+ readl(info->base + TEGRA_RTC_REG_SECONDS_ALARM0));
- dev_vdbg(dev, "Suspend (device_may_wakeup=%d) irq:%d\n",
- device_may_wakeup(dev), info->tegra_rtc_irq);
+ dev_vdbg(dev, "Suspend (device_may_wakeup=%d) IRQ:%d\n",
+ device_may_wakeup(dev), info->irq);
- /* leave the alarms on as a wake source. */
+ /* leave the alarms on as a wake source */
if (device_may_wakeup(dev))
- enable_irq_wake(info->tegra_rtc_irq);
+ enable_irq_wake(info->irq);
return 0;
}
struct tegra_rtc_info *info = dev_get_drvdata(dev);
dev_vdbg(dev, "Resume (device_may_wakeup=%d)\n",
- device_may_wakeup(dev));
- /* alarms were left on as a wake source, turn them off. */
+ device_may_wakeup(dev));
+
+ /* alarms were left on as a wake source, turn them off */
if (device_may_wakeup(dev))
- disable_irq_wake(info->tegra_rtc_irq);
+ disable_irq_wake(info->irq);
return 0;
}
static void tegra_rtc_shutdown(struct platform_device *pdev)
{
- dev_vdbg(&pdev->dev, "disabling interrupts.\n");
+ dev_vdbg(&pdev->dev, "disabling interrupts\n");
tegra_rtc_alarm_irq_enable(&pdev->dev, 0);
}
-MODULE_ALIAS("platform:tegra_rtc");
static struct platform_driver tegra_rtc_driver = {
- .remove = tegra_rtc_remove,
- .shutdown = tegra_rtc_shutdown,
- .driver = {
- .name = "tegra_rtc",
+ .probe = tegra_rtc_probe,
+ .remove = tegra_rtc_remove,
+ .shutdown = tegra_rtc_shutdown,
+ .driver = {
+ .name = "tegra_rtc",
.of_match_table = tegra_rtc_dt_match,
- .pm = &tegra_rtc_pm_ops,
+ .pm = &tegra_rtc_pm_ops,
},
};
-
-module_platform_driver_probe(tegra_rtc_driver, tegra_rtc_probe);
+module_platform_driver(tegra_rtc_driver);
MODULE_AUTHOR("Jon Mayo <jmayo@nvidia.com>");
MODULE_DESCRIPTION("driver for Tegra internal RTC");
break;
default:
rtd->rtc->ops = &test_rtc_ops;
+ device_init_wakeup(&plat_dev->dev, 1);
}
timer_setup(&rtd->alarm, test_rtc_alarm_handler, 0);
struct tps65910 *tps = dev_get_drvdata(dev->parent);
int ret;
- ret = regmap_bulk_read(tps->regmap, TPS65910_SECONDS, alarm_data,
+ ret = regmap_bulk_read(tps->regmap, TPS65910_ALARM_SECONDS, alarm_data,
NUM_TIME_REGS);
if (ret < 0) {
dev_err(dev, "rtc_read_alarm error %d\n", ret);
ret = devm_request_threaded_irq(&pdev->dev, alm_irq, NULL,
wm831x_alm_irq,
- IRQF_TRIGGER_RISING, "RTC alarm",
+ IRQF_TRIGGER_RISING | IRQF_ONESHOT,
+ "RTC alarm",
wm831x_rtc);
if (ret != 0) {
dev_err(&pdev->dev, "Failed to request alarm IRQ %d: %d\n",
goto put_dev;
dev_info->dax_dev = alloc_dax(dev_info, dev_info->gd->disk_name,
- &dcssblk_dax_ops);
+ &dcssblk_dax_ops, DAXDEV_F_SYNC);
if (!dev_info->dax_dev) {
rc = -ENOMEM;
goto put_dev;
#include <linux/idr.h>
#include <linux/module.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/poll.h>
#include <linux/sched/signal.h>
static int ocxlflash_fs_cnt;
static struct vfsmount *ocxlflash_vfs_mount;
-static const struct dentry_operations ocxlflash_fs_dops = {
- .d_dname = simple_dname,
-};
-
-/*
- * ocxlflash_fs_mount() - mount the pseudo-filesystem
- * @fs_type: File system type.
- * @flags: Flags for the filesystem.
- * @dev_name: Device name associated with the filesystem.
- * @data: Data pointer.
- *
- * Return: pointer to the directory entry structure
- */
-static struct dentry *ocxlflash_fs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name,
- void *data)
+static int ocxlflash_fs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "ocxlflash:", NULL, &ocxlflash_fs_dops,
- OCXLFLASH_FS_MAGIC);
+ return init_pseudo(fc, OCXLFLASH_FS_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type ocxlflash_fs_type = {
.name = "ocxlflash",
.owner = THIS_MODULE,
- .mount = ocxlflash_fs_mount,
+ .init_fs_context = ocxlflash_fs_init_fs_context,
.kill_sb = kill_anon_super,
};
num_targets = virtscsi_config_get(vdev, max_target) + 1;
shost = scsi_host_alloc(&virtscsi_host_template,
- sizeof(*vscsi) + sizeof(vscsi->req_vqs[0]) * num_queues);
+ struct_size(vscsi, req_vqs, num_queues));
if (!shost)
return -ENOMEM;
}
EXPORT_SYMBOL_GPL(qcom_mdt_get_size);
+/**
+ * qcom_mdt_read_metadata() - read header and metadata from mdt or mbn
+ * @fw: firmware of mdt header or mbn
+ * @data_len: length of the read metadata blob
+ *
+ * The mechanism that performs the authentication of the loading firmware
+ * expects an ELF header directly followed by the segment of hashes, with no
+ * padding inbetween. This function allocates a chunk of memory for this pair
+ * and copy the two pieces into the buffer.
+ *
+ * In the case of split firmware the hash is found directly following the ELF
+ * header, rather than at p_offset described by the second program header.
+ *
+ * The caller is responsible to free (kfree()) the returned pointer.
+ *
+ * Return: pointer to data, or ERR_PTR()
+ */
+void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len)
+{
+ const struct elf32_phdr *phdrs;
+ const struct elf32_hdr *ehdr;
+ size_t hash_offset;
+ size_t hash_size;
+ size_t ehdr_size;
+ void *data;
+
+ ehdr = (struct elf32_hdr *)fw->data;
+ phdrs = (struct elf32_phdr *)(ehdr + 1);
+
+ if (ehdr->e_phnum < 2)
+ return ERR_PTR(-EINVAL);
+
+ if (phdrs[0].p_type == PT_LOAD || phdrs[1].p_type == PT_LOAD)
+ return ERR_PTR(-EINVAL);
+
+ if ((phdrs[1].p_flags & QCOM_MDT_TYPE_MASK) != QCOM_MDT_TYPE_HASH)
+ return ERR_PTR(-EINVAL);
+
+ ehdr_size = phdrs[0].p_filesz;
+ hash_size = phdrs[1].p_filesz;
+
+ data = kmalloc(ehdr_size + hash_size, GFP_KERNEL);
+ if (!data)
+ return ERR_PTR(-ENOMEM);
+
+ /* Is the header and hash already packed */
+ if (ehdr_size + hash_size == fw->size)
+ hash_offset = phdrs[0].p_filesz;
+ else
+ hash_offset = phdrs[1].p_offset;
+
+ memcpy(data, fw->data, ehdr_size);
+ memcpy(data + ehdr_size, fw->data + hash_offset, hash_size);
+
+ *data_len = ehdr_size + hash_size;
+
+ return data;
+}
+EXPORT_SYMBOL_GPL(qcom_mdt_read_metadata);
+
static int __qcom_mdt_load(struct device *dev, const struct firmware *fw,
const char *firmware, int pas_id, void *mem_region,
phys_addr_t mem_phys, size_t mem_size,
phys_addr_t mem_reloc;
phys_addr_t min_addr = PHYS_ADDR_MAX;
phys_addr_t max_addr = 0;
+ size_t metadata_len;
size_t fw_name_len;
ssize_t offset;
+ void *metadata;
char *fw_name;
bool relocate = false;
void *ptr;
- int ret;
+ int ret = 0;
int i;
if (!fw || !mem_region || !mem_phys || !mem_size)
return -ENOMEM;
if (pas_init) {
- ret = qcom_scm_pas_init_image(pas_id, fw->data, fw->size);
+ metadata = qcom_mdt_read_metadata(fw, &metadata_len);
+ if (IS_ERR(metadata)) {
+ ret = PTR_ERR(metadata);
+ goto out;
+ }
+
+ ret = qcom_scm_pas_init_image(pas_id, metadata, metadata_len);
+
+ kfree(metadata);
if (ret) {
dev_err(dev, "invalid firmware metadata\n");
goto out;
ptr = mem_region + offset;
- if (phdr->p_filesz) {
+ if (phdr->p_filesz && phdr->p_offset < fw->size) {
+ /* Firmware is large enough to be non-split */
+ if (phdr->p_offset + phdr->p_filesz > fw->size) {
+ dev_err(dev,
+ "failed to load segment %d from truncated file %s\n",
+ i, firmware);
+ ret = -EINVAL;
+ break;
+ }
+
+ memcpy(ptr, fw->data + phdr->p_offset, phdr->p_filesz);
+ } else if (phdr->p_filesz) {
+ /* Firmware not large enough, load split-out segments */
sprintf(fw_name + fw_name_len - 3, "b%02d", i);
ret = request_firmware_into_buf(&seg_fw, fw_name, dev,
ptr, phdr->p_filesz);
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
- fuse->apbdma.chan = __dma_request_channel(&mask, dma_filter, NULL);
+ fuse->apbdma.chan = dma_request_channel(mask, dma_filter, NULL);
if (!fuse->apbdma.chan)
return -EPROBE_DEFER;
* visorinput - keyboard and mouse
These drivers conform to the standard Linux bus/device model described
-within Documentation/driver-model/, and utilize a driver named visorbus to
+within Documentation/driver-api/driver-model/, and utilize a driver named visorbus to
present the virtual busses involved. Drivers in the 'visor*' driver set are
commonly referred to as "guest drivers" or "client drivers". All drivers
except visorbus expose a device of a specific usable class to the Linux guest
-----------------------------------
Because visorbus is a standard Linux bus driver in the model described in
-Documentation/driver-model/, the hierarchy of s-Par virtual devices is
+Documentation/driver-api/driver-model/, the hierarchy of s-Par virtual devices is
published in the sysfs tree beneath /bus/visorbus/, e.g.,
/sys/bus/visorbus/devices/ might look like:
.name = "fair_share",
.throttle = fair_share_throttle,
};
-
-int thermal_gov_fair_share_register(void)
-{
- return thermal_register_governor(&thermal_gov_fair_share);
-}
-
-void thermal_gov_fair_share_unregister(void)
-{
- thermal_unregister_governor(&thermal_gov_fair_share);
-}
-
+THERMAL_GOVERNOR_DECLARE(thermal_gov_fair_share);
.name = "bang_bang",
.throttle = bang_bang_control,
};
-
-int thermal_gov_bang_bang_register(void)
-{
- return thermal_register_governor(&thermal_gov_bang_bang);
-}
-
-void thermal_gov_bang_bang_unregister(void)
-{
- thermal_unregister_governor(&thermal_gov_bang_bang);
-}
+THERMAL_GOVERNOR_DECLARE(thermal_gov_bang_bang);
brightness in order to address a thermal condition or to reduce
power consumed by display device.
+config PROC_THERMAL_MMIO_RAPL
+ bool
+ depends on 64BIT
+ depends on POWERCAP
+ select INTEL_RAPL_CORE
+ default y
endif
#include <linux/platform_device.h>
#include <linux/acpi.h>
#include <linux/thermal.h>
+#include <linux/cpuhotplug.h>
+#include <linux/intel_rapl.h>
#include "int340x_thermal_zone.h"
#include "../intel_soc_dts_iosf.h"
/* GeminiLake thermal reporting device */
#define PCI_DEVICE_ID_PROC_GLK_THERMAL 0x318C
+#define DRV_NAME "proc_thermal"
+
struct power_config {
u32 index;
u32 min_uw;
struct power_config power_limits[2];
struct int34x_thermal_zone *int340x_zone;
struct intel_soc_dts_sensors *soc_dts;
+ void __iomem *mmio_base;
};
enum proc_thermal_emum_mode_type {
PROC_THERMAL_PLATFORM_DEV
};
+struct rapl_mmio_regs {
+ u64 reg_unit;
+ u64 regs[RAPL_DOMAIN_MAX][RAPL_DOMAIN_REG_MAX];
+ int limits[RAPL_DOMAIN_MAX];
+};
+
/*
* We can have only one type of enumeration, PCI or Platform,
* not both. So we don't need instance specific data.
return IRQ_HANDLED;
}
+#ifdef CONFIG_PROC_THERMAL_MMIO_RAPL
+
+#define MCHBAR 0
+
+/* RAPL Support via MMIO interface */
+static struct rapl_if_priv rapl_mmio_priv;
+
+static int rapl_mmio_cpu_online(unsigned int cpu)
+{
+ struct rapl_package *rp;
+
+ /* mmio rapl supports package 0 only for now */
+ if (topology_physical_package_id(cpu))
+ return 0;
+
+ rp = rapl_find_package_domain(cpu, &rapl_mmio_priv);
+ if (!rp) {
+ rp = rapl_add_package(cpu, &rapl_mmio_priv);
+ if (IS_ERR(rp))
+ return PTR_ERR(rp);
+ }
+ cpumask_set_cpu(cpu, &rp->cpumask);
+ return 0;
+}
+
+static int rapl_mmio_cpu_down_prep(unsigned int cpu)
+{
+ struct rapl_package *rp;
+ int lead_cpu;
+
+ rp = rapl_find_package_domain(cpu, &rapl_mmio_priv);
+ if (!rp)
+ return 0;
+
+ cpumask_clear_cpu(cpu, &rp->cpumask);
+ lead_cpu = cpumask_first(&rp->cpumask);
+ if (lead_cpu >= nr_cpu_ids)
+ rapl_remove_package(rp);
+ else if (rp->lead_cpu == cpu)
+ rp->lead_cpu = lead_cpu;
+ return 0;
+}
+
+static int rapl_mmio_read_raw(int cpu, struct reg_action *ra)
+{
+ if (!ra->reg)
+ return -EINVAL;
+
+ ra->value = readq((void __iomem *)ra->reg);
+ ra->value &= ra->mask;
+ return 0;
+}
+
+static int rapl_mmio_write_raw(int cpu, struct reg_action *ra)
+{
+ u64 val;
+
+ if (!ra->reg)
+ return -EINVAL;
+
+ val = readq((void __iomem *)ra->reg);
+ val &= ~ra->mask;
+ val |= ra->value;
+ writeq(val, (void __iomem *)ra->reg);
+ return 0;
+}
+
+static int proc_thermal_rapl_add(struct pci_dev *pdev,
+ struct proc_thermal_device *proc_priv,
+ struct rapl_mmio_regs *rapl_regs)
+{
+ enum rapl_domain_reg_id reg;
+ enum rapl_domain_type domain;
+ int ret;
+
+ if (!rapl_regs)
+ return 0;
+
+ ret = pcim_iomap_regions(pdev, 1 << MCHBAR, DRV_NAME);
+ if (ret) {
+ dev_err(&pdev->dev, "cannot reserve PCI memory region\n");
+ return -ENOMEM;
+ }
+
+ proc_priv->mmio_base = pcim_iomap_table(pdev)[MCHBAR];
+
+ for (domain = RAPL_DOMAIN_PACKAGE; domain < RAPL_DOMAIN_MAX; domain++) {
+ for (reg = RAPL_DOMAIN_REG_LIMIT; reg < RAPL_DOMAIN_REG_MAX; reg++)
+ if (rapl_regs->regs[domain][reg])
+ rapl_mmio_priv.regs[domain][reg] =
+ (u64)proc_priv->mmio_base +
+ rapl_regs->regs[domain][reg];
+ rapl_mmio_priv.limits[domain] = rapl_regs->limits[domain];
+ }
+ rapl_mmio_priv.reg_unit = (u64)proc_priv->mmio_base + rapl_regs->reg_unit;
+
+ rapl_mmio_priv.read_raw = rapl_mmio_read_raw;
+ rapl_mmio_priv.write_raw = rapl_mmio_write_raw;
+
+ rapl_mmio_priv.control_type = powercap_register_control_type(NULL, "intel-rapl-mmio", NULL);
+ if (IS_ERR(rapl_mmio_priv.control_type)) {
+ pr_debug("failed to register powercap control_type.\n");
+ return PTR_ERR(rapl_mmio_priv.control_type);
+ }
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "powercap/rapl:online",
+ rapl_mmio_cpu_online, rapl_mmio_cpu_down_prep);
+ if (ret < 0) {
+ powercap_unregister_control_type(rapl_mmio_priv.control_type);
+ return ret;
+ }
+ rapl_mmio_priv.pcap_rapl_online = ret;
+
+ return 0;
+}
+
+static void proc_thermal_rapl_remove(void)
+{
+ cpuhp_remove_state(rapl_mmio_priv.pcap_rapl_online);
+ powercap_unregister_control_type(rapl_mmio_priv.control_type);
+}
+
+static const struct rapl_mmio_regs rapl_mmio_hsw = {
+ .reg_unit = 0x5938,
+ .regs[RAPL_DOMAIN_PACKAGE] = { 0x59a0, 0x593c, 0x58f0, 0, 0x5930},
+ .regs[RAPL_DOMAIN_DRAM] = { 0x58e0, 0x58e8, 0x58ec, 0, 0},
+ .limits[RAPL_DOMAIN_PACKAGE] = 2,
+ .limits[RAPL_DOMAIN_DRAM] = 2,
+};
+
+#else
+
+static int proc_thermal_rapl_add(struct pci_dev *pdev,
+ struct proc_thermal_device *proc_priv,
+ struct rapl_mmio_regs *rapl_regs)
+{
+ return 0;
+}
+static void proc_thermal_rapl_remove(void) {}
+static const struct rapl_mmio_regs rapl_mmio_hsw;
+
+#endif /* CONFIG_MMIO_RAPL */
+
static int proc_thermal_pci_probe(struct pci_dev *pdev,
- const struct pci_device_id *unused)
+ const struct pci_device_id *id)
{
struct proc_thermal_device *proc_priv;
int ret;
return -ENODEV;
}
- ret = pci_enable_device(pdev);
+ ret = pcim_enable_device(pdev);
if (ret < 0) {
dev_err(&pdev->dev, "error: could not enable device\n");
return ret;
}
ret = proc_thermal_add(&pdev->dev, &proc_priv);
+ if (ret)
+ return ret;
+
+ ret = proc_thermal_rapl_add(pdev, proc_priv,
+ (struct rapl_mmio_regs *)id->driver_data);
if (ret) {
- pci_disable_device(pdev);
+ dev_err(&pdev->dev, "failed to add RAPL MMIO interface\n");
+ proc_thermal_remove(proc_priv);
return ret;
}
pci_disable_msi(pdev);
}
}
+ proc_thermal_rapl_remove();
proc_thermal_remove(proc_priv);
- pci_disable_device(pdev);
}
+#ifdef CONFIG_PM_SLEEP
+static int proc_thermal_resume(struct device *dev)
+{
+ struct proc_thermal_device *proc_dev;
+
+ proc_dev = dev_get_drvdata(dev);
+ proc_thermal_read_ppcc(proc_dev);
+
+ return 0;
+}
+#else
+#define proc_thermal_resume NULL
+#endif
+
+static SIMPLE_DEV_PM_OPS(proc_thermal_pm, NULL, proc_thermal_resume);
+
static const struct pci_device_id proc_thermal_pci_ids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_BDW_THERMAL)},
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_HSB_THERMAL)},
- { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_SKL_THERMAL)},
+ { PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_SKL_THERMAL),
+ .driver_data = (kernel_ulong_t)&rapl_mmio_hsw, },
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_BSW_THERMAL)},
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_BXT0_THERMAL)},
{ PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_PROC_BXT1_THERMAL)},
MODULE_DEVICE_TABLE(pci, proc_thermal_pci_ids);
static struct pci_driver proc_thermal_pci_driver = {
- .name = "proc_thermal",
+ .name = DRV_NAME,
.probe = proc_thermal_pci_probe,
.remove = proc_thermal_pci_remove,
.id_table = proc_thermal_pci_ids,
+ .driver.pm = &proc_thermal_pm,
};
static const struct acpi_device_id int3401_device_ids[] = {
.driver = {
.name = "int3401 thermal",
.acpi_match_table = int3401_device_ids,
+ .pm = &proc_thermal_pm,
},
};
.unbind_from_tz = power_allocator_unbind,
.throttle = power_allocator_throttle,
};
-
-int thermal_gov_power_allocator_register(void)
-{
- return thermal_register_governor(&thermal_gov_power_allocator);
-}
-
-void thermal_gov_power_allocator_unregister(void)
-{
- thermal_unregister_governor(&thermal_gov_power_allocator);
-}
+THERMAL_GOVERNOR_DECLARE(thermal_gov_power_allocator);
.name = "step_wise",
.throttle = step_wise_throttle,
};
-
-int thermal_gov_step_wise_register(void)
-{
- return thermal_register_governor(&thermal_gov_step_wise);
-}
-
-void thermal_gov_step_wise_unregister(void)
-{
- thermal_unregister_governor(&thermal_gov_step_wise);
-}
+THERMAL_GOVERNOR_DECLARE(thermal_gov_step_wise);
return count;
}
-static int __init thermal_register_governors(void)
+static void __init thermal_unregister_governors(void)
{
- int result;
+ struct thermal_governor **governor;
- result = thermal_gov_step_wise_register();
- if (result)
- return result;
+ for_each_governor_table(governor)
+ thermal_unregister_governor(*governor);
+}
- result = thermal_gov_fair_share_register();
- if (result)
- return result;
+static int __init thermal_register_governors(void)
+{
+ int ret = 0;
+ struct thermal_governor **governor;
- result = thermal_gov_bang_bang_register();
- if (result)
- return result;
+ for_each_governor_table(governor) {
+ ret = thermal_register_governor(*governor);
+ if (ret) {
+ pr_err("Failed to register governor: '%s'",
+ (*governor)->name);
+ break;
+ }
- result = thermal_gov_user_space_register();
- if (result)
- return result;
+ pr_info("Registered thermal governor '%s'",
+ (*governor)->name);
+ }
- return thermal_gov_power_allocator_register();
-}
+ if (ret) {
+ struct thermal_governor **gov;
-static void __init thermal_unregister_governors(void)
-{
- thermal_gov_step_wise_unregister();
- thermal_gov_fair_share_unregister();
- thermal_gov_bang_bang_unregister();
- thermal_gov_user_space_unregister();
- thermal_gov_power_allocator_unregister();
+ for_each_governor_table(gov) {
+ if (gov == governor)
+ break;
+ thermal_unregister_governor(*gov);
+ }
+ }
+
+ return ret;
}
/*
/* Initial state of a cooling device during binding */
#define THERMAL_NO_TARGET -1UL
+/* Init section thermal table */
+extern struct thermal_governor *__governor_thermal_table[];
+extern struct thermal_governor *__governor_thermal_table_end[];
+
+#define THERMAL_TABLE_ENTRY(table, name) \
+ static typeof(name) *__thermal_table_entry_##name \
+ __used __section(__##table##_thermal_table) = &name
+
+#define THERMAL_GOVERNOR_DECLARE(name) THERMAL_TABLE_ENTRY(governor, name)
+
+#define for_each_governor_table(__governor) \
+ for (__governor = __governor_thermal_table; \
+ __governor < __governor_thermal_table_end; \
+ __governor++)
+
/*
* This structure is used to describe the behavior of
* a certain cooling device on a certain trip point
unsigned long new_state) {}
#endif /* CONFIG_THERMAL_STATISTICS */
-#ifdef CONFIG_THERMAL_GOV_STEP_WISE
-int thermal_gov_step_wise_register(void);
-void thermal_gov_step_wise_unregister(void);
-#else
-static inline int thermal_gov_step_wise_register(void) { return 0; }
-static inline void thermal_gov_step_wise_unregister(void) {}
-#endif /* CONFIG_THERMAL_GOV_STEP_WISE */
-
-#ifdef CONFIG_THERMAL_GOV_FAIR_SHARE
-int thermal_gov_fair_share_register(void);
-void thermal_gov_fair_share_unregister(void);
-#else
-static inline int thermal_gov_fair_share_register(void) { return 0; }
-static inline void thermal_gov_fair_share_unregister(void) {}
-#endif /* CONFIG_THERMAL_GOV_FAIR_SHARE */
-
-#ifdef CONFIG_THERMAL_GOV_BANG_BANG
-int thermal_gov_bang_bang_register(void);
-void thermal_gov_bang_bang_unregister(void);
-#else
-static inline int thermal_gov_bang_bang_register(void) { return 0; }
-static inline void thermal_gov_bang_bang_unregister(void) {}
-#endif /* CONFIG_THERMAL_GOV_BANG_BANG */
-
-#ifdef CONFIG_THERMAL_GOV_USER_SPACE
-int thermal_gov_user_space_register(void);
-void thermal_gov_user_space_unregister(void);
-#else
-static inline int thermal_gov_user_space_register(void) { return 0; }
-static inline void thermal_gov_user_space_unregister(void) {}
-#endif /* CONFIG_THERMAL_GOV_USER_SPACE */
-
-#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
-int thermal_gov_power_allocator_register(void);
-void thermal_gov_power_allocator_unregister(void);
-#else
-static inline int thermal_gov_power_allocator_register(void) { return 0; }
-static inline void thermal_gov_power_allocator_unregister(void) {}
-#endif /* CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
-
/* device tree support */
#ifdef CONFIG_THERMAL_OF
int of_parse_thermal_zones(void);
.name = "user_space",
.throttle = notify_user_space,
};
-
-int thermal_gov_user_space_register(void)
-{
- return thermal_register_governor(&thermal_gov_user_space);
-}
-
-void thermal_gov_user_space_unregister(void)
-{
- thermal_unregister_governor(&thermal_gov_user_space);
-}
-
+THERMAL_GOVERNOR_DECLARE(thermal_gov_user_space);
select the console driver that will serve as the backend for the
virtual terminals.
- See <file:Documentation/console/console.txt> for more
+ See <file:Documentation/driver-api/console.rst> for more
information. For framebuffer console users, please refer to
<file:Documentation/fb/fbcon.rst>.
This driver supports Comtrol RocketPort and RocketModem PCI boards.
These boards provide 2, 4, 8, 16, or 32 high-speed serial ports or
modems. For information about the RocketPort/RocketModem boards
- and this driver read <file:Documentation/serial/rocket.rst>.
+ and this driver read <file:Documentation/driver-api/serial/rocket.rst>.
To compile this driver as a module, choose M here: the
module will be called rocket.
your Linux box, for instance in order to become a dial-in server.
For information about the Cyclades-Z card, read
- <file:Documentation/serial/cyclades_z.rst>.
+ <file:Documentation/driver-api/serial/cyclades_z.rst>.
To compile this driver as a module, choose M here: the
module will be called cyclades.
help
If you have a machine based on a SA1100/SA1110 StrongARM(R) CPU you
can enable its onboard serial port by enabling this option.
- Please read <file:Documentation/arm/SA1100/serial_UART> for further
+ Please read <file:Documentation/arm/sa1100/serial_uart.rst> for further
info.
config SERIAL_SA1100_CONSOLE
}
/* UART operations
*
- * Details on these functions can be found in Documentation/serial/driver.rst
+ * Details on these functions can be found in Documentation/driver-api/serial/driver.rst
*/
static const struct uart_ops qe_uart_pops = {
.tx_empty = qe_uart_tx_empty,
tty->ldisc = NULL;
}
-static int zero;
-static int one = 1;
static struct ctl_table tty_table[] = {
{
.procname = "ldisc_autoload",
.maxlen = sizeof(tty_ldisc_autoload),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{ }
};
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/pagemap.h>
#include <linux/uts.h>
#include <linux/wait.h>
};
static int
-gadgetfs_fill_super (struct super_block *sb, void *opts, int silent)
+gadgetfs_fill_super (struct super_block *sb, struct fs_context *fc)
{
struct inode *inode;
struct dev_data *dev;
}
/* "mount -t gadgetfs path /dev/gadget" ends up here */
-static struct dentry *
-gadgetfs_mount (struct file_system_type *t, int flags,
- const char *path, void *opts)
+static int gadgetfs_get_tree(struct fs_context *fc)
{
- return mount_single (t, flags, opts, gadgetfs_fill_super);
+ return get_tree_single(fc, gadgetfs_fill_super);
+}
+
+static const struct fs_context_operations gadgetfs_context_ops = {
+ .get_tree = gadgetfs_get_tree,
+};
+
+static int gadgetfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &gadgetfs_context_ops;
+ return 0;
}
static void
static struct file_system_type gadgetfs_type = {
.owner = THIS_MODULE,
.name = shortname,
- .mount = gadgetfs_mount,
+ .init_fs_context = gadgetfs_init_fs_context,
.kill_sb = gadgetfs_kill_sb,
};
MODULE_ALIAS_FS("gadgetfs");
select VFIO_IOMMU_TYPE1 if (X86 || S390 || ARM || ARM64)
help
VFIO provides a framework for secure userspace device drivers.
- See Documentation/vfio.txt for more details.
+ See Documentation/driver-api/vfio.rst for more details.
If you don't know what to do here, say N.
default n
help
Provides a framework to virtualize devices.
- See Documentation/vfio-mediated-device.txt for more details.
+ See Documentation/driver-api/vfio-mediated-device.rst for more details.
If you don't know what do here, say N.
{
int ret;
struct mdev_parent *parent;
+ char *env_string = "MDEV_STATE=registered";
+ char *envp[] = { env_string, NULL };
/* check for mandatory ops */
if (!ops || !ops->create || !ops->remove || !ops->supported_type_groups)
mutex_unlock(&parent_list_lock);
dev_info(dev, "MDEV: Registered\n");
+ kobject_uevent_env(&dev->kobj, KOBJ_CHANGE, envp);
+
return 0;
add_dev_err:
void mdev_unregister_device(struct device *dev)
{
struct mdev_parent *parent;
+ char *env_string = "MDEV_STATE=unregistered";
+ char *envp[] = { env_string, NULL };
mutex_lock(&parent_list_lock);
parent = __find_parent_device(dev);
up_write(&parent->unreg_sem);
mdev_put_parent(parent);
+
+ /* We still have the caller's reference to use for the uevent */
+ kobject_uevent_env(&dev->kobj, KOBJ_CHANGE, envp);
}
EXPORT_SYMBOL(mdev_unregister_device);
atomic_inc(&data->mm->mm_count);
ret = (int) mm_iommu_newdev(data->mm, data->useraddr,
- (vma->vm_end - vma->vm_start) >> PAGE_SHIFT,
- data->gpu_hpa, &data->mem);
+ vma_pages(vma), data->gpu_hpa, &data->mem);
trace_vfio_pci_nvgpu_mmap(vdev->pdev, data->gpu_hpa, data->useraddr,
vma->vm_end - vma->vm_start, ret);
#include <linux/vmalloc.h>
#include <linux/sched/mm.h>
#include <linux/sched/signal.h>
+#include <linux/mm.h>
#include <asm/iommu.h>
#include <asm/tce.h>
static void tce_iommu_detach_group(void *iommu_data,
struct iommu_group *iommu_group);
-static long try_increment_locked_vm(struct mm_struct *mm, long npages)
-{
- long ret = 0, locked, lock_limit;
-
- if (WARN_ON_ONCE(!mm))
- return -EPERM;
-
- if (!npages)
- return 0;
-
- down_write(&mm->mmap_sem);
- locked = mm->locked_vm + npages;
- lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
- if (locked > lock_limit && !capable(CAP_IPC_LOCK))
- ret = -ENOMEM;
- else
- mm->locked_vm += npages;
-
- pr_debug("[%d] RLIMIT_MEMLOCK +%ld %ld/%ld%s\n", current->pid,
- npages << PAGE_SHIFT,
- mm->locked_vm << PAGE_SHIFT,
- rlimit(RLIMIT_MEMLOCK),
- ret ? " - exceeded" : "");
-
- up_write(&mm->mmap_sem);
-
- return ret;
-}
-
-static void decrement_locked_vm(struct mm_struct *mm, long npages)
-{
- if (!mm || !npages)
- return;
-
- down_write(&mm->mmap_sem);
- if (WARN_ON_ONCE(npages > mm->locked_vm))
- npages = mm->locked_vm;
- mm->locked_vm -= npages;
- pr_debug("[%d] RLIMIT_MEMLOCK -%ld %ld/%ld\n", current->pid,
- npages << PAGE_SHIFT,
- mm->locked_vm << PAGE_SHIFT,
- rlimit(RLIMIT_MEMLOCK));
- up_write(&mm->mmap_sem);
-}
-
/*
* VFIO IOMMU fd for SPAPR_TCE IOMMU implementation
*
return ret;
locked = table_group->tce32_size >> PAGE_SHIFT;
- ret = try_increment_locked_vm(container->mm, locked);
+ ret = account_locked_vm(container->mm, locked, true);
if (ret)
return ret;
container->enabled = false;
BUG_ON(!container->mm);
- decrement_locked_vm(container->mm, container->locked_pages);
+ account_locked_vm(container->mm, container->locked_pages, false);
}
static void *tce_iommu_open(unsigned long arg)
if (!table_size)
return -EINVAL;
- ret = try_increment_locked_vm(container->mm, table_size >> PAGE_SHIFT);
+ ret = account_locked_vm(container->mm, table_size >> PAGE_SHIFT, true);
if (ret)
return ret;
unsigned long pages = tbl->it_allocated_size >> PAGE_SHIFT;
iommu_tce_table_put(tbl);
- decrement_locked_vm(container->mm, pages);
+ account_locked_vm(container->mm, pages, false);
}
static long tce_iommu_create_window(struct tce_container *container,
ret = down_write_killable(&mm->mmap_sem);
if (!ret) {
- if (npage > 0) {
- if (!dma->lock_cap) {
- unsigned long limit;
-
- limit = task_rlimit(dma->task,
- RLIMIT_MEMLOCK) >> PAGE_SHIFT;
-
- if (mm->locked_vm + npage > limit)
- ret = -ENOMEM;
- }
- }
-
- if (!ret)
- mm->locked_vm += npage;
-
+ ret = __account_locked_vm(mm, abs(npage), npage > 0, dma->task,
+ dma->lock_cap);
up_write(&mm->mmap_sem);
}
if (!sock)
goto out;
- if (!vq_iotlb_prefetch(vq))
+ if (!vq_meta_prefetch(vq))
goto out;
vhost_disable_notify(&net->dev, vq);
if (!sock)
goto out;
- if (!vq_iotlb_prefetch(vq))
+ if (!vq_meta_prefetch(vq))
goto out;
vhost_disable_notify(&net->dev, vq);
__vhost_vq_meta_reset(d->vqs[i]);
}
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+static void vhost_map_unprefetch(struct vhost_map *map)
+{
+ kfree(map->pages);
+ map->pages = NULL;
+ map->npages = 0;
+ map->addr = NULL;
+}
+
+static void vhost_uninit_vq_maps(struct vhost_virtqueue *vq)
+{
+ struct vhost_map *map[VHOST_NUM_ADDRS];
+ int i;
+
+ spin_lock(&vq->mmu_lock);
+ for (i = 0; i < VHOST_NUM_ADDRS; i++) {
+ map[i] = rcu_dereference_protected(vq->maps[i],
+ lockdep_is_held(&vq->mmu_lock));
+ if (map[i])
+ rcu_assign_pointer(vq->maps[i], NULL);
+ }
+ spin_unlock(&vq->mmu_lock);
+
+ synchronize_rcu();
+
+ for (i = 0; i < VHOST_NUM_ADDRS; i++)
+ if (map[i])
+ vhost_map_unprefetch(map[i]);
+
+}
+
+static void vhost_reset_vq_maps(struct vhost_virtqueue *vq)
+{
+ int i;
+
+ vhost_uninit_vq_maps(vq);
+ for (i = 0; i < VHOST_NUM_ADDRS; i++)
+ vq->uaddrs[i].size = 0;
+}
+
+static bool vhost_map_range_overlap(struct vhost_uaddr *uaddr,
+ unsigned long start,
+ unsigned long end)
+{
+ if (unlikely(!uaddr->size))
+ return false;
+
+ return !(end < uaddr->uaddr || start > uaddr->uaddr - 1 + uaddr->size);
+}
+
+static void vhost_invalidate_vq_start(struct vhost_virtqueue *vq,
+ int index,
+ unsigned long start,
+ unsigned long end)
+{
+ struct vhost_uaddr *uaddr = &vq->uaddrs[index];
+ struct vhost_map *map;
+ int i;
+
+ if (!vhost_map_range_overlap(uaddr, start, end))
+ return;
+
+ spin_lock(&vq->mmu_lock);
+ ++vq->invalidate_count;
+
+ map = rcu_dereference_protected(vq->maps[index],
+ lockdep_is_held(&vq->mmu_lock));
+ if (map) {
+ if (uaddr->write) {
+ for (i = 0; i < map->npages; i++)
+ set_page_dirty(map->pages[i]);
+ }
+ rcu_assign_pointer(vq->maps[index], NULL);
+ }
+ spin_unlock(&vq->mmu_lock);
+
+ if (map) {
+ synchronize_rcu();
+ vhost_map_unprefetch(map);
+ }
+}
+
+static void vhost_invalidate_vq_end(struct vhost_virtqueue *vq,
+ int index,
+ unsigned long start,
+ unsigned long end)
+{
+ if (!vhost_map_range_overlap(&vq->uaddrs[index], start, end))
+ return;
+
+ spin_lock(&vq->mmu_lock);
+ --vq->invalidate_count;
+ spin_unlock(&vq->mmu_lock);
+}
+
+static int vhost_invalidate_range_start(struct mmu_notifier *mn,
+ const struct mmu_notifier_range *range)
+{
+ struct vhost_dev *dev = container_of(mn, struct vhost_dev,
+ mmu_notifier);
+ int i, j;
+
+ if (!mmu_notifier_range_blockable(range))
+ return -EAGAIN;
+
+ for (i = 0; i < dev->nvqs; i++) {
+ struct vhost_virtqueue *vq = dev->vqs[i];
+
+ for (j = 0; j < VHOST_NUM_ADDRS; j++)
+ vhost_invalidate_vq_start(vq, j,
+ range->start,
+ range->end);
+ }
+
+ return 0;
+}
+
+static void vhost_invalidate_range_end(struct mmu_notifier *mn,
+ const struct mmu_notifier_range *range)
+{
+ struct vhost_dev *dev = container_of(mn, struct vhost_dev,
+ mmu_notifier);
+ int i, j;
+
+ for (i = 0; i < dev->nvqs; i++) {
+ struct vhost_virtqueue *vq = dev->vqs[i];
+
+ for (j = 0; j < VHOST_NUM_ADDRS; j++)
+ vhost_invalidate_vq_end(vq, j,
+ range->start,
+ range->end);
+ }
+}
+
+static const struct mmu_notifier_ops vhost_mmu_notifier_ops = {
+ .invalidate_range_start = vhost_invalidate_range_start,
+ .invalidate_range_end = vhost_invalidate_range_end,
+};
+
+static void vhost_init_maps(struct vhost_dev *dev)
+{
+ struct vhost_virtqueue *vq;
+ int i, j;
+
+ dev->mmu_notifier.ops = &vhost_mmu_notifier_ops;
+
+ for (i = 0; i < dev->nvqs; ++i) {
+ vq = dev->vqs[i];
+ for (j = 0; j < VHOST_NUM_ADDRS; j++)
+ RCU_INIT_POINTER(vq->maps[j], NULL);
+ }
+}
+#endif
+
static void vhost_vq_reset(struct vhost_dev *dev,
struct vhost_virtqueue *vq)
{
vq->busyloop_timeout = 0;
vq->umem = NULL;
vq->iotlb = NULL;
+ vq->invalidate_count = 0;
__vhost_vq_meta_reset(vq);
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ vhost_reset_vq_maps(vq);
+#endif
}
static int vhost_worker(void *data)
}
EXPORT_SYMBOL_GPL(vhost_exceeds_weight);
+static size_t vhost_get_avail_size(struct vhost_virtqueue *vq,
+ unsigned int num)
+{
+ size_t event __maybe_unused =
+ vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0;
+
+ return sizeof(*vq->avail) +
+ sizeof(*vq->avail->ring) * num + event;
+}
+
+static size_t vhost_get_used_size(struct vhost_virtqueue *vq,
+ unsigned int num)
+{
+ size_t event __maybe_unused =
+ vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0;
+
+ return sizeof(*vq->used) +
+ sizeof(*vq->used->ring) * num + event;
+}
+
+static size_t vhost_get_desc_size(struct vhost_virtqueue *vq,
+ unsigned int num)
+{
+ return sizeof(*vq->desc) * num;
+}
+
void vhost_dev_init(struct vhost_dev *dev,
struct vhost_virtqueue **vqs, int nvqs,
int iov_limit, int weight, int byte_weight)
INIT_LIST_HEAD(&dev->read_list);
INIT_LIST_HEAD(&dev->pending_list);
spin_lock_init(&dev->iotlb_lock);
-
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ vhost_init_maps(dev);
+#endif
for (i = 0; i < dev->nvqs; ++i) {
vq = dev->vqs[i];
vq->heads = NULL;
vq->dev = dev;
mutex_init(&vq->mutex);
+ spin_lock_init(&vq->mmu_lock);
vhost_vq_reset(dev, vq);
if (vq->handle_kick)
vhost_poll_init(&vq->poll, vq->handle_kick,
if (err)
goto err_cgroup;
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ err = mmu_notifier_register(&dev->mmu_notifier, dev->mm);
+ if (err)
+ goto err_mmu_notifier;
+#endif
+
return 0;
+
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+err_mmu_notifier:
+ vhost_dev_free_iovecs(dev);
+#endif
err_cgroup:
kthread_stop(worker);
dev->worker = NULL;
spin_unlock(&dev->iotlb_lock);
}
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+static void vhost_setup_uaddr(struct vhost_virtqueue *vq,
+ int index, unsigned long uaddr,
+ size_t size, bool write)
+{
+ struct vhost_uaddr *addr = &vq->uaddrs[index];
+
+ addr->uaddr = uaddr;
+ addr->size = size;
+ addr->write = write;
+}
+
+static void vhost_setup_vq_uaddr(struct vhost_virtqueue *vq)
+{
+ vhost_setup_uaddr(vq, VHOST_ADDR_DESC,
+ (unsigned long)vq->desc,
+ vhost_get_desc_size(vq, vq->num),
+ false);
+ vhost_setup_uaddr(vq, VHOST_ADDR_AVAIL,
+ (unsigned long)vq->avail,
+ vhost_get_avail_size(vq, vq->num),
+ false);
+ vhost_setup_uaddr(vq, VHOST_ADDR_USED,
+ (unsigned long)vq->used,
+ vhost_get_used_size(vq, vq->num),
+ true);
+}
+
+static int vhost_map_prefetch(struct vhost_virtqueue *vq,
+ int index)
+{
+ struct vhost_map *map;
+ struct vhost_uaddr *uaddr = &vq->uaddrs[index];
+ struct page **pages;
+ int npages = DIV_ROUND_UP(uaddr->size, PAGE_SIZE);
+ int npinned;
+ void *vaddr, *v;
+ int err;
+ int i;
+
+ spin_lock(&vq->mmu_lock);
+
+ err = -EFAULT;
+ if (vq->invalidate_count)
+ goto err;
+
+ err = -ENOMEM;
+ map = kmalloc(sizeof(*map), GFP_ATOMIC);
+ if (!map)
+ goto err;
+
+ pages = kmalloc_array(npages, sizeof(struct page *), GFP_ATOMIC);
+ if (!pages)
+ goto err_pages;
+
+ err = EFAULT;
+ npinned = __get_user_pages_fast(uaddr->uaddr, npages,
+ uaddr->write, pages);
+ if (npinned > 0)
+ release_pages(pages, npinned);
+ if (npinned != npages)
+ goto err_gup;
+
+ for (i = 0; i < npinned; i++)
+ if (PageHighMem(pages[i]))
+ goto err_gup;
+
+ vaddr = v = page_address(pages[0]);
+
+ /* For simplicity, fallback to userspace address if VA is not
+ * contigious.
+ */
+ for (i = 1; i < npinned; i++) {
+ v += PAGE_SIZE;
+ if (v != page_address(pages[i]))
+ goto err_gup;
+ }
+
+ map->addr = vaddr + (uaddr->uaddr & (PAGE_SIZE - 1));
+ map->npages = npages;
+ map->pages = pages;
+
+ rcu_assign_pointer(vq->maps[index], map);
+ /* No need for a synchronize_rcu(). This function should be
+ * called by dev->worker so we are serialized with all
+ * readers.
+ */
+ spin_unlock(&vq->mmu_lock);
+
+ return 0;
+
+err_gup:
+ kfree(pages);
+err_pages:
+ kfree(map);
+err:
+ spin_unlock(&vq->mmu_lock);
+ return err;
+}
+#endif
+
void vhost_dev_cleanup(struct vhost_dev *dev)
{
int i;
kthread_stop(dev->worker);
dev->worker = NULL;
}
- if (dev->mm)
+ if (dev->mm) {
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ mmu_notifier_unregister(&dev->mmu_notifier, dev->mm);
+#endif
mmput(dev->mm);
+ }
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ for (i = 0; i < dev->nvqs; i++)
+ vhost_uninit_vq_maps(dev->vqs[i]);
+#endif
dev->mm = NULL;
}
EXPORT_SYMBOL_GPL(vhost_dev_cleanup);
ret; \
})
+static inline int vhost_put_avail_event(struct vhost_virtqueue *vq)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_used *used;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_USED]);
+ if (likely(map)) {
+ used = map->addr;
+ *((__virtio16 *)&used->ring[vq->num]) =
+ cpu_to_vhost16(vq, vq->avail_idx);
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_put_user(vq, cpu_to_vhost16(vq, vq->avail_idx),
+ vhost_avail_event(vq));
+}
+
+static inline int vhost_put_used(struct vhost_virtqueue *vq,
+ struct vring_used_elem *head, int idx,
+ int count)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_used *used;
+ size_t size;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_USED]);
+ if (likely(map)) {
+ used = map->addr;
+ size = count * sizeof(*head);
+ memcpy(used->ring + idx, head, size);
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_copy_to_user(vq, vq->used->ring + idx, head,
+ count * sizeof(*head));
+}
+
+static inline int vhost_put_used_flags(struct vhost_virtqueue *vq)
+
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_used *used;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_USED]);
+ if (likely(map)) {
+ used = map->addr;
+ used->flags = cpu_to_vhost16(vq, vq->used_flags);
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_put_user(vq, cpu_to_vhost16(vq, vq->used_flags),
+ &vq->used->flags);
+}
+
+static inline int vhost_put_used_idx(struct vhost_virtqueue *vq)
+
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_used *used;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_USED]);
+ if (likely(map)) {
+ used = map->addr;
+ used->idx = cpu_to_vhost16(vq, vq->last_used_idx);
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_put_user(vq, cpu_to_vhost16(vq, vq->last_used_idx),
+ &vq->used->idx);
+}
+
#define vhost_get_user(vq, x, ptr, type) \
({ \
int ret; \
mutex_unlock(&d->vqs[i]->mutex);
}
+static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq,
+ __virtio16 *idx)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_avail *avail;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_AVAIL]);
+ if (likely(map)) {
+ avail = map->addr;
+ *idx = avail->idx;
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_get_avail(vq, *idx, &vq->avail->idx);
+}
+
+static inline int vhost_get_avail_head(struct vhost_virtqueue *vq,
+ __virtio16 *head, int idx)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_avail *avail;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_AVAIL]);
+ if (likely(map)) {
+ avail = map->addr;
+ *head = avail->ring[idx & (vq->num - 1)];
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_get_avail(vq, *head,
+ &vq->avail->ring[idx & (vq->num - 1)]);
+}
+
+static inline int vhost_get_avail_flags(struct vhost_virtqueue *vq,
+ __virtio16 *flags)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_avail *avail;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_AVAIL]);
+ if (likely(map)) {
+ avail = map->addr;
+ *flags = avail->flags;
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_get_avail(vq, *flags, &vq->avail->flags);
+}
+
+static inline int vhost_get_used_event(struct vhost_virtqueue *vq,
+ __virtio16 *event)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_avail *avail;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+ map = rcu_dereference(vq->maps[VHOST_ADDR_AVAIL]);
+ if (likely(map)) {
+ avail = map->addr;
+ *event = (__virtio16)avail->ring[vq->num];
+ rcu_read_unlock();
+ return 0;
+ }
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_get_avail(vq, *event, vhost_used_event(vq));
+}
+
+static inline int vhost_get_used_idx(struct vhost_virtqueue *vq,
+ __virtio16 *idx)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_used *used;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_USED]);
+ if (likely(map)) {
+ used = map->addr;
+ *idx = used->idx;
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_get_used(vq, *idx, &vq->used->idx);
+}
+
+static inline int vhost_get_desc(struct vhost_virtqueue *vq,
+ struct vring_desc *desc, int idx)
+{
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ struct vhost_map *map;
+ struct vring_desc *d;
+
+ if (!vq->iotlb) {
+ rcu_read_lock();
+
+ map = rcu_dereference(vq->maps[VHOST_ADDR_DESC]);
+ if (likely(map)) {
+ d = map->addr;
+ *desc = *(d + idx);
+ rcu_read_unlock();
+ return 0;
+ }
+
+ rcu_read_unlock();
+ }
+#endif
+
+ return vhost_copy_from_user(vq, desc, vq->desc + idx, sizeof(*desc));
+}
+
static int vhost_new_umem_range(struct vhost_umem *umem,
u64 start, u64 size, u64 end,
u64 userspace_addr, int perm)
struct vring_used __user *used)
{
- size_t s __maybe_unused = vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0;
-
- return access_ok(desc, num * sizeof *desc) &&
- access_ok(avail,
- sizeof *avail + num * sizeof *avail->ring + s) &&
- access_ok(used,
- sizeof *used + num * sizeof *used->ring + s);
+ return access_ok(desc, vhost_get_desc_size(vq, num)) &&
+ access_ok(avail, vhost_get_avail_size(vq, num)) &&
+ access_ok(used, vhost_get_used_size(vq, num));
}
static void vhost_vq_meta_update(struct vhost_virtqueue *vq,
return true;
}
-int vq_iotlb_prefetch(struct vhost_virtqueue *vq)
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+static void vhost_vq_map_prefetch(struct vhost_virtqueue *vq)
+{
+ struct vhost_map __rcu *map;
+ int i;
+
+ for (i = 0; i < VHOST_NUM_ADDRS; i++) {
+ rcu_read_lock();
+ map = rcu_dereference(vq->maps[i]);
+ rcu_read_unlock();
+ if (unlikely(!map))
+ vhost_map_prefetch(vq, i);
+ }
+}
+#endif
+
+int vq_meta_prefetch(struct vhost_virtqueue *vq)
{
- size_t s = vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0;
unsigned int num = vq->num;
- if (!vq->iotlb)
+ if (!vq->iotlb) {
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ vhost_vq_map_prefetch(vq);
+#endif
return 1;
+ }
return iotlb_access_ok(vq, VHOST_ACCESS_RO, (u64)(uintptr_t)vq->desc,
- num * sizeof(*vq->desc), VHOST_ADDR_DESC) &&
+ vhost_get_desc_size(vq, num), VHOST_ADDR_DESC) &&
iotlb_access_ok(vq, VHOST_ACCESS_RO, (u64)(uintptr_t)vq->avail,
- sizeof *vq->avail +
- num * sizeof(*vq->avail->ring) + s,
+ vhost_get_avail_size(vq, num),
VHOST_ADDR_AVAIL) &&
iotlb_access_ok(vq, VHOST_ACCESS_WO, (u64)(uintptr_t)vq->used,
- sizeof *vq->used +
- num * sizeof(*vq->used->ring) + s,
- VHOST_ADDR_USED);
+ vhost_get_used_size(vq, num), VHOST_ADDR_USED);
}
-EXPORT_SYMBOL_GPL(vq_iotlb_prefetch);
+EXPORT_SYMBOL_GPL(vq_meta_prefetch);
/* Can we log writes? */
/* Caller should have device mutex but not vq mutex */
static bool vq_log_access_ok(struct vhost_virtqueue *vq,
void __user *log_base)
{
- size_t s = vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0;
-
return vq_memory_access_ok(log_base, vq->umem,
vhost_has_feature(vq, VHOST_F_LOG_ALL)) &&
(!vq->log_used || log_access_ok(log_base, vq->log_addr,
- sizeof *vq->used +
- vq->num * sizeof *vq->used->ring + s));
+ vhost_get_used_size(vq, vq->num)));
}
/* Can we start vq? */
return -EFAULT;
}
+static long vhost_vring_set_num(struct vhost_dev *d,
+ struct vhost_virtqueue *vq,
+ void __user *argp)
+{
+ struct vhost_vring_state s;
+
+ /* Resizing ring with an active backend?
+ * You don't want to do that. */
+ if (vq->private_data)
+ return -EBUSY;
+
+ if (copy_from_user(&s, argp, sizeof s))
+ return -EFAULT;
+
+ if (!s.num || s.num > 0xffff || (s.num & (s.num - 1)))
+ return -EINVAL;
+ vq->num = s.num;
+
+ return 0;
+}
+
+static long vhost_vring_set_addr(struct vhost_dev *d,
+ struct vhost_virtqueue *vq,
+ void __user *argp)
+{
+ struct vhost_vring_addr a;
+
+ if (copy_from_user(&a, argp, sizeof a))
+ return -EFAULT;
+ if (a.flags & ~(0x1 << VHOST_VRING_F_LOG))
+ return -EOPNOTSUPP;
+
+ /* For 32bit, verify that the top 32bits of the user
+ data are set to zero. */
+ if ((u64)(unsigned long)a.desc_user_addr != a.desc_user_addr ||
+ (u64)(unsigned long)a.used_user_addr != a.used_user_addr ||
+ (u64)(unsigned long)a.avail_user_addr != a.avail_user_addr)
+ return -EFAULT;
+
+ /* Make sure it's safe to cast pointers to vring types. */
+ BUILD_BUG_ON(__alignof__ *vq->avail > VRING_AVAIL_ALIGN_SIZE);
+ BUILD_BUG_ON(__alignof__ *vq->used > VRING_USED_ALIGN_SIZE);
+ if ((a.avail_user_addr & (VRING_AVAIL_ALIGN_SIZE - 1)) ||
+ (a.used_user_addr & (VRING_USED_ALIGN_SIZE - 1)) ||
+ (a.log_guest_addr & (VRING_USED_ALIGN_SIZE - 1)))
+ return -EINVAL;
+
+ /* We only verify access here if backend is configured.
+ * If it is not, we don't as size might not have been setup.
+ * We will verify when backend is configured. */
+ if (vq->private_data) {
+ if (!vq_access_ok(vq, vq->num,
+ (void __user *)(unsigned long)a.desc_user_addr,
+ (void __user *)(unsigned long)a.avail_user_addr,
+ (void __user *)(unsigned long)a.used_user_addr))
+ return -EINVAL;
+
+ /* Also validate log access for used ring if enabled. */
+ if ((a.flags & (0x1 << VHOST_VRING_F_LOG)) &&
+ !log_access_ok(vq->log_base, a.log_guest_addr,
+ sizeof *vq->used +
+ vq->num * sizeof *vq->used->ring))
+ return -EINVAL;
+ }
+
+ vq->log_used = !!(a.flags & (0x1 << VHOST_VRING_F_LOG));
+ vq->desc = (void __user *)(unsigned long)a.desc_user_addr;
+ vq->avail = (void __user *)(unsigned long)a.avail_user_addr;
+ vq->log_addr = a.log_guest_addr;
+ vq->used = (void __user *)(unsigned long)a.used_user_addr;
+
+ return 0;
+}
+
+static long vhost_vring_set_num_addr(struct vhost_dev *d,
+ struct vhost_virtqueue *vq,
+ unsigned int ioctl,
+ void __user *argp)
+{
+ long r;
+
+ mutex_lock(&vq->mutex);
+
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ /* Unregister MMU notifer to allow invalidation callback
+ * can access vq->uaddrs[] without holding a lock.
+ */
+ if (d->mm)
+ mmu_notifier_unregister(&d->mmu_notifier, d->mm);
+
+ vhost_uninit_vq_maps(vq);
+#endif
+
+ switch (ioctl) {
+ case VHOST_SET_VRING_NUM:
+ r = vhost_vring_set_num(d, vq, argp);
+ break;
+ case VHOST_SET_VRING_ADDR:
+ r = vhost_vring_set_addr(d, vq, argp);
+ break;
+ default:
+ BUG();
+ }
+
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ vhost_setup_vq_uaddr(vq);
+
+ if (d->mm)
+ mmu_notifier_register(&d->mmu_notifier, d->mm);
+#endif
+
+ mutex_unlock(&vq->mutex);
+
+ return r;
+}
long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp)
{
struct file *eventfp, *filep = NULL;
struct vhost_virtqueue *vq;
struct vhost_vring_state s;
struct vhost_vring_file f;
- struct vhost_vring_addr a;
u32 idx;
long r;
idx = array_index_nospec(idx, d->nvqs);
vq = d->vqs[idx];
+ if (ioctl == VHOST_SET_VRING_NUM ||
+ ioctl == VHOST_SET_VRING_ADDR) {
+ return vhost_vring_set_num_addr(d, vq, ioctl, argp);
+ }
+
mutex_lock(&vq->mutex);
switch (ioctl) {
- case VHOST_SET_VRING_NUM:
- /* Resizing ring with an active backend?
- * You don't want to do that. */
- if (vq->private_data) {
- r = -EBUSY;
- break;
- }
- if (copy_from_user(&s, argp, sizeof s)) {
- r = -EFAULT;
- break;
- }
- if (!s.num || s.num > 0xffff || (s.num & (s.num - 1))) {
- r = -EINVAL;
- break;
- }
- vq->num = s.num;
- break;
case VHOST_SET_VRING_BASE:
/* Moving base with an active backend?
* You don't want to do that. */
if (copy_to_user(argp, &s, sizeof s))
r = -EFAULT;
break;
- case VHOST_SET_VRING_ADDR:
- if (copy_from_user(&a, argp, sizeof a)) {
- r = -EFAULT;
- break;
- }
- if (a.flags & ~(0x1 << VHOST_VRING_F_LOG)) {
- r = -EOPNOTSUPP;
- break;
- }
- /* For 32bit, verify that the top 32bits of the user
- data are set to zero. */
- if ((u64)(unsigned long)a.desc_user_addr != a.desc_user_addr ||
- (u64)(unsigned long)a.used_user_addr != a.used_user_addr ||
- (u64)(unsigned long)a.avail_user_addr != a.avail_user_addr) {
- r = -EFAULT;
- break;
- }
-
- /* Make sure it's safe to cast pointers to vring types. */
- BUILD_BUG_ON(__alignof__ *vq->avail > VRING_AVAIL_ALIGN_SIZE);
- BUILD_BUG_ON(__alignof__ *vq->used > VRING_USED_ALIGN_SIZE);
- if ((a.avail_user_addr & (VRING_AVAIL_ALIGN_SIZE - 1)) ||
- (a.used_user_addr & (VRING_USED_ALIGN_SIZE - 1)) ||
- (a.log_guest_addr & (VRING_USED_ALIGN_SIZE - 1))) {
- r = -EINVAL;
- break;
- }
-
- /* We only verify access here if backend is configured.
- * If it is not, we don't as size might not have been setup.
- * We will verify when backend is configured. */
- if (vq->private_data) {
- if (!vq_access_ok(vq, vq->num,
- (void __user *)(unsigned long)a.desc_user_addr,
- (void __user *)(unsigned long)a.avail_user_addr,
- (void __user *)(unsigned long)a.used_user_addr)) {
- r = -EINVAL;
- break;
- }
-
- /* Also validate log access for used ring if enabled. */
- if ((a.flags & (0x1 << VHOST_VRING_F_LOG)) &&
- !log_access_ok(vq->log_base, a.log_guest_addr,
- sizeof *vq->used +
- vq->num * sizeof *vq->used->ring)) {
- r = -EINVAL;
- break;
- }
- }
-
- vq->log_used = !!(a.flags & (0x1 << VHOST_VRING_F_LOG));
- vq->desc = (void __user *)(unsigned long)a.desc_user_addr;
- vq->avail = (void __user *)(unsigned long)a.avail_user_addr;
- vq->log_addr = a.log_guest_addr;
- vq->used = (void __user *)(unsigned long)a.used_user_addr;
- break;
case VHOST_SET_VRING_KICK:
if (copy_from_user(&f, argp, sizeof f)) {
r = -EFAULT;
static int vhost_update_used_flags(struct vhost_virtqueue *vq)
{
void __user *used;
- if (vhost_put_user(vq, cpu_to_vhost16(vq, vq->used_flags),
- &vq->used->flags) < 0)
+ if (vhost_put_used_flags(vq))
return -EFAULT;
if (unlikely(vq->log_used)) {
/* Make sure the flag is seen before log. */
static int vhost_update_avail_event(struct vhost_virtqueue *vq, u16 avail_event)
{
- if (vhost_put_user(vq, cpu_to_vhost16(vq, vq->avail_idx),
- vhost_avail_event(vq)))
+ if (vhost_put_avail_event(vq))
return -EFAULT;
if (unlikely(vq->log_used)) {
void __user *used;
r = -EFAULT;
goto err;
}
- r = vhost_get_used(vq, last_used_idx, &vq->used->idx);
+ r = vhost_get_used_idx(vq, &last_used_idx);
if (r) {
vq_err(vq, "Can't access used idx at %p\n",
&vq->used->idx);
last_avail_idx = vq->last_avail_idx;
if (vq->avail_idx == vq->last_avail_idx) {
- if (unlikely(vhost_get_avail(vq, avail_idx, &vq->avail->idx))) {
+ if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) {
vq_err(vq, "Failed to access avail idx at %p\n",
&vq->avail->idx);
return -EFAULT;
/* Grab the next descriptor number they're advertising, and increment
* the index we've seen. */
- if (unlikely(vhost_get_avail(vq, ring_head,
- &vq->avail->ring[last_avail_idx & (vq->num - 1)]))) {
+ if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) {
vq_err(vq, "Failed to read head: idx %d address %p\n",
last_avail_idx,
&vq->avail->ring[last_avail_idx % vq->num]);
i, vq->num, head);
return -EINVAL;
}
- ret = vhost_copy_from_user(vq, &desc, vq->desc + i,
- sizeof desc);
+ ret = vhost_get_desc(vq, &desc, i);
if (unlikely(ret)) {
vq_err(vq, "Failed to get descriptor: idx %d addr %p\n",
i, vq->desc + i);
start = vq->last_used_idx & (vq->num - 1);
used = vq->used->ring + start;
- if (count == 1) {
- if (vhost_put_user(vq, heads[0].id, &used->id)) {
- vq_err(vq, "Failed to write used id");
- return -EFAULT;
- }
- if (vhost_put_user(vq, heads[0].len, &used->len)) {
- vq_err(vq, "Failed to write used len");
- return -EFAULT;
- }
- } else if (vhost_copy_to_user(vq, used, heads, count * sizeof *used)) {
+ if (vhost_put_used(vq, heads, start, count)) {
vq_err(vq, "Failed to write used");
return -EFAULT;
}
/* Make sure buffer is written before we update index. */
smp_wmb();
- if (vhost_put_user(vq, cpu_to_vhost16(vq, vq->last_used_idx),
- &vq->used->idx)) {
+ if (vhost_put_used_idx(vq)) {
vq_err(vq, "Failed to increment used idx");
return -EFAULT;
}
if (!vhost_has_feature(vq, VIRTIO_RING_F_EVENT_IDX)) {
__virtio16 flags;
- if (vhost_get_avail(vq, flags, &vq->avail->flags)) {
+ if (vhost_get_avail_flags(vq, &flags)) {
vq_err(vq, "Failed to get flags");
return true;
}
if (unlikely(!v))
return true;
- if (vhost_get_avail(vq, event, vhost_used_event(vq))) {
+ if (vhost_get_used_event(vq, &event)) {
vq_err(vq, "Failed to get used event idx");
return true;
}
if (vq->avail_idx != vq->last_avail_idx)
return false;
- r = vhost_get_avail(vq, avail_idx, &vq->avail->idx);
+ r = vhost_get_avail_idx(vq, &avail_idx);
if (unlikely(r))
return false;
vq->avail_idx = vhost16_to_cpu(vq, avail_idx);
/* They could have slipped one in as we were doing that: make
* sure it's written, then check again. */
smp_mb();
- r = vhost_get_avail(vq, avail_idx, &vq->avail->idx);
+ r = vhost_get_avail_idx(vq, &avail_idx);
if (r) {
vq_err(vq, "Failed to check avail idx at %p: %d\n",
&vq->avail->idx, r);
#include <linux/virtio_config.h>
#include <linux/virtio_ring.h>
#include <linux/atomic.h>
+#include <linux/pagemap.h>
+#include <linux/mmu_notifier.h>
+#include <asm/cacheflush.h>
struct vhost_work;
typedef void (*vhost_work_fn_t)(struct vhost_work *work);
VHOST_NUM_ADDRS = 3,
};
+struct vhost_map {
+ int npages;
+ void *addr;
+ struct page **pages;
+};
+
+struct vhost_uaddr {
+ unsigned long uaddr;
+ size_t size;
+ bool write;
+};
+
+#if defined(CONFIG_MMU_NOTIFIER) && ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE == 0
+#define VHOST_ARCH_CAN_ACCEL_UACCESS 1
+#else
+#define VHOST_ARCH_CAN_ACCEL_UACCESS 0
+#endif
+
/* The virtqueue structure describes a queue attached to a device. */
struct vhost_virtqueue {
struct vhost_dev *dev;
struct vring_desc __user *desc;
struct vring_avail __user *avail;
struct vring_used __user *used;
+
+#if VHOST_ARCH_CAN_ACCEL_UACCESS
+ /* Read by memory accessors, modified by meta data
+ * prefetching, MMU notifier and vring ioctl().
+ * Synchonrized through mmu_lock (writers) and RCU (writers
+ * and readers).
+ */
+ struct vhost_map __rcu *maps[VHOST_NUM_ADDRS];
+ /* Read by MMU notifier, modified by vring ioctl(),
+ * synchronized through MMU notifier
+ * registering/unregistering.
+ */
+ struct vhost_uaddr uaddrs[VHOST_NUM_ADDRS];
+#endif
const struct vhost_umem_node *meta_iotlb[VHOST_NUM_ADDRS];
+
struct file *kick;
struct eventfd_ctx *call_ctx;
struct eventfd_ctx *error_ctx;
bool user_be;
#endif
u32 busyloop_timeout;
+ spinlock_t mmu_lock;
+ int invalidate_count;
};
struct vhost_msg_node {
struct vhost_dev {
struct mm_struct *mm;
+#ifdef CONFIG_MMU_NOTIFIER
+ struct mmu_notifier mmu_notifier;
+#endif
struct mutex mutex;
struct vhost_virtqueue **vqs;
int nvqs;
int vhost_log_write(struct vhost_virtqueue *vq, struct vhost_log *log,
unsigned int log_num, u64 len,
struct iovec *iov, int count);
-int vq_iotlb_prefetch(struct vhost_virtqueue *vq);
+int vq_meta_prefetch(struct vhost_virtqueue *vq);
struct vhost_msg_node *vhost_new_msg(struct vhost_virtqueue *vq, int type);
void vhost_enqueue_msg(struct vhost_dev *dev,
#include <linux/of_gpio.h>
#include <linux/platform_data/gpio_backlight.h>
#include <linux/platform_device.h>
+#include <linux/property.h>
#include <linux/slab.h>
struct gpio_backlight {
struct gpio_backlight *gbl)
{
struct device *dev = &pdev->dev;
- struct device_node *np = dev->of_node;
enum gpiod_flags flags;
int ret;
- gbl->def_value = of_property_read_bool(np, "default-on");
+ gbl->def_value = device_property_read_bool(dev, "default-on");
flags = gbl->def_value ? GPIOD_OUT_HIGH : GPIOD_OUT_LOW;
gbl->gpiod = devm_gpiod_get(dev, NULL, flags);
struct backlight_properties props;
struct backlight_device *bl;
struct gpio_backlight *gbl;
- struct device_node *np = pdev->dev.of_node;
int ret;
- if (!pdata && !np) {
- dev_err(&pdev->dev,
- "failed to find platform data or device tree node.\n");
- return -ENODEV;
- }
-
gbl = devm_kzalloc(&pdev->dev, sizeof(*gbl), GFP_KERNEL);
if (gbl == NULL)
return -ENOMEM;
gbl->dev = &pdev->dev;
- if (np) {
+ if (pdev->dev.fwnode) {
ret = gpio_backlight_probe_dt(pdev, gbl);
if (ret)
return ret;
- } else {
+ } else if (pdata) {
/*
* Legacy platform data GPIO retrieveal. Do not expand
* the use of this code path, currently only used by one
gbl->gpiod = gpio_to_desc(pdata->gpio);
if (!gbl->gpiod)
return -EINVAL;
+ } else {
+ dev_err(&pdev->dev,
+ "failed to find platform data or device tree node.\n");
+ return -ENODEV;
}
memset(&props, 0, sizeof(props));
return 0;
}
-#ifdef CONFIG_OF
static struct of_device_id gpio_backlight_of_match[] = {
{ .compatible = "gpio-backlight" },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, gpio_backlight_of_match);
-#endif
static struct platform_driver gpio_backlight_driver = {
.driver = {
.name = "gpio-backlight",
- .of_match_table = of_match_ptr(gpio_backlight_of_match),
+ .of_match_table = gpio_backlight_of_match,
},
.probe = gpio_backlight_probe,
};
// SPDX-License-Identifier: GPL-2.0-only
/*
- * linux/drivers/video/backlight/pwm_bl.c
- *
- * simple PWM based backlight control, board code has to setup
+ * Simple PWM based backlight control, board code has to setup
* 1) pin configuration so PWM waveforms can output
* 2) platform_data being correctly configured
*/
struct platform_pwm_backlight_data *data,
unsigned int period)
{
- unsigned int counter = 0;
- unsigned int i, n;
+ unsigned int i;
u64 retval;
/*
- * Count the number of bits needed to represent the period number. The
- * number of bits is used to calculate the number of levels used for the
- * brightness-levels table, the purpose of this calculation is have a
- * pre-computed table with enough levels to get linear brightness
- * perception. The period is divided by the number of bits so for a
- * 8-bit PWM we have 255 / 8 = 32 brightness levels or for a 16-bit PWM
- * we have 65535 / 16 = 4096 brightness levels.
- *
- * Note that this method is based on empirical testing on different
- * devices with PWM of 8 and 16 bits of resolution.
+ * Once we have 4096 levels there's little point going much higher...
+ * neither interactive sliders nor animation benefits from having
+ * more values in the table.
*/
- n = period;
- while (n) {
- counter += n % 2;
- n >>= 1;
- }
+ data->max_brightness =
+ min((int)DIV_ROUND_UP(period, fls(period)), 4096);
- data->max_brightness = DIV_ROUND_UP(period, counter);
data->levels = devm_kcalloc(dev, data->max_brightness,
sizeof(*data->levels), GFP_KERNEL);
if (!data->levels)
module_platform_driver(pwm_backlight_driver);
MODULE_DESCRIPTION("PWM based Backlight Driver");
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
MODULE_ALIAS("platform:pwm-backlight");
If unsure, say Y.
+config VIRTIO_PMEM
+ tristate "Support for virtio pmem driver"
+ depends on VIRTIO
+ depends on LIBNVDIMM
+ help
+ This driver provides access to virtio-pmem devices, storage devices
+ that are mapped into the physical address space - similar to NVDIMMs
+ - with a virtio-based flushing interface.
+
+ If unsure, say Y.
+
config VIRTIO_BALLOON
tristate "Virtio balloon driver"
depends on VIRTIO
#include <linux/mm.h>
#include <linux/mount.h>
#include <linux/magic.h>
+#include <linux/pseudo_fs.h>
/*
* Balloon device works in 4K page units. So each page is pointed to by
return MIGRATEPAGE_SUCCESS;
}
-static struct dentry *balloon_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int balloon_init_fs_context(struct fs_context *fc)
{
- static const struct dentry_operations ops = {
- .d_dname = simple_dname,
- };
-
- return mount_pseudo(fs_type, "balloon-kvm:", NULL, &ops,
- BALLOON_KVM_MAGIC);
+ return init_pseudo(fc, BALLOON_KVM_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type balloon_fs = {
.name = "balloon-kvm",
- .mount = balloon_mount,
+ .init_fs_context = balloon_init_fs_context,
.kill_sb = kill_anon_super,
};
struct irq_affinity *desc)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
- unsigned int irq = platform_get_irq(vm_dev->pdev, 0);
+ int irq = platform_get_irq(vm_dev->pdev, 0);
int i, err, queue_idx = 0;
+ if (irq < 0) {
+ dev_err(&vdev->dev, "Cannot get IRQ resource\n");
+ return irq;
+ }
+
err = request_irq(irq, vm_interrupt, IRQF_SHARED,
dev_name(&vdev->dev), vm_dev);
if (err)
default y
---help---
This allows to communicate with userspace using connector. For more
- information see <file:Documentation/connector/connector.txt>.
+ information see <file:Documentation/driver-api/connector.rst>.
There are three types of messages between w1 core and userspace:
1. Events. They are generated each time new master or slave device found
either due to automatic or requested search.
the watchdog on its own. Thus if your userspace does not start fast
enough your device will reboot.
+config WATCHDOG_OPEN_TIMEOUT
+ int "Timeout value for opening watchdog device"
+ default 0
+ help
+ The maximum time, in seconds, for which the watchdog framework takes
+ care of pinging a hardware watchdog. A value of 0 means infinite. The
+ value set here can be overridden by the commandline parameter
+ "watchdog.open_timeout".
+
config WATCHDOG_SYSFS
bool "Read different watchdog information through sysfs"
help
config IMX_SC_WDT
tristate "IMX SC Watchdog"
depends on HAVE_ARM_SMCCC
+ depends on IMX_SCU
select WATCHDOG_CORE
help
This is the driver for the system controller watchdog
+// SPDX-License-Identifier: GPL-2.0+
/*
* Acquire Single Board Computer Watchdog Timer driver
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* Advantech Single Board Computer WDT driver
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
if (status & WDT_TIMEOUT_STATUS_BOOT_SECONDARY)
wdt->wdd.bootstatus = WDIOF_CARDRESET;
- ret = devm_watchdog_register_device(dev, &wdt->wdd);
- if (ret) {
- dev_err(dev, "failed to register\n");
- return ret;
- }
-
- return 0;
+ return devm_watchdog_register_device(dev, &wdt->wdd);
}
static struct platform_driver aspeed_watchdog_driver = {
watchdog_stop_on_reboot(&bcm2835_wdt_wdd);
err = devm_watchdog_register_device(dev, &bcm2835_wdt_wdd);
- if (err) {
- dev_err(dev, "Failed to register watchdog device");
+ if (err)
return err;
- }
if (pm_power_off == NULL) {
pm_power_off = bcm2835_power_off;
MODULE_PARM_DESC(nowayout, "Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+MODULE_ALIAS("platform:bcm2835-wdt");
MODULE_AUTHOR("Lubomir Rintel <lkundrak@v3.sk>");
MODULE_DESCRIPTION("Driver for Broadcom BCM2835 watchdog timer");
MODULE_LICENSE("GPL");
watchdog_stop_on_reboot(&wdt->wdd);
watchdog_stop_on_unregister(&wdt->wdd);
err = devm_watchdog_register_device(dev, &wdt->wdd);
- if (err) {
- dev_err(dev, "Failed to register watchdog device\n");
+ if (err)
return err;
- }
dev_info(dev, "Registered BCM7038 Watchdog\n");
watchdog_stop_on_reboot(&bcm_kona_wdt_wdd);
watchdog_stop_on_unregister(&bcm_kona_wdt_wdd);
ret = devm_watchdog_register_device(dev, &bcm_kona_wdt_wdd);
- if (ret) {
- dev_err(dev, "Failed to register watchdog device");
+ if (ret)
return ret;
- }
bcm_kona_wdt_debug_init(pdev);
dev_dbg(dev, "Broadcom Kona Watchdog Timer");
watchdog_stop_on_reboot(cdns_wdt_device);
watchdog_stop_on_unregister(cdns_wdt_device);
ret = devm_watchdog_register_device(dev, cdns_wdt_device);
- if (ret) {
- dev_err(dev, "Failed to register wdt device\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
dev_info(dev, "Xilinx Watchdog Timer at %p with timeout %ds%s\n",
return ret;
}
- ret = devm_watchdog_register_device(dev, &driver_data->wdt);
- if (ret != 0) {
- dev_err(da9052->dev, "watchdog_register_device() failed: %d\n",
- ret);
- return ret;
- }
-
- return ret;
+ return devm_watchdog_register_device(dev, &driver_data->wdt);
}
static struct platform_driver da9052_wdt_driver = {
watchdog_set_drvdata(&wdt->wdtdev, wdt);
ret = devm_watchdog_register_device(dev, &wdt->wdtdev);
- if (ret < 0) {
- dev_err(wdt->hw->dev,
- "watchdog registration failed (%d)\n", ret);
+ if (ret < 0)
return ret;
- }
return da9062_wdt_ping(&wdt->wdtdev);
}
+// SPDX-License-Identifier: GPL-2.0
/*
* drivers/char/watchdog/davinci_wdt.c
*
*
* Copyright (C) 2006-2013 Texas Instruments.
*
- * 2007 (c) MontaVista Software, Inc. This file is licensed under
- * the terms of the GNU General Public License version 2. This program
- * is licensed "as is" without any warranty of any kind, whether express
- * or implied.
+ * 2007 (c) MontaVista Software, Inc.
*/
#include <linux/module.h>
if (IS_ERR(davinci_wdt->base))
return PTR_ERR(davinci_wdt->base);
- ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "cannot register watchdog device\n");
- return ret;
- }
-
- return 0;
+ return devm_watchdog_register_device(dev, wdd);
}
static const struct of_device_id davinci_wdt_of_match[] = {
{
struct device *dev = &pdev->dev;
struct dc_wdt *wdt;
- int ret;
wdt = devm_kzalloc(dev, sizeof(struct dc_wdt), GFP_KERNEL);
if (!wdt)
watchdog_set_restart_priority(&dc_wdt_wdd, 128);
watchdog_init_timeout(&dc_wdt_wdd, timeout, dev);
watchdog_stop_on_reboot(&dc_wdt_wdd);
- ret = devm_watchdog_register_device(dev, &dc_wdt_wdd);
- if (ret) {
- dev_err(dev, "Failed to register watchdog device");
- return ret;
- }
-
- return 0;
+ return devm_watchdog_register_device(dev, &dc_wdt_wdd);
}
static const struct of_device_id dc_wdt_of_match[] = {
/*
* Watchdog timer driver for the WinSystems EBC-C384
* Copyright (C) 2016 William Breathitt Gray
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License, version 2, as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
*/
#include <linux/device.h>
#include <linux/dmi.h>
+// SPDX-License-Identifier: GPL-2.0+
/*
* Eurotech CPU-1220/1410/1420 on board WDT driver
*
* (c) Copyright 1996-1997 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
}
ret = devm_watchdog_register_device(dev, &gwdt->wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog\n");
+ if (ret)
return ret;
- }
/* Set up platform driver data */
platform_set_drvdata(pdev, gwdt);
#include <linux/platform_device.h>
#include <linux/watchdog.h>
+static bool nowayout = WATCHDOG_NOWAYOUT;
+module_param(nowayout, bool, 0);
+MODULE_PARM_DESC(nowayout,
+ "Watchdog cannot be stopped once started (default="
+ __MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+
#define SOFT_TIMEOUT_MIN 1
#define SOFT_TIMEOUT_DEF 60
priv->wdd.timeout = SOFT_TIMEOUT_DEF;
watchdog_init_timeout(&priv->wdd, 0, dev);
+ watchdog_set_nowayout(&priv->wdd, nowayout);
watchdog_stop_on_reboot(&priv->wdd);
#include <linux/watchdog.h>
#include <asm/nmi.h>
-#define HPWDT_VERSION "2.0.2"
+#define HPWDT_VERSION "2.0.3"
#define SECS_TO_TICKS(secs) ((secs) * 1000 / 128)
#define TICKS_TO_SECS(ticks) ((ticks) * 128 / 1000)
-#define HPWDT_MAX_TIMER TICKS_TO_SECS(65535)
+#define HPWDT_MAX_TICKS 65535
+#define HPWDT_MAX_TIMER TICKS_TO_SECS(HPWDT_MAX_TICKS)
#define DEFAULT_MARGIN 30
#define PRETIMEOUT_SEC 9
static unsigned int soft_margin = DEFAULT_MARGIN; /* in seconds */
static bool nowayout = WATCHDOG_NOWAYOUT;
static bool pretimeout = IS_ENABLED(CONFIG_HPWDT_NMI_DECODING);
+static int kdumptimeout = -1;
static void __iomem *pci_mem_addr; /* the PCI-memory address */
static unsigned long __iomem *hpwdt_nmistat;
{0}, /* terminate list */
};
+static struct watchdog_device hpwdt_dev;
/*
* Watchdog operations
*/
+static int hpwdt_hw_is_running(void)
+{
+ return ioread8(hpwdt_timer_con) & 0x01;
+}
+
static int hpwdt_start(struct watchdog_device *wdd)
{
int control = 0x81 | (pretimeout ? 0x4 : 0);
- int reload = SECS_TO_TICKS(wdd->timeout);
+ int reload = SECS_TO_TICKS(min(wdd->timeout, wdd->max_hw_heartbeat_ms/1000));
- dev_dbg(wdd->parent, "start watchdog 0x%08x:0x%02x\n", reload, control);
+ dev_dbg(wdd->parent, "start watchdog 0x%08x:0x%08x:0x%02x\n", wdd->timeout, reload, control);
iowrite16(reload, hpwdt_timer_reg);
iowrite8(control, hpwdt_timer_con);
return 0;
}
+static void hpwdt_ping_ticks(int val)
+{
+ val = min(val, HPWDT_MAX_TICKS);
+ iowrite16(val, hpwdt_timer_reg);
+}
+
static int hpwdt_ping(struct watchdog_device *wdd)
{
- int reload = SECS_TO_TICKS(wdd->timeout);
+ int reload = SECS_TO_TICKS(min(wdd->timeout, wdd->max_hw_heartbeat_ms/1000));
- dev_dbg(wdd->parent, "ping watchdog 0x%08x\n", reload);
- iowrite16(reload, hpwdt_timer_reg);
+ dev_dbg(wdd->parent, "ping watchdog 0x%08x:0x%08x\n", wdd->timeout, reload);
+ hpwdt_ping_ticks(reload);
return 0;
}
if (ilo5 && !pretimeout && !mynmi)
return NMI_DONE;
- hpwdt_stop();
+ if (kdumptimeout < 0)
+ hpwdt_stop();
+ else if (kdumptimeout == 0)
+ ;
+ else {
+ unsigned int val = max((unsigned int)kdumptimeout, hpwdt_dev.timeout);
+ hpwdt_ping_ticks(SECS_TO_TICKS(val));
+ }
hex_byte_pack(panic_msg, mynmi);
nmi_panic(regs, panic_msg);
.info = &ident,
.ops = &hpwdt_ops,
.min_timeout = 1,
- .max_timeout = HPWDT_MAX_TIMER,
.timeout = DEFAULT_MARGIN,
.pretimeout = PRETIMEOUT_SEC,
+ .max_hw_heartbeat_ms = HPWDT_MAX_TIMER * 1000,
};
hpwdt_timer_reg = pci_mem_addr + 0x70;
hpwdt_timer_con = pci_mem_addr + 0x72;
- /* Make sure that timer is disabled until /dev/watchdog is opened */
- hpwdt_stop();
+ /* Have the core update running timer until user space is ready */
+ if (hpwdt_hw_is_running()) {
+ dev_info(&dev->dev, "timer is running\n");
+ set_bit(WDOG_HW_RUNNING, &hpwdt_dev.status);
+ }
/* Initialize NMI Decoding functionality */
retval = hpwdt_init_nmi_decoding(dev);
if (retval != 0)
goto error_init_nmi_decoding;
+ watchdog_stop_on_unregister(&hpwdt_dev);
watchdog_set_nowayout(&hpwdt_dev, nowayout);
watchdog_init_timeout(&hpwdt_dev, soft_margin, NULL);
pretimeout = 0;
}
hpwdt_dev.pretimeout = pretimeout ? PRETIMEOUT_SEC : 0;
+ kdumptimeout = min(kdumptimeout, HPWDT_MAX_TIMER);
hpwdt_dev.parent = &dev->dev;
retval = watchdog_register_device(&hpwdt_dev);
- if (retval < 0) {
- dev_err(&dev->dev, "watchdog register failed: %d.\n", retval);
+ if (retval < 0)
goto error_wd_register;
- }
dev_info(&dev->dev, "HPE Watchdog Timer Driver: Version: %s\n",
HPWDT_VERSION);
hpwdt_dev.timeout, nowayout);
dev_info(&dev->dev, "pretimeout: %s.\n",
pretimeout ? "on" : "off");
+ dev_info(&dev->dev, "kdumptimeout: %d.\n", kdumptimeout);
if (dev->subsystem_vendor == PCI_VENDOR_ID_HP_3PAR)
ilo5 = true;
static void hpwdt_exit(struct pci_dev *dev)
{
- if (!nowayout)
- hpwdt_stop();
-
watchdog_unregister_device(&hpwdt_dev);
hpwdt_exit_nmi_decoding();
pci_iounmap(dev, pci_mem_addr);
MODULE_PARM_DESC(nowayout, "Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+module_param(kdumptimeout, int, 0444);
+MODULE_PARM_DESC(kdumptimeout, "Timeout applied for crash kernel transition in seconds");
+
#ifdef CONFIG_HPWDT_NMI_DECODING
module_param(pretimeout, bool, 0);
MODULE_PARM_DESC(pretimeout, "Watchdog pretimeout enabled");
/* Register the watchdog so that userspace has access to it */
ret = watchdog_register_device(&edev->wdd);
- if (ret != 0) {
- dev_err(&pdev->dev,
- "cannot register watchdog device (err=%d)\n", ret);
+ if (ret != 0)
goto err_unmap;
- }
dev_info(&pdev->dev,
"initialized. heartbeat=%d sec (nowayout=%d)\n",
edev->wdd.timeout, nowayout);
+// SPDX-License-Identifier: GPL-2.0+
/*
* intel TCO vendor specific watchdog driver support
*
* (c) Copyright 2006-2009 Wim Van Sebroeck <wim@iguana.be>.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Wim Van Sebroeck nor Iguana vzw. admit liability nor
* provide warranty for any of this software. This material is
* provided "AS-IS" and at no charge.
MODULE_DESCRIPTION("Intel TCO Vendor Specific WatchDog Timer Driver Support");
MODULE_VERSION(DRV_VERSION);
MODULE_LICENSE("GPL");
-
+// SPDX-License-Identifier: GPL-2.0+
/*
* intel TCO Watchdog Driver
*
* (c) Copyright 2006-2011 Wim Van Sebroeck <wim@iguana.be>.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Wim Van Sebroeck nor Iguana vzw. admit liability nor
* provide warranty for any of this software. This material is
* provided "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* IB700 Single Board Computer WDT driver
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
static struct {
unsigned short sch_wdtba;
- struct spinlock unlock_sequence;
+ spinlock_t unlock_sequence;
#ifdef CONFIG_DEBUG_FS
struct dentry *debugfs;
#endif
ie6xx_wdt_debugfs_init();
ret = watchdog_register_device(&ie6xx_wdt_dev);
- if (ret) {
- dev_err(&pdev->dev,
- "Watchdog timer: cannot register device (err =%d)\n",
- ret);
+ if (ret)
goto misc_register_error;
- }
return 0;
regmap_write(wdev->regmap, IMX2_WDT_WMCR, 0);
ret = watchdog_register_device(wdog);
- if (ret) {
- dev_err(&pdev->dev, "cannot register watchdog device\n");
+ if (ret)
goto disable_clk;
- }
dev_info(&pdev->dev, "timeout %d sec (nowayout=%d)\n",
wdog->timeout, nowayout);
*/
#include <linux/arm-smccc.h>
+#include <linux/firmware/imx/sci.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/kernel.h>
#define SC_TIMER_WDOG_ACTION_PARTITION 0
+#define SC_IRQ_WDOG 1
+#define SC_IRQ_GROUP_WDOG 1
+
static bool nowayout = WATCHDOG_NOWAYOUT;
module_param(nowayout, bool, 0000);
MODULE_PARM_DESC(nowayout, "Watchdog cannot be stopped once started (default="
__MODULE_STRING(WATCHDOG_NOWAYOUT) ")");
+struct imx_sc_wdt_device {
+ struct watchdog_device wdd;
+ struct notifier_block wdt_notifier;
+};
+
static int imx_sc_wdt_ping(struct watchdog_device *wdog)
{
struct arm_smccc_res res;
return res.a0 ? -EACCES : 0;
}
+static int imx_sc_wdt_set_pretimeout(struct watchdog_device *wdog,
+ unsigned int pretimeout)
+{
+ struct arm_smccc_res res;
+
+ arm_smccc_smc(IMX_SIP_TIMER, IMX_SIP_TIMER_SET_PRETIME_WDOG,
+ pretimeout * 1000, 0, 0, 0, 0, 0, &res);
+ if (res.a0)
+ return -EACCES;
+
+ wdog->pretimeout = pretimeout;
+
+ return 0;
+}
+
+static int imx_sc_wdt_notify(struct notifier_block *nb,
+ unsigned long event, void *group)
+{
+ struct imx_sc_wdt_device *imx_sc_wdd =
+ container_of(nb,
+ struct imx_sc_wdt_device,
+ wdt_notifier);
+
+ if (event & SC_IRQ_WDOG &&
+ *(u8 *)group == SC_IRQ_GROUP_WDOG)
+ watchdog_notify_pretimeout(&imx_sc_wdd->wdd);
+
+ return 0;
+}
+
+static void imx_sc_wdt_action(void *data)
+{
+ struct notifier_block *wdt_notifier = data;
+
+ imx_scu_irq_unregister_notifier(wdt_notifier);
+ imx_scu_irq_group_enable(SC_IRQ_GROUP_WDOG,
+ SC_IRQ_WDOG,
+ false);
+}
+
static const struct watchdog_ops imx_sc_wdt_ops = {
.owner = THIS_MODULE,
.start = imx_sc_wdt_start,
.stop = imx_sc_wdt_stop,
.ping = imx_sc_wdt_ping,
.set_timeout = imx_sc_wdt_set_timeout,
+ .set_pretimeout = imx_sc_wdt_set_pretimeout,
};
-static const struct watchdog_info imx_sc_wdt_info = {
+static struct watchdog_info imx_sc_wdt_info = {
.identity = "i.MX SC watchdog timer",
.options = WDIOF_SETTIMEOUT | WDIOF_KEEPALIVEPING |
- WDIOF_MAGICCLOSE | WDIOF_PRETIMEOUT,
+ WDIOF_MAGICCLOSE,
};
static int imx_sc_wdt_probe(struct platform_device *pdev)
{
+ struct imx_sc_wdt_device *imx_sc_wdd;
+ struct watchdog_device *wdog;
struct device *dev = &pdev->dev;
- struct watchdog_device *imx_sc_wdd;
int ret;
imx_sc_wdd = devm_kzalloc(dev, sizeof(*imx_sc_wdd), GFP_KERNEL);
platform_set_drvdata(pdev, imx_sc_wdd);
- imx_sc_wdd->info = &imx_sc_wdt_info;
- imx_sc_wdd->ops = &imx_sc_wdt_ops;
- imx_sc_wdd->min_timeout = 1;
- imx_sc_wdd->max_timeout = MAX_TIMEOUT;
- imx_sc_wdd->parent = dev;
- imx_sc_wdd->timeout = DEFAULT_TIMEOUT;
-
- watchdog_init_timeout(imx_sc_wdd, 0, dev);
- watchdog_stop_on_reboot(imx_sc_wdd);
- watchdog_stop_on_unregister(imx_sc_wdd);
+ wdog = &imx_sc_wdd->wdd;
+ wdog->info = &imx_sc_wdt_info;
+ wdog->ops = &imx_sc_wdt_ops;
+ wdog->min_timeout = 1;
+ wdog->max_timeout = MAX_TIMEOUT;
+ wdog->parent = dev;
+ wdog->timeout = DEFAULT_TIMEOUT;
+
+ watchdog_init_timeout(wdog, 0, dev);
+ watchdog_stop_on_reboot(wdog);
+ watchdog_stop_on_unregister(wdog);
+
+ ret = devm_watchdog_register_device(dev, wdog);
+
+ if (ret) {
+ dev_err(dev, "Failed to register watchdog device\n");
+ return ret;
+ }
+
+ ret = imx_scu_irq_group_enable(SC_IRQ_GROUP_WDOG,
+ SC_IRQ_WDOG,
+ true);
+ if (ret) {
+ dev_warn(dev, "Enable irq failed, pretimeout NOT supported\n");
+ return 0;
+ }
- ret = devm_watchdog_register_device(dev, imx_sc_wdd);
+ imx_sc_wdd->wdt_notifier.notifier_call = imx_sc_wdt_notify;
+ ret = imx_scu_irq_register_notifier(&imx_sc_wdd->wdt_notifier);
if (ret) {
- dev_err(dev, "Failed to register watchdog device\n");
- return ret;
+ imx_scu_irq_group_enable(SC_IRQ_GROUP_WDOG,
+ SC_IRQ_WDOG,
+ false);
+ dev_warn(dev,
+ "Register irq notifier failed, pretimeout NOT supported\n");
+ return 0;
}
+ ret = devm_add_action_or_reset(dev, imx_sc_wdt_action,
+ &imx_sc_wdd->wdt_notifier);
+ if (!ret)
+ imx_sc_wdt_info.options |= WDIOF_PRETIMEOUT;
+ else
+ dev_warn(dev, "Add action failed, pretimeout NOT supported\n");
+
return 0;
}
static int __maybe_unused imx_sc_wdt_suspend(struct device *dev)
{
- struct watchdog_device *imx_sc_wdd = dev_get_drvdata(dev);
+ struct imx_sc_wdt_device *imx_sc_wdd = dev_get_drvdata(dev);
- if (watchdog_active(imx_sc_wdd))
- imx_sc_wdt_stop(imx_sc_wdd);
+ if (watchdog_active(&imx_sc_wdd->wdd))
+ imx_sc_wdt_stop(&imx_sc_wdd->wdd);
return 0;
}
static int __maybe_unused imx_sc_wdt_resume(struct device *dev)
{
- struct watchdog_device *imx_sc_wdd = dev_get_drvdata(dev);
+ struct imx_sc_wdt_device *imx_sc_wdd = dev_get_drvdata(dev);
- if (watchdog_active(imx_sc_wdd))
- imx_sc_wdt_start(imx_sc_wdd);
+ if (watchdog_active(&imx_sc_wdd->wdd))
+ imx_sc_wdt_start(&imx_sc_wdd->wdd);
return 0;
}
set_bit(WDOG_HW_RUNNING, &wdt_dev->status);
ret = devm_watchdog_register_device(dev, wdt_dev);
- if (ret) {
- dev_err(dev, "error registering watchdog device\n");
+ if (ret)
return ret;
- }
dev_info(dev, "Intel MID watchdog device probed\n");
* JZ4740 Watchdog driver
*/
+#include <linux/mfd/ingenic-tcu.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <asm/mach-jz4740/timer.h>
-#define JZ_REG_WDT_TIMER_DATA 0x0
-#define JZ_REG_WDT_COUNTER_ENABLE 0x4
-#define JZ_REG_WDT_TIMER_COUNTER 0x8
-#define JZ_REG_WDT_TIMER_CONTROL 0xC
-
#define JZ_WDT_CLOCK_PCLK 0x1
#define JZ_WDT_CLOCK_RTC 0x2
#define JZ_WDT_CLOCK_EXT 0x4
-#define JZ_WDT_CLOCK_DIV_SHIFT 3
-
-#define JZ_WDT_CLOCK_DIV_1 (0 << JZ_WDT_CLOCK_DIV_SHIFT)
-#define JZ_WDT_CLOCK_DIV_4 (1 << JZ_WDT_CLOCK_DIV_SHIFT)
-#define JZ_WDT_CLOCK_DIV_16 (2 << JZ_WDT_CLOCK_DIV_SHIFT)
-#define JZ_WDT_CLOCK_DIV_64 (3 << JZ_WDT_CLOCK_DIV_SHIFT)
-#define JZ_WDT_CLOCK_DIV_256 (4 << JZ_WDT_CLOCK_DIV_SHIFT)
-#define JZ_WDT_CLOCK_DIV_1024 (5 << JZ_WDT_CLOCK_DIV_SHIFT)
+#define JZ_WDT_CLOCK_DIV_1 (0 << TCU_TCSR_PRESCALE_LSB)
+#define JZ_WDT_CLOCK_DIV_4 (1 << TCU_TCSR_PRESCALE_LSB)
+#define JZ_WDT_CLOCK_DIV_16 (2 << TCU_TCSR_PRESCALE_LSB)
+#define JZ_WDT_CLOCK_DIV_64 (3 << TCU_TCSR_PRESCALE_LSB)
+#define JZ_WDT_CLOCK_DIV_256 (4 << TCU_TCSR_PRESCALE_LSB)
+#define JZ_WDT_CLOCK_DIV_1024 (5 << TCU_TCSR_PRESCALE_LSB)
#define DEFAULT_HEARTBEAT 5
#define MAX_HEARTBEAT 2048
{
struct jz4740_wdt_drvdata *drvdata = watchdog_get_drvdata(wdt_dev);
- writew(0x0, drvdata->base + JZ_REG_WDT_TIMER_COUNTER);
+ writew(0x0, drvdata->base + TCU_REG_WDT_TCNT);
return 0;
}
unsigned int rtc_clk_rate;
unsigned int timeout_value;
unsigned short clock_div = JZ_WDT_CLOCK_DIV_1;
+ u8 tcer;
rtc_clk_rate = clk_get_rate(drvdata->rtc_clk);
break;
}
timeout_value >>= 2;
- clock_div += (1 << JZ_WDT_CLOCK_DIV_SHIFT);
+ clock_div += (1 << TCU_TCSR_PRESCALE_LSB);
}
- writeb(0x0, drvdata->base + JZ_REG_WDT_COUNTER_ENABLE);
- writew(clock_div, drvdata->base + JZ_REG_WDT_TIMER_CONTROL);
+ tcer = readb(drvdata->base + TCU_REG_WDT_TCER);
+ writeb(0x0, drvdata->base + TCU_REG_WDT_TCER);
+ writew(clock_div, drvdata->base + TCU_REG_WDT_TCSR);
- writew((u16)timeout_value, drvdata->base + JZ_REG_WDT_TIMER_DATA);
- writew(0x0, drvdata->base + JZ_REG_WDT_TIMER_COUNTER);
- writew(clock_div | JZ_WDT_CLOCK_RTC,
- drvdata->base + JZ_REG_WDT_TIMER_CONTROL);
+ writew((u16)timeout_value, drvdata->base + TCU_REG_WDT_TDR);
+ writew(0x0, drvdata->base + TCU_REG_WDT_TCNT);
+ writew(clock_div | JZ_WDT_CLOCK_RTC, drvdata->base + TCU_REG_WDT_TCSR);
- writeb(0x1, drvdata->base + JZ_REG_WDT_COUNTER_ENABLE);
+ if (tcer & TCU_WDT_TCER_TCEN)
+ writeb(TCU_WDT_TCER_TCEN, drvdata->base + TCU_REG_WDT_TCER);
wdt_dev->timeout = new_timeout;
return 0;
static int jz4740_wdt_start(struct watchdog_device *wdt_dev)
{
+ struct jz4740_wdt_drvdata *drvdata = watchdog_get_drvdata(wdt_dev);
+ u8 tcer;
+
+ tcer = readb(drvdata->base + TCU_REG_WDT_TCER);
+
jz4740_timer_enable_watchdog();
jz4740_wdt_set_timeout(wdt_dev, wdt_dev->timeout);
+ /* Start watchdog if it wasn't started already */
+ if (!(tcer & TCU_WDT_TCER_TCEN))
+ writeb(TCU_WDT_TCER_TCEN, drvdata->base + TCU_REG_WDT_TCER);
+
return 0;
}
{
struct jz4740_wdt_drvdata *drvdata = watchdog_get_drvdata(wdt_dev);
- writeb(0x0, drvdata->base + JZ_REG_WDT_COUNTER_ENABLE);
+ writeb(0x0, drvdata->base + TCU_REG_WDT_TCER);
jz4740_timer_disable_watchdog();
return 0;
return PTR_ERR(drvdata->rtc_clk);
}
- ret = devm_watchdog_register_device(dev, &drvdata->wdt);
- if (ret < 0)
- return ret;
-
- return 0;
+ return devm_watchdog_register_device(dev, &drvdata->wdt);
}
static struct platform_driver jz4740_wdt_driver = {
watchdog_set_drvdata(ls1x_wdt, drvdata);
err = devm_watchdog_register_device(dev, &drvdata->wdt);
- if (err) {
- dev_err(dev, "failed to register watchdog device\n");
+ if (err)
return err;
- }
platform_set_drvdata(pdev, drvdata);
watchdog_set_drvdata(wdt_dev, wdt);
watchdog_stop_on_unregister(wdt_dev);
- ret = devm_watchdog_register_device(dev, wdt_dev);
- if (ret < 0) {
- dev_err(dev, "watchdog registration failed: %d\n", ret);
- return ret;
- }
-
- return 0;
+ return devm_watchdog_register_device(dev, wdt_dev);
}
static const struct platform_device_id max77620_wdt_devtype[] = {
watchdog_stop_on_reboot(&wdt->wdd);
ret = watchdog_register_device(&wdt->wdd);
- if (ret) {
- dev_err(dev, "unable to register watchdog device = %d.\n", ret);
+ if (ret)
watchdog_set_drvdata(&wdt->wdd, NULL);
- }
wdt->state = MEI_WDT_IDLE;
dev_set_drvdata(dev, drv);
ret = devm_watchdog_register_device(dev, &a21_wdt);
- if (ret) {
- dev_err(dev, "Cannot register watchdog device\n");
+ if (ret)
return ret;
- }
dev_info(dev, "MEN A21 watchdog timer driver enabled\n");
}
ret = devm_watchdog_register_device(dev, &drv_data->wdt);
- if (ret) {
- dev_err(dev, "failed to register Watchdog device\n");
+ if (ret)
return ret;
- }
dev_info(dev, "MEN 14F021P00 BMC Watchdog device enabled\n");
ddata->wdd.timeout = ddata->wdd.min_timeout;
ret = devm_watchdog_register_device(dev, &ddata->wdd);
- if (ret) {
- dev_err(dev, "cannot register watchdog device (err=%d)\n",
- ret);
+ if (ret)
return ret;
- }
dev_info(dev,
"WDT driver for MPC8xxx initialized. mode:%s timeout=%d sec\n",
+// SPDX-License-Identifier: GPL-2.0
/*
* mv64x60_wdt.c - MV64X60 (Marvell Discovery) watchdog userspace interface
*
*
* Derived from mpc8xx_wdt.c, with the following copyright.
*
- * 2002 (c) Florian Schirmer <jolt@tuxbox.org> This file is licensed under
- * the terms of the GNU General Public License version 2. This program
- * is licensed "as is" without any warranty of any kind, whether express
- * or implied.
+ * 2002 (c) Florian Schirmer <jolt@tuxbox.org>
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
watchdog_init_timeout(wdd, timeout, dev);
ret = watchdog_register_device(wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog\n");
+ if (ret)
return ret;
- }
/* Switch from boot mode to user mode */
outb(NIWD_CONTROL_RESET | NIWD_CONTROL_MODE,
ret = watchdog_register_device(wdd);
if (ret) {
outb(LOCK, wdt->io_base + WDT_REG_LOCK);
- dev_err(dev, "failed to register watchdog\n");
return ret;
}
return ret;
ret = devm_watchdog_register_device(dev, &wdt->wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog\n");
+ if (ret)
return ret;
- }
dev_info(dev, "NPCM watchdog driver enabled\n");
+/* SPDX-License-Identifier: GPL-2.0+ */
/*
* nv_tco: TCO timer driver for nVidia chipsets.
*
* Reserved.
* http://www.kernelconcepts.de
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither kernel concepts nor Nils Faerber admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* Octeon Watchdog driver
*
* (c) Copyright 1996-1997 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
*
* (c) Copyright 1995 Alan Cox <alan@lxorguk.ukuu.org.uk>
*
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- *
- *
* The OCTEON watchdog has a maximum timeout of 2^32 * io_clock.
* For most systems this is less than 10 seconds, so to allow for
* software to request longer watchdog heartbeats, we maintain software
}
rc = devm_watchdog_register_device(dev, xilinx_wdt_wdd);
- if (rc) {
- dev_err(dev, "Cannot register watchdog (err=%d)\n", rc);
+ if (rc)
return rc;
- }
clk_disable(xdev->clk);
+// SPDX-License-Identifier: GPL-2.0
/*
* omap_wdt.c
*
* Author: MontaVista Software, Inc.
* <gdavis@mvista.com> or <source@mvista.com>
*
- * 2003 (c) MontaVista Software, Inc. This file is licensed under the
- * terms of the GNU General Public License version 2. This program is
- * licensed "as is" without any warranty of any kind, whether express
- * or implied.
+ * 2003 (c) MontaVista Software, Inc.
*
* History:
*
+/* SPDX-License-Identifier: GPL-2.0+ */
/*
* linux/drivers/char/watchdog/omap_wdt.h
*
* OMAP Watchdog timer register definitions
*
* Copyright (C) 2004 Texas Instruments.
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- *
- * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
- * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
- * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
- * NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
- * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
- * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
- * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
- * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef _OMAP_WATCHDOG_H
+// SPDX-License-Identifier: GPL-2.0+
/*
* NS pc87413-wdt Watchdog Timer driver for Linux 2.6.x.x
*
* (C) Copyright 2006 Sven Anders, <anders@anduras.de>
* and Marcus Junker, <junker@anduras.de>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Sven Anders, Marcus Junker nor ANDURAS AG
* admit liability nor provide warranty for any of this software.
* This material is provided "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* Berkshire PCI-PC Watchdog Card Driver
*
* Matt Domsch <Matt_Domsch@dell.com>,
* Rob Radez <rob@osinvestor.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Wim Van Sebroeck nor Iguana vzw. admit liability nor
* provide warranty for any of this software. This material is
* provided "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* Berkshire USB-PC Watchdog Card Driver
*
* Rob Radez <rob@osinvestor.com>,
* Greg Kroah-Hartman <greg@kroah.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Wim Van Sebroeck nor Iguana vzw. admit liability nor
* provide warranty for any of this software. This material is
* provided "AS-IS" and at no charge.
watchdog_set_drvdata(wdd, dmt);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "watchdog register failed, err %d\n", ret);
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdd);
return 0;
watchdog_set_drvdata(wdd, wdt);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "watchdog register failed, err %d\n", ret);
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdd);
+// SPDX-License-Identifier: GPL-2.0
/*
* drivers/char/watchdog/pnx4008_wdt.c
*
* 2005-2006 (c) MontaVista Software, Inc.
*
* (C) 2012 Wolfram Sang, Pengutronix
- *
- * This file is licensed under the terms of the GNU General Public License
- * version 2. This program is licensed "as is" without any warranty of any
- * kind, whether express or implied.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
set_bit(WDOG_HW_RUNNING, &pnx4008_wdd.status);
ret = devm_watchdog_register_device(dev, &pnx4008_wdd);
- if (ret < 0) {
- dev_err(dev, "cannot register watchdog device\n");
+ if (ret < 0)
return ret;
- }
dev_info(dev, "heartbeat %d sec\n", pnx4008_wdd.timeout);
watchdog_init_timeout(&wdt->wdd, 0, dev);
ret = devm_watchdog_register_device(dev, &wdt->wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
return 0;
ret = devm_watchdog_register_device(dev, wdd);
if (ret) {
- dev_err(dev, "Failed to register watchdog device\n");
rave_sp_wdt_stop(wdd);
return ret;
}
*/
#include <linux/bitops.h>
#include <linux/clk.h>
+#include <linux/delay.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
return 0;
}
+static void rwdt_wait_cycles(struct rwdt_priv *priv, unsigned int cycles)
+{
+ unsigned int delay;
+
+ delay = DIV_ROUND_UP(cycles * 1000000, priv->clk_rate);
+
+ usleep_range(delay, 2 * delay);
+}
+
static int rwdt_start(struct watchdog_device *wdev)
{
struct rwdt_priv *priv = watchdog_get_drvdata(wdev);
/* Stop the timer before we modify any register */
val = readb_relaxed(priv->base + RWTCSRA) & ~RWTCSRA_TME;
rwdt_write(priv, val, RWTCSRA);
+ /* Delay 2 cycles before setting watchdog counter */
+ rwdt_wait_cycles(priv, 2);
rwdt_init_timeout(wdev);
rwdt_write(priv, priv->cks, RWTCSRA);
struct rwdt_priv *priv = watchdog_get_drvdata(wdev);
rwdt_write(priv, priv->cks, RWTCSRA);
+ /* Delay 3 cycles before disabling module clock */
+ rwdt_wait_cycles(priv, 3);
pm_runtime_put(wdev->parent);
return 0;
static int rwdt_probe(struct platform_device *pdev)
{
+ struct device *dev = &pdev->dev;
struct rwdt_priv *priv;
struct clk *clk;
unsigned long clks_per_sec;
int ret, i;
- if (rwdt_blacklisted(&pdev->dev))
+ if (rwdt_blacklisted(dev))
return -ENODEV;
- priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
if (IS_ERR(priv->base))
return PTR_ERR(priv->base);
- clk = devm_clk_get(&pdev->dev, NULL);
+ clk = devm_clk_get(dev, NULL);
if (IS_ERR(clk))
return PTR_ERR(clk);
- pm_runtime_enable(&pdev->dev);
- pm_runtime_get_sync(&pdev->dev);
+ pm_runtime_enable(dev);
+ pm_runtime_get_sync(dev);
priv->clk_rate = clk_get_rate(clk);
priv->wdev.bootstatus = (readb_relaxed(priv->base + RWTCSRA) &
RWTCSRA_WOVF) ? WDIOF_CARDRESET : 0;
- pm_runtime_put(&pdev->dev);
+ pm_runtime_put(dev);
if (!priv->clk_rate) {
ret = -ENOENT;
}
if (i < 0) {
- dev_err(&pdev->dev, "Can't find suitable clock divider\n");
+ dev_err(dev, "Can't find suitable clock divider\n");
ret = -ERANGE;
goto out_pm_disable;
}
priv->wdev.info = &rwdt_ident;
priv->wdev.ops = &rwdt_ops;
- priv->wdev.parent = &pdev->dev;
+ priv->wdev.parent = dev;
priv->wdev.min_timeout = 1;
priv->wdev.max_timeout = DIV_BY_CLKS_PER_SEC(priv, 65536);
priv->wdev.timeout = min(priv->wdev.max_timeout, RWDT_DEFAULT_TIMEOUT);
watchdog_stop_on_unregister(&priv->wdev);
/* This overrides the default timeout only if DT configuration was found */
- watchdog_init_timeout(&priv->wdev, 0, &pdev->dev);
+ watchdog_init_timeout(&priv->wdev, 0, dev);
ret = watchdog_register_device(&priv->wdev);
if (ret < 0)
return 0;
out_pm_disable:
- pm_runtime_disable(&pdev->dev);
+ pm_runtime_disable(dev);
return ret;
}
+// SPDX-License-Identifier: GPL-2.0+
/*
* Retu watchdog driver
*
*
* Based on code written by Amit Kucheria and Michael Buesch.
* Rewritten by Aaro Koskinen.
- *
- * This file is subject to the terms and conditions of the GNU General
- * Public License. See the file "COPYING" in the main directory of this
- * archive for more details.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
*/
#include <linux/slab.h>
wdt->wdt_device.parent = dev;
ret = watchdog_register_device(&wdt->wdt_device);
- if (ret) {
- dev_err(dev, "cannot register watchdog (%d)\n", ret);
+ if (ret)
goto err_cpufreq;
- }
ret = s3c2410wdt_mask_and_disable_reset(wdt, false);
if (ret < 0)
+// SPDX-License-Identifier: GPL-2.0+
/*
* Watchdog driver for the SA11x0/PXA2xx
*
* (c) Copyright 2000 Oleg Drokin <green@crimea.edu>
* Based on SoftDog driver by Alan Cox <alan@lxorguk.ukuu.org.uk>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Oleg Drokin nor iXcelerator.com admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
u32 value = WDT_SEC2TICKS(timeout);
wdt->mr &= ~AT91_WDT_WDV;
- wdt->mr &= ~AT91_WDT_WDD;
wdt->mr |= AT91_WDT_SET_WDV(value);
- wdt->mr |= AT91_WDT_SET_WDD(value);
/*
* WDDIS has to be 0 when updating WDD/WDV. The datasheet states: When
timeout = WDT_SEC2TICKS(wdd->timeout);
- wdt->mr |= AT91_WDT_SET_WDD(timeout);
+ wdt->mr |= AT91_WDT_SET_WDD(WDT_SEC2TICKS(MAX_WDT_TIMEOUT));
wdt->mr |= AT91_WDT_SET_WDV(timeout);
ret = sama5d4_wdt_init(wdt);
watchdog_stop_on_unregister(wdd);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog device\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
MODULE_DEVICE_TABLE(of, sama5d4_wdt_of_match);
#ifdef CONFIG_PM_SLEEP
-static int sama5d4_wdt_resume(struct device *dev)
+static int sama5d4_wdt_suspend_late(struct device *dev)
+{
+ struct sama5d4_wdt *wdt = dev_get_drvdata(dev);
+
+ if (watchdog_active(&wdt->wdd))
+ sama5d4_wdt_stop(&wdt->wdd);
+
+ return 0;
+}
+
+static int sama5d4_wdt_resume_early(struct device *dev)
{
struct sama5d4_wdt *wdt = dev_get_drvdata(dev);
*/
sama5d4_wdt_init(wdt);
+ if (watchdog_active(&wdt->wdd))
+ sama5d4_wdt_start(&wdt->wdd);
+
return 0;
}
#endif
-static SIMPLE_DEV_PM_OPS(sama5d4_wdt_pm_ops, NULL,
- sama5d4_wdt_resume);
+static const struct dev_pm_ops sama5d4_wdt_pm_ops = {
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(sama5d4_wdt_suspend_late,
+ sama5d4_wdt_resume_early)
+};
static struct platform_driver sama5d4_wdt_driver = {
.probe = sama5d4_wdt_probe,
+// SPDX-License-Identifier: GPL-2.0
/*
* NANO7240 SBC Watchdog device driver
*
* Based on w83877f.c by Scott Jennings,
*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation;
- *
- * Software distributed under the License is distributed on an "AS IS"
- * basis, WITHOUT WARRANTY OF ANY KIND, either express or
- * implied. See the License for the specific language governing
- * rights and limitations under the License.
- *
* (c) Copyright 2007 Gilles GIGAN <gilles.gigan@jcu.edu.au>
- *
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+// SPDX-License-Identifier: GPL-2.0+
/*
* SBC8360 Watchdog driver
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* sch311x_wdt.c - Driver for the SCH311x Super-I/O chips
* integrated watchdog.
*
* (c) Copyright 2008 Wim Van Sebroeck <wim@iguana.be>.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Wim Van Sebroeck nor Iguana vzw. admit liability nor
* provide warranty for any of this software. This material is
* provided "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* SoftDog: A Software Watchdog Device
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
return ret;
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "cannot register watchdog device (err=%d)\n", ret);
+ if (ret)
return ret;
- }
/* Show module parameters */
dev_info(dev, "initialized. heartbeat=%d sec (nowayout=%d)\n",
}
ret = watchdog_register_device(&wdt->wdd);
- if (ret) {
- dev_err(&adev->dev, "watchdog_register_device() failed: %d\n",
- ret);
+ if (ret)
goto err;
- }
amba_set_drvdata(adev, wdt);
dev_info(&adev->dev, "registration successful\n");
ret = devm_watchdog_register_device(dev, &wdt->wdd);
if (ret) {
sprd_wdt_disable(wdt);
- dev_err(dev, "failed to register watchdog\n");
return ret;
}
platform_set_drvdata(pdev, wdt);
return ret;
ret = devm_watchdog_register_device(dev, &st_wdog_dev);
- if (ret) {
- dev_err(dev, "Unable to register watchdog\n");
+ if (ret)
return ret;
- }
st_wdog_setup(st_wdog, true);
watchdog_init_timeout(wdd, 0, dev);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog device\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
stmp3xxx_wdd.parent = dev;
ret = devm_watchdog_register_device(dev, &stmp3xxx_wdd);
- if (ret < 0) {
- dev_err(dev, "cannot register watchdog device\n");
+ if (ret < 0)
return ret;
- }
if (register_reboot_notifier(&wdt_notifier))
dev_warn(dev, "cannot register reboot notifier\n");
watchdog_stop_on_unregister(wdd);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog device\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
ts4800_wdt_stop(wdd);
ret = devm_watchdog_register_device(dev, wdd);
- if (ret) {
- dev_err(dev, "failed to register watchdog device\n");
+ if (ret)
return ret;
- }
platform_set_drvdata(pdev, wdt);
+// SPDX-License-Identifier: GPL-2.0+
/*
* w83627hf/thf WDT driver
*
* (c) Copyright 1996 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* ICP Wafer 5823 Single Board Computer WDT driver
* http://www.icpamerica.com/wafer_5823.php
* (c) Copyright 1996-1997 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* watchdog_core.c
*
* Satyam Sharma <satyam@infradead.org>
* Randy Dunlap <randy.dunlap@oracle.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox, CymruNet Ltd., Wim Van Sebroeck nor Iguana vzw.
* admit liability nor provide warranty for any of this software.
* This material is provided "AS-IS" and at no charge.
static LIST_HEAD(wtd_deferred_reg_list);
static bool wtd_deferred_reg_done;
-static int watchdog_deferred_registration_add(struct watchdog_device *wdd)
+static void watchdog_deferred_registration_add(struct watchdog_device *wdd)
{
list_add_tail(&wdd->deferred,
&wtd_deferred_reg_list);
- return 0;
}
static void watchdog_deferred_registration_del(struct watchdog_device *wdd)
int watchdog_register_device(struct watchdog_device *wdd)
{
- int ret;
+ const char *dev_str;
+ int ret = 0;
mutex_lock(&wtd_deferred_reg_mutex);
if (wtd_deferred_reg_done)
ret = __watchdog_register_device(wdd);
else
- ret = watchdog_deferred_registration_add(wdd);
+ watchdog_deferred_registration_add(wdd);
mutex_unlock(&wtd_deferred_reg_mutex);
+
+ if (ret) {
+ dev_str = wdd->parent ? dev_name(wdd->parent) :
+ (const char *)wdd->info->identity;
+ pr_err("%s: failed to register watchdog device (err = %d)\n",
+ dev_str, ret);
+ }
+
return ret;
}
EXPORT_SYMBOL_GPL(watchdog_register_device);
+/* SPDX-License-Identifier: GPL-2.0+ */
/*
* watchdog_core.h
*
* Satyam Sharma <satyam@infradead.org>
* Randy Dunlap <randy.dunlap@oracle.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox, CymruNet Ltd., Wim Van Sebroeck nor Iguana vzw.
* admit liability nor provide warranty for any of this software.
* This material is provided "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* watchdog_dev.c
*
* Satyam Sharma <satyam@infradead.org>
* Randy Dunlap <randy.dunlap@oracle.com>
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox, CymruNet Ltd., Wim Van Sebroeck nor Iguana vzw.
* admit liability nor provide warranty for any of this software.
* This material is provided "AS-IS" and at no charge.
struct mutex lock;
ktime_t last_keepalive;
ktime_t last_hw_keepalive;
+ ktime_t open_deadline;
struct hrtimer timer;
struct kthread_work work;
unsigned long status; /* Internal status bits */
static bool handle_boot_enabled =
IS_ENABLED(CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED);
+static unsigned open_timeout = CONFIG_WATCHDOG_OPEN_TIMEOUT;
+
+static bool watchdog_past_open_deadline(struct watchdog_core_data *data)
+{
+ return ktime_after(ktime_get(), data->open_deadline);
+}
+
+static void watchdog_set_open_deadline(struct watchdog_core_data *data)
+{
+ data->open_deadline = open_timeout ?
+ ktime_get() + ktime_set(open_timeout, 0) : KTIME_MAX;
+}
+
static inline bool watchdog_need_worker(struct watchdog_device *wdd)
{
/* All variables in milli-seconds */
ktime_t virt_timeout;
unsigned int hw_heartbeat_ms;
- virt_timeout = ktime_add(wd_data->last_keepalive,
- ms_to_ktime(timeout_ms));
+ if (watchdog_active(wdd))
+ virt_timeout = ktime_add(wd_data->last_keepalive,
+ ms_to_ktime(timeout_ms));
+ else
+ virt_timeout = wd_data->open_deadline;
+
hw_heartbeat_ms = min_not_zero(timeout_ms, wdd->max_hw_heartbeat_ms);
keepalive_interval = ms_to_ktime(hw_heartbeat_ms / 2);
- if (!watchdog_active(wdd))
- return keepalive_interval;
-
/*
* To ensure that the watchdog times out wdd->timeout seconds
* after the most recent ping from userspace, the last
{
struct watchdog_device *wdd = wd_data->wdd;
- return wdd && (watchdog_active(wdd) || watchdog_hw_running(wdd));
+ if (!wdd)
+ return false;
+
+ if (watchdog_active(wdd))
+ return true;
+
+ return watchdog_hw_running(wdd) && !watchdog_past_open_deadline(wd_data);
}
static void watchdog_ping_work(struct kthread_work *work)
if (!hw_running)
kref_get(&wd_data->kref);
+ /*
+ * open_timeout only applies for the first open from
+ * userspace. Set open_deadline to infinity so that the kernel
+ * will take care of an always-running hardware watchdog in
+ * case the device gets magic-closed or WDIOS_DISABLECARD is
+ * applied.
+ */
+ wd_data->open_deadline = KTIME_MAX;
+
/* dev/watchdog is a virtual (and thus non-seekable) filesystem */
return stream_open(inode, file);
/* Record time of most recent heartbeat as 'just before now'. */
wd_data->last_hw_keepalive = ktime_sub(ktime_get(), 1);
+ watchdog_set_open_deadline(wd_data);
/*
* If the watchdog is running, prevent its driver from being unloaded,
MODULE_PARM_DESC(handle_boot_enabled,
"Watchdog core auto-updates boot enabled watchdogs before userspace takes over (default="
__MODULE_STRING(IS_ENABLED(CONFIG_WATCHDOG_HANDLE_BOOT_ENABLED)) ")");
+
+module_param(open_timeout, uint, 0644);
+MODULE_PARM_DESC(open_timeout,
+ "Maximum time (in seconds, 0 means infinity) for userspace to take over a running watchdog (default="
+ __MODULE_STRING(CONFIG_WATCHDOG_OPEN_TIMEOUT) ")");
+/* SPDX-License-Identifier: GPL-1.0+ */
/*
* Industrial Computer Source WDT500/501 driver
*
*
* http://www.cymru.net
*
- * This driver is provided under the GNU General Public License,
- * incorporated herein by reference. The driver is provided without
- * warranty or support.
- *
* Release 0.04.
- *
*/
+// SPDX-License-Identifier: GPL-2.0+
/*
* Industrial Computer Source WDT501 driver
*
* (c) Copyright 1996-1997 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
+// SPDX-License-Identifier: GPL-2.0+
/*
* Industrial Computer Source PCI-WDT500/501 driver
*
* (c) Copyright 1996-1997 Alan Cox <alan@lxorguk.ukuu.org.uk>,
* All Rights Reserved.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; either version
- * 2 of the License, or (at your option) any later version.
- *
* Neither Alan Cox nor CymruNet Ltd. admit liability nor provide
* warranty for any of this software. This material is provided
* "AS-IS" and at no charge.
}
}
- ret = devm_watchdog_register_device(dev, &driver_data->wdt);
- if (ret != 0) {
- dev_err(wm831x->dev, "watchdog_register_device() failed: %d\n",
- ret);
- return ret;
- }
-
- return 0;
+ return devm_watchdog_register_device(dev, &driver_data->wdt);
}
static struct platform_driver wm831x_wdt_driver = {
watchdog_stop_on_unregister(&xen_wdt_dev);
ret = devm_watchdog_register_device(dev, &xen_wdt_dev);
- if (ret) {
- dev_err(dev, "cannot register watchdog device (%d)\n", ret);
+ if (ret)
return ret;
- }
dev_info(dev, "initialized (timeout=%ds, nowayout=%d)\n",
xen_wdt_dev.timeout, nowayout);
#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG
-static int zero;
-static int one = 1;
-
static struct ctl_table balloon_table[] = {
{
.procname = "hotplug_unpopulated",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{ }
};
map = swiotlb_tbl_map_single(dev, start_dma_addr, phys, size, dir,
attrs);
- if (map == DMA_MAPPING_ERROR)
+ if (map == (phys_addr_t)DMA_MAPPING_ERROR)
return DMA_MAPPING_ERROR;
dev_addr = xen_phys_to_bus(map);
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/magic.h>
#include <xen/xen.h>
.llseek = default_llseek,
};
-static int xenfs_fill_super(struct super_block *sb, void *data, int silent)
+static int xenfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
static const struct tree_descr xenfs_files[] = {
[2] = { "xenbus", &xen_xenbus_fops, S_IRUSR|S_IWUSR },
xen_initial_domain() ? xenfs_init_files : xenfs_files);
}
-static struct dentry *xenfs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name,
- void *data)
+static int xenfs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, xenfs_fill_super);
+ return get_tree_single(fc, xenfs_fill_super);
+}
+
+static const struct fs_context_operations xenfs_context_ops = {
+ .get_tree = xenfs_get_tree,
+};
+
+static int xenfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &xenfs_context_ops;
+ return 0;
}
static struct file_system_type xenfs_type = {
.owner = THIS_MODULE,
.name = "xenfs",
- .mount = xenfs_mount,
+ .init_fs_context = xenfs_init_fs_context,
.kill_sb = kill_litter_super,
};
MODULE_ALIAS_FS("xenfs");
obj-$(CONFIG_SYSCTL) += drop_caches.o
obj-$(CONFIG_FHANDLE) += fhandle.o
-obj-$(CONFIG_FS_IOMAP) += iomap.o
+obj-y += iomap/
obj-y += quota/
/* SPDX-License-Identifier: GPL-2.0 */
+#include <linux/buffer_head.h>
#include <linux/fs.h>
#include <linux/adfs_fs.h>
#define ADFS_BAD_FRAG 1
#define ADFS_ROOT_FRAG 2
+#define ADFS_FILETYPE_NONE ((u16)~0)
+
+/* RISC OS 12-bit filetype is stored in load_address[19:8] */
+static inline u16 adfs_filetype(u32 loadaddr)
+{
+ return (loadaddr & 0xfff00000) == 0xfff00000 ?
+ (loadaddr >> 8) & 0xfff : ADFS_FILETYPE_NONE;
+}
+
#define ADFS_NDA_OWNER_READ (1 << 0)
#define ADFS_NDA_OWNER_WRITE (1 << 1)
#define ADFS_NDA_LOCKED (1 << 2)
#include "dir_f.h"
-struct buffer_head;
-
/*
* adfs file system inode data in memory
*/
struct adfs_inode_info {
loff_t mmu_private;
- unsigned long parent_id; /* object id of parent */
+ __u32 parent_id; /* parent indirect disc address */
__u32 loadaddr; /* RISC OS load address */
__u32 execaddr; /* RISC OS exec address */
- unsigned int filetype; /* RISC OS file type */
unsigned int attr; /* RISC OS permissions */
- unsigned int stamped:1; /* RISC OS file has date/time */
struct inode vfs_inode;
};
+static inline struct adfs_inode_info *ADFS_I(struct inode *inode)
+{
+ return container_of(inode, struct adfs_inode_info, vfs_inode);
+}
+
+static inline bool adfs_inode_is_stamped(struct inode *inode)
+{
+ return (ADFS_I(inode)->loadaddr & 0xfff00000) == 0xfff00000;
+}
+
/*
* Forward-declare this
*/
__u32 s_ids_per_zone; /* max. no ids in one zone */
__u32 s_idlen; /* length of ID in map */
__u32 s_map_size; /* sector size of a map */
- unsigned long s_size; /* total size (in blocks) of this fs */
signed int s_map2blk; /* shift left by this for map->sector*/
unsigned int s_log2sharesize;/* log2 share size */
- __le32 s_version; /* disc format version */
unsigned int s_namelen; /* maximum number of characters in name */
};
return sb->s_fs_info;
}
-static inline struct adfs_inode_info *ADFS_I(struct inode *inode)
-{
- return container_of(inode, struct adfs_inode_info, vfs_inode);
-}
-
/*
* Directory handling
*/
struct buffer_head **bh_fplus;
unsigned int pos;
- unsigned int parent_id;
+ __u32 parent_id;
struct adfs_dirheader dirhead;
union adfs_dirtail dirtail;
#define ADFS_MAX_NAME_LEN (256 + 4) /* +4 for ,xyz hex filetype suffix */
struct object_info {
__u32 parent_id; /* parent object id */
- __u32 file_id; /* object id */
+ __u32 indaddr; /* indirect disc addr */
__u32 loadaddr; /* load address */
__u32 execaddr; /* execution address */
__u32 size; /* size */
__u8 attr; /* RISC OS attributes */
unsigned int name_len; /* name length */
char name[ADFS_MAX_NAME_LEN];/* file name */
-
- /* RISC OS file type (12-bit: derived from loadaddr) */
- __u16 filetype;
};
struct adfs_dir_ops {
- int (*read)(struct super_block *sb, unsigned int id, unsigned int sz, struct adfs_dir *dir);
+ int (*read)(struct super_block *sb, unsigned int indaddr,
+ unsigned int size, struct adfs_dir *dir);
int (*setpos)(struct adfs_dir *dir, unsigned int fpos);
int (*getnext)(struct adfs_dir *dir, struct object_info *obj);
int (*update)(struct adfs_dir *dir, struct object_info *obj);
int adfs_notify_change(struct dentry *dentry, struct iattr *attr);
/* map.c */
-extern int adfs_map_lookup(struct super_block *sb, unsigned int frag_id, unsigned int offset);
+int adfs_map_lookup(struct super_block *sb, u32 frag_id, unsigned int offset);
extern unsigned int adfs_map_free(struct super_block *sb);
/* Misc */
void __adfs_error(struct super_block *sb, const char *function,
const char *fmt, ...);
#define adfs_error(sb, fmt...) __adfs_error(sb, __func__, fmt)
+void adfs_msg(struct super_block *sb, const char *pfx, const char *fmt, ...);
/* super.c */
*
* The root directory ID should always be looked up in the map [3.4]
*/
-static inline int
-__adfs_block_map(struct super_block *sb, unsigned int object_id,
- unsigned int block)
+static inline int __adfs_block_map(struct super_block *sb, u32 indaddr,
+ unsigned int block)
{
- if (object_id & 255) {
+ if (indaddr & 255) {
unsigned int off;
- off = (object_id & 255) - 1;
+ off = (indaddr & 255) - 1;
block += off << ADFS_SB(sb)->s_log2sharesize;
}
- return adfs_map_lookup(sb, object_id >> 8, block);
+ return adfs_map_lookup(sb, indaddr >> 8, block);
+}
+
+/* Return the disc record from the map */
+static inline
+struct adfs_discrecord *adfs_map_discrecord(struct adfs_discmap *dm)
+{
+ return (void *)(dm[0].dm_bh->b_data + 4);
+}
+
+static inline u64 adfs_disc_size(const struct adfs_discrecord *dr)
+{
+ return (u64)le32_to_cpu(dr->disc_size_high) << 32 |
+ le32_to_cpu(dr->disc_size);
}
if (obj->name_len <= 2 && dots == obj->name_len)
obj->name[0] = '^';
- obj->filetype = -1;
-
/*
- * object is a file and is filetyped and timestamped?
- * RISC OS 12-bit filetype is stored in load_address[19:8]
+ * If the object is a file, and the user requested the ,xyz hex
+ * filetype suffix to the name, check the filetype and append.
*/
- if ((0 == (obj->attr & ADFS_NDA_DIRECTORY)) &&
- (0xfff00000 == (0xfff00000 & obj->loadaddr))) {
- obj->filetype = (__u16) ((0x000fff00 & obj->loadaddr) >> 8);
-
- /* optionally append the ,xyz hex filetype suffix */
- if (ADFS_SB(dir->sb)->s_ftsuffix) {
- __u16 filetype = obj->filetype;
+ if (!(obj->attr & ADFS_NDA_DIRECTORY) && ADFS_SB(dir->sb)->s_ftsuffix) {
+ u16 filetype = adfs_filetype(obj->loadaddr);
+ if (filetype != ADFS_FILETYPE_NONE) {
obj->name[obj->name_len++] = ',';
obj->name[obj->name_len++] = hex_asc_lo(filetype >> 8);
obj->name[obj->name_len++] = hex_asc_lo(filetype >> 4);
goto unlock_out;
while (ops->getnext(&dir, &obj) == 0) {
if (!dir_emit(ctx, obj.name, obj.name_len,
- obj.file_id, DT_UNKNOWN))
+ obj.indaddr, DT_UNKNOWN))
break;
ctx->pos++;
}
const struct adfs_dir_ops *ops = ADFS_SB(sb)->s_dir;
struct adfs_dir dir;
- printk(KERN_INFO "adfs_dir_update: object %06X in dir %06X\n",
- obj->file_id, obj->parent_id);
+ printk(KERN_INFO "adfs_dir_update: object %06x in dir %06x\n",
+ obj->indaddr, obj->parent_id);
if (!ops->update) {
ret = -EINVAL;
goto out;
if (ADFS_I(inode)->parent_id != dir.parent_id) {
- adfs_error(sb, "parent directory changed under me! (%lx but got %x)\n",
+ adfs_error(sb,
+ "parent directory changed under me! (%06x but got %06x)\n",
ADFS_I(inode)->parent_id, dir.parent_id);
ret = -EIO;
goto free_out;
*
* E and F format directory handling
*/
-#include <linux/buffer_head.h>
#include "adfs.h"
#include "dir_f.h"
return (dircheck ^ (dircheck >> 8) ^ (dircheck >> 16) ^ (dircheck >> 24)) & 0xff;
}
-/*
- * Read and check that a directory is valid
- */
-static int
-adfs_dir_read(struct super_block *sb, unsigned long object_id,
- unsigned int size, struct adfs_dir *dir)
+/* Read and check that a directory is valid */
+static int adfs_dir_read(struct super_block *sb, u32 indaddr,
+ unsigned int size, struct adfs_dir *dir)
{
const unsigned int blocksize_bits = sb->s_blocksize_bits;
int blk = 0;
for (blk = 0; blk < size; blk++) {
int phys;
- phys = __adfs_block_map(sb, object_id, blk);
+ phys = __adfs_block_map(sb, indaddr, blk);
if (!phys) {
- adfs_error(sb, "dir object %lX has a hole at offset %d",
- object_id, blk);
+ adfs_error(sb, "dir %06x has a hole at offset %d",
+ indaddr, blk);
goto release_buffers;
}
return 0;
bad_dir:
- adfs_error(sb, "corrupted directory fragment %lX",
- object_id);
+ adfs_error(sb, "dir %06x is corrupted", indaddr);
release_buffers:
for (blk -= 1; blk >= 0; blk -= 1)
brelse(dir->bh[blk]);
}
obj->name_len = name_len;
- obj->file_id = adfs_readval(de->dirinddiscadd, 3);
+ obj->indaddr = adfs_readval(de->dirinddiscadd, 3);
obj->loadaddr = adfs_readval(de->dirload, 4);
obj->execaddr = adfs_readval(de->direxec, 4);
obj->size = adfs_readval(de->dirlen, 4);
static inline void
adfs_obj2dir(struct adfs_direntry *de, struct object_info *obj)
{
- adfs_writeval(de->dirinddiscadd, 3, obj->file_id);
+ adfs_writeval(de->dirinddiscadd, 3, obj->indaddr);
adfs_writeval(de->dirload, 4, obj->loadaddr);
adfs_writeval(de->direxec, 4, obj->execaddr);
adfs_writeval(de->dirlen, 4, obj->size);
* the caller is responsible for holding the necessary
* locks.
*/
-static int
-adfs_dir_find_entry(struct adfs_dir *dir, unsigned long object_id)
+static int adfs_dir_find_entry(struct adfs_dir *dir, u32 indaddr)
{
int pos, ret;
if (!__adfs_dir_get(dir, pos, &obj))
break;
- if (obj.file_id == object_id) {
+ if (obj.indaddr == indaddr) {
ret = pos;
break;
}
return ret;
}
-static int
-adfs_f_read(struct super_block *sb, unsigned int id, unsigned int sz, struct adfs_dir *dir)
+static int adfs_f_read(struct super_block *sb, u32 indaddr, unsigned int size,
+ struct adfs_dir *dir)
{
int ret;
- if (sz != ADFS_NEWDIR_SIZE)
+ if (size != ADFS_NEWDIR_SIZE)
return -EIO;
- ret = adfs_dir_read(sb, id, sz, dir);
+ ret = adfs_dir_read(sb, indaddr, size, dir);
if (ret)
adfs_error(sb, "unable to read directory");
else
struct super_block *sb = dir->sb;
int ret, i;
- ret = adfs_dir_find_entry(dir, obj->file_id);
+ ret = adfs_dir_find_entry(dir, obj->indaddr);
if (ret < 0) {
adfs_error(dir->sb, "unable to locate entry to update");
goto out;
*
* Copyright (C) 1997-1999 Russell King
*/
-#include <linux/buffer_head.h>
#include <linux/slab.h>
#include "adfs.h"
#include "dir_fplus.h"
h = (struct adfs_bigdirheader *)dir->bh_fplus[0]->b_data;
size = le32_to_cpu(h->bigdirsize);
if (size != sz) {
- printk(KERN_WARNING "adfs: adfs_fplus_read:"
- " directory header size %X\n"
- " does not match directory size %X\n",
- size, sz);
+ adfs_msg(sb, KERN_WARNING,
+ "directory header size %X does not match directory size %X",
+ size, sz);
}
if (h->bigdirversion[0] != 0 || h->bigdirversion[1] != 0 ||
h->bigdirversion[2] != 0 || size & 2047 ||
h->bigdirstartname != cpu_to_le32(BIGDIRSTARTNAME)) {
- printk(KERN_WARNING "adfs: dir object %X has"
- " malformed dir header\n", id);
+ adfs_error(sb, "dir %06x has malformed header", id);
goto out;
}
kcalloc(size, sizeof(struct buffer_head *),
GFP_KERNEL);
if (!bh_fplus) {
+ adfs_msg(sb, KERN_ERR,
+ "not enough memory for dir object %X (%d blocks)",
+ id, size);
ret = -ENOMEM;
- adfs_error(sb, "not enough memory for"
- " dir object %X (%d blocks)", id, size);
goto out;
}
dir->bh_fplus = bh_fplus;
if (t->bigdirendname != cpu_to_le32(BIGDIRENDNAME) ||
t->bigdirendmasseq != h->startmasseq ||
t->reserved[0] != 0 || t->reserved[1] != 0) {
- printk(KERN_WARNING "adfs: dir object %X has "
- "malformed dir end\n", id);
+ adfs_error(sb, "dir %06x has malformed tail", id);
goto out;
}
obj->loadaddr = le32_to_cpu(bde.bigdirload);
obj->execaddr = le32_to_cpu(bde.bigdirexec);
obj->size = le32_to_cpu(bde.bigdirlen);
- obj->file_id = le32_to_cpu(bde.bigdirindaddr);
+ obj->indaddr = le32_to_cpu(bde.bigdirindaddr);
obj->attr = le32_to_cpu(bde.bigdirattr);
obj->name_len = le32_to_cpu(bde.bigdirobnamelen);
return S_IFDIR | S_IXUGO | mode;
}
- switch (ADFS_I(inode)->filetype) {
+ switch (adfs_filetype(ADFS_I(inode)->loadaddr)) {
case 0xfc0: /* LinkFS */
return S_IFLNK|S_IRWXUGO;
2208988800000000000LL;
s64 nsec;
- if (ADFS_I(inode)->stamped == 0)
+ if (!adfs_inode_is_stamped(inode))
goto cur_time;
high = ADFS_I(inode)->loadaddr & 0xFF; /* top 8 bits of timestamp */
{
unsigned int high, low;
- if (ADFS_I(inode)->stamped) {
+ if (adfs_inode_is_stamped(inode)) {
/* convert 32-bit seconds to 40-bit centi-seconds */
low = (secs & 255) * 100;
high = (secs / 256) * 100 + (low >> 8) + 0x336e996a;
inode->i_uid = ADFS_SB(sb)->s_uid;
inode->i_gid = ADFS_SB(sb)->s_gid;
- inode->i_ino = obj->file_id;
+ inode->i_ino = obj->indaddr;
inode->i_size = obj->size;
set_nlink(inode, 2);
inode->i_blocks = (inode->i_size + sb->s_blocksize - 1) >>
ADFS_I(inode)->loadaddr = obj->loadaddr;
ADFS_I(inode)->execaddr = obj->execaddr;
ADFS_I(inode)->attr = obj->attr;
- ADFS_I(inode)->filetype = obj->filetype;
- ADFS_I(inode)->stamped = ((obj->loadaddr & 0xfff00000) == 0xfff00000);
inode->i_mode = adfs_atts2mode(sb, inode);
adfs_adfs2unix_time(&inode->i_mtime, inode);
struct object_info obj;
int ret;
- obj.file_id = inode->i_ino;
+ obj.indaddr = inode->i_ino;
obj.name_len = 0;
obj.parent_id = ADFS_I(inode)->parent_id;
obj.loadaddr = ADFS_I(inode)->loadaddr;
*
* Copyright (C) 1997-2002 Russell King
*/
-#include <linux/buffer_head.h>
#include <asm/unaligned.h>
#include "adfs.h"
* output of:
* gcc -D__KERNEL__ -O2 -I../../include -o - -S map.c
*/
-static int
-lookup_zone(const struct adfs_discmap *dm, const unsigned int idlen,
- const unsigned int frag_id, unsigned int *offset)
+static int lookup_zone(const struct adfs_discmap *dm, const unsigned int idlen,
+ const u32 frag_id, unsigned int *offset)
{
const unsigned int mapsize = dm->dm_endbit;
const u32 idmask = (1 << idlen) - 1;
return 0;
}
-static int
-scan_map(struct adfs_sb_info *asb, unsigned int zone,
- const unsigned int frag_id, unsigned int mapoff)
+static int scan_map(struct adfs_sb_info *asb, unsigned int zone,
+ const u32 frag_id, unsigned int mapoff)
{
const unsigned int idlen = asb->s_idlen;
struct adfs_discmap *dm, *dm_end;
return signed_asl(total, asb->s_map2blk);
}
-int
-adfs_map_lookup(struct super_block *sb, unsigned int frag_id,
- unsigned int offset)
+int adfs_map_lookup(struct super_block *sb, u32 frag_id, unsigned int offset)
{
struct adfs_sb_info *asb = ADFS_SB(sb);
unsigned int zone, mapoff;
*/
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/buffer_head.h>
#include <linux/parser.h>
#include <linux/mount.h>
#include <linux/seq_file.h>
#include "dir_f.h"
#include "dir_fplus.h"
+#define ADFS_SB_FLAGS SB_NOATIME
+
#define ADFS_DEFAULT_OWNER_MASK S_IRWXU
#define ADFS_DEFAULT_OTHER_MASK (S_IRWXG | S_IRWXO)
void __adfs_error(struct super_block *sb, const char *function, const char *fmt, ...)
{
- char error_buf[128];
+ struct va_format vaf;
va_list args;
va_start(args, fmt);
- vsnprintf(error_buf, sizeof(error_buf), fmt, args);
- va_end(args);
+ vaf.fmt = fmt;
+ vaf.va = &args;
- printk(KERN_CRIT "ADFS-fs error (device %s)%s%s: %s\n",
+ printk(KERN_CRIT "ADFS-fs error (device %s)%s%s: %pV\n",
sb->s_id, function ? ": " : "",
- function ? function : "", error_buf);
+ function ? function : "", &vaf);
+
+ va_end(args);
+}
+
+void adfs_msg(struct super_block *sb, const char *pfx, const char *fmt, ...)
+{
+ struct va_format vaf;
+ va_list args;
+
+ va_start(args, fmt);
+ vaf.fmt = fmt;
+ vaf.va = &args;
+ printk("%sADFS-fs (%s): %pV\n", pfx, sb->s_id, &vaf);
+ va_end(args);
}
static int adfs_checkdiscrecord(struct adfs_discrecord *dr)
{
+ unsigned int max_idlen;
int i;
/* sector size must be 256, 512 or 1024 bytes */
if (le32_to_cpu(dr->disc_size_high) >> dr->log2secsize)
return 1;
- /* idlen must be no greater than 19 v2 [1.0] */
- if (dr->idlen > 19)
+ /*
+ * Maximum idlen is limited to 16 bits for new directories by
+ * the three-byte storage of an indirect disc address. For
+ * big directories, idlen must be no greater than 19 v2 [1.0]
+ */
+ max_idlen = dr->format_version ? 19 : 16;
+ if (dr->idlen > max_idlen)
return 1;
/* reserved bytes should be zero */
{Opt_err, NULL}
};
-static int parse_options(struct super_block *sb, char *options)
+static int parse_options(struct super_block *sb, struct adfs_sb_info *asb,
+ char *options)
{
char *p;
- struct adfs_sb_info *asb = ADFS_SB(sb);
int option;
if (!options)
asb->s_ftsuffix = option;
break;
default:
- printk("ADFS-fs: unrecognised mount option \"%s\" "
- "or missing value\n", p);
+ adfs_msg(sb, KERN_ERR,
+ "unrecognised mount option \"%s\" or missing value",
+ p);
return -EINVAL;
}
}
static int adfs_remount(struct super_block *sb, int *flags, char *data)
{
+ struct adfs_sb_info temp_asb;
+ int ret;
+
sync_filesystem(sb);
- *flags |= SB_NODIRATIME;
- return parse_options(sb, data);
+ *flags |= ADFS_SB_FLAGS;
+
+ temp_asb = *ADFS_SB(sb);
+ ret = parse_options(sb, &temp_asb, data);
+ if (ret == 0)
+ *ADFS_SB(sb) = temp_asb;
+
+ return ret;
}
static int adfs_statfs(struct dentry *dentry, struct kstatfs *buf)
{
struct super_block *sb = dentry->d_sb;
struct adfs_sb_info *sbi = ADFS_SB(sb);
+ struct adfs_discrecord *dr = adfs_map_discrecord(sbi->s_map);
u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
buf->f_type = ADFS_SUPER_MAGIC;
buf->f_namelen = sbi->s_namelen;
buf->f_bsize = sb->s_blocksize;
- buf->f_blocks = sbi->s_size;
+ buf->f_blocks = adfs_disc_size(dr) >> sb->s_blocksize_bits;
buf->f_files = sbi->s_ids_per_zone * sbi->s_map_size;
buf->f_bavail =
buf->f_bfree = adfs_map_free(sb);
i = zone - 1;
dm[0].dm_startblk = 0;
dm[0].dm_startbit = ADFS_DR_SIZE_BITS;
- dm[i].dm_endbit = (le32_to_cpu(dr->disc_size_high) << (32 - dr->log2bpmb)) +
- (le32_to_cpu(dr->disc_size) >> dr->log2bpmb) +
+ dm[i].dm_endbit = (adfs_disc_size(dr) >> dr->log2bpmb) +
(ADFS_DR_SIZE_BITS - i * zone_size);
if (adfs_checkmap(sb, dm))
return ERR_PTR(-EIO);
}
-static inline unsigned long adfs_discsize(struct adfs_discrecord *dr, int block_bits)
-{
- unsigned long discsize;
-
- discsize = le32_to_cpu(dr->disc_size_high) << (32 - block_bits);
- discsize |= le32_to_cpu(dr->disc_size) >> block_bits;
-
- return discsize;
-}
-
static int adfs_fill_super(struct super_block *sb, void *data, int silent)
{
struct adfs_discrecord *dr;
struct buffer_head *bh;
struct object_info root_obj;
unsigned char *b_data;
+ unsigned int blocksize;
struct adfs_sb_info *asb;
struct inode *root;
int ret = -EINVAL;
- sb->s_flags |= SB_NODIRATIME;
+ sb->s_flags |= ADFS_SB_FLAGS;
asb = kzalloc(sizeof(*asb), GFP_KERNEL);
if (!asb)
asb->s_other_mask = ADFS_DEFAULT_OTHER_MASK;
asb->s_ftsuffix = 0;
- if (parse_options(sb, data))
+ if (parse_options(sb, asb, data))
goto error;
sb_set_blocksize(sb, BLOCK_SIZE);
if (!(bh = sb_bread(sb, ADFS_DISCRECORD / BLOCK_SIZE))) {
- adfs_error(sb, "unable to read superblock");
+ adfs_msg(sb, KERN_ERR, "error: unable to read superblock");
ret = -EIO;
goto error;
}
b_data = bh->b_data + (ADFS_DISCRECORD % BLOCK_SIZE);
if (adfs_checkbblk(b_data)) {
- if (!silent)
- printk("VFS: Can't find an adfs filesystem on dev "
- "%s.\n", sb->s_id);
ret = -EINVAL;
- goto error_free_bh;
+ goto error_badfs;
}
dr = (struct adfs_discrecord *)(b_data + ADFS_DR_OFFSET);
* Do some sanity checks on the ADFS disc record
*/
if (adfs_checkdiscrecord(dr)) {
- if (!silent)
- printk("VPS: Can't find an adfs filesystem on dev "
- "%s.\n", sb->s_id);
ret = -EINVAL;
- goto error_free_bh;
+ goto error_badfs;
}
+ blocksize = 1 << dr->log2secsize;
brelse(bh);
- if (sb_set_blocksize(sb, 1 << dr->log2secsize)) {
+
+ if (sb_set_blocksize(sb, blocksize)) {
bh = sb_bread(sb, ADFS_DISCRECORD / sb->s_blocksize);
if (!bh) {
- adfs_error(sb, "couldn't read superblock on "
- "2nd try.");
+ adfs_msg(sb, KERN_ERR,
+ "error: couldn't read superblock on 2nd try.");
ret = -EIO;
goto error;
}
b_data = bh->b_data + (ADFS_DISCRECORD % sb->s_blocksize);
if (adfs_checkbblk(b_data)) {
- adfs_error(sb, "disc record mismatch, very weird!");
+ adfs_msg(sb, KERN_ERR,
+ "error: disc record mismatch, very weird!");
ret = -EINVAL;
goto error_free_bh;
}
dr = (struct adfs_discrecord *)(b_data + ADFS_DR_OFFSET);
} else {
if (!silent)
- printk(KERN_ERR "VFS: Unsupported blocksize on dev "
- "%s.\n", sb->s_id);
+ adfs_msg(sb, KERN_ERR,
+ "error: unsupported blocksize");
ret = -EINVAL;
goto error;
}
asb->s_idlen = dr->idlen;
asb->s_map_size = dr->nzones | (dr->nzones_high << 8);
asb->s_map2blk = dr->log2bpmb - dr->log2secsize;
- asb->s_size = adfs_discsize(dr, sb->s_blocksize_bits);
- asb->s_version = dr->format_version;
asb->s_log2sharesize = dr->log2sharesize;
asb->s_map = adfs_read_map(sb, dr);
*/
sb->s_op = &adfs_sops;
- dr = (struct adfs_discrecord *)(asb->s_map[0].dm_bh->b_data + 4);
+ dr = adfs_map_discrecord(asb->s_map);
- root_obj.parent_id = root_obj.file_id = le32_to_cpu(dr->root);
+ root_obj.parent_id = root_obj.indaddr = le32_to_cpu(dr->root);
root_obj.name_len = 0;
/* Set root object date as 01 Jan 1987 00:00:00 */
root_obj.loadaddr = 0xfff0003f;
root_obj.size = ADFS_NEWDIR_SIZE;
root_obj.attr = ADFS_NDA_DIRECTORY | ADFS_NDA_OWNER_READ |
ADFS_NDA_OWNER_WRITE | ADFS_NDA_PUBLIC_READ;
- root_obj.filetype = -1;
/*
* If this is a F+ disk with variable length directories,
* get the root_size from the disc record.
*/
- if (asb->s_version) {
+ if (dr->format_version) {
root_obj.size = le32_to_cpu(dr->root_size);
asb->s_dir = &adfs_fplus_dir_ops;
asb->s_namelen = ADFS_FPLUS_NAME_LEN;
}
return 0;
+error_badfs:
+ if (!silent)
+ adfs_msg(sb, KERN_ERR,
+ "error: can't find an ADFS filesystem on dev %s.",
+ sb->s_id);
error_free_bh:
brelse(bh);
error:
#include <linux/ramfs.h>
#include <linux/percpu-refcount.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <asm/kmap_types.h>
#include <linux/uaccess.h>
return file;
}
-static struct dentry *aio_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int aio_init_fs_context(struct fs_context *fc)
{
- struct dentry *root = mount_pseudo(fs_type, "aio:", NULL, NULL,
- AIO_RING_MAGIC);
-
- if (!IS_ERR(root))
- root->d_sb->s_iflags |= SB_I_NOEXEC;
- return root;
+ if (!init_pseudo(fc, AIO_RING_MAGIC))
+ return -ENOMEM;
+ fc->s_iflags |= SB_I_NOEXEC;
+ return 0;
}
/* aio_setup
{
static struct file_system_type aio_fs = {
.name = "aio",
- .mount = aio_mount,
+ .init_fs_context = aio_init_fs_context,
.kill_sb = kill_anon_super,
};
aio_mnt = kern_mount(&aio_fs);
BUG_ON(PageWriteback(old));
get_page(new);
- rc = migrate_page_move_mapping(mapping, new, old, mode, 1);
+ rc = migrate_page_move_mapping(mapping, new, old, 1);
if (rc != MIGRATEPAGE_SUCCESS) {
put_page(new);
goto out_unlock;
const struct __aio_sigset __user *, usig)
{
struct __aio_sigset ksig = { NULL, };
- sigset_t ksigmask, sigsaved;
struct timespec64 ts;
bool interrupted;
int ret;
if (usig && copy_from_user(&ksig, usig, sizeof(ksig)))
return -EFAULT;
- ret = set_user_sigmask(ksig.sigmask, &ksigmask, &sigsaved, ksig.sigsetsize);
+ ret = set_user_sigmask(ksig.sigmask, ksig.sigsetsize);
if (ret)
return ret;
ret = do_io_getevents(ctx_id, min_nr, nr, events, timeout ? &ts : NULL);
interrupted = signal_pending(current);
- restore_user_sigmask(ksig.sigmask, &sigsaved, interrupted);
+ restore_saved_sigmask_unless(interrupted);
if (interrupted && !ret)
ret = -ERESTARTNOHAND;
const struct __aio_sigset __user *, usig)
{
struct __aio_sigset ksig = { NULL, };
- sigset_t ksigmask, sigsaved;
struct timespec64 ts;
bool interrupted;
int ret;
return -EFAULT;
- ret = set_user_sigmask(ksig.sigmask, &ksigmask, &sigsaved, ksig.sigsetsize);
+ ret = set_user_sigmask(ksig.sigmask, ksig.sigsetsize);
if (ret)
return ret;
ret = do_io_getevents(ctx_id, min_nr, nr, events, timeout ? &ts : NULL);
interrupted = signal_pending(current);
- restore_user_sigmask(ksig.sigmask, &sigsaved, interrupted);
+ restore_saved_sigmask_unless(interrupted);
if (interrupted && !ret)
ret = -ERESTARTNOHAND;
const struct __compat_aio_sigset __user *, usig)
{
struct __compat_aio_sigset ksig = { NULL, };
- sigset_t ksigmask, sigsaved;
struct timespec64 t;
bool interrupted;
int ret;
if (usig && copy_from_user(&ksig, usig, sizeof(ksig)))
return -EFAULT;
- ret = set_compat_user_sigmask(ksig.sigmask, &ksigmask, &sigsaved, ksig.sigsetsize);
+ ret = set_compat_user_sigmask(ksig.sigmask, ksig.sigsetsize);
if (ret)
return ret;
ret = do_io_getevents(ctx_id, min_nr, nr, events, timeout ? &t : NULL);
interrupted = signal_pending(current);
- restore_user_sigmask(ksig.sigmask, &sigsaved, interrupted);
+ restore_saved_sigmask_unless(interrupted);
if (interrupted && !ret)
ret = -ERESTARTNOHAND;
const struct __compat_aio_sigset __user *, usig)
{
struct __compat_aio_sigset ksig = { NULL, };
- sigset_t ksigmask, sigsaved;
struct timespec64 t;
bool interrupted;
int ret;
if (usig && copy_from_user(&ksig, usig, sizeof(ksig)))
return -EFAULT;
- ret = set_compat_user_sigmask(ksig.sigmask, &ksigmask, &sigsaved, ksig.sigsetsize);
+ ret = set_compat_user_sigmask(ksig.sigmask, ksig.sigsetsize);
if (ret)
return ret;
ret = do_io_getevents(ctx_id, min_nr, nr, events, timeout ? &t : NULL);
interrupted = signal_pending(current);
- restore_user_sigmask(ksig.sigmask, &sigsaved, interrupted);
+ restore_saved_sigmask_unless(interrupted);
if (interrupted && !ret)
ret = -ERESTARTNOHAND;
#include <linux/kernel.h>
#include <linux/magic.h>
#include <linux/anon_inodes.h>
+#include <linux/pseudo_fs.h>
#include <linux/uaccess.h>
.d_dname = anon_inodefs_dname,
};
-static struct dentry *anon_inodefs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int anon_inodefs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "anon_inode:", NULL,
- &anon_inodefs_dentry_operations, ANON_INODE_FS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, ANON_INODE_FS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->dops = &anon_inodefs_dentry_operations;
+ return 0;
}
static struct file_system_type anon_inode_fs_type = {
.name = "anon_inodefs",
- .mount = anon_inodefs_mount,
+ .init_fs_context = anon_inodefs_init_fs_context,
.kill_sb = kill_anon_super,
};
load_addr, interp_load_addr);
if (retval < 0)
goto out;
- /* N.B. passed_fileno might not be initialized? */
current->mm->end_code = end_code;
current->mm->start_code = start_code;
current->mm->start_data = start_data;
unsigned long len, memp, memp_size, extra, rlim;
__be32 __user *reloc;
u32 __user *rp;
- struct inode *inode;
int i, rev, relocs;
loff_t fpos;
unsigned long start_code, end_code;
int ret;
hdr = ((struct flat_hdr *) bprm->buf); /* exec-header */
- inode = file_inode(bprm->file);
text_len = ntohl(hdr->data_start);
data_len = ntohl(hdr->data_end) - ntohl(hdr->data_start);
#include <linux/pagemap.h>
#include <linux/namei.h>
#include <linux/mount.h>
+#include <linux/fs_context.h>
#include <linux/syscalls.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
.evict_inode = bm_evict_inode,
};
-static int bm_fill_super(struct super_block *sb, void *data, int silent)
+static int bm_fill_super(struct super_block *sb, struct fs_context *fc)
{
int err;
static const struct tree_descr bm_files[] = {
return err;
}
-static struct dentry *bm_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int bm_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, bm_fill_super);
+ return get_tree_single(fc, bm_fill_super);
+}
+
+static const struct fs_context_operations bm_context_ops = {
+ .get_tree = bm_get_tree,
+};
+
+static int bm_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &bm_context_ops;
+ return 0;
}
static struct linux_binfmt misc_format = {
static struct file_system_type bm_fs_type = {
.owner = THIS_MODULE,
.name = "binfmt_misc",
- .mount = bm_mount,
+ .init_fs_context = bm_init_fs_context,
.kill_sb = kill_litter_super,
};
MODULE_ALIAS_FS("binfmt_misc");
#include <linux/writeback.h>
#include <linux/mpage.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/uio.h>
#include <linux/namei.h>
#include <linux/log2.h>
.evict_inode = bdev_evict_inode,
};
-static struct dentry *bd_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int bd_init_fs_context(struct fs_context *fc)
{
- struct dentry *dent;
- dent = mount_pseudo(fs_type, "bdev:", &bdev_sops, NULL, BDEVFS_MAGIC);
- if (!IS_ERR(dent))
- dent->d_sb->s_iflags |= SB_I_CGROUPWB;
- return dent;
+ struct pseudo_fs_context *ctx = init_pseudo(fc, BDEVFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ fc->s_iflags |= SB_I_CGROUPWB;
+ ctx->ops = &bdev_sops;
+ return 0;
}
static struct file_system_type bd_type = {
.name = "bdev",
- .mount = bd_mount,
+ .init_fs_context = bd_init_fs_context,
.kill_sb = kill_anon_super,
};
config BTRFS_FS
tristate "Btrfs filesystem support"
- select LIBCRC32C
+ select CRYPTO
+ select CRYPTO_CRC32C
select ZLIB_INFLATE
select ZLIB_DEFLATE
select LZO_COMPRESS
export.o tree-log.o free-space-cache.o zlib.o lzo.o zstd.o \
compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o \
reada.o backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \
- uuid-tree.o props.o free-space-tree.o tree-checker.o
+ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \
+ block-rsv.o delalloc-space.o
btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o
btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o
*
* Return: 0 if extent is not shared, 1 if it is shared, < 0 on error.
*/
-int btrfs_check_shared(struct btrfs_root *root, u64 inum, u64 bytenr)
+int btrfs_check_shared(struct btrfs_root *root, u64 inum, u64 bytenr,
+ struct ulist *roots, struct ulist *tmp)
{
struct btrfs_fs_info *fs_info = root->fs_info;
struct btrfs_trans_handle *trans;
- struct ulist *tmp = NULL;
- struct ulist *roots = NULL;
struct ulist_iterator uiter;
struct ulist_node *node;
struct seq_list elem = SEQ_LIST_INIT(elem);
.share_count = 0,
};
- tmp = ulist_alloc(GFP_NOFS);
- roots = ulist_alloc(GFP_NOFS);
- if (!tmp || !roots) {
- ret = -ENOMEM;
- goto out;
- }
+ ulist_init(roots);
+ ulist_init(tmp);
trans = btrfs_attach_transaction(root);
if (IS_ERR(trans)) {
up_read(&fs_info->commit_root_sem);
}
out:
- ulist_free(tmp);
- ulist_free(roots);
+ ulist_release(roots);
+ ulist_release(tmp);
return ret;
}
u64 start_off, struct btrfs_path *path,
struct btrfs_inode_extref **ret_extref,
u64 *found_off);
-int btrfs_check_shared(struct btrfs_root *root, u64 inum, u64 bytenr);
+int btrfs_check_shared(struct btrfs_root *root, u64 inum, u64 bytenr,
+ struct ulist *roots, struct ulist *tmp_ulist);
int __init btrfs_prelim_ref_init(void);
void __cold btrfs_prelim_ref_exit(void);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include "ctree.h"
+#include "block-rsv.h"
+#include "space-info.h"
+#include "math.h"
+#include "transaction.h"
+
+static u64 block_rsv_release_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *block_rsv,
+ struct btrfs_block_rsv *dest, u64 num_bytes,
+ u64 *qgroup_to_release_ret)
+{
+ struct btrfs_space_info *space_info = block_rsv->space_info;
+ u64 qgroup_to_release = 0;
+ u64 ret;
+
+ spin_lock(&block_rsv->lock);
+ if (num_bytes == (u64)-1) {
+ num_bytes = block_rsv->size;
+ qgroup_to_release = block_rsv->qgroup_rsv_size;
+ }
+ block_rsv->size -= num_bytes;
+ if (block_rsv->reserved >= block_rsv->size) {
+ num_bytes = block_rsv->reserved - block_rsv->size;
+ block_rsv->reserved = block_rsv->size;
+ block_rsv->full = 1;
+ } else {
+ num_bytes = 0;
+ }
+ if (block_rsv->qgroup_rsv_reserved >= block_rsv->qgroup_rsv_size) {
+ qgroup_to_release = block_rsv->qgroup_rsv_reserved -
+ block_rsv->qgroup_rsv_size;
+ block_rsv->qgroup_rsv_reserved = block_rsv->qgroup_rsv_size;
+ } else {
+ qgroup_to_release = 0;
+ }
+ spin_unlock(&block_rsv->lock);
+
+ ret = num_bytes;
+ if (num_bytes > 0) {
+ if (dest) {
+ spin_lock(&dest->lock);
+ if (!dest->full) {
+ u64 bytes_to_add;
+
+ bytes_to_add = dest->size - dest->reserved;
+ bytes_to_add = min(num_bytes, bytes_to_add);
+ dest->reserved += bytes_to_add;
+ if (dest->reserved >= dest->size)
+ dest->full = 1;
+ num_bytes -= bytes_to_add;
+ }
+ spin_unlock(&dest->lock);
+ }
+ if (num_bytes)
+ btrfs_space_info_add_old_bytes(fs_info, space_info,
+ num_bytes);
+ }
+ if (qgroup_to_release_ret)
+ *qgroup_to_release_ret = qgroup_to_release;
+ return ret;
+}
+
+int btrfs_block_rsv_migrate(struct btrfs_block_rsv *src,
+ struct btrfs_block_rsv *dst, u64 num_bytes,
+ bool update_size)
+{
+ int ret;
+
+ ret = btrfs_block_rsv_use_bytes(src, num_bytes);
+ if (ret)
+ return ret;
+
+ btrfs_block_rsv_add_bytes(dst, num_bytes, update_size);
+ return 0;
+}
+
+void btrfs_init_block_rsv(struct btrfs_block_rsv *rsv, unsigned short type)
+{
+ memset(rsv, 0, sizeof(*rsv));
+ spin_lock_init(&rsv->lock);
+ rsv->type = type;
+}
+
+void btrfs_init_metadata_block_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *rsv,
+ unsigned short type)
+{
+ btrfs_init_block_rsv(rsv, type);
+ rsv->space_info = btrfs_find_space_info(fs_info,
+ BTRFS_BLOCK_GROUP_METADATA);
+}
+
+struct btrfs_block_rsv *btrfs_alloc_block_rsv(struct btrfs_fs_info *fs_info,
+ unsigned short type)
+{
+ struct btrfs_block_rsv *block_rsv;
+
+ block_rsv = kmalloc(sizeof(*block_rsv), GFP_NOFS);
+ if (!block_rsv)
+ return NULL;
+
+ btrfs_init_metadata_block_rsv(fs_info, block_rsv, type);
+ return block_rsv;
+}
+
+void btrfs_free_block_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *rsv)
+{
+ if (!rsv)
+ return;
+ btrfs_block_rsv_release(fs_info, rsv, (u64)-1);
+ kfree(rsv);
+}
+
+int btrfs_block_rsv_add(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, u64 num_bytes,
+ enum btrfs_reserve_flush_enum flush)
+{
+ int ret;
+
+ if (num_bytes == 0)
+ return 0;
+
+ ret = btrfs_reserve_metadata_bytes(root, block_rsv, num_bytes, flush);
+ if (!ret)
+ btrfs_block_rsv_add_bytes(block_rsv, num_bytes, true);
+
+ return ret;
+}
+
+int btrfs_block_rsv_check(struct btrfs_block_rsv *block_rsv, int min_factor)
+{
+ u64 num_bytes = 0;
+ int ret = -ENOSPC;
+
+ if (!block_rsv)
+ return 0;
+
+ spin_lock(&block_rsv->lock);
+ num_bytes = div_factor(block_rsv->size, min_factor);
+ if (block_rsv->reserved >= num_bytes)
+ ret = 0;
+ spin_unlock(&block_rsv->lock);
+
+ return ret;
+}
+
+int btrfs_block_rsv_refill(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, u64 min_reserved,
+ enum btrfs_reserve_flush_enum flush)
+{
+ u64 num_bytes = 0;
+ int ret = -ENOSPC;
+
+ if (!block_rsv)
+ return 0;
+
+ spin_lock(&block_rsv->lock);
+ num_bytes = min_reserved;
+ if (block_rsv->reserved >= num_bytes)
+ ret = 0;
+ else
+ num_bytes -= block_rsv->reserved;
+ spin_unlock(&block_rsv->lock);
+
+ if (!ret)
+ return 0;
+
+ ret = btrfs_reserve_metadata_bytes(root, block_rsv, num_bytes, flush);
+ if (!ret) {
+ btrfs_block_rsv_add_bytes(block_rsv, num_bytes, false);
+ return 0;
+ }
+
+ return ret;
+}
+
+u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *block_rsv,
+ u64 num_bytes, u64 *qgroup_to_release)
+{
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv;
+ struct btrfs_block_rsv *target = NULL;
+
+ /*
+ * If we are the delayed_rsv then push to the global rsv, otherwise dump
+ * into the delayed rsv if it is not full.
+ */
+ if (block_rsv == delayed_rsv)
+ target = global_rsv;
+ else if (block_rsv != global_rsv && !delayed_rsv->full)
+ target = delayed_rsv;
+
+ if (target && block_rsv->space_info != target->space_info)
+ target = NULL;
+
+ return block_rsv_release_bytes(fs_info, block_rsv, target, num_bytes,
+ qgroup_to_release);
+}
+
+int btrfs_block_rsv_use_bytes(struct btrfs_block_rsv *block_rsv, u64 num_bytes)
+{
+ int ret = -ENOSPC;
+
+ spin_lock(&block_rsv->lock);
+ if (block_rsv->reserved >= num_bytes) {
+ block_rsv->reserved -= num_bytes;
+ if (block_rsv->reserved < block_rsv->size)
+ block_rsv->full = 0;
+ ret = 0;
+ }
+ spin_unlock(&block_rsv->lock);
+ return ret;
+}
+
+void btrfs_block_rsv_add_bytes(struct btrfs_block_rsv *block_rsv,
+ u64 num_bytes, bool update_size)
+{
+ spin_lock(&block_rsv->lock);
+ block_rsv->reserved += num_bytes;
+ if (update_size)
+ block_rsv->size += num_bytes;
+ else if (block_rsv->reserved >= block_rsv->size)
+ block_rsv->full = 1;
+ spin_unlock(&block_rsv->lock);
+}
+
+int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *dest, u64 num_bytes,
+ int min_factor)
+{
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ u64 min_bytes;
+
+ if (global_rsv->space_info != dest->space_info)
+ return -ENOSPC;
+
+ spin_lock(&global_rsv->lock);
+ min_bytes = div_factor(global_rsv->size, min_factor);
+ if (global_rsv->reserved < min_bytes + num_bytes) {
+ spin_unlock(&global_rsv->lock);
+ return -ENOSPC;
+ }
+ global_rsv->reserved -= num_bytes;
+ if (global_rsv->reserved < global_rsv->size)
+ global_rsv->full = 0;
+ spin_unlock(&global_rsv->lock);
+
+ btrfs_block_rsv_add_bytes(dest, num_bytes, true);
+ return 0;
+}
+
+void btrfs_update_global_block_rsv(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv;
+ struct btrfs_space_info *sinfo = block_rsv->space_info;
+ u64 num_bytes;
+
+ /*
+ * The global block rsv is based on the size of the extent tree, the
+ * checksum tree and the root tree. If the fs is empty we want to set
+ * it to a minimal amount for safety.
+ */
+ num_bytes = btrfs_root_used(&fs_info->extent_root->root_item) +
+ btrfs_root_used(&fs_info->csum_root->root_item) +
+ btrfs_root_used(&fs_info->tree_root->root_item);
+ num_bytes = max_t(u64, num_bytes, SZ_16M);
+
+ spin_lock(&sinfo->lock);
+ spin_lock(&block_rsv->lock);
+
+ block_rsv->size = min_t(u64, num_bytes, SZ_512M);
+
+ if (block_rsv->reserved < block_rsv->size) {
+ num_bytes = btrfs_space_info_used(sinfo, true);
+ if (sinfo->total_bytes > num_bytes) {
+ num_bytes = sinfo->total_bytes - num_bytes;
+ num_bytes = min(num_bytes,
+ block_rsv->size - block_rsv->reserved);
+ block_rsv->reserved += num_bytes;
+ btrfs_space_info_update_bytes_may_use(fs_info, sinfo,
+ num_bytes);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ sinfo->flags, num_bytes,
+ 1);
+ }
+ } else if (block_rsv->reserved > block_rsv->size) {
+ num_bytes = block_rsv->reserved - block_rsv->size;
+ btrfs_space_info_update_bytes_may_use(fs_info, sinfo,
+ -num_bytes);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ sinfo->flags, num_bytes, 0);
+ block_rsv->reserved = block_rsv->size;
+ }
+
+ if (block_rsv->reserved == block_rsv->size)
+ block_rsv->full = 1;
+ else
+ block_rsv->full = 0;
+
+ spin_unlock(&block_rsv->lock);
+ spin_unlock(&sinfo->lock);
+}
+
+void btrfs_init_global_block_rsv(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_space_info *space_info;
+
+ space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
+ fs_info->chunk_block_rsv.space_info = space_info;
+
+ space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
+ fs_info->global_block_rsv.space_info = space_info;
+ fs_info->trans_block_rsv.space_info = space_info;
+ fs_info->empty_block_rsv.space_info = space_info;
+ fs_info->delayed_block_rsv.space_info = space_info;
+ fs_info->delayed_refs_rsv.space_info = space_info;
+
+ fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv;
+ fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv;
+ fs_info->dev_root->block_rsv = &fs_info->global_block_rsv;
+ fs_info->tree_root->block_rsv = &fs_info->global_block_rsv;
+ if (fs_info->quota_root)
+ fs_info->quota_root->block_rsv = &fs_info->global_block_rsv;
+ fs_info->chunk_root->block_rsv = &fs_info->chunk_block_rsv;
+
+ btrfs_update_global_block_rsv(fs_info);
+}
+
+void btrfs_release_global_block_rsv(struct btrfs_fs_info *fs_info)
+{
+ btrfs_block_rsv_release(fs_info, &fs_info->global_block_rsv, (u64)-1);
+ WARN_ON(fs_info->trans_block_rsv.size > 0);
+ WARN_ON(fs_info->trans_block_rsv.reserved > 0);
+ WARN_ON(fs_info->chunk_block_rsv.size > 0);
+ WARN_ON(fs_info->chunk_block_rsv.reserved > 0);
+ WARN_ON(fs_info->delayed_block_rsv.size > 0);
+ WARN_ON(fs_info->delayed_block_rsv.reserved > 0);
+ WARN_ON(fs_info->delayed_refs_rsv.reserved > 0);
+ WARN_ON(fs_info->delayed_refs_rsv.size > 0);
+}
+
+static struct btrfs_block_rsv *get_block_rsv(
+ const struct btrfs_trans_handle *trans,
+ const struct btrfs_root *root)
+{
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_block_rsv *block_rsv = NULL;
+
+ if (test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
+ (root == fs_info->csum_root && trans->adding_csums) ||
+ (root == fs_info->uuid_root))
+ block_rsv = trans->block_rsv;
+
+ if (!block_rsv)
+ block_rsv = root->block_rsv;
+
+ if (!block_rsv)
+ block_rsv = &fs_info->empty_block_rsv;
+
+ return block_rsv;
+}
+
+struct btrfs_block_rsv *btrfs_use_block_rsv(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u32 blocksize)
+{
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_block_rsv *block_rsv;
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ int ret;
+ bool global_updated = false;
+
+ block_rsv = get_block_rsv(trans, root);
+
+ if (unlikely(block_rsv->size == 0))
+ goto try_reserve;
+again:
+ ret = btrfs_block_rsv_use_bytes(block_rsv, blocksize);
+ if (!ret)
+ return block_rsv;
+
+ if (block_rsv->failfast)
+ return ERR_PTR(ret);
+
+ if (block_rsv->type == BTRFS_BLOCK_RSV_GLOBAL && !global_updated) {
+ global_updated = true;
+ btrfs_update_global_block_rsv(fs_info);
+ goto again;
+ }
+
+ /*
+ * The global reserve still exists to save us from ourselves, so don't
+ * warn_on if we are short on our delayed refs reserve.
+ */
+ if (block_rsv->type != BTRFS_BLOCK_RSV_DELREFS &&
+ btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
+ static DEFINE_RATELIMIT_STATE(_rs,
+ DEFAULT_RATELIMIT_INTERVAL * 10,
+ /*DEFAULT_RATELIMIT_BURST*/ 1);
+ if (__ratelimit(&_rs))
+ WARN(1, KERN_DEBUG
+ "BTRFS: block rsv returned %d\n", ret);
+ }
+try_reserve:
+ ret = btrfs_reserve_metadata_bytes(root, block_rsv, blocksize,
+ BTRFS_RESERVE_NO_FLUSH);
+ if (!ret)
+ return block_rsv;
+ /*
+ * If we couldn't reserve metadata bytes try and use some from
+ * the global reserve if its space type is the same as the global
+ * reservation.
+ */
+ if (block_rsv->type != BTRFS_BLOCK_RSV_GLOBAL &&
+ block_rsv->space_info == global_rsv->space_info) {
+ ret = btrfs_block_rsv_use_bytes(global_rsv, blocksize);
+ if (!ret)
+ return global_rsv;
+ }
+ return ERR_PTR(ret);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef BTRFS_BLOCK_RSV_H
+#define BTRFS_BLOCK_RSV_H
+
+struct btrfs_trans_handle;
+enum btrfs_reserve_flush_enum;
+
+/*
+ * Types of block reserves
+ */
+enum {
+ BTRFS_BLOCK_RSV_GLOBAL,
+ BTRFS_BLOCK_RSV_DELALLOC,
+ BTRFS_BLOCK_RSV_TRANS,
+ BTRFS_BLOCK_RSV_CHUNK,
+ BTRFS_BLOCK_RSV_DELOPS,
+ BTRFS_BLOCK_RSV_DELREFS,
+ BTRFS_BLOCK_RSV_EMPTY,
+ BTRFS_BLOCK_RSV_TEMP,
+};
+
+struct btrfs_block_rsv {
+ u64 size;
+ u64 reserved;
+ struct btrfs_space_info *space_info;
+ spinlock_t lock;
+ unsigned short full;
+ unsigned short type;
+ unsigned short failfast;
+
+ /*
+ * Qgroup equivalent for @size @reserved
+ *
+ * Unlike normal @size/@reserved for inode rsv, qgroup doesn't care
+ * about things like csum size nor how many tree blocks it will need to
+ * reserve.
+ *
+ * Qgroup cares more about net change of the extent usage.
+ *
+ * So for one newly inserted file extent, in worst case it will cause
+ * leaf split and level increase, nodesize for each file extent is
+ * already too much.
+ *
+ * In short, qgroup_size/reserved is the upper limit of possible needed
+ * qgroup metadata reservation.
+ */
+ u64 qgroup_rsv_size;
+ u64 qgroup_rsv_reserved;
+};
+
+void btrfs_init_block_rsv(struct btrfs_block_rsv *rsv, unsigned short type);
+struct btrfs_block_rsv *btrfs_alloc_block_rsv(struct btrfs_fs_info *fs_info,
+ unsigned short type);
+void btrfs_init_metadata_block_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *rsv,
+ unsigned short type);
+void btrfs_free_block_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *rsv);
+int btrfs_block_rsv_add(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, u64 num_bytes,
+ enum btrfs_reserve_flush_enum flush);
+int btrfs_block_rsv_check(struct btrfs_block_rsv *block_rsv, int min_factor);
+int btrfs_block_rsv_refill(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, u64 min_reserved,
+ enum btrfs_reserve_flush_enum flush);
+int btrfs_block_rsv_migrate(struct btrfs_block_rsv *src_rsv,
+ struct btrfs_block_rsv *dst_rsv, u64 num_bytes,
+ bool update_size);
+int btrfs_block_rsv_use_bytes(struct btrfs_block_rsv *block_rsv, u64 num_bytes);
+int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *dest, u64 num_bytes,
+ int min_factor);
+void btrfs_block_rsv_add_bytes(struct btrfs_block_rsv *block_rsv,
+ u64 num_bytes, bool update_size);
+u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *block_rsv,
+ u64 num_bytes, u64 *qgroup_to_release);
+void btrfs_update_global_block_rsv(struct btrfs_fs_info *fs_info);
+void btrfs_init_global_block_rsv(struct btrfs_fs_info *fs_info);
+void btrfs_release_global_block_rsv(struct btrfs_fs_info *fs_info);
+struct btrfs_block_rsv *btrfs_use_block_rsv(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u32 blocksize);
+
+static inline void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *block_rsv,
+ u64 num_bytes)
+{
+ __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL);
+}
+
+static inline void btrfs_unuse_block_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *block_rsv,
+ u32 blocksize)
+{
+ btrfs_block_rsv_add_bytes(block_rsv, blocksize, false);
+ btrfs_block_rsv_release(fs_info, block_rsv, 0);
+}
+
+#endif /* BTRFS_BLOCK_RSV_H */
clear_bit(BTRFS_INODE_READDIO_NEED_LOCK, &inode->runtime_flags);
}
+/* Array of bytes with variable length, hexadecimal format 0x1234 */
+#define CSUM_FMT "0x%*phN"
+#define CSUM_FMT_VALUE(size, bytes) size, bytes
+
static inline void btrfs_print_data_csum_error(struct btrfs_inode *inode,
- u64 logical_start, u32 csum, u32 csum_expected, int mirror_num)
+ u64 logical_start, u8 *csum, u8 *csum_expected, int mirror_num)
{
struct btrfs_root *root = inode->root;
+ struct btrfs_super_block *sb = root->fs_info->super_copy;
+ const u16 csum_size = btrfs_super_csum_size(sb);
/* Output minus objectid, which is more meaningful */
if (root->root_key.objectid >= BTRFS_LAST_FREE_OBJECTID)
btrfs_warn_rl(root->fs_info,
- "csum failed root %lld ino %lld off %llu csum 0x%08x expected csum 0x%08x mirror %d",
+"csum failed root %lld ino %lld off %llu csum " CSUM_FMT " expected csum " CSUM_FMT " mirror %d",
root->root_key.objectid, btrfs_ino(inode),
- logical_start, csum, csum_expected, mirror_num);
+ logical_start,
+ CSUM_FMT_VALUE(csum_size, csum),
+ CSUM_FMT_VALUE(csum_size, csum_expected),
+ mirror_num);
else
btrfs_warn_rl(root->fs_info,
- "csum failed root %llu ino %llu off %llu csum 0x%08x expected csum 0x%08x mirror %d",
+"csum failed root %llu ino %llu off %llu csum " CSUM_FMT " expected csum " CSUM_FMT " mirror %d",
root->root_key.objectid, btrfs_ino(inode),
- logical_start, csum, csum_expected, mirror_num);
+ logical_start,
+ CSUM_FMT_VALUE(csum_size, csum),
+ CSUM_FMT_VALUE(csum_size, csum_expected),
+ mirror_num);
}
#endif
#include <linux/blkdev.h>
#include <linux/mm.h>
#include <linux/string.h>
-#include <linux/crc32c.h>
+#include <crypto/hash.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
char **datav, unsigned int num_pages)
{
struct btrfs_fs_info *fs_info = state->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
struct btrfs_header *h;
u8 csum[BTRFS_CSUM_SIZE];
- u32 crc = ~(u32)0;
unsigned int i;
if (num_pages * PAGE_SIZE < state->metablock_size)
if (memcmp(h->fsid, fs_info->fs_devices->fsid, BTRFS_FSID_SIZE))
return 1;
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
+
for (i = 0; i < num_pages; i++) {
u8 *data = i ? datav[i] : (datav[i] + BTRFS_CSUM_SIZE);
size_t sublen = i ? PAGE_SIZE :
(PAGE_SIZE - BTRFS_CSUM_SIZE);
- crc = crc32c(crc, data, sublen);
+ crypto_shash_update(shash, data, sublen);
}
- btrfs_csum_final(crc, csum);
+ crypto_shash_final(shash, csum);
if (memcmp(csum, h->csum, state->csum_size))
return 1;
#include <linux/slab.h>
#include <linux/sched/mm.h>
#include <linux/log2.h>
+#include <crypto/hash.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
return NULL;
}
+bool btrfs_compress_is_valid_type(const char *str, size_t len)
+{
+ int i;
+
+ for (i = 1; i < ARRAY_SIZE(btrfs_compress_types); i++) {
+ size_t comp_len = strlen(btrfs_compress_types[i]);
+
+ if (len < comp_len)
+ continue;
+
+ if (!strncmp(btrfs_compress_types[i], str, comp_len))
+ return true;
+ }
+ return false;
+}
+
static int btrfs_decompress_bio(struct compressed_bio *cb);
static inline int compressed_bio_size(struct btrfs_fs_info *fs_info,
struct compressed_bio *cb,
u64 disk_start)
{
+ struct btrfs_fs_info *fs_info = inode->root->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
+ const u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
int ret;
struct page *page;
unsigned long i;
char *kaddr;
- u32 csum;
- u32 *cb_sum = &cb->sums;
+ u8 csum[BTRFS_CSUM_SIZE];
+ u8 *cb_sum = cb->sums;
if (inode->flags & BTRFS_INODE_NODATASUM)
return 0;
+ shash->tfm = fs_info->csum_shash;
+
for (i = 0; i < cb->nr_pages; i++) {
page = cb->compressed_pages[i];
- csum = ~(u32)0;
+ crypto_shash_init(shash);
kaddr = kmap_atomic(page);
- csum = btrfs_csum_data(kaddr, csum, PAGE_SIZE);
- btrfs_csum_final(csum, (u8 *)&csum);
+ crypto_shash_update(shash, kaddr, PAGE_SIZE);
kunmap_atomic(kaddr);
+ crypto_shash_final(shash, (u8 *)&csum);
- if (csum != *cb_sum) {
- btrfs_print_data_csum_error(inode, disk_start, csum,
- *cb_sum, cb->mirror_num);
+ if (memcmp(&csum, cb_sum, csum_size)) {
+ btrfs_print_data_csum_error(inode, disk_start,
+ csum, cb_sum, cb->mirror_num);
ret = -EIO;
goto fail;
}
- cb_sum++;
+ cb_sum += csum_size;
}
ret = 0;
bdev = fs_info->fs_devices->latest_bdev;
- bio = btrfs_bio_alloc(bdev, first_byte);
+ bio = btrfs_bio_alloc(first_byte);
+ bio_set_dev(bio, bdev);
bio->bi_opf = REQ_OP_WRITE | write_flags;
bio->bi_private = cb;
bio->bi_end_io = end_compressed_bio_write;
bio_endio(bio);
}
- bio = btrfs_bio_alloc(bdev, first_byte);
+ bio = btrfs_bio_alloc(first_byte);
+ bio_set_dev(bio, bdev);
bio->bi_opf = REQ_OP_WRITE | write_flags;
bio->bi_private = cb;
bio->bi_end_io = end_compressed_bio_write;
struct extent_map *em;
blk_status_t ret = BLK_STS_RESOURCE;
int faili = 0;
- u32 *sums;
+ const u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
+ u8 *sums;
em_tree = &BTRFS_I(inode)->extent_tree;
cb->errors = 0;
cb->inode = inode;
cb->mirror_num = mirror_num;
- sums = &cb->sums;
+ sums = cb->sums;
cb->start = em->orig_start;
em_len = em->len;
/* include any pages we added in add_ra-bio_pages */
cb->len = bio->bi_iter.bi_size;
- comp_bio = btrfs_bio_alloc(bdev, cur_disk_byte);
+ comp_bio = btrfs_bio_alloc(cur_disk_byte);
+ bio_set_dev(comp_bio, bdev);
comp_bio->bi_opf = REQ_OP_READ;
comp_bio->bi_private = cb;
comp_bio->bi_end_io = end_compressed_bio_read;
page->mapping = NULL;
if (submit || bio_add_page(comp_bio, page, PAGE_SIZE, 0) <
PAGE_SIZE) {
+ unsigned int nr_sectors;
+
ret = btrfs_bio_wq_end_io(fs_info, comp_bio,
BTRFS_WQ_ENDIO_DATA);
BUG_ON(ret); /* -ENOMEM */
sums);
BUG_ON(ret); /* -ENOMEM */
}
- sums += DIV_ROUND_UP(comp_bio->bi_iter.bi_size,
- fs_info->sectorsize);
+
+ nr_sectors = DIV_ROUND_UP(comp_bio->bi_iter.bi_size,
+ fs_info->sectorsize);
+ sums += csum_size * nr_sectors;
ret = btrfs_map_bio(fs_info, comp_bio, mirror_num, 0);
if (ret) {
bio_endio(comp_bio);
}
- comp_bio = btrfs_bio_alloc(bdev, cur_disk_byte);
+ comp_bio = btrfs_bio_alloc(cur_disk_byte);
+ bio_set_dev(comp_bio, bdev);
comp_bio->bi_opf = REQ_OP_READ;
comp_bio->bi_private = cb;
comp_bio->bi_end_io = end_compressed_bio_read;
* the start of a variable length array of checksums only
* used by reads
*/
- u32 sums;
+ u8 sums[];
};
static inline unsigned int btrfs_compress_type(unsigned int type_level)
extern const struct btrfs_compress_op btrfs_zstd_compress;
const char* btrfs_compress_type2str(enum btrfs_compression_type type);
+bool btrfs_compress_is_valid_type(const char *str, size_t len);
int btrfs_compress_heuristic(struct inode *inode, u64 start, u64 end);
#include <linux/kobject.h>
#include <trace/events/btrfs.h>
#include <asm/kmap_types.h>
+#include <asm/unaligned.h>
#include <linux/pagemap.h>
#include <linux/btrfs.h>
#include <linux/btrfs_tree.h>
#include "extent_io.h"
#include "extent_map.h"
#include "async-thread.h"
+#include "block-rsv.h"
struct btrfs_trans_handle;
struct btrfs_transaction;
struct btrfs_pending_snapshot;
struct btrfs_delayed_ref_root;
+struct btrfs_space_info;
extern struct kmem_cache *btrfs_trans_handle_cachep;
extern struct kmem_cache *btrfs_bit_radix_cachep;
extern struct kmem_cache *btrfs_path_cachep;
#define BTRFS_MAGIC 0x4D5F53665248425FULL /* ascii _BHRfS_M, no null */
-#define BTRFS_MAX_MIRRORS 3
+/*
+ * Maximum number of mirrors that can be available for all profiles counting
+ * the target device of dev-replace as one. During an active device replace
+ * procedure, the target device of the copy operation is a mirror for the
+ * filesystem data as well that can be used to read data in order to repair
+ * read errors on other disks.
+ *
+ * Current value is derived from RAID1 with 2 copies.
+ */
+#define BTRFS_MAX_MIRRORS (2 + 1)
#define BTRFS_MAX_LEVEL 8
/* four bytes for CRC32 */
static const int btrfs_csum_sizes[] = { 4 };
+static const char *btrfs_csum_names[] = { "crc32c" };
#define BTRFS_EMPTY_DIR_SIZE 0
return div_u64(size + BTRFS_MAX_EXTENT_SIZE - 1, BTRFS_MAX_EXTENT_SIZE);
}
-struct btrfs_mapping_tree {
- struct extent_map_tree map_tree;
-};
-
static inline unsigned long btrfs_chunk_item_size(int num_stripes)
{
BUG_ON(num_stripes == 0);
struct list_head list;
};
-struct btrfs_space_info {
- spinlock_t lock;
-
- u64 total_bytes; /* total bytes in the space,
- this doesn't take mirrors into account */
- u64 bytes_used; /* total bytes used,
- this doesn't take mirrors into account */
- u64 bytes_pinned; /* total bytes pinned, will be freed when the
- transaction finishes */
- u64 bytes_reserved; /* total bytes the allocator has reserved for
- current allocations */
- u64 bytes_may_use; /* number of bytes that may be used for
- delalloc/allocations */
- u64 bytes_readonly; /* total bytes that are read only */
-
- u64 max_extent_size; /* This will hold the maximum extent size of
- the space info if we had an ENOSPC in the
- allocator. */
-
- unsigned int full:1; /* indicates that we cannot allocate any more
- chunks for this space */
- unsigned int chunk_alloc:1; /* set if we are allocating a chunk */
-
- unsigned int flush:1; /* set if we are trying to make space */
-
- unsigned int force_alloc; /* set if we need to force a chunk
- alloc for this space */
-
- u64 disk_used; /* total bytes used on disk */
- u64 disk_total; /* total bytes on disk, takes mirrors into
- account */
-
- u64 flags;
-
- /*
- * bytes_pinned is kept in line with what is actually pinned, as in
- * we've called update_block_group and dropped the bytes_used counter
- * and increased the bytes_pinned counter. However this means that
- * bytes_pinned does not reflect the bytes that will be pinned once the
- * delayed refs are flushed, so this counter is inc'ed every time we
- * call btrfs_free_extent so it is a realtime count of what will be
- * freed once the transaction is committed. It will be zeroed every
- * time the transaction commits.
- */
- struct percpu_counter total_bytes_pinned;
-
- struct list_head list;
- /* Protected by the spinlock 'lock'. */
- struct list_head ro_bgs;
- struct list_head priority_tickets;
- struct list_head tickets;
- /*
- * tickets_id just indicates the next ticket will be handled, so note
- * it's not stored per ticket.
- */
- u64 tickets_id;
-
- struct rw_semaphore groups_sem;
- /* for block groups in our same type */
- struct list_head block_groups[BTRFS_NR_RAID_TYPES];
- wait_queue_head_t wait;
-
- struct kobject kobj;
- struct kobject *block_group_kobjs[BTRFS_NR_RAID_TYPES];
-};
-
-/*
- * Types of block reserves
- */
-enum {
- BTRFS_BLOCK_RSV_GLOBAL,
- BTRFS_BLOCK_RSV_DELALLOC,
- BTRFS_BLOCK_RSV_TRANS,
- BTRFS_BLOCK_RSV_CHUNK,
- BTRFS_BLOCK_RSV_DELOPS,
- BTRFS_BLOCK_RSV_DELREFS,
- BTRFS_BLOCK_RSV_EMPTY,
- BTRFS_BLOCK_RSV_TEMP,
-};
-
-struct btrfs_block_rsv {
- u64 size;
- u64 reserved;
- struct btrfs_space_info *space_info;
- spinlock_t lock;
- unsigned short full;
- unsigned short type;
- unsigned short failfast;
-
- /*
- * Qgroup equivalent for @size @reserved
- *
- * Unlike normal @size/@reserved for inode rsv, qgroup doesn't care
- * about things like csum size nor how many tree blocks it will need to
- * reserve.
- *
- * Qgroup cares more about net change of the extent usage.
- *
- * So for one newly inserted file extent, in worst case it will cause
- * leaf split and level increase, nodesize for each file extent is
- * already too much.
- *
- * In short, qgroup_size/reserved is the upper limit of possible needed
- * qgroup metadata reservation.
- */
- u64 qgroup_rsv_size;
- u64 qgroup_rsv_reserved;
-};
-
/*
* free clusters are used to claim free space in relatively large chunks,
* allowing us to do less seeky writes. They are used for all metadata
/*
* Indicate that balance has been set up from the ioctl and is in the
* main phase. The fs_info::balance_ctl is initialized.
+ * Set and cleared while holding fs_info::balance_mutex.
*/
BTRFS_FS_BALANCE_RUNNING,
/* Indicate that the cleaner thread is awake and doing something. */
BTRFS_FS_CLEANER_RUNNING,
+
+ /*
+ * The checksumming has an optimized version and is considered fast,
+ * so we don't need to offload checksums to workqueues.
+ */
+ BTRFS_FS_CSUM_IMPL_FAST,
};
struct btrfs_fs_info {
struct extent_io_tree *pinned_extents;
/* logical->physical extent mapping */
- struct btrfs_mapping_tree mapping_tree;
+ struct extent_map_tree mapping_tree;
/*
* block reservation for extent, checksum, root tree and
spinlock_t swapfile_pins_lock;
struct rb_root swapfile_pins;
+ struct crypto_shash *csum_shash;
+
+ /*
+ * Number of send operations in progress.
+ * Updated while holding fs_info::balance_mutex.
+ */
+ int send_in_progress;
+
#ifdef CONFIG_BTRFS_FS_REF_VERIFY
spinlock_t ref_verify_lock;
struct rb_root block_tree;
return btrfs_csum_sizes[t];
}
+static inline const char *btrfs_super_csum_name(u16 csum_type)
+{
+ /* csum type is validated at mount time */
+ return btrfs_csum_names[csum_type];
+}
/*
* The leaf data grows from end-to-front in the node.
((unsigned long)(BTRFS_LEAF_DATA_OFFSET + \
btrfs_item_offset_nr(leaf, slot)))
+static inline u32 btrfs_crc32c(u32 crc, const void *address, unsigned length)
+{
+ return crc32c(crc, address, length);
+}
+
+static inline void btrfs_crc32c_final(u32 crc, u8 *result)
+{
+ put_unaligned_le32(~crc, result);
+}
+
static inline u64 btrfs_name_hash(const char *name, int len)
{
return crc32c((u32)~1, name, len);
return (u64) crc32c(parent_objectid, name, len);
}
-static inline bool btrfs_mixed_space_info(struct btrfs_space_info *space_info)
-{
- return ((space_info->flags & BTRFS_BLOCK_GROUP_METADATA) &&
- (space_info->flags & BTRFS_BLOCK_GROUP_DATA));
-}
-
static inline gfp_t btrfs_alloc_write_mask(struct address_space *mapping)
{
return mapping_gfp_constraint(mapping, ~__GFP_FS);
return (u64)fs_info->nodesize * BTRFS_MAX_LEVEL * num_items;
}
-int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans);
-bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info);
void btrfs_dec_block_group_reservations(struct btrfs_fs_info *fs_info,
const u64 start);
void btrfs_wait_block_group_reservations(struct btrfs_block_group_cache *bg);
COMMIT_TRANS = 9,
};
-int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes);
-int btrfs_check_data_free_space(struct inode *inode,
- struct extent_changeset **reserved, u64 start, u64 len);
-void btrfs_free_reserved_data_space(struct inode *inode,
- struct extent_changeset *reserved, u64 start, u64 len);
-void btrfs_delalloc_release_space(struct inode *inode,
- struct extent_changeset *reserved,
- u64 start, u64 len, bool qgroup_free);
-void btrfs_free_reserved_data_space_noquota(struct inode *inode, u64 start,
- u64 len);
-void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans);
+/*
+ * control flags for do_chunk_alloc's force field
+ * CHUNK_ALLOC_NO_FORCE means to only allocate a chunk
+ * if we really need one.
+ *
+ * CHUNK_ALLOC_LIMITED means to only try and allocate one
+ * if we have very few chunks already allocated. This is
+ * used as part of the clustering code to help make sure
+ * we have a good pool of storage to cluster in, without
+ * filling the FS with empty chunks
+ *
+ * CHUNK_ALLOC_FORCE means it must try to allocate one
+ *
+ */
+enum btrfs_chunk_alloc_enum {
+ CHUNK_ALLOC_NO_FORCE,
+ CHUNK_ALLOC_LIMITED,
+ CHUNK_ALLOC_FORCE,
+};
+
+int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
+ enum btrfs_chunk_alloc_enum force);
int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,
struct btrfs_block_rsv *rsv,
int nitems, bool use_global_rsv);
bool qgroup_free);
int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes);
-void btrfs_delalloc_release_metadata(struct btrfs_inode *inode, u64 num_bytes,
- bool qgroup_free);
-int btrfs_delalloc_reserve_space(struct inode *inode,
- struct extent_changeset **reserved, u64 start, u64 len);
-void btrfs_init_block_rsv(struct btrfs_block_rsv *rsv, unsigned short type);
-struct btrfs_block_rsv *btrfs_alloc_block_rsv(struct btrfs_fs_info *fs_info,
- unsigned short type);
-void btrfs_init_metadata_block_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *rsv,
- unsigned short type);
-void btrfs_free_block_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *rsv);
-int btrfs_block_rsv_add(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, u64 num_bytes,
- enum btrfs_reserve_flush_enum flush);
-int btrfs_block_rsv_check(struct btrfs_block_rsv *block_rsv, int min_factor);
-int btrfs_block_rsv_refill(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, u64 min_reserved,
- enum btrfs_reserve_flush_enum flush);
-int btrfs_block_rsv_migrate(struct btrfs_block_rsv *src_rsv,
- struct btrfs_block_rsv *dst_rsv, u64 num_bytes,
- bool update_size);
-int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *dest, u64 num_bytes,
- int min_factor);
-void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *block_rsv,
- u64 num_bytes);
-void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr);
-void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans);
-int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,
- enum btrfs_reserve_flush_enum flush);
-void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *src,
- u64 num_bytes);
int btrfs_inc_block_group_ro(struct btrfs_block_group_cache *cache);
void btrfs_dec_block_group_ro(struct btrfs_block_group_cache *cache);
void btrfs_put_block_group_cache(struct btrfs_fs_info *info);
struct btrfs_dio_private;
int btrfs_del_csums(struct btrfs_trans_handle *trans,
struct btrfs_fs_info *fs_info, u64 bytenr, u64 len);
-blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio, u32 *dst);
+blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio,
+ u8 *dst);
blk_status_t btrfs_lookup_bio_sums_dio(struct inode *inode, struct bio *bio,
u64 logical_offset);
int btrfs_insert_file_extent(struct btrfs_trans_handle *trans,
static inline void assfail(const char *expr, const char *file, int line)
{
if (IS_ENABLED(CONFIG_BTRFS_ASSERT)) {
- pr_err("assertion failed: %s, file: %s, line: %d\n",
- expr, file, line);
+ pr_err("assertion failed: %s, in %s:%d\n", expr, file, line);
BUG();
}
}
/* compatibility and incompatibility defines */
#define btrfs_set_fs_incompat(__fs_info, opt) \
- __btrfs_set_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt)
+ __btrfs_set_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt, \
+ #opt)
static inline void __btrfs_set_fs_incompat(struct btrfs_fs_info *fs_info,
- u64 flag)
+ u64 flag, const char* name)
{
struct btrfs_super_block *disk_super;
u64 features;
if (!(features & flag)) {
features |= flag;
btrfs_set_super_incompat_flags(disk_super, features);
- btrfs_info(fs_info, "setting %llu feature flag",
- flag);
+ btrfs_info(fs_info,
+ "setting incompat feature flag for %s (0x%llx)",
+ name, flag);
}
spin_unlock(&fs_info->super_lock);
}
}
#define btrfs_clear_fs_incompat(__fs_info, opt) \
- __btrfs_clear_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt)
+ __btrfs_clear_fs_incompat((__fs_info), BTRFS_FEATURE_INCOMPAT_##opt, \
+ #opt)
static inline void __btrfs_clear_fs_incompat(struct btrfs_fs_info *fs_info,
- u64 flag)
+ u64 flag, const char* name)
{
struct btrfs_super_block *disk_super;
u64 features;
if (features & flag) {
features &= ~flag;
btrfs_set_super_incompat_flags(disk_super, features);
- btrfs_info(fs_info, "clearing %llu feature flag",
- flag);
+ btrfs_info(fs_info,
+ "clearing incompat feature flag for %s (0x%llx)",
+ name, flag);
}
spin_unlock(&fs_info->super_lock);
}
}
#define btrfs_set_fs_compat_ro(__fs_info, opt) \
- __btrfs_set_fs_compat_ro((__fs_info), BTRFS_FEATURE_COMPAT_RO_##opt)
+ __btrfs_set_fs_compat_ro((__fs_info), BTRFS_FEATURE_COMPAT_RO_##opt, \
+ #opt)
static inline void __btrfs_set_fs_compat_ro(struct btrfs_fs_info *fs_info,
- u64 flag)
+ u64 flag, const char *name)
{
struct btrfs_super_block *disk_super;
u64 features;
if (!(features & flag)) {
features |= flag;
btrfs_set_super_compat_ro_flags(disk_super, features);
- btrfs_info(fs_info, "setting %llu ro feature flag",
- flag);
+ btrfs_info(fs_info,
+ "setting compat-ro feature flag for %s (0x%llx)",
+ name, flag);
}
spin_unlock(&fs_info->super_lock);
}
}
#define btrfs_clear_fs_compat_ro(__fs_info, opt) \
- __btrfs_clear_fs_compat_ro((__fs_info), BTRFS_FEATURE_COMPAT_RO_##opt)
+ __btrfs_clear_fs_compat_ro((__fs_info), BTRFS_FEATURE_COMPAT_RO_##opt, \
+ #opt)
static inline void __btrfs_clear_fs_compat_ro(struct btrfs_fs_info *fs_info,
- u64 flag)
+ u64 flag, const char *name)
{
struct btrfs_super_block *disk_super;
u64 features;
if (features & flag) {
features &= ~flag;
btrfs_set_super_compat_ro_flags(disk_super, features);
- btrfs_info(fs_info, "clearing %llu ro feature flag",
- flag);
+ btrfs_info(fs_info,
+ "clearing compat-ro feature flag for %s (0x%llx)",
+ name, flag);
}
spin_unlock(&fs_info->super_lock);
}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include "ctree.h"
+#include "delalloc-space.h"
+#include "block-rsv.h"
+#include "btrfs_inode.h"
+#include "space-info.h"
+#include "transaction.h"
+#include "qgroup.h"
+
+int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes)
+{
+ struct btrfs_root *root = inode->root;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_space_info *data_sinfo = fs_info->data_sinfo;
+ u64 used;
+ int ret = 0;
+ int need_commit = 2;
+ int have_pinned_space;
+
+ /* Make sure bytes are sectorsize aligned */
+ bytes = ALIGN(bytes, fs_info->sectorsize);
+
+ if (btrfs_is_free_space_inode(inode)) {
+ need_commit = 0;
+ ASSERT(current->journal_info);
+ }
+
+again:
+ /* Make sure we have enough space to handle the data first */
+ spin_lock(&data_sinfo->lock);
+ used = btrfs_space_info_used(data_sinfo, true);
+
+ if (used + bytes > data_sinfo->total_bytes) {
+ struct btrfs_trans_handle *trans;
+
+ /*
+ * If we don't have enough free bytes in this space then we need
+ * to alloc a new chunk.
+ */
+ if (!data_sinfo->full) {
+ u64 alloc_target;
+
+ data_sinfo->force_alloc = CHUNK_ALLOC_FORCE;
+ spin_unlock(&data_sinfo->lock);
+
+ alloc_target = btrfs_data_alloc_profile(fs_info);
+ /*
+ * It is ugly that we don't call nolock join
+ * transaction for the free space inode case here.
+ * But it is safe because we only do the data space
+ * reservation for the free space cache in the
+ * transaction context, the common join transaction
+ * just increase the counter of the current transaction
+ * handler, doesn't try to acquire the trans_lock of
+ * the fs.
+ */
+ trans = btrfs_join_transaction(root);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+
+ ret = btrfs_chunk_alloc(trans, alloc_target,
+ CHUNK_ALLOC_NO_FORCE);
+ btrfs_end_transaction(trans);
+ if (ret < 0) {
+ if (ret != -ENOSPC)
+ return ret;
+ else {
+ have_pinned_space = 1;
+ goto commit_trans;
+ }
+ }
+
+ goto again;
+ }
+
+ /*
+ * If we don't have enough pinned space to deal with this
+ * allocation, and no removed chunk in current transaction,
+ * don't bother committing the transaction.
+ */
+ have_pinned_space = __percpu_counter_compare(
+ &data_sinfo->total_bytes_pinned,
+ used + bytes - data_sinfo->total_bytes,
+ BTRFS_TOTAL_BYTES_PINNED_BATCH);
+ spin_unlock(&data_sinfo->lock);
+
+ /* Commit the current transaction and try again */
+commit_trans:
+ if (need_commit) {
+ need_commit--;
+
+ if (need_commit > 0) {
+ btrfs_start_delalloc_roots(fs_info, -1);
+ btrfs_wait_ordered_roots(fs_info, U64_MAX, 0,
+ (u64)-1);
+ }
+
+ trans = btrfs_join_transaction(root);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+ if (have_pinned_space >= 0 ||
+ test_bit(BTRFS_TRANS_HAVE_FREE_BGS,
+ &trans->transaction->flags) ||
+ need_commit > 0) {
+ ret = btrfs_commit_transaction(trans);
+ if (ret)
+ return ret;
+ /*
+ * The cleaner kthread might still be doing iput
+ * operations. Wait for it to finish so that
+ * more space is released. We don't need to
+ * explicitly run the delayed iputs here because
+ * the commit_transaction would have woken up
+ * the cleaner.
+ */
+ ret = btrfs_wait_on_delayed_iputs(fs_info);
+ if (ret)
+ return ret;
+ goto again;
+ } else {
+ btrfs_end_transaction(trans);
+ }
+ }
+
+ trace_btrfs_space_reservation(fs_info,
+ "space_info:enospc",
+ data_sinfo->flags, bytes, 1);
+ return -ENOSPC;
+ }
+ btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, bytes);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ data_sinfo->flags, bytes, 1);
+ spin_unlock(&data_sinfo->lock);
+
+ return 0;
+}
+
+int btrfs_check_data_free_space(struct inode *inode,
+ struct extent_changeset **reserved, u64 start, u64 len)
+{
+ struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
+ int ret;
+
+ /* align the range */
+ len = round_up(start + len, fs_info->sectorsize) -
+ round_down(start, fs_info->sectorsize);
+ start = round_down(start, fs_info->sectorsize);
+
+ ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode), len);
+ if (ret < 0)
+ return ret;
+
+ /* Use new btrfs_qgroup_reserve_data to reserve precious data space. */
+ ret = btrfs_qgroup_reserve_data(inode, reserved, start, len);
+ if (ret < 0)
+ btrfs_free_reserved_data_space_noquota(inode, start, len);
+ else
+ ret = 0;
+ return ret;
+}
+
+/*
+ * Called if we need to clear a data reservation for this inode
+ * Normally in a error case.
+ *
+ * This one will *NOT* use accurate qgroup reserved space API, just for case
+ * which we can't sleep and is sure it won't affect qgroup reserved space.
+ * Like clear_bit_hook().
+ */
+void btrfs_free_reserved_data_space_noquota(struct inode *inode, u64 start,
+ u64 len)
+{
+ struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
+ struct btrfs_space_info *data_sinfo;
+
+ /* Make sure the range is aligned to sectorsize */
+ len = round_up(start + len, fs_info->sectorsize) -
+ round_down(start, fs_info->sectorsize);
+ start = round_down(start, fs_info->sectorsize);
+
+ data_sinfo = fs_info->data_sinfo;
+ spin_lock(&data_sinfo->lock);
+ btrfs_space_info_update_bytes_may_use(fs_info, data_sinfo, -len);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ data_sinfo->flags, len, 0);
+ spin_unlock(&data_sinfo->lock);
+}
+
+/*
+ * Called if we need to clear a data reservation for this inode
+ * Normally in a error case.
+ *
+ * This one will handle the per-inode data rsv map for accurate reserved
+ * space framework.
+ */
+void btrfs_free_reserved_data_space(struct inode *inode,
+ struct extent_changeset *reserved, u64 start, u64 len)
+{
+ struct btrfs_root *root = BTRFS_I(inode)->root;
+
+ /* Make sure the range is aligned to sectorsize */
+ len = round_up(start + len, root->fs_info->sectorsize) -
+ round_down(start, root->fs_info->sectorsize);
+ start = round_down(start, root->fs_info->sectorsize);
+
+ btrfs_free_reserved_data_space_noquota(inode, start, len);
+ btrfs_qgroup_free_data(inode, reserved, start, len);
+}
+
+/**
+ * btrfs_inode_rsv_release - release any excessive reservation.
+ * @inode - the inode we need to release from.
+ * @qgroup_free - free or convert qgroup meta.
+ * Unlike normal operation, qgroup meta reservation needs to know if we are
+ * freeing qgroup reservation or just converting it into per-trans. Normally
+ * @qgroup_free is true for error handling, and false for normal release.
+ *
+ * This is the same as btrfs_block_rsv_release, except that it handles the
+ * tracepoint for the reservation.
+ */
+static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free)
+{
+ struct btrfs_fs_info *fs_info = inode->root->fs_info;
+ struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
+ u64 released = 0;
+ u64 qgroup_to_release = 0;
+
+ /*
+ * Since we statically set the block_rsv->size we just want to say we
+ * are releasing 0 bytes, and then we'll just get the reservation over
+ * the size free'd.
+ */
+ released = __btrfs_block_rsv_release(fs_info, block_rsv, 0,
+ &qgroup_to_release);
+ if (released > 0)
+ trace_btrfs_space_reservation(fs_info, "delalloc",
+ btrfs_ino(inode), released, 0);
+ if (qgroup_free)
+ btrfs_qgroup_free_meta_prealloc(inode->root, qgroup_to_release);
+ else
+ btrfs_qgroup_convert_reserved_meta(inode->root,
+ qgroup_to_release);
+}
+
+static void btrfs_calculate_inode_block_rsv_size(struct btrfs_fs_info *fs_info,
+ struct btrfs_inode *inode)
+{
+ struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
+ u64 reserve_size = 0;
+ u64 qgroup_rsv_size = 0;
+ u64 csum_leaves;
+ unsigned outstanding_extents;
+
+ lockdep_assert_held(&inode->lock);
+ outstanding_extents = inode->outstanding_extents;
+ if (outstanding_extents)
+ reserve_size = btrfs_calc_trans_metadata_size(fs_info,
+ outstanding_extents + 1);
+ csum_leaves = btrfs_csum_bytes_to_leaves(fs_info,
+ inode->csum_bytes);
+ reserve_size += btrfs_calc_trans_metadata_size(fs_info,
+ csum_leaves);
+ /*
+ * For qgroup rsv, the calculation is very simple:
+ * account one nodesize for each outstanding extent
+ *
+ * This is overestimating in most cases.
+ */
+ qgroup_rsv_size = (u64)outstanding_extents * fs_info->nodesize;
+
+ spin_lock(&block_rsv->lock);
+ block_rsv->size = reserve_size;
+ block_rsv->qgroup_rsv_size = qgroup_rsv_size;
+ spin_unlock(&block_rsv->lock);
+}
+
+static void calc_inode_reservations(struct btrfs_fs_info *fs_info,
+ u64 num_bytes, u64 *meta_reserve,
+ u64 *qgroup_reserve)
+{
+ u64 nr_extents = count_max_extents(num_bytes);
+ u64 csum_leaves = btrfs_csum_bytes_to_leaves(fs_info, num_bytes);
+
+ /* We add one for the inode update at finish ordered time */
+ *meta_reserve = btrfs_calc_trans_metadata_size(fs_info,
+ nr_extents + csum_leaves + 1);
+ *qgroup_reserve = nr_extents * fs_info->nodesize;
+}
+
+int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes)
+{
+ struct btrfs_root *root = inode->root;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
+ u64 meta_reserve, qgroup_reserve;
+ unsigned nr_extents;
+ enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_ALL;
+ int ret = 0;
+ bool delalloc_lock = true;
+
+ /*
+ * If we are a free space inode we need to not flush since we will be in
+ * the middle of a transaction commit. We also don't need the delalloc
+ * mutex since we won't race with anybody. We need this mostly to make
+ * lockdep shut its filthy mouth.
+ *
+ * If we have a transaction open (can happen if we call truncate_block
+ * from truncate), then we need FLUSH_LIMIT so we don't deadlock.
+ */
+ if (btrfs_is_free_space_inode(inode)) {
+ flush = BTRFS_RESERVE_NO_FLUSH;
+ delalloc_lock = false;
+ } else {
+ if (current->journal_info)
+ flush = BTRFS_RESERVE_FLUSH_LIMIT;
+
+ if (btrfs_transaction_in_commit(fs_info))
+ schedule_timeout(1);
+ }
+
+ if (delalloc_lock)
+ mutex_lock(&inode->delalloc_mutex);
+
+ num_bytes = ALIGN(num_bytes, fs_info->sectorsize);
+
+ /*
+ * We always want to do it this way, every other way is wrong and ends
+ * in tears. Pre-reserving the amount we are going to add will always
+ * be the right way, because otherwise if we have enough parallelism we
+ * could end up with thousands of inodes all holding little bits of
+ * reservations they were able to make previously and the only way to
+ * reclaim that space is to ENOSPC out the operations and clear
+ * everything out and try again, which is bad. This way we just
+ * over-reserve slightly, and clean up the mess when we are done.
+ */
+ calc_inode_reservations(fs_info, num_bytes, &meta_reserve,
+ &qgroup_reserve);
+ ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_reserve, true);
+ if (ret)
+ goto out_fail;
+ ret = btrfs_reserve_metadata_bytes(root, block_rsv, meta_reserve, flush);
+ if (ret)
+ goto out_qgroup;
+
+ /*
+ * Now we need to update our outstanding extents and csum bytes _first_
+ * and then add the reservation to the block_rsv. This keeps us from
+ * racing with an ordered completion or some such that would think it
+ * needs to free the reservation we just made.
+ */
+ spin_lock(&inode->lock);
+ nr_extents = count_max_extents(num_bytes);
+ btrfs_mod_outstanding_extents(inode, nr_extents);
+ inode->csum_bytes += num_bytes;
+ btrfs_calculate_inode_block_rsv_size(fs_info, inode);
+ spin_unlock(&inode->lock);
+
+ /* Now we can safely add our space to our block rsv */
+ btrfs_block_rsv_add_bytes(block_rsv, meta_reserve, false);
+ trace_btrfs_space_reservation(root->fs_info, "delalloc",
+ btrfs_ino(inode), meta_reserve, 1);
+
+ spin_lock(&block_rsv->lock);
+ block_rsv->qgroup_rsv_reserved += qgroup_reserve;
+ spin_unlock(&block_rsv->lock);
+
+ if (delalloc_lock)
+ mutex_unlock(&inode->delalloc_mutex);
+ return 0;
+out_qgroup:
+ btrfs_qgroup_free_meta_prealloc(root, qgroup_reserve);
+out_fail:
+ btrfs_inode_rsv_release(inode, true);
+ if (delalloc_lock)
+ mutex_unlock(&inode->delalloc_mutex);
+ return ret;
+}
+
+/**
+ * btrfs_delalloc_release_metadata - release a metadata reservation for an inode
+ * @inode: the inode to release the reservation for.
+ * @num_bytes: the number of bytes we are releasing.
+ * @qgroup_free: free qgroup reservation or convert it to per-trans reservation
+ *
+ * This will release the metadata reservation for an inode. This can be called
+ * once we complete IO for a given set of bytes to release their metadata
+ * reservations, or on error for the same reason.
+ */
+void btrfs_delalloc_release_metadata(struct btrfs_inode *inode, u64 num_bytes,
+ bool qgroup_free)
+{
+ struct btrfs_fs_info *fs_info = inode->root->fs_info;
+
+ num_bytes = ALIGN(num_bytes, fs_info->sectorsize);
+ spin_lock(&inode->lock);
+ inode->csum_bytes -= num_bytes;
+ btrfs_calculate_inode_block_rsv_size(fs_info, inode);
+ spin_unlock(&inode->lock);
+
+ if (btrfs_is_testing(fs_info))
+ return;
+
+ btrfs_inode_rsv_release(inode, qgroup_free);
+}
+
+/**
+ * btrfs_delalloc_release_extents - release our outstanding_extents
+ * @inode: the inode to balance the reservation for.
+ * @num_bytes: the number of bytes we originally reserved with
+ * @qgroup_free: do we need to free qgroup meta reservation or convert them.
+ *
+ * When we reserve space we increase outstanding_extents for the extents we may
+ * add. Once we've set the range as delalloc or created our ordered extents we
+ * have outstanding_extents to track the real usage, so we use this to free our
+ * temporarily tracked outstanding_extents. This _must_ be used in conjunction
+ * with btrfs_delalloc_reserve_metadata.
+ */
+void btrfs_delalloc_release_extents(struct btrfs_inode *inode, u64 num_bytes,
+ bool qgroup_free)
+{
+ struct btrfs_fs_info *fs_info = inode->root->fs_info;
+ unsigned num_extents;
+
+ spin_lock(&inode->lock);
+ num_extents = count_max_extents(num_bytes);
+ btrfs_mod_outstanding_extents(inode, -num_extents);
+ btrfs_calculate_inode_block_rsv_size(fs_info, inode);
+ spin_unlock(&inode->lock);
+
+ if (btrfs_is_testing(fs_info))
+ return;
+
+ btrfs_inode_rsv_release(inode, qgroup_free);
+}
+
+/**
+ * btrfs_delalloc_reserve_space - reserve data and metadata space for
+ * delalloc
+ * @inode: inode we're writing to
+ * @start: start range we are writing to
+ * @len: how long the range we are writing to
+ * @reserved: mandatory parameter, record actually reserved qgroup ranges of
+ * current reservation.
+ *
+ * This will do the following things
+ *
+ * - reserve space in data space info for num bytes
+ * and reserve precious corresponding qgroup space
+ * (Done in check_data_free_space)
+ *
+ * - reserve space for metadata space, based on the number of outstanding
+ * extents and how much csums will be needed
+ * also reserve metadata space in a per root over-reserve method.
+ * - add to the inodes->delalloc_bytes
+ * - add it to the fs_info's delalloc inodes list.
+ * (Above 3 all done in delalloc_reserve_metadata)
+ *
+ * Return 0 for success
+ * Return <0 for error(-ENOSPC or -EQUOT)
+ */
+int btrfs_delalloc_reserve_space(struct inode *inode,
+ struct extent_changeset **reserved, u64 start, u64 len)
+{
+ int ret;
+
+ ret = btrfs_check_data_free_space(inode, reserved, start, len);
+ if (ret < 0)
+ return ret;
+ ret = btrfs_delalloc_reserve_metadata(BTRFS_I(inode), len);
+ if (ret < 0)
+ btrfs_free_reserved_data_space(inode, *reserved, start, len);
+ return ret;
+}
+
+/**
+ * btrfs_delalloc_release_space - release data and metadata space for delalloc
+ * @inode: inode we're releasing space for
+ * @start: start position of the space already reserved
+ * @len: the len of the space already reserved
+ * @release_bytes: the len of the space we consumed or didn't use
+ *
+ * This function will release the metadata space that was not used and will
+ * decrement ->delalloc_bytes and remove it from the fs_info delalloc_inodes
+ * list if there are no delalloc bytes left.
+ * Also it will handle the qgroup reserved space.
+ */
+void btrfs_delalloc_release_space(struct inode *inode,
+ struct extent_changeset *reserved,
+ u64 start, u64 len, bool qgroup_free)
+{
+ btrfs_delalloc_release_metadata(BTRFS_I(inode), len, qgroup_free);
+ btrfs_free_reserved_data_space(inode, reserved, start, len);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef BTRFS_DELALLOC_SPACE_H
+#define BTRFS_DELALLOC_SPACE_H
+
+struct extent_changeset;
+
+int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes);
+int btrfs_check_data_free_space(struct inode *inode,
+ struct extent_changeset **reserved, u64 start, u64 len);
+void btrfs_free_reserved_data_space(struct inode *inode,
+ struct extent_changeset *reserved, u64 start, u64 len);
+void btrfs_delalloc_release_space(struct inode *inode,
+ struct extent_changeset *reserved,
+ u64 start, u64 len, bool qgroup_free);
+void btrfs_free_reserved_data_space_noquota(struct inode *inode, u64 start,
+ u64 len);
+void btrfs_delalloc_release_metadata(struct btrfs_inode *inode, u64 num_bytes,
+ bool qgroup_free);
+int btrfs_delalloc_reserve_space(struct inode *inode,
+ struct extent_changeset **reserved, u64 start, u64 len);
+
+#endif /* BTRFS_DELALLOC_SPACE_H */
#include "delayed-ref.h"
#include "transaction.h"
#include "qgroup.h"
+#include "space-info.h"
struct kmem_cache *btrfs_delayed_ref_head_cachep;
struct kmem_cache *btrfs_delayed_tree_ref_cachep;
* of hammering updates on the extent allocation tree.
*/
+bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ bool ret = false;
+ u64 reserved;
+
+ spin_lock(&global_rsv->lock);
+ reserved = global_rsv->reserved;
+ spin_unlock(&global_rsv->lock);
+
+ /*
+ * Since the global reserve is just kind of magic we don't really want
+ * to rely on it to save our bacon, so if our size is more than the
+ * delayed_refs_rsv and the global rsv then it's time to think about
+ * bailing.
+ */
+ spin_lock(&delayed_refs_rsv->lock);
+ reserved += delayed_refs_rsv->reserved;
+ if (delayed_refs_rsv->size >= reserved)
+ ret = true;
+ spin_unlock(&delayed_refs_rsv->lock);
+ return ret;
+}
+
+int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans)
+{
+ u64 num_entries =
+ atomic_read(&trans->transaction->delayed_refs.num_entries);
+ u64 avg_runtime;
+ u64 val;
+
+ smp_mb();
+ avg_runtime = trans->fs_info->avg_delayed_ref_runtime;
+ val = num_entries * avg_runtime;
+ if (val >= NSEC_PER_SEC)
+ return 1;
+ if (val >= NSEC_PER_SEC / 2)
+ return 2;
+
+ return btrfs_check_space_for_delayed_refs(trans->fs_info);
+}
+
+/**
+ * btrfs_delayed_refs_rsv_release - release a ref head's reservation.
+ * @fs_info - the fs_info for our fs.
+ * @nr - the number of items to drop.
+ *
+ * This drops the delayed ref head's count from the delayed refs rsv and frees
+ * any excess reservation we had.
+ */
+void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr)
+{
+ struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv;
+ u64 num_bytes = btrfs_calc_trans_metadata_size(fs_info, nr);
+ u64 released = 0;
+
+ released = __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes,
+ NULL);
+ if (released)
+ trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
+ 0, released, 0);
+}
+
+/*
+ * btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv
+ * @trans - the trans that may have generated delayed refs
+ *
+ * This is to be called anytime we may have adjusted trans->delayed_ref_updates,
+ * it'll calculate the additional size and add it to the delayed_refs_rsv.
+ */
+void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv;
+ u64 num_bytes;
+
+ if (!trans->delayed_ref_updates)
+ return;
+
+ num_bytes = btrfs_calc_trans_metadata_size(fs_info,
+ trans->delayed_ref_updates);
+ spin_lock(&delayed_rsv->lock);
+ delayed_rsv->size += num_bytes;
+ delayed_rsv->full = 0;
+ spin_unlock(&delayed_rsv->lock);
+ trans->delayed_ref_updates = 0;
+}
+
+/**
+ * btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv.
+ * @fs_info - the fs info for our fs.
+ * @src - the source block rsv to transfer from.
+ * @num_bytes - the number of bytes to transfer.
+ *
+ * This transfers up to the num_bytes amount from the src rsv to the
+ * delayed_refs_rsv. Any extra bytes are returned to the space info.
+ */
+void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *src,
+ u64 num_bytes)
+{
+ struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
+ u64 to_free = 0;
+
+ spin_lock(&src->lock);
+ src->reserved -= num_bytes;
+ src->size -= num_bytes;
+ spin_unlock(&src->lock);
+
+ spin_lock(&delayed_refs_rsv->lock);
+ if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) {
+ u64 delta = delayed_refs_rsv->size -
+ delayed_refs_rsv->reserved;
+ if (num_bytes > delta) {
+ to_free = num_bytes - delta;
+ num_bytes = delta;
+ }
+ } else {
+ to_free = num_bytes;
+ num_bytes = 0;
+ }
+
+ if (num_bytes)
+ delayed_refs_rsv->reserved += num_bytes;
+ if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size)
+ delayed_refs_rsv->full = 1;
+ spin_unlock(&delayed_refs_rsv->lock);
+
+ if (num_bytes)
+ trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
+ 0, num_bytes, 1);
+ if (to_free)
+ btrfs_space_info_add_old_bytes(fs_info,
+ delayed_refs_rsv->space_info, to_free);
+}
+
+/**
+ * btrfs_delayed_refs_rsv_refill - refill based on our delayed refs usage.
+ * @fs_info - the fs_info for our fs.
+ * @flush - control how we can flush for this reservation.
+ *
+ * This will refill the delayed block_rsv up to 1 items size worth of space and
+ * will return -ENOSPC if we can't make the reservation.
+ */
+int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,
+ enum btrfs_reserve_flush_enum flush)
+{
+ struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv;
+ u64 limit = btrfs_calc_trans_metadata_size(fs_info, 1);
+ u64 num_bytes = 0;
+ int ret = -ENOSPC;
+
+ spin_lock(&block_rsv->lock);
+ if (block_rsv->reserved < block_rsv->size) {
+ num_bytes = block_rsv->size - block_rsv->reserved;
+ num_bytes = min(num_bytes, limit);
+ }
+ spin_unlock(&block_rsv->lock);
+
+ if (!num_bytes)
+ return 0;
+
+ ret = btrfs_reserve_metadata_bytes(fs_info->extent_root, block_rsv,
+ num_bytes, flush);
+ if (ret)
+ return ret;
+ btrfs_block_rsv_add_bytes(block_rsv, num_bytes, 0);
+ trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
+ 0, num_bytes, 1);
+ return 0;
+}
+
/*
* compare two delayed tree backrefs with same bytenr and type
*/
}
/*
- * this does a simple search for the head node for a given extent.
- * It must be called with the delayed ref spinlock held, and it returns
- * the head node if any where found, or NULL if not.
+ * This does a simple search for the head node for a given extent. Returns the
+ * head node if found, or NULL if not.
*/
struct btrfs_delayed_ref_head *
btrfs_find_delayed_ref_head(struct btrfs_delayed_ref_root *delayed_refs, u64 bytenr)
{
+ lockdep_assert_held(&delayed_refs->lock);
+
return find_ref_head(delayed_refs, bytenr, false);
}
int btrfs_check_delayed_seq(struct btrfs_fs_info *fs_info, u64 seq);
+void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr);
+void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans);
+int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,
+ enum btrfs_reserve_flush_enum flush);
+void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *src,
+ u64 num_bytes);
+int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans);
+bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info);
+
/*
* helper functions to cast a node into its container
*/
return PTR_ERR(bdev);
}
- filemap_write_and_wait(bdev->bd_inode->i_mapping);
+ sync_blockdev(bdev);
devices = &fs_info->fs_devices->devices;
list_for_each_entry(device, devices, dev_list) {
}
rcu_assign_pointer(device->name, name);
- mutex_lock(&fs_info->fs_devices->device_list_mutex);
set_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state);
device->generation = 0;
device->io_width = fs_info->sectorsize;
device->dev_stats_valid = 1;
set_blocksize(device->bdev, BTRFS_BDEV_BLOCKSIZE);
device->fs_devices = fs_info->fs_devices;
+
+ mutex_lock(&fs_info->fs_devices->device_list_mutex);
list_add(&device->dev_list, &fs_info->fs_devices->devices);
fs_info->fs_devices->num_devices++;
fs_info->fs_devices->open_devices++;
int ret;
struct btrfs_device *tgt_device = NULL;
struct btrfs_device *src_device = NULL;
- bool need_unlock;
src_device = btrfs_find_device_by_devspec(fs_info, srcdevid,
srcdev_name);
return -ETXTBSY;
}
- ret = btrfs_init_dev_replace_tgtdev(fs_info, tgtdev_name,
- src_device, &tgt_device);
- if (ret)
- return ret;
-
/*
* Here we commit the transaction to make sure commit_total_bytes
* of all the devices are updated.
return PTR_ERR(trans);
}
- need_unlock = true;
+ ret = btrfs_init_dev_replace_tgtdev(fs_info, tgtdev_name,
+ src_device, &tgt_device);
+ if (ret)
+ return ret;
+
down_write(&dev_replace->rwsem);
switch (dev_replace->replace_state) {
case BTRFS_IOCTL_DEV_REPLACE_STATE_NEVER_STARTED:
case BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED:
ASSERT(0);
ret = BTRFS_IOCTL_DEV_REPLACE_RESULT_ALREADY_STARTED;
+ up_write(&dev_replace->rwsem);
goto leave;
}
dev_replace->cont_reading_from_srcdev_mode = read_src;
- WARN_ON(!src_device);
dev_replace->srcdev = src_device;
dev_replace->tgtdev = tgt_device;
atomic64_set(&dev_replace->num_write_errors, 0);
atomic64_set(&dev_replace->num_uncorrectable_read_errors, 0);
up_write(&dev_replace->rwsem);
- need_unlock = false;
ret = btrfs_sysfs_add_device_link(tgt_device->fs_devices, tgt_device);
if (ret)
btrfs_wait_ordered_roots(fs_info, U64_MAX, 0, (u64)-1);
- /* force writing the updated state information to disk */
- trans = btrfs_start_transaction(root, 0);
+ /* Commit dev_replace state and reserve 1 item for it. */
+ trans = btrfs_start_transaction(root, 1);
if (IS_ERR(trans)) {
ret = PTR_ERR(trans);
- need_unlock = true;
down_write(&dev_replace->rwsem);
dev_replace->replace_state =
BTRFS_IOCTL_DEV_REPLACE_STATE_NEVER_STARTED;
dev_replace->srcdev = NULL;
dev_replace->tgtdev = NULL;
+ up_write(&dev_replace->rwsem);
goto leave;
}
return ret;
leave:
- if (need_unlock)
- up_write(&dev_replace->rwsem);
btrfs_destroy_dev_replace_tgtdev(tgt_device);
return ret;
}
btrfs_device_set_disk_total_bytes(tgt_device,
src_device->disk_total_bytes);
btrfs_device_set_bytes_used(tgt_device, src_device->bytes_used);
- tgt_device->commit_total_bytes = src_device->commit_total_bytes;
tgt_device->commit_bytes_used = src_device->bytes_used;
btrfs_assign_next_active_device(src_device, tgt_device);
struct btrfs_device *srcdev,
struct btrfs_device *tgtdev)
{
- struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
+ struct extent_map_tree *em_tree = &fs_info->mapping_tree;
struct extent_map *em;
struct map_lookup *map;
u64 start = 0;
#include <linux/crc32c.h>
#include <linux/sched/mm.h>
#include <asm/unaligned.h>
+#include <crypto/hash.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
#include "tree-checker.h"
#include "ref-verify.h"
-#ifdef CONFIG_X86
-#include <asm/cpufeature.h>
-#endif
-
#define BTRFS_SUPER_FLAG_SUPP (BTRFS_HEADER_FLAG_WRITTEN |\
BTRFS_HEADER_FLAG_RELOC |\
BTRFS_SUPER_FLAG_ERROR |\
return em;
}
-u32 btrfs_csum_data(const char *data, u32 seed, size_t len)
-{
- return crc32c(seed, data, len);
-}
-
-void btrfs_csum_final(u32 crc, u8 *result)
-{
- put_unaligned_le32(~crc, result);
-}
-
/*
* Compute the csum of a btree block and store the result to provided buffer.
*
*/
static int csum_tree_block(struct extent_buffer *buf, u8 *result)
{
+ struct btrfs_fs_info *fs_info = buf->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
unsigned long len;
unsigned long cur_len;
unsigned long offset = BTRFS_CSUM_SIZE;
unsigned long map_start;
unsigned long map_len;
int err;
- u32 crc = ~(u32)0;
+
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
len = buf->len - offset;
+
while (len > 0) {
/*
* Note: we don't need to check for the err == 1 case here, as
if (WARN_ON(err))
return err;
cur_len = min(len, map_len - (offset - map_start));
- crc = btrfs_csum_data(kaddr + offset - map_start,
- crc, cur_len);
+ crypto_shash_update(shash, kaddr + offset - map_start, cur_len);
len -= cur_len;
offset += cur_len;
}
memset(result, 0, BTRFS_CSUM_SIZE);
- btrfs_csum_final(crc, result);
+ crypto_shash_final(shash, result);
return 0;
}
return ret;
}
+static bool btrfs_supported_super_csum(u16 csum_type)
+{
+ switch (csum_type) {
+ case BTRFS_CSUM_TYPE_CRC32:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* Return 0 if the superblock checksum type matches the checksum value of that
* algorithm. Pass the raw disk superblock data.
{
struct btrfs_super_block *disk_sb =
(struct btrfs_super_block *)raw_disk_sb;
- u16 csum_type = btrfs_super_csum_type(disk_sb);
- int ret = 0;
+ char result[BTRFS_CSUM_SIZE];
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
- if (csum_type == BTRFS_CSUM_TYPE_CRC32) {
- u32 crc = ~(u32)0;
- char result[sizeof(crc)];
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
- /*
- * The super_block structure does not span the whole
- * BTRFS_SUPER_INFO_SIZE range, we expect that the unused space
- * is filled with zeros and is included in the checksum.
- */
- crc = btrfs_csum_data(raw_disk_sb + BTRFS_CSUM_SIZE,
- crc, BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
- btrfs_csum_final(crc, result);
+ /*
+ * The super_block structure does not span the whole
+ * BTRFS_SUPER_INFO_SIZE range, we expect that the unused space is
+ * filled with zeros and is included in the checksum.
+ */
+ crypto_shash_update(shash, raw_disk_sb + BTRFS_CSUM_SIZE,
+ BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
+ crypto_shash_final(shash, result);
- if (memcmp(raw_disk_sb, result, sizeof(result)))
- ret = 1;
- }
+ if (memcmp(disk_sb->csum, result, btrfs_super_csum_size(disk_sb)))
+ return 1;
- if (csum_type >= ARRAY_SIZE(btrfs_csum_sizes)) {
- btrfs_err(fs_info, "unsupported checksum algorithm %u",
- csum_type);
- ret = 1;
- }
-
- return ret;
+ return 0;
}
int btrfs_verify_level_key(struct extent_buffer *eb, int level,
return btree_csum_one_bio(bio);
}
-static int check_async_write(struct btrfs_inode *bi)
+static int check_async_write(struct btrfs_fs_info *fs_info,
+ struct btrfs_inode *bi)
{
if (atomic_read(&bi->sync_writers))
return 0;
-#ifdef CONFIG_X86
- if (static_cpu_has(X86_FEATURE_XMM4_2))
+ if (test_bit(BTRFS_FS_CSUM_IMPL_FAST, &fs_info->flags))
return 0;
-#endif
return 1;
}
unsigned long bio_flags)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
- int async = check_async_write(BTRFS_I(inode));
+ int async = check_async_write(fs_info, BTRFS_I(inode));
blk_status_t ret;
if (bio_op(bio) != REQ_OP_WRITE) {
return 0;
}
+static int btrfs_init_csum_hash(struct btrfs_fs_info *fs_info, u16 csum_type)
+{
+ struct crypto_shash *csum_shash;
+ const char *csum_name = btrfs_super_csum_name(csum_type);
+
+ csum_shash = crypto_alloc_shash(csum_name, 0, 0);
+
+ if (IS_ERR(csum_shash)) {
+ btrfs_err(fs_info, "error allocating %s hash for checksum",
+ csum_name);
+ return PTR_ERR(csum_shash);
+ }
+
+ fs_info->csum_shash = csum_shash;
+
+ return 0;
+}
+
+static void btrfs_free_csum_hash(struct btrfs_fs_info *fs_info)
+{
+ crypto_free_shash(fs_info->csum_shash);
+}
+
static int btrfs_replay_log(struct btrfs_fs_info *fs_info,
struct btrfs_fs_devices *fs_devices)
{
ret = validate_super(fs_info, sb, -1);
if (ret < 0)
goto out;
- if (btrfs_super_csum_type(sb) != BTRFS_CSUM_TYPE_CRC32) {
+ if (!btrfs_supported_super_csum(btrfs_super_csum_type(sb))) {
ret = -EUCLEAN;
btrfs_err(fs_info, "invalid csum type, has %u want %u",
btrfs_super_csum_type(sb), BTRFS_CSUM_TYPE_CRC32);
u32 stripesize;
u64 generation;
u64 features;
+ u16 csum_type;
struct btrfs_key location;
struct buffer_head *bh;
struct btrfs_super_block *disk_super;
INIT_LIST_HEAD(&fs_info->space_info);
INIT_LIST_HEAD(&fs_info->tree_mod_seq_list);
INIT_LIST_HEAD(&fs_info->unused_bgs);
- btrfs_mapping_init(&fs_info->mapping_tree);
+ extent_map_tree_init(&fs_info->mapping_tree);
btrfs_init_block_rsv(&fs_info->global_block_rsv,
BTRFS_BLOCK_RSV_GLOBAL);
btrfs_init_block_rsv(&fs_info->trans_block_rsv, BTRFS_BLOCK_RSV_TRANS);
spin_lock_init(&fs_info->swapfile_pins_lock);
fs_info->swapfile_pins = RB_ROOT;
+ fs_info->send_in_progress = 0;
+
ret = btrfs_alloc_stripe_hash_table(fs_info);
if (ret) {
err = ret;
}
/*
+ * Verify the type first, if that or the the checksum value are
+ * corrupted, we'll find out
+ */
+ csum_type = btrfs_super_csum_type((struct btrfs_super_block *)bh->b_data);
+ if (!btrfs_supported_super_csum(csum_type)) {
+ btrfs_err(fs_info, "unsupported checksum algorithm: %u",
+ csum_type);
+ err = -EINVAL;
+ brelse(bh);
+ goto fail_alloc;
+ }
+
+ ret = btrfs_init_csum_hash(fs_info, csum_type);
+ if (ret) {
+ err = ret;
+ goto fail_alloc;
+ }
+
+ /*
* We want to check superblock checksum, the type is stored inside.
* Pass the whole disk block of size BTRFS_SUPER_INFO_SIZE (4k).
*/
btrfs_err(fs_info, "superblock checksum mismatch");
err = -EINVAL;
brelse(bh);
- goto fail_alloc;
+ goto fail_csum;
}
/*
if (ret) {
btrfs_err(fs_info, "superblock contains fatal errors");
err = -EINVAL;
- goto fail_alloc;
+ goto fail_csum;
}
if (!btrfs_super_root(disk_super))
- goto fail_alloc;
+ goto fail_csum;
/* check FS state, whether FS is broken. */
if (btrfs_super_flags(disk_super) & BTRFS_SUPER_FLAG_ERROR)
ret = btrfs_parse_options(fs_info, options, sb->s_flags);
if (ret) {
err = ret;
- goto fail_alloc;
+ goto fail_csum;
}
features = btrfs_super_incompat_flags(disk_super) &
"cannot mount because of unsupported optional features (%llx)",
features);
err = -EINVAL;
- goto fail_alloc;
+ goto fail_csum;
}
features = btrfs_super_incompat_flags(disk_super);
btrfs_err(fs_info,
"unequal nodesize/sectorsize (%u != %u) are not allowed for mixed block groups",
nodesize, sectorsize);
- goto fail_alloc;
+ goto fail_csum;
}
/*
"cannot mount read-write because of unsupported optional features (%llx)",
features);
err = -EINVAL;
- goto fail_alloc;
+ goto fail_csum;
}
ret = btrfs_init_workqueues(fs_info, fs_devices);
fail_sb_buffer:
btrfs_stop_all_workers(fs_info);
btrfs_free_block_groups(fs_info);
+fail_csum:
+ btrfs_free_csum_hash(fs_info);
fail_alloc:
fail_iput:
btrfs_mapping_tree_free(&fs_info->mapping_tree);
static int write_dev_supers(struct btrfs_device *device,
struct btrfs_super_block *sb, int max_mirrors)
{
+ struct btrfs_fs_info *fs_info = device->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
struct buffer_head *bh;
int i;
int ret;
int errors = 0;
- u32 crc;
u64 bytenr;
int op_flags;
if (max_mirrors == 0)
max_mirrors = BTRFS_SUPER_MIRROR_MAX;
+ shash->tfm = fs_info->csum_shash;
+
for (i = 0; i < max_mirrors; i++) {
bytenr = btrfs_sb_offset(i);
if (bytenr + BTRFS_SUPER_INFO_SIZE >=
btrfs_set_super_bytenr(sb, bytenr);
- crc = ~(u32)0;
- crc = btrfs_csum_data((const char *)sb + BTRFS_CSUM_SIZE, crc,
- BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
- btrfs_csum_final(crc, sb->csum);
+ crypto_shash_init(shash);
+ crypto_shash_update(shash, (const char *)sb + BTRFS_CSUM_SIZE,
+ BTRFS_SUPER_INFO_SIZE - BTRFS_CSUM_SIZE);
+ crypto_shash_final(shash, sb->csum);
/* One reference for us, and we leave it for the caller */
bh = __getblk(device->bdev, bytenr / BTRFS_BDEV_BLOCKSIZE,
if ((flags & BTRFS_BLOCK_GROUP_PROFILE_MASK) == 0 ||
(flags & BTRFS_AVAIL_ALLOC_BIT_SINGLE))
- min_tolerated = min(min_tolerated,
+ min_tolerated = min_t(int, min_tolerated,
btrfs_raid_array[BTRFS_RAID_SINGLE].
tolerated_failures);
continue;
if (!(flags & btrfs_raid_array[raid_type].bg_flag))
continue;
- min_tolerated = min(min_tolerated,
+ min_tolerated = min_t(int, min_tolerated,
btrfs_raid_array[raid_type].
tolerated_failures);
}
int atomic);
int btrfs_read_buffer(struct extent_buffer *buf, u64 parent_transid, int level,
struct btrfs_key *first_key);
-u32 btrfs_csum_data(const char *data, u32 seed, size_t len);
-void btrfs_csum_final(u32 crc, u8 *result);
blk_status_t btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio,
enum btrfs_wq_endio_type metadata);
blk_status_t btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct bio *bio,
#include "sysfs.h"
#include "qgroup.h"
#include "ref-verify.h"
+#include "space-info.h"
+#include "block-rsv.h"
+#include "delalloc-space.h"
#undef SCRAMBLE_DELAYED_REFS
-/*
- * control flags for do_chunk_alloc's force field
- * CHUNK_ALLOC_NO_FORCE means to only allocate a chunk
- * if we really need one.
- *
- * CHUNK_ALLOC_LIMITED means to only try and allocate one
- * if we have very few chunks already allocated. This is
- * used as part of the clustering code to help make sure
- * we have a good pool of storage to cluster in, without
- * filling the FS with empty chunks
- *
- * CHUNK_ALLOC_FORCE means it must try to allocate one
- *
- */
-enum {
- CHUNK_ALLOC_NO_FORCE = 0,
- CHUNK_ALLOC_LIMITED = 1,
- CHUNK_ALLOC_FORCE = 2,
-};
-
-/*
- * Declare a helper function to detect underflow of various space info members
- */
-#define DECLARE_SPACE_INFO_UPDATE(name) \
-static inline void update_##name(struct btrfs_space_info *sinfo, \
- s64 bytes) \
-{ \
- if (bytes < 0 && sinfo->name < -bytes) { \
- WARN_ON(1); \
- sinfo->name = 0; \
- return; \
- } \
- sinfo->name += bytes; \
-}
-
-DECLARE_SPACE_INFO_UPDATE(bytes_may_use);
-DECLARE_SPACE_INFO_UPDATE(bytes_pinned);
static int __btrfs_free_extent(struct btrfs_trans_handle *trans,
struct btrfs_delayed_ref_node *node, u64 parent,
static int alloc_reserved_tree_block(struct btrfs_trans_handle *trans,
struct btrfs_delayed_ref_node *node,
struct btrfs_delayed_extent_op *extent_op);
-static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
- int force);
static int find_next_key(struct btrfs_path *path, int level,
struct btrfs_key *key);
-static void dump_space_info(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *info, u64 bytes,
- int dump_block_groups);
-static int block_rsv_use_bytes(struct btrfs_block_rsv *block_rsv,
- u64 num_bytes);
-static void space_info_add_new_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 num_bytes);
-static void space_info_add_old_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 num_bytes);
static noinline int
block_group_cache_done(struct btrfs_block_group_cache *cache)
return block_group_cache_tree_search(info, bytenr, 1);
}
-static struct btrfs_space_info *__find_space_info(struct btrfs_fs_info *info,
- u64 flags)
+static u64 generic_ref_to_space_flags(struct btrfs_ref *ref)
{
- struct list_head *head = &info->space_info;
- struct btrfs_space_info *found;
-
- flags &= BTRFS_BLOCK_GROUP_TYPE_MASK;
-
- rcu_read_lock();
- list_for_each_entry_rcu(found, head, list) {
- if (found->flags & flags) {
- rcu_read_unlock();
- return found;
- }
+ if (ref->type == BTRFS_REF_METADATA) {
+ if (ref->tree_ref.root == BTRFS_CHUNK_TREE_OBJECTID)
+ return BTRFS_BLOCK_GROUP_SYSTEM;
+ else
+ return BTRFS_BLOCK_GROUP_METADATA;
}
- rcu_read_unlock();
- return NULL;
+ return BTRFS_BLOCK_GROUP_DATA;
}
static void add_pinned_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_ref *ref, int sign)
+ struct btrfs_ref *ref)
{
struct btrfs_space_info *space_info;
- s64 num_bytes;
- u64 flags;
-
- ASSERT(sign == 1 || sign == -1);
- num_bytes = sign * ref->len;
- if (ref->type == BTRFS_REF_METADATA) {
- if (ref->tree_ref.root == BTRFS_CHUNK_TREE_OBJECTID)
- flags = BTRFS_BLOCK_GROUP_SYSTEM;
- else
- flags = BTRFS_BLOCK_GROUP_METADATA;
- } else {
- flags = BTRFS_BLOCK_GROUP_DATA;
- }
+ u64 flags = generic_ref_to_space_flags(ref);
- space_info = __find_space_info(fs_info, flags);
+ space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
- percpu_counter_add_batch(&space_info->total_bytes_pinned, num_bytes,
+ percpu_counter_add_batch(&space_info->total_bytes_pinned, ref->len,
BTRFS_TOTAL_BYTES_PINNED_BATCH);
}
-/*
- * after adding space to the filesystem, we need to clear the full flags
- * on all the space infos.
- */
-void btrfs_clear_space_info_full(struct btrfs_fs_info *info)
+static void sub_pinned_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_ref *ref)
{
- struct list_head *head = &info->space_info;
- struct btrfs_space_info *found;
+ struct btrfs_space_info *space_info;
+ u64 flags = generic_ref_to_space_flags(ref);
- rcu_read_lock();
- list_for_each_entry_rcu(found, head, list)
- found->full = 0;
- rcu_read_unlock();
+ space_info = btrfs_find_space_info(fs_info, flags);
+ ASSERT(space_info);
+ percpu_counter_add_batch(&space_info->total_bytes_pinned, -ref->len,
+ BTRFS_TOTAL_BYTES_PINNED_BATCH);
}
/* simple helper to search for an existing data extent at a given offset */
__le64 lenum;
lenum = cpu_to_le64(root_objectid);
- high_crc = crc32c(high_crc, &lenum, sizeof(lenum));
+ high_crc = btrfs_crc32c(high_crc, &lenum, sizeof(lenum));
lenum = cpu_to_le64(owner);
- low_crc = crc32c(low_crc, &lenum, sizeof(lenum));
+ low_crc = btrfs_crc32c(low_crc, &lenum, sizeof(lenum));
lenum = cpu_to_le64(offset);
- low_crc = crc32c(low_crc, &lenum, sizeof(lenum));
+ low_crc = btrfs_crc32c(low_crc, &lenum, sizeof(lenum));
return ((u64)high_crc << 31) ^ (u64)low_crc;
}
btrfs_ref_tree_mod(fs_info, generic_ref);
if (ret == 0 && old_ref_mod < 0 && new_ref_mod >= 0)
- add_pinned_bytes(fs_info, generic_ref, -1);
+ sub_pinned_bytes(fs_info, generic_ref);
return ret;
}
flags = BTRFS_BLOCK_GROUP_SYSTEM;
else
flags = BTRFS_BLOCK_GROUP_METADATA;
- space_info = __find_space_info(fs_info, flags);
+ space_info = btrfs_find_space_info(fs_info, flags);
ASSERT(space_info);
percpu_counter_add_batch(&space_info->total_bytes_pinned,
-head->num_bytes,
return num_csums;
}
-bool btrfs_check_space_for_delayed_refs(struct btrfs_fs_info *fs_info)
-{
- struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- bool ret = false;
- u64 reserved;
-
- spin_lock(&global_rsv->lock);
- reserved = global_rsv->reserved;
- spin_unlock(&global_rsv->lock);
-
- /*
- * Since the global reserve is just kind of magic we don't really want
- * to rely on it to save our bacon, so if our size is more than the
- * delayed_refs_rsv and the global rsv then it's time to think about
- * bailing.
- */
- spin_lock(&delayed_refs_rsv->lock);
- reserved += delayed_refs_rsv->reserved;
- if (delayed_refs_rsv->size >= reserved)
- ret = true;
- spin_unlock(&delayed_refs_rsv->lock);
- return ret;
-}
-
-int btrfs_should_throttle_delayed_refs(struct btrfs_trans_handle *trans)
-{
- u64 num_entries =
- atomic_read(&trans->transaction->delayed_refs.num_entries);
- u64 avg_runtime;
- u64 val;
-
- smp_mb();
- avg_runtime = trans->fs_info->avg_delayed_ref_runtime;
- val = num_entries * avg_runtime;
- if (val >= NSEC_PER_SEC)
- return 1;
- if (val >= NSEC_PER_SEC / 2)
- return 2;
-
- return btrfs_check_space_for_delayed_refs(trans->fs_info);
-}
-
/*
* this starts processing the delayed reference count updates and
* extent insertions we have queued up so far. count can be
wait_var_event(&bg->nocow_writers, !atomic_read(&bg->nocow_writers));
}
-static const char *alloc_name(u64 flags)
-{
- switch (flags) {
- case BTRFS_BLOCK_GROUP_METADATA|BTRFS_BLOCK_GROUP_DATA:
- return "mixed";
- case BTRFS_BLOCK_GROUP_METADATA:
- return "metadata";
- case BTRFS_BLOCK_GROUP_DATA:
- return "data";
- case BTRFS_BLOCK_GROUP_SYSTEM:
- return "system";
- default:
- WARN_ON(1);
- return "invalid-combination";
- };
-}
-
-static int create_space_info(struct btrfs_fs_info *info, u64 flags)
-{
-
- struct btrfs_space_info *space_info;
- int i;
- int ret;
-
- space_info = kzalloc(sizeof(*space_info), GFP_NOFS);
- if (!space_info)
- return -ENOMEM;
-
- ret = percpu_counter_init(&space_info->total_bytes_pinned, 0,
- GFP_KERNEL);
- if (ret) {
- kfree(space_info);
- return ret;
- }
-
- for (i = 0; i < BTRFS_NR_RAID_TYPES; i++)
- INIT_LIST_HEAD(&space_info->block_groups[i]);
- init_rwsem(&space_info->groups_sem);
- spin_lock_init(&space_info->lock);
- space_info->flags = flags & BTRFS_BLOCK_GROUP_TYPE_MASK;
- space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
- init_waitqueue_head(&space_info->wait);
- INIT_LIST_HEAD(&space_info->ro_bgs);
- INIT_LIST_HEAD(&space_info->tickets);
- INIT_LIST_HEAD(&space_info->priority_tickets);
-
- ret = kobject_init_and_add(&space_info->kobj, &space_info_ktype,
- info->space_info_kobj, "%s",
- alloc_name(space_info->flags));
- if (ret) {
- kobject_put(&space_info->kobj);
- return ret;
- }
-
- list_add_rcu(&space_info->list, &info->space_info);
- if (flags & BTRFS_BLOCK_GROUP_DATA)
- info->data_sinfo = space_info;
-
- return ret;
-}
-
-static void update_space_info(struct btrfs_fs_info *info, u64 flags,
- u64 total_bytes, u64 bytes_used,
- u64 bytes_readonly,
- struct btrfs_space_info **space_info)
-{
- struct btrfs_space_info *found;
- int factor;
-
- factor = btrfs_bg_type_to_factor(flags);
-
- found = __find_space_info(info, flags);
- ASSERT(found);
- spin_lock(&found->lock);
- found->total_bytes += total_bytes;
- found->disk_total += total_bytes * factor;
- found->bytes_used += bytes_used;
- found->disk_used += bytes_used * factor;
- found->bytes_readonly += bytes_readonly;
- if (total_bytes > 0)
- found->full = 0;
- space_info_add_new_bytes(info, found, total_bytes -
- bytes_used - bytes_readonly);
- spin_unlock(&found->lock);
- *space_info = found;
-}
-
static void set_avail_alloc_bits(struct btrfs_fs_info *fs_info, u64 flags)
{
u64 extra_flags = chunk_to_extended(flags) &
return get_alloc_profile(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
}
-static u64 btrfs_space_info_used(struct btrfs_space_info *s_info,
- bool may_use_included)
-{
- ASSERT(s_info);
- return s_info->bytes_used + s_info->bytes_reserved +
- s_info->bytes_pinned + s_info->bytes_readonly +
- (may_use_included ? s_info->bytes_may_use : 0);
-}
-
-int btrfs_alloc_data_chunk_ondemand(struct btrfs_inode *inode, u64 bytes)
-{
- struct btrfs_root *root = inode->root;
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_space_info *data_sinfo = fs_info->data_sinfo;
- u64 used;
- int ret = 0;
- int need_commit = 2;
- int have_pinned_space;
-
- /* make sure bytes are sectorsize aligned */
- bytes = ALIGN(bytes, fs_info->sectorsize);
-
- if (btrfs_is_free_space_inode(inode)) {
- need_commit = 0;
- ASSERT(current->journal_info);
- }
-
-again:
- /* make sure we have enough space to handle the data first */
- spin_lock(&data_sinfo->lock);
- used = btrfs_space_info_used(data_sinfo, true);
-
- if (used + bytes > data_sinfo->total_bytes) {
- struct btrfs_trans_handle *trans;
-
- /*
- * if we don't have enough free bytes in this space then we need
- * to alloc a new chunk.
- */
- if (!data_sinfo->full) {
- u64 alloc_target;
-
- data_sinfo->force_alloc = CHUNK_ALLOC_FORCE;
- spin_unlock(&data_sinfo->lock);
-
- alloc_target = btrfs_data_alloc_profile(fs_info);
- /*
- * It is ugly that we don't call nolock join
- * transaction for the free space inode case here.
- * But it is safe because we only do the data space
- * reservation for the free space cache in the
- * transaction context, the common join transaction
- * just increase the counter of the current transaction
- * handler, doesn't try to acquire the trans_lock of
- * the fs.
- */
- trans = btrfs_join_transaction(root);
- if (IS_ERR(trans))
- return PTR_ERR(trans);
-
- ret = do_chunk_alloc(trans, alloc_target,
- CHUNK_ALLOC_NO_FORCE);
- btrfs_end_transaction(trans);
- if (ret < 0) {
- if (ret != -ENOSPC)
- return ret;
- else {
- have_pinned_space = 1;
- goto commit_trans;
- }
- }
-
- goto again;
- }
-
- /*
- * If we don't have enough pinned space to deal with this
- * allocation, and no removed chunk in current transaction,
- * don't bother committing the transaction.
- */
- have_pinned_space = __percpu_counter_compare(
- &data_sinfo->total_bytes_pinned,
- used + bytes - data_sinfo->total_bytes,
- BTRFS_TOTAL_BYTES_PINNED_BATCH);
- spin_unlock(&data_sinfo->lock);
-
- /* commit the current transaction and try again */
-commit_trans:
- if (need_commit) {
- need_commit--;
-
- if (need_commit > 0) {
- btrfs_start_delalloc_roots(fs_info, -1);
- btrfs_wait_ordered_roots(fs_info, U64_MAX, 0,
- (u64)-1);
- }
-
- trans = btrfs_join_transaction(root);
- if (IS_ERR(trans))
- return PTR_ERR(trans);
- if (have_pinned_space >= 0 ||
- test_bit(BTRFS_TRANS_HAVE_FREE_BGS,
- &trans->transaction->flags) ||
- need_commit > 0) {
- ret = btrfs_commit_transaction(trans);
- if (ret)
- return ret;
- /*
- * The cleaner kthread might still be doing iput
- * operations. Wait for it to finish so that
- * more space is released. We don't need to
- * explicitly run the delayed iputs here because
- * the commit_transaction would have woken up
- * the cleaner.
- */
- ret = btrfs_wait_on_delayed_iputs(fs_info);
- if (ret)
- return ret;
- goto again;
- } else {
- btrfs_end_transaction(trans);
- }
- }
-
- trace_btrfs_space_reservation(fs_info,
- "space_info:enospc",
- data_sinfo->flags, bytes, 1);
- return -ENOSPC;
- }
- update_bytes_may_use(data_sinfo, bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- data_sinfo->flags, bytes, 1);
- spin_unlock(&data_sinfo->lock);
-
- return 0;
-}
-
-int btrfs_check_data_free_space(struct inode *inode,
- struct extent_changeset **reserved, u64 start, u64 len)
-{
- struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
- int ret;
-
- /* align the range */
- len = round_up(start + len, fs_info->sectorsize) -
- round_down(start, fs_info->sectorsize);
- start = round_down(start, fs_info->sectorsize);
-
- ret = btrfs_alloc_data_chunk_ondemand(BTRFS_I(inode), len);
- if (ret < 0)
- return ret;
-
- /* Use new btrfs_qgroup_reserve_data to reserve precious data space. */
- ret = btrfs_qgroup_reserve_data(inode, reserved, start, len);
- if (ret < 0)
- btrfs_free_reserved_data_space_noquota(inode, start, len);
- else
- ret = 0;
- return ret;
-}
-
-/*
- * Called if we need to clear a data reservation for this inode
- * Normally in a error case.
- *
- * This one will *NOT* use accurate qgroup reserved space API, just for case
- * which we can't sleep and is sure it won't affect qgroup reserved space.
- * Like clear_bit_hook().
- */
-void btrfs_free_reserved_data_space_noquota(struct inode *inode, u64 start,
- u64 len)
-{
- struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
- struct btrfs_space_info *data_sinfo;
-
- /* Make sure the range is aligned to sectorsize */
- len = round_up(start + len, fs_info->sectorsize) -
- round_down(start, fs_info->sectorsize);
- start = round_down(start, fs_info->sectorsize);
-
- data_sinfo = fs_info->data_sinfo;
- spin_lock(&data_sinfo->lock);
- update_bytes_may_use(data_sinfo, -len);
- trace_btrfs_space_reservation(fs_info, "space_info",
- data_sinfo->flags, len, 0);
- spin_unlock(&data_sinfo->lock);
-}
-
-/*
- * Called if we need to clear a data reservation for this inode
- * Normally in a error case.
- *
- * This one will handle the per-inode data rsv map for accurate reserved
- * space framework.
- */
-void btrfs_free_reserved_data_space(struct inode *inode,
- struct extent_changeset *reserved, u64 start, u64 len)
-{
- struct btrfs_root *root = BTRFS_I(inode)->root;
-
- /* Make sure the range is aligned to sectorsize */
- len = round_up(start + len, root->fs_info->sectorsize) -
- round_down(start, root->fs_info->sectorsize);
- start = round_down(start, root->fs_info->sectorsize);
-
- btrfs_free_reserved_data_space_noquota(inode, start, len);
- btrfs_qgroup_free_data(inode, reserved, start, len);
-}
-
static void force_metadata_allocation(struct btrfs_fs_info *info)
{
struct list_head *head = &info->space_info;
rcu_read_unlock();
}
-static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global)
-{
- return (global->size << 1);
-}
-
static int should_alloc_chunk(struct btrfs_fs_info *fs_info,
struct btrfs_space_info *sinfo, int force)
{
{
u64 num_dev;
- if (type & (BTRFS_BLOCK_GROUP_RAID10 |
- BTRFS_BLOCK_GROUP_RAID0 |
- BTRFS_BLOCK_GROUP_RAID5 |
- BTRFS_BLOCK_GROUP_RAID6))
+ num_dev = btrfs_raid_array[btrfs_bg_flags_to_raid_index(type)].devs_max;
+ if (!num_dev)
num_dev = fs_info->fs_devices->rw_devices;
- else if (type & BTRFS_BLOCK_GROUP_RAID1)
- num_dev = 2;
- else
- num_dev = 1; /* DUP or single */
return num_dev;
}
*/
lockdep_assert_held(&fs_info->chunk_mutex);
- info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
+ info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
spin_lock(&info->lock);
left = info->total_bytes - btrfs_space_info_used(info, true);
spin_unlock(&info->lock);
btrfs_calc_trans_metadata_size(fs_info, 1);
if (left < thresh && btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
- btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
- left, thresh, type);
- dump_space_info(fs_info, info, 0, 0);
- }
-
- if (left < thresh) {
- u64 flags = btrfs_system_alloc_profile(fs_info);
-
- /*
- * Ignore failure to create system chunk. We might end up not
- * needing it, as we might not need to COW all nodes/leafs from
- * the paths we visit in the chunk tree (they were already COWed
- * or created in the current transaction for example).
- */
- ret = btrfs_alloc_chunk(trans, flags);
- }
-
- if (!ret) {
- ret = btrfs_block_rsv_add(fs_info->chunk_root,
- &fs_info->chunk_block_rsv,
- thresh, BTRFS_RESERVE_NO_FLUSH);
- if (!ret)
- trans->chunk_bytes_reserved += thresh;
- }
-}
-
-/*
- * If force is CHUNK_ALLOC_FORCE:
- * - return 1 if it successfully allocates a chunk,
- * - return errors including -ENOSPC otherwise.
- * If force is NOT CHUNK_ALLOC_FORCE:
- * - return 0 if it doesn't need to allocate a new chunk,
- * - return 1 if it successfully allocates a chunk,
- * - return errors including -ENOSPC otherwise.
- */
-static int do_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
- int force)
-{
- struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_space_info *space_info;
- bool wait_for_alloc = false;
- bool should_alloc = false;
- int ret = 0;
-
- /* Don't re-enter if we're already allocating a chunk */
- if (trans->allocating_chunk)
- return -ENOSPC;
-
- space_info = __find_space_info(fs_info, flags);
- ASSERT(space_info);
-
- do {
- spin_lock(&space_info->lock);
- if (force < space_info->force_alloc)
- force = space_info->force_alloc;
- should_alloc = should_alloc_chunk(fs_info, space_info, force);
- if (space_info->full) {
- /* No more free physical space */
- if (should_alloc)
- ret = -ENOSPC;
- else
- ret = 0;
- spin_unlock(&space_info->lock);
- return ret;
- } else if (!should_alloc) {
- spin_unlock(&space_info->lock);
- return 0;
- } else if (space_info->chunk_alloc) {
- /*
- * Someone is already allocating, so we need to block
- * until this someone is finished and then loop to
- * recheck if we should continue with our allocation
- * attempt.
- */
- wait_for_alloc = true;
- spin_unlock(&space_info->lock);
- mutex_lock(&fs_info->chunk_mutex);
- mutex_unlock(&fs_info->chunk_mutex);
- } else {
- /* Proceed with allocation */
- space_info->chunk_alloc = 1;
- wait_for_alloc = false;
- spin_unlock(&space_info->lock);
- }
-
- cond_resched();
- } while (wait_for_alloc);
-
- mutex_lock(&fs_info->chunk_mutex);
- trans->allocating_chunk = true;
-
- /*
- * If we have mixed data/metadata chunks we want to make sure we keep
- * allocating mixed chunks instead of individual chunks.
- */
- if (btrfs_mixed_space_info(space_info))
- flags |= (BTRFS_BLOCK_GROUP_DATA | BTRFS_BLOCK_GROUP_METADATA);
-
- /*
- * if we're doing a data chunk, go ahead and make sure that
- * we keep a reasonable number of metadata chunks allocated in the
- * FS as well.
- */
- if (flags & BTRFS_BLOCK_GROUP_DATA && fs_info->metadata_ratio) {
- fs_info->data_chunk_allocations++;
- if (!(fs_info->data_chunk_allocations %
- fs_info->metadata_ratio))
- force_metadata_allocation(fs_info);
- }
-
- /*
- * Check if we have enough space in SYSTEM chunk because we may need
- * to update devices.
- */
- check_system_chunk(trans, flags);
-
- ret = btrfs_alloc_chunk(trans, flags);
- trans->allocating_chunk = false;
-
- spin_lock(&space_info->lock);
- if (ret < 0) {
- if (ret == -ENOSPC)
- space_info->full = 1;
- else
- goto out;
- } else {
- ret = 1;
- space_info->max_extent_size = 0;
- }
-
- space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
-out:
- space_info->chunk_alloc = 0;
- spin_unlock(&space_info->lock);
- mutex_unlock(&fs_info->chunk_mutex);
- /*
- * When we allocate a new chunk we reserve space in the chunk block
- * reserve to make sure we can COW nodes/leafs in the chunk tree or
- * add new nodes/leafs to it if we end up needing to do it when
- * inserting the chunk item and updating device items as part of the
- * second phase of chunk allocation, performed by
- * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
- * large number of new block groups to create in our transaction
- * handle's new_bgs list to avoid exhausting the chunk block reserve
- * in extreme cases - like having a single transaction create many new
- * block groups when starting to write out the free space caches of all
- * the block groups that were made dirty during the lifetime of the
- * transaction.
- */
- if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
- btrfs_create_pending_block_groups(trans);
-
- return ret;
-}
-
-static int can_overcommit(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info, u64 bytes,
- enum btrfs_reserve_flush_enum flush,
- bool system_chunk)
-{
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- u64 profile;
- u64 space_size;
- u64 avail;
- u64 used;
- int factor;
-
- /* Don't overcommit when in mixed mode. */
- if (space_info->flags & BTRFS_BLOCK_GROUP_DATA)
- return 0;
-
- if (system_chunk)
- profile = btrfs_system_alloc_profile(fs_info);
- else
- profile = btrfs_metadata_alloc_profile(fs_info);
-
- used = btrfs_space_info_used(space_info, false);
-
- /*
- * We only want to allow over committing if we have lots of actual space
- * free, but if we don't have enough space to handle the global reserve
- * space then we could end up having a real enospc problem when trying
- * to allocate a chunk or some other such important allocation.
- */
- spin_lock(&global_rsv->lock);
- space_size = calc_global_rsv_need_space(global_rsv);
- spin_unlock(&global_rsv->lock);
- if (used + space_size >= space_info->total_bytes)
- return 0;
-
- used += space_info->bytes_may_use;
-
- avail = atomic64_read(&fs_info->free_chunk_space);
-
- /*
- * If we have dup, raid1 or raid10 then only half of the free
- * space is actually usable. For raid56, the space info used
- * doesn't include the parity drive, so we don't have to
- * change the math
- */
- factor = btrfs_bg_type_to_factor(profile);
- avail = div_u64(avail, factor);
-
- /*
- * If we aren't flushing all things, let us overcommit up to
- * 1/2th of the space. If we can flush, don't let us overcommit
- * too much, let it overcommit up to 1/8 of the space.
- */
- if (flush == BTRFS_RESERVE_FLUSH_ALL)
- avail >>= 3;
- else
- avail >>= 1;
-
- if (used + bytes < space_info->total_bytes + avail)
- return 1;
- return 0;
-}
-
-static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info,
- unsigned long nr_pages, int nr_items)
-{
- struct super_block *sb = fs_info->sb;
-
- if (down_read_trylock(&sb->s_umount)) {
- writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE);
- up_read(&sb->s_umount);
- } else {
- /*
- * We needn't worry the filesystem going from r/w to r/o though
- * we don't acquire ->s_umount mutex, because the filesystem
- * should guarantee the delalloc inodes list be empty after
- * the filesystem is readonly(all dirty pages are written to
- * the disk).
- */
- btrfs_start_delalloc_roots(fs_info, nr_items);
- if (!current->journal_info)
- btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1);
- }
-}
-
-static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info,
- u64 to_reclaim)
-{
- u64 bytes;
- u64 nr;
-
- bytes = btrfs_calc_trans_metadata_size(fs_info, 1);
- nr = div64_u64(to_reclaim, bytes);
- if (!nr)
- nr = 1;
- return nr;
-}
-
-#define EXTENT_SIZE_PER_ITEM SZ_256K
-
-/*
- * shrink metadata reservation for delalloc
- */
-static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim,
- u64 orig, bool wait_ordered)
-{
- struct btrfs_space_info *space_info;
- struct btrfs_trans_handle *trans;
- u64 delalloc_bytes;
- u64 dio_bytes;
- u64 async_pages;
- u64 items;
- long time_left;
- unsigned long nr_pages;
- int loops;
-
- /* Calc the number of the pages we need flush for space reservation */
- items = calc_reclaim_items_nr(fs_info, to_reclaim);
- to_reclaim = items * EXTENT_SIZE_PER_ITEM;
-
- trans = (struct btrfs_trans_handle *)current->journal_info;
- space_info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
-
- delalloc_bytes = percpu_counter_sum_positive(
- &fs_info->delalloc_bytes);
- dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes);
- if (delalloc_bytes == 0 && dio_bytes == 0) {
- if (trans)
- return;
- if (wait_ordered)
- btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1);
- return;
- }
-
- /*
- * If we are doing more ordered than delalloc we need to just wait on
- * ordered extents, otherwise we'll waste time trying to flush delalloc
- * that likely won't give us the space back we need.
- */
- if (dio_bytes > delalloc_bytes)
- wait_ordered = true;
-
- loops = 0;
- while ((delalloc_bytes || dio_bytes) && loops < 3) {
- nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT;
-
- /*
- * Triggers inode writeback for up to nr_pages. This will invoke
- * ->writepages callback and trigger delalloc filling
- * (btrfs_run_delalloc_range()).
- */
- btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items);
-
- /*
- * We need to wait for the compressed pages to start before
- * we continue.
- */
- async_pages = atomic_read(&fs_info->async_delalloc_pages);
- if (!async_pages)
- goto skip_async;
-
- /*
- * Calculate how many compressed pages we want to be written
- * before we continue. I.e if there are more async pages than we
- * require wait_event will wait until nr_pages are written.
- */
- if (async_pages <= nr_pages)
- async_pages = 0;
- else
- async_pages -= nr_pages;
-
- wait_event(fs_info->async_submit_wait,
- atomic_read(&fs_info->async_delalloc_pages) <=
- (int)async_pages);
-skip_async:
- spin_lock(&space_info->lock);
- if (list_empty(&space_info->tickets) &&
- list_empty(&space_info->priority_tickets)) {
- spin_unlock(&space_info->lock);
- break;
- }
- spin_unlock(&space_info->lock);
-
- loops++;
- if (wait_ordered && !trans) {
- btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1);
- } else {
- time_left = schedule_timeout_killable(1);
- if (time_left)
- break;
- }
- delalloc_bytes = percpu_counter_sum_positive(
- &fs_info->delalloc_bytes);
- dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes);
- }
-}
-
-struct reserve_ticket {
- u64 orig_bytes;
- u64 bytes;
- int error;
- struct list_head list;
- wait_queue_head_t wait;
-};
-
-/**
- * maybe_commit_transaction - possibly commit the transaction if its ok to
- * @root - the root we're allocating for
- * @bytes - the number of bytes we want to reserve
- * @force - force the commit
- *
- * This will check to make sure that committing the transaction will actually
- * get us somewhere and then commit the transaction if it does. Otherwise it
- * will return -ENOSPC.
- */
-static int may_commit_transaction(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info)
-{
- struct reserve_ticket *ticket = NULL;
- struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv;
- struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
- struct btrfs_trans_handle *trans;
- u64 bytes_needed;
- u64 reclaim_bytes = 0;
-
- trans = (struct btrfs_trans_handle *)current->journal_info;
- if (trans)
- return -EAGAIN;
-
- spin_lock(&space_info->lock);
- if (!list_empty(&space_info->priority_tickets))
- ticket = list_first_entry(&space_info->priority_tickets,
- struct reserve_ticket, list);
- else if (!list_empty(&space_info->tickets))
- ticket = list_first_entry(&space_info->tickets,
- struct reserve_ticket, list);
- bytes_needed = (ticket) ? ticket->bytes : 0;
- spin_unlock(&space_info->lock);
-
- if (!bytes_needed)
- return 0;
-
- trans = btrfs_join_transaction(fs_info->extent_root);
- if (IS_ERR(trans))
- return PTR_ERR(trans);
-
- /*
- * See if there is enough pinned space to make this reservation, or if
- * we have block groups that are going to be freed, allowing us to
- * possibly do a chunk allocation the next loop through.
- */
- if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) ||
- __percpu_counter_compare(&space_info->total_bytes_pinned,
- bytes_needed,
- BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0)
- goto commit;
-
- /*
- * See if there is some space in the delayed insertion reservation for
- * this reservation.
- */
- if (space_info != delayed_rsv->space_info)
- goto enospc;
-
- spin_lock(&delayed_rsv->lock);
- reclaim_bytes += delayed_rsv->reserved;
- spin_unlock(&delayed_rsv->lock);
-
- spin_lock(&delayed_refs_rsv->lock);
- reclaim_bytes += delayed_refs_rsv->reserved;
- spin_unlock(&delayed_refs_rsv->lock);
- if (reclaim_bytes >= bytes_needed)
- goto commit;
- bytes_needed -= reclaim_bytes;
-
- if (__percpu_counter_compare(&space_info->total_bytes_pinned,
- bytes_needed,
- BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0)
- goto enospc;
-
-commit:
- return btrfs_commit_transaction(trans);
-enospc:
- btrfs_end_transaction(trans);
- return -ENOSPC;
-}
-
-/*
- * Try to flush some data based on policy set by @state. This is only advisory
- * and may fail for various reasons. The caller is supposed to examine the
- * state of @space_info to detect the outcome.
- */
-static void flush_space(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info, u64 num_bytes,
- int state)
-{
- struct btrfs_root *root = fs_info->extent_root;
- struct btrfs_trans_handle *trans;
- int nr;
- int ret = 0;
-
- switch (state) {
- case FLUSH_DELAYED_ITEMS_NR:
- case FLUSH_DELAYED_ITEMS:
- if (state == FLUSH_DELAYED_ITEMS_NR)
- nr = calc_reclaim_items_nr(fs_info, num_bytes) * 2;
- else
- nr = -1;
-
- trans = btrfs_join_transaction(root);
- if (IS_ERR(trans)) {
- ret = PTR_ERR(trans);
- break;
- }
- ret = btrfs_run_delayed_items_nr(trans, nr);
- btrfs_end_transaction(trans);
- break;
- case FLUSH_DELALLOC:
- case FLUSH_DELALLOC_WAIT:
- shrink_delalloc(fs_info, num_bytes * 2, num_bytes,
- state == FLUSH_DELALLOC_WAIT);
- break;
- case FLUSH_DELAYED_REFS_NR:
- case FLUSH_DELAYED_REFS:
- trans = btrfs_join_transaction(root);
- if (IS_ERR(trans)) {
- ret = PTR_ERR(trans);
- break;
- }
- if (state == FLUSH_DELAYED_REFS_NR)
- nr = calc_reclaim_items_nr(fs_info, num_bytes);
- else
- nr = 0;
- btrfs_run_delayed_refs(trans, nr);
- btrfs_end_transaction(trans);
- break;
- case ALLOC_CHUNK:
- case ALLOC_CHUNK_FORCE:
- trans = btrfs_join_transaction(root);
- if (IS_ERR(trans)) {
- ret = PTR_ERR(trans);
- break;
- }
- ret = do_chunk_alloc(trans,
- btrfs_metadata_alloc_profile(fs_info),
- (state == ALLOC_CHUNK) ?
- CHUNK_ALLOC_NO_FORCE : CHUNK_ALLOC_FORCE);
- btrfs_end_transaction(trans);
- if (ret > 0 || ret == -ENOSPC)
- ret = 0;
- break;
- case COMMIT_TRANS:
- /*
- * If we have pending delayed iputs then we could free up a
- * bunch of pinned space, so make sure we run the iputs before
- * we do our pinned bytes check below.
- */
- btrfs_run_delayed_iputs(fs_info);
- btrfs_wait_on_delayed_iputs(fs_info);
-
- ret = may_commit_transaction(fs_info, space_info);
- break;
- default:
- ret = -ENOSPC;
- break;
- }
-
- trace_btrfs_flush_space(fs_info, space_info->flags, num_bytes, state,
- ret);
- return;
-}
-
-static inline u64
-btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- bool system_chunk)
-{
- struct reserve_ticket *ticket;
- u64 used;
- u64 expected;
- u64 to_reclaim = 0;
-
- list_for_each_entry(ticket, &space_info->tickets, list)
- to_reclaim += ticket->bytes;
- list_for_each_entry(ticket, &space_info->priority_tickets, list)
- to_reclaim += ticket->bytes;
- if (to_reclaim)
- return to_reclaim;
-
- to_reclaim = min_t(u64, num_online_cpus() * SZ_1M, SZ_16M);
- if (can_overcommit(fs_info, space_info, to_reclaim,
- BTRFS_RESERVE_FLUSH_ALL, system_chunk))
- return 0;
-
- used = btrfs_space_info_used(space_info, true);
-
- if (can_overcommit(fs_info, space_info, SZ_1M,
- BTRFS_RESERVE_FLUSH_ALL, system_chunk))
- expected = div_factor_fine(space_info->total_bytes, 95);
- else
- expected = div_factor_fine(space_info->total_bytes, 90);
-
- if (used > expected)
- to_reclaim = used - expected;
- else
- to_reclaim = 0;
- to_reclaim = min(to_reclaim, space_info->bytes_may_use +
- space_info->bytes_reserved);
- return to_reclaim;
-}
-
-static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 used, bool system_chunk)
-{
- u64 thresh = div_factor_fine(space_info->total_bytes, 98);
-
- /* If we're just plain full then async reclaim just slows us down. */
- if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh)
- return 0;
-
- if (!btrfs_calc_reclaim_metadata_size(fs_info, space_info,
- system_chunk))
- return 0;
-
- return (used >= thresh && !btrfs_fs_closing(fs_info) &&
- !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state));
-}
-
-static bool wake_all_tickets(struct list_head *head)
-{
- struct reserve_ticket *ticket;
-
- while (!list_empty(head)) {
- ticket = list_first_entry(head, struct reserve_ticket, list);
- list_del_init(&ticket->list);
- ticket->error = -ENOSPC;
- wake_up(&ticket->wait);
- if (ticket->bytes != ticket->orig_bytes)
- return true;
- }
- return false;
-}
-
-/*
- * This is for normal flushers, we can wait all goddamned day if we want to. We
- * will loop and continuously try to flush as long as we are making progress.
- * We count progress as clearing off tickets each time we have to loop.
- */
-static void btrfs_async_reclaim_metadata_space(struct work_struct *work)
-{
- struct btrfs_fs_info *fs_info;
- struct btrfs_space_info *space_info;
- u64 to_reclaim;
- int flush_state;
- int commit_cycles = 0;
- u64 last_tickets_id;
-
- fs_info = container_of(work, struct btrfs_fs_info, async_reclaim_work);
- space_info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
-
- spin_lock(&space_info->lock);
- to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
- false);
- if (!to_reclaim) {
- space_info->flush = 0;
- spin_unlock(&space_info->lock);
- return;
- }
- last_tickets_id = space_info->tickets_id;
- spin_unlock(&space_info->lock);
-
- flush_state = FLUSH_DELAYED_ITEMS_NR;
- do {
- flush_space(fs_info, space_info, to_reclaim, flush_state);
- spin_lock(&space_info->lock);
- if (list_empty(&space_info->tickets)) {
- space_info->flush = 0;
- spin_unlock(&space_info->lock);
- return;
- }
- to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info,
- space_info,
- false);
- if (last_tickets_id == space_info->tickets_id) {
- flush_state++;
- } else {
- last_tickets_id = space_info->tickets_id;
- flush_state = FLUSH_DELAYED_ITEMS_NR;
- if (commit_cycles)
- commit_cycles--;
- }
-
- /*
- * We don't want to force a chunk allocation until we've tried
- * pretty hard to reclaim space. Think of the case where we
- * freed up a bunch of space and so have a lot of pinned space
- * to reclaim. We would rather use that than possibly create a
- * underutilized metadata chunk. So if this is our first run
- * through the flushing state machine skip ALLOC_CHUNK_FORCE and
- * commit the transaction. If nothing has changed the next go
- * around then we can force a chunk allocation.
- */
- if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
- flush_state++;
-
- if (flush_state > COMMIT_TRANS) {
- commit_cycles++;
- if (commit_cycles > 2) {
- if (wake_all_tickets(&space_info->tickets)) {
- flush_state = FLUSH_DELAYED_ITEMS_NR;
- commit_cycles--;
- } else {
- space_info->flush = 0;
- }
- } else {
- flush_state = FLUSH_DELAYED_ITEMS_NR;
- }
- }
- spin_unlock(&space_info->lock);
- } while (flush_state <= COMMIT_TRANS);
-}
-
-void btrfs_init_async_reclaim_work(struct work_struct *work)
-{
- INIT_WORK(work, btrfs_async_reclaim_metadata_space);
-}
-
-static const enum btrfs_flush_state priority_flush_states[] = {
- FLUSH_DELAYED_ITEMS_NR,
- FLUSH_DELAYED_ITEMS,
- ALLOC_CHUNK,
-};
-
-static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- struct reserve_ticket *ticket)
-{
- u64 to_reclaim;
- int flush_state;
-
- spin_lock(&space_info->lock);
- to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
- false);
- if (!to_reclaim) {
- spin_unlock(&space_info->lock);
- return;
- }
- spin_unlock(&space_info->lock);
-
- flush_state = 0;
- do {
- flush_space(fs_info, space_info, to_reclaim,
- priority_flush_states[flush_state]);
- flush_state++;
- spin_lock(&space_info->lock);
- if (ticket->bytes == 0) {
- spin_unlock(&space_info->lock);
- return;
- }
- spin_unlock(&space_info->lock);
- } while (flush_state < ARRAY_SIZE(priority_flush_states));
-}
-
-static int wait_reserve_ticket(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- struct reserve_ticket *ticket)
-
-{
- DEFINE_WAIT(wait);
- u64 reclaim_bytes = 0;
- int ret = 0;
-
- spin_lock(&space_info->lock);
- while (ticket->bytes > 0 && ticket->error == 0) {
- ret = prepare_to_wait_event(&ticket->wait, &wait, TASK_KILLABLE);
- if (ret) {
- ret = -EINTR;
- break;
- }
- spin_unlock(&space_info->lock);
-
- schedule();
-
- finish_wait(&ticket->wait, &wait);
- spin_lock(&space_info->lock);
- }
- if (!ret)
- ret = ticket->error;
- if (!list_empty(&ticket->list))
- list_del_init(&ticket->list);
- if (ticket->bytes && ticket->bytes < ticket->orig_bytes)
- reclaim_bytes = ticket->orig_bytes - ticket->bytes;
- spin_unlock(&space_info->lock);
-
- if (reclaim_bytes)
- space_info_add_old_bytes(fs_info, space_info, reclaim_bytes);
- return ret;
-}
-
-/**
- * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
- * @root - the root we're allocating for
- * @space_info - the space info we want to allocate from
- * @orig_bytes - the number of bytes we want
- * @flush - whether or not we can flush to make our reservation
- *
- * This will reserve orig_bytes number of bytes from the space info associated
- * with the block_rsv. If there is not enough space it will make an attempt to
- * flush out space to make room. It will do this by flushing delalloc if
- * possible or committing the transaction. If flush is 0 then no attempts to
- * regain reservations will be made and this will fail if there is not enough
- * space already.
- */
-static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 orig_bytes,
- enum btrfs_reserve_flush_enum flush,
- bool system_chunk)
-{
- struct reserve_ticket ticket;
- u64 used;
- u64 reclaim_bytes = 0;
- int ret = 0;
-
- ASSERT(orig_bytes);
- ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL);
-
- spin_lock(&space_info->lock);
- ret = -ENOSPC;
- used = btrfs_space_info_used(space_info, true);
-
- /*
- * If we have enough space then hooray, make our reservation and carry
- * on. If not see if we can overcommit, and if we can, hooray carry on.
- * If not things get more complicated.
- */
- if (used + orig_bytes <= space_info->total_bytes) {
- update_bytes_may_use(space_info, orig_bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- space_info->flags, orig_bytes, 1);
- ret = 0;
- } else if (can_overcommit(fs_info, space_info, orig_bytes, flush,
- system_chunk)) {
- update_bytes_may_use(space_info, orig_bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- space_info->flags, orig_bytes, 1);
- ret = 0;
- }
-
- /*
- * If we couldn't make a reservation then setup our reservation ticket
- * and kick the async worker if it's not already running.
- *
- * If we are a priority flusher then we just need to add our ticket to
- * the list and we will do our own flushing further down.
- */
- if (ret && flush != BTRFS_RESERVE_NO_FLUSH) {
- ticket.orig_bytes = orig_bytes;
- ticket.bytes = orig_bytes;
- ticket.error = 0;
- init_waitqueue_head(&ticket.wait);
- if (flush == BTRFS_RESERVE_FLUSH_ALL) {
- list_add_tail(&ticket.list, &space_info->tickets);
- if (!space_info->flush) {
- space_info->flush = 1;
- trace_btrfs_trigger_flush(fs_info,
- space_info->flags,
- orig_bytes, flush,
- "enospc");
- queue_work(system_unbound_wq,
- &fs_info->async_reclaim_work);
- }
- } else {
- list_add_tail(&ticket.list,
- &space_info->priority_tickets);
- }
- } else if (!ret && space_info->flags & BTRFS_BLOCK_GROUP_METADATA) {
- used += orig_bytes;
- /*
- * We will do the space reservation dance during log replay,
- * which means we won't have fs_info->fs_root set, so don't do
- * the async reclaim as we will panic.
- */
- if (!test_bit(BTRFS_FS_LOG_RECOVERING, &fs_info->flags) &&
- need_do_async_reclaim(fs_info, space_info,
- used, system_chunk) &&
- !work_busy(&fs_info->async_reclaim_work)) {
- trace_btrfs_trigger_flush(fs_info, space_info->flags,
- orig_bytes, flush, "preempt");
- queue_work(system_unbound_wq,
- &fs_info->async_reclaim_work);
- }
- }
- spin_unlock(&space_info->lock);
- if (!ret || flush == BTRFS_RESERVE_NO_FLUSH)
- return ret;
-
- if (flush == BTRFS_RESERVE_FLUSH_ALL)
- return wait_reserve_ticket(fs_info, space_info, &ticket);
-
- ret = 0;
- priority_reclaim_metadata_space(fs_info, space_info, &ticket);
- spin_lock(&space_info->lock);
- if (ticket.bytes) {
- if (ticket.bytes < orig_bytes)
- reclaim_bytes = orig_bytes - ticket.bytes;
- list_del_init(&ticket.list);
- ret = -ENOSPC;
- }
- spin_unlock(&space_info->lock);
-
- if (reclaim_bytes)
- space_info_add_old_bytes(fs_info, space_info, reclaim_bytes);
- ASSERT(list_empty(&ticket.list));
- return ret;
-}
-
-/**
- * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
- * @root - the root we're allocating for
- * @block_rsv - the block_rsv we're allocating for
- * @orig_bytes - the number of bytes we want
- * @flush - whether or not we can flush to make our reservation
- *
- * This will reserve orig_bytes number of bytes from the space info associated
- * with the block_rsv. If there is not enough space it will make an attempt to
- * flush out space to make room. It will do this by flushing delalloc if
- * possible or committing the transaction. If flush is 0 then no attempts to
- * regain reservations will be made and this will fail if there is not enough
- * space already.
- */
-static int reserve_metadata_bytes(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv,
- u64 orig_bytes,
- enum btrfs_reserve_flush_enum flush)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- int ret;
- bool system_chunk = (root == fs_info->chunk_root);
-
- ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info,
- orig_bytes, flush, system_chunk);
- if (ret == -ENOSPC &&
- unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) {
- if (block_rsv != global_rsv &&
- !block_rsv_use_bytes(global_rsv, orig_bytes))
- ret = 0;
- }
- if (ret == -ENOSPC) {
- trace_btrfs_space_reservation(fs_info, "space_info:enospc",
- block_rsv->space_info->flags,
- orig_bytes, 1);
-
- if (btrfs_test_opt(fs_info, ENOSPC_DEBUG))
- dump_space_info(fs_info, block_rsv->space_info,
- orig_bytes, 0);
- }
- return ret;
-}
-
-static struct btrfs_block_rsv *get_block_rsv(
- const struct btrfs_trans_handle *trans,
- const struct btrfs_root *root)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_block_rsv *block_rsv = NULL;
-
- if (test_bit(BTRFS_ROOT_REF_COWS, &root->state) ||
- (root == fs_info->csum_root && trans->adding_csums) ||
- (root == fs_info->uuid_root))
- block_rsv = trans->block_rsv;
-
- if (!block_rsv)
- block_rsv = root->block_rsv;
-
- if (!block_rsv)
- block_rsv = &fs_info->empty_block_rsv;
-
- return block_rsv;
-}
-
-static int block_rsv_use_bytes(struct btrfs_block_rsv *block_rsv,
- u64 num_bytes)
-{
- int ret = -ENOSPC;
- spin_lock(&block_rsv->lock);
- if (block_rsv->reserved >= num_bytes) {
- block_rsv->reserved -= num_bytes;
- if (block_rsv->reserved < block_rsv->size)
- block_rsv->full = 0;
- ret = 0;
- }
- spin_unlock(&block_rsv->lock);
- return ret;
-}
-
-static void block_rsv_add_bytes(struct btrfs_block_rsv *block_rsv,
- u64 num_bytes, bool update_size)
-{
- spin_lock(&block_rsv->lock);
- block_rsv->reserved += num_bytes;
- if (update_size)
- block_rsv->size += num_bytes;
- else if (block_rsv->reserved >= block_rsv->size)
- block_rsv->full = 1;
- spin_unlock(&block_rsv->lock);
-}
-
-int btrfs_cond_migrate_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *dest, u64 num_bytes,
- int min_factor)
-{
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- u64 min_bytes;
-
- if (global_rsv->space_info != dest->space_info)
- return -ENOSPC;
-
- spin_lock(&global_rsv->lock);
- min_bytes = div_factor(global_rsv->size, min_factor);
- if (global_rsv->reserved < min_bytes + num_bytes) {
- spin_unlock(&global_rsv->lock);
- return -ENOSPC;
- }
- global_rsv->reserved -= num_bytes;
- if (global_rsv->reserved < global_rsv->size)
- global_rsv->full = 0;
- spin_unlock(&global_rsv->lock);
-
- block_rsv_add_bytes(dest, num_bytes, true);
- return 0;
-}
-
-/**
- * btrfs_migrate_to_delayed_refs_rsv - transfer bytes to our delayed refs rsv.
- * @fs_info - the fs info for our fs.
- * @src - the source block rsv to transfer from.
- * @num_bytes - the number of bytes to transfer.
- *
- * This transfers up to the num_bytes amount from the src rsv to the
- * delayed_refs_rsv. Any extra bytes are returned to the space info.
- */
-void btrfs_migrate_to_delayed_refs_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *src,
- u64 num_bytes)
-{
- struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
- u64 to_free = 0;
-
- spin_lock(&src->lock);
- src->reserved -= num_bytes;
- src->size -= num_bytes;
- spin_unlock(&src->lock);
-
- spin_lock(&delayed_refs_rsv->lock);
- if (delayed_refs_rsv->size > delayed_refs_rsv->reserved) {
- u64 delta = delayed_refs_rsv->size -
- delayed_refs_rsv->reserved;
- if (num_bytes > delta) {
- to_free = num_bytes - delta;
- num_bytes = delta;
- }
- } else {
- to_free = num_bytes;
- num_bytes = 0;
- }
-
- if (num_bytes)
- delayed_refs_rsv->reserved += num_bytes;
- if (delayed_refs_rsv->reserved >= delayed_refs_rsv->size)
- delayed_refs_rsv->full = 1;
- spin_unlock(&delayed_refs_rsv->lock);
-
- if (num_bytes)
- trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
- 0, num_bytes, 1);
- if (to_free)
- space_info_add_old_bytes(fs_info, delayed_refs_rsv->space_info,
- to_free);
-}
-
-/**
- * btrfs_delayed_refs_rsv_refill - refill based on our delayed refs usage.
- * @fs_info - the fs_info for our fs.
- * @flush - control how we can flush for this reservation.
- *
- * This will refill the delayed block_rsv up to 1 items size worth of space and
- * will return -ENOSPC if we can't make the reservation.
- */
-int btrfs_delayed_refs_rsv_refill(struct btrfs_fs_info *fs_info,
- enum btrfs_reserve_flush_enum flush)
-{
- struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv;
- u64 limit = btrfs_calc_trans_metadata_size(fs_info, 1);
- u64 num_bytes = 0;
- int ret = -ENOSPC;
-
- spin_lock(&block_rsv->lock);
- if (block_rsv->reserved < block_rsv->size) {
- num_bytes = block_rsv->size - block_rsv->reserved;
- num_bytes = min(num_bytes, limit);
- }
- spin_unlock(&block_rsv->lock);
-
- if (!num_bytes)
- return 0;
-
- ret = reserve_metadata_bytes(fs_info->extent_root, block_rsv,
- num_bytes, flush);
- if (ret)
- return ret;
- block_rsv_add_bytes(block_rsv, num_bytes, 0);
- trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
- 0, num_bytes, 1);
- return 0;
-}
-
-/*
- * This is for space we already have accounted in space_info->bytes_may_use, so
- * basically when we're returning space from block_rsv's.
- */
-static void space_info_add_old_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 num_bytes)
-{
- struct reserve_ticket *ticket;
- struct list_head *head;
- u64 used;
- enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_NO_FLUSH;
- bool check_overcommit = false;
-
- spin_lock(&space_info->lock);
- head = &space_info->priority_tickets;
-
- /*
- * If we are over our limit then we need to check and see if we can
- * overcommit, and if we can't then we just need to free up our space
- * and not satisfy any requests.
- */
- used = btrfs_space_info_used(space_info, true);
- if (used - num_bytes >= space_info->total_bytes)
- check_overcommit = true;
-again:
- while (!list_empty(head) && num_bytes) {
- ticket = list_first_entry(head, struct reserve_ticket,
- list);
- /*
- * We use 0 bytes because this space is already reserved, so
- * adding the ticket space would be a double count.
- */
- if (check_overcommit &&
- !can_overcommit(fs_info, space_info, 0, flush, false))
- break;
- if (num_bytes >= ticket->bytes) {
- list_del_init(&ticket->list);
- num_bytes -= ticket->bytes;
- ticket->bytes = 0;
- space_info->tickets_id++;
- wake_up(&ticket->wait);
- } else {
- ticket->bytes -= num_bytes;
- num_bytes = 0;
- }
- }
-
- if (num_bytes && head == &space_info->priority_tickets) {
- head = &space_info->tickets;
- flush = BTRFS_RESERVE_FLUSH_ALL;
- goto again;
- }
- update_bytes_may_use(space_info, -num_bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- space_info->flags, num_bytes, 0);
- spin_unlock(&space_info->lock);
-}
-
-/*
- * This is for newly allocated space that isn't accounted in
- * space_info->bytes_may_use yet. So if we allocate a chunk or unpin an extent
- * we use this helper.
- */
-static void space_info_add_new_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *space_info,
- u64 num_bytes)
-{
- struct reserve_ticket *ticket;
- struct list_head *head = &space_info->priority_tickets;
-
-again:
- while (!list_empty(head) && num_bytes) {
- ticket = list_first_entry(head, struct reserve_ticket,
- list);
- if (num_bytes >= ticket->bytes) {
- trace_btrfs_space_reservation(fs_info, "space_info",
- space_info->flags,
- ticket->bytes, 1);
- list_del_init(&ticket->list);
- num_bytes -= ticket->bytes;
- update_bytes_may_use(space_info, ticket->bytes);
- ticket->bytes = 0;
- space_info->tickets_id++;
- wake_up(&ticket->wait);
- } else {
- trace_btrfs_space_reservation(fs_info, "space_info",
- space_info->flags,
- num_bytes, 1);
- update_bytes_may_use(space_info, num_bytes);
- ticket->bytes -= num_bytes;
- num_bytes = 0;
- }
- }
-
- if (num_bytes && head == &space_info->priority_tickets) {
- head = &space_info->tickets;
- goto again;
- }
-}
-
-static u64 block_rsv_release_bytes(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *block_rsv,
- struct btrfs_block_rsv *dest, u64 num_bytes,
- u64 *qgroup_to_release_ret)
-{
- struct btrfs_space_info *space_info = block_rsv->space_info;
- u64 qgroup_to_release = 0;
- u64 ret;
-
- spin_lock(&block_rsv->lock);
- if (num_bytes == (u64)-1) {
- num_bytes = block_rsv->size;
- qgroup_to_release = block_rsv->qgroup_rsv_size;
- }
- block_rsv->size -= num_bytes;
- if (block_rsv->reserved >= block_rsv->size) {
- num_bytes = block_rsv->reserved - block_rsv->size;
- block_rsv->reserved = block_rsv->size;
- block_rsv->full = 1;
- } else {
- num_bytes = 0;
- }
- if (block_rsv->qgroup_rsv_reserved >= block_rsv->qgroup_rsv_size) {
- qgroup_to_release = block_rsv->qgroup_rsv_reserved -
- block_rsv->qgroup_rsv_size;
- block_rsv->qgroup_rsv_reserved = block_rsv->qgroup_rsv_size;
- } else {
- qgroup_to_release = 0;
- }
- spin_unlock(&block_rsv->lock);
-
- ret = num_bytes;
- if (num_bytes > 0) {
- if (dest) {
- spin_lock(&dest->lock);
- if (!dest->full) {
- u64 bytes_to_add;
-
- bytes_to_add = dest->size - dest->reserved;
- bytes_to_add = min(num_bytes, bytes_to_add);
- dest->reserved += bytes_to_add;
- if (dest->reserved >= dest->size)
- dest->full = 1;
- num_bytes -= bytes_to_add;
- }
- spin_unlock(&dest->lock);
- }
- if (num_bytes)
- space_info_add_old_bytes(fs_info, space_info,
- num_bytes);
- }
- if (qgroup_to_release_ret)
- *qgroup_to_release_ret = qgroup_to_release;
- return ret;
-}
-
-int btrfs_block_rsv_migrate(struct btrfs_block_rsv *src,
- struct btrfs_block_rsv *dst, u64 num_bytes,
- bool update_size)
-{
- int ret;
-
- ret = block_rsv_use_bytes(src, num_bytes);
- if (ret)
- return ret;
-
- block_rsv_add_bytes(dst, num_bytes, update_size);
- return 0;
-}
-
-void btrfs_init_block_rsv(struct btrfs_block_rsv *rsv, unsigned short type)
-{
- memset(rsv, 0, sizeof(*rsv));
- spin_lock_init(&rsv->lock);
- rsv->type = type;
-}
-
-void btrfs_init_metadata_block_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *rsv,
- unsigned short type)
-{
- btrfs_init_block_rsv(rsv, type);
- rsv->space_info = __find_space_info(fs_info,
- BTRFS_BLOCK_GROUP_METADATA);
-}
-
-struct btrfs_block_rsv *btrfs_alloc_block_rsv(struct btrfs_fs_info *fs_info,
- unsigned short type)
-{
- struct btrfs_block_rsv *block_rsv;
-
- block_rsv = kmalloc(sizeof(*block_rsv), GFP_NOFS);
- if (!block_rsv)
- return NULL;
-
- btrfs_init_metadata_block_rsv(fs_info, block_rsv, type);
- return block_rsv;
-}
-
-void btrfs_free_block_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *rsv)
-{
- if (!rsv)
- return;
- btrfs_block_rsv_release(fs_info, rsv, (u64)-1);
- kfree(rsv);
-}
-
-int btrfs_block_rsv_add(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, u64 num_bytes,
- enum btrfs_reserve_flush_enum flush)
-{
- int ret;
-
- if (num_bytes == 0)
- return 0;
-
- ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush);
- if (!ret)
- block_rsv_add_bytes(block_rsv, num_bytes, true);
-
- return ret;
-}
-
-int btrfs_block_rsv_check(struct btrfs_block_rsv *block_rsv, int min_factor)
-{
- u64 num_bytes = 0;
- int ret = -ENOSPC;
-
- if (!block_rsv)
- return 0;
-
- spin_lock(&block_rsv->lock);
- num_bytes = div_factor(block_rsv->size, min_factor);
- if (block_rsv->reserved >= num_bytes)
- ret = 0;
- spin_unlock(&block_rsv->lock);
-
- return ret;
-}
-
-int btrfs_block_rsv_refill(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, u64 min_reserved,
- enum btrfs_reserve_flush_enum flush)
-{
- u64 num_bytes = 0;
- int ret = -ENOSPC;
-
- if (!block_rsv)
- return 0;
-
- spin_lock(&block_rsv->lock);
- num_bytes = min_reserved;
- if (block_rsv->reserved >= num_bytes)
- ret = 0;
- else
- num_bytes -= block_rsv->reserved;
- spin_unlock(&block_rsv->lock);
-
- if (!ret)
- return 0;
-
- ret = reserve_metadata_bytes(root, block_rsv, num_bytes, flush);
- if (!ret) {
- block_rsv_add_bytes(block_rsv, num_bytes, false);
- return 0;
- }
-
- return ret;
-}
-
-static u64 __btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *block_rsv,
- u64 num_bytes, u64 *qgroup_to_release)
-{
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv;
- struct btrfs_block_rsv *target = delayed_rsv;
-
- if (target->full || target == block_rsv)
- target = global_rsv;
-
- if (block_rsv->space_info != target->space_info)
- target = NULL;
-
- return block_rsv_release_bytes(fs_info, block_rsv, target, num_bytes,
- qgroup_to_release);
-}
-
-void btrfs_block_rsv_release(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *block_rsv,
- u64 num_bytes)
-{
- __btrfs_block_rsv_release(fs_info, block_rsv, num_bytes, NULL);
-}
-
-/**
- * btrfs_inode_rsv_release - release any excessive reservation.
- * @inode - the inode we need to release from.
- * @qgroup_free - free or convert qgroup meta.
- * Unlike normal operation, qgroup meta reservation needs to know if we are
- * freeing qgroup reservation or just converting it into per-trans. Normally
- * @qgroup_free is true for error handling, and false for normal release.
- *
- * This is the same as btrfs_block_rsv_release, except that it handles the
- * tracepoint for the reservation.
- */
-static void btrfs_inode_rsv_release(struct btrfs_inode *inode, bool qgroup_free)
-{
- struct btrfs_fs_info *fs_info = inode->root->fs_info;
- struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
- u64 released = 0;
- u64 qgroup_to_release = 0;
-
- /*
- * Since we statically set the block_rsv->size we just want to say we
- * are releasing 0 bytes, and then we'll just get the reservation over
- * the size free'd.
- */
- released = __btrfs_block_rsv_release(fs_info, block_rsv, 0,
- &qgroup_to_release);
- if (released > 0)
- trace_btrfs_space_reservation(fs_info, "delalloc",
- btrfs_ino(inode), released, 0);
- if (qgroup_free)
- btrfs_qgroup_free_meta_prealloc(inode->root, qgroup_to_release);
- else
- btrfs_qgroup_convert_reserved_meta(inode->root,
- qgroup_to_release);
-}
-
-/**
- * btrfs_delayed_refs_rsv_release - release a ref head's reservation.
- * @fs_info - the fs_info for our fs.
- * @nr - the number of items to drop.
- *
- * This drops the delayed ref head's count from the delayed refs rsv and frees
- * any excess reservation we had.
- */
-void btrfs_delayed_refs_rsv_release(struct btrfs_fs_info *fs_info, int nr)
-{
- struct btrfs_block_rsv *block_rsv = &fs_info->delayed_refs_rsv;
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- u64 num_bytes = btrfs_calc_trans_metadata_size(fs_info, nr);
- u64 released = 0;
-
- released = block_rsv_release_bytes(fs_info, block_rsv, global_rsv,
- num_bytes, NULL);
- if (released)
- trace_btrfs_space_reservation(fs_info, "delayed_refs_rsv",
- 0, released, 0);
-}
-
-static void update_global_block_rsv(struct btrfs_fs_info *fs_info)
-{
- struct btrfs_block_rsv *block_rsv = &fs_info->global_block_rsv;
- struct btrfs_space_info *sinfo = block_rsv->space_info;
- u64 num_bytes;
-
- /*
- * The global block rsv is based on the size of the extent tree, the
- * checksum tree and the root tree. If the fs is empty we want to set
- * it to a minimal amount for safety.
- */
- num_bytes = btrfs_root_used(&fs_info->extent_root->root_item) +
- btrfs_root_used(&fs_info->csum_root->root_item) +
- btrfs_root_used(&fs_info->tree_root->root_item);
- num_bytes = max_t(u64, num_bytes, SZ_16M);
-
- spin_lock(&sinfo->lock);
- spin_lock(&block_rsv->lock);
-
- block_rsv->size = min_t(u64, num_bytes, SZ_512M);
-
- if (block_rsv->reserved < block_rsv->size) {
- num_bytes = btrfs_space_info_used(sinfo, true);
- if (sinfo->total_bytes > num_bytes) {
- num_bytes = sinfo->total_bytes - num_bytes;
- num_bytes = min(num_bytes,
- block_rsv->size - block_rsv->reserved);
- block_rsv->reserved += num_bytes;
- update_bytes_may_use(sinfo, num_bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- sinfo->flags, num_bytes,
- 1);
- }
- } else if (block_rsv->reserved > block_rsv->size) {
- num_bytes = block_rsv->reserved - block_rsv->size;
- update_bytes_may_use(sinfo, -num_bytes);
- trace_btrfs_space_reservation(fs_info, "space_info",
- sinfo->flags, num_bytes, 0);
- block_rsv->reserved = block_rsv->size;
- }
-
- if (block_rsv->reserved == block_rsv->size)
- block_rsv->full = 1;
- else
- block_rsv->full = 0;
-
- spin_unlock(&block_rsv->lock);
- spin_unlock(&sinfo->lock);
-}
-
-static void init_global_block_rsv(struct btrfs_fs_info *fs_info)
-{
- struct btrfs_space_info *space_info;
-
- space_info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM);
- fs_info->chunk_block_rsv.space_info = space_info;
-
- space_info = __find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
- fs_info->global_block_rsv.space_info = space_info;
- fs_info->trans_block_rsv.space_info = space_info;
- fs_info->empty_block_rsv.space_info = space_info;
- fs_info->delayed_block_rsv.space_info = space_info;
- fs_info->delayed_refs_rsv.space_info = space_info;
-
- fs_info->extent_root->block_rsv = &fs_info->delayed_refs_rsv;
- fs_info->csum_root->block_rsv = &fs_info->delayed_refs_rsv;
- fs_info->dev_root->block_rsv = &fs_info->global_block_rsv;
- fs_info->tree_root->block_rsv = &fs_info->global_block_rsv;
- if (fs_info->quota_root)
- fs_info->quota_root->block_rsv = &fs_info->global_block_rsv;
- fs_info->chunk_root->block_rsv = &fs_info->chunk_block_rsv;
-
- update_global_block_rsv(fs_info);
-}
-
-static void release_global_block_rsv(struct btrfs_fs_info *fs_info)
-{
- block_rsv_release_bytes(fs_info, &fs_info->global_block_rsv, NULL,
- (u64)-1, NULL);
- WARN_ON(fs_info->trans_block_rsv.size > 0);
- WARN_ON(fs_info->trans_block_rsv.reserved > 0);
- WARN_ON(fs_info->chunk_block_rsv.size > 0);
- WARN_ON(fs_info->chunk_block_rsv.reserved > 0);
- WARN_ON(fs_info->delayed_block_rsv.size > 0);
- WARN_ON(fs_info->delayed_block_rsv.reserved > 0);
- WARN_ON(fs_info->delayed_refs_rsv.reserved > 0);
- WARN_ON(fs_info->delayed_refs_rsv.size > 0);
-}
-
-/*
- * btrfs_update_delayed_refs_rsv - adjust the size of the delayed refs rsv
- * @trans - the trans that may have generated delayed refs
- *
- * This is to be called anytime we may have adjusted trans->delayed_ref_updates,
- * it'll calculate the additional size and add it to the delayed_refs_rsv.
- */
-void btrfs_update_delayed_refs_rsv(struct btrfs_trans_handle *trans)
-{
- struct btrfs_fs_info *fs_info = trans->fs_info;
- struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_refs_rsv;
- u64 num_bytes;
-
- if (!trans->delayed_ref_updates)
- return;
-
- num_bytes = btrfs_calc_trans_metadata_size(fs_info,
- trans->delayed_ref_updates);
- spin_lock(&delayed_rsv->lock);
- delayed_rsv->size += num_bytes;
- delayed_rsv->full = 0;
- spin_unlock(&delayed_rsv->lock);
- trans->delayed_ref_updates = 0;
-}
-
-/*
- * To be called after all the new block groups attached to the transaction
- * handle have been created (btrfs_create_pending_block_groups()).
- */
-void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
-{
- struct btrfs_fs_info *fs_info = trans->fs_info;
-
- if (!trans->chunk_bytes_reserved)
- return;
-
- WARN_ON_ONCE(!list_empty(&trans->new_bgs));
-
- block_rsv_release_bytes(fs_info, &fs_info->chunk_block_rsv, NULL,
- trans->chunk_bytes_reserved, NULL);
- trans->chunk_bytes_reserved = 0;
-}
-
-/*
- * btrfs_subvolume_reserve_metadata() - reserve space for subvolume operation
- * root: the root of the parent directory
- * rsv: block reservation
- * items: the number of items that we need do reservation
- * use_global_rsv: allow fallback to the global block reservation
- *
- * This function is used to reserve the space for snapshot/subvolume
- * creation and deletion. Those operations are different with the
- * common file/directory operations, they change two fs/file trees
- * and root tree, the number of items that the qgroup reserves is
- * different with the free space reservation. So we can not use
- * the space reservation mechanism in start_transaction().
- */
-int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,
- struct btrfs_block_rsv *rsv, int items,
- bool use_global_rsv)
-{
- u64 qgroup_num_bytes = 0;
- u64 num_bytes;
- int ret;
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
-
- if (test_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags)) {
- /* One for parent inode, two for dir entries */
- qgroup_num_bytes = 3 * fs_info->nodesize;
- ret = btrfs_qgroup_reserve_meta_prealloc(root,
- qgroup_num_bytes, true);
- if (ret)
- return ret;
- }
-
- num_bytes = btrfs_calc_trans_metadata_size(fs_info, items);
- rsv->space_info = __find_space_info(fs_info,
- BTRFS_BLOCK_GROUP_METADATA);
- ret = btrfs_block_rsv_add(root, rsv, num_bytes,
- BTRFS_RESERVE_FLUSH_ALL);
-
- if (ret == -ENOSPC && use_global_rsv)
- ret = btrfs_block_rsv_migrate(global_rsv, rsv, num_bytes, true);
-
- if (ret && qgroup_num_bytes)
- btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes);
-
- return ret;
-}
-
-void btrfs_subvolume_release_metadata(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *rsv)
-{
- btrfs_block_rsv_release(fs_info, rsv, (u64)-1);
-}
-
-static void btrfs_calculate_inode_block_rsv_size(struct btrfs_fs_info *fs_info,
- struct btrfs_inode *inode)
-{
- struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
- u64 reserve_size = 0;
- u64 qgroup_rsv_size = 0;
- u64 csum_leaves;
- unsigned outstanding_extents;
-
- lockdep_assert_held(&inode->lock);
- outstanding_extents = inode->outstanding_extents;
- if (outstanding_extents)
- reserve_size = btrfs_calc_trans_metadata_size(fs_info,
- outstanding_extents + 1);
- csum_leaves = btrfs_csum_bytes_to_leaves(fs_info,
- inode->csum_bytes);
- reserve_size += btrfs_calc_trans_metadata_size(fs_info,
- csum_leaves);
- /*
- * For qgroup rsv, the calculation is very simple:
- * account one nodesize for each outstanding extent
- *
- * This is overestimating in most cases.
- */
- qgroup_rsv_size = (u64)outstanding_extents * fs_info->nodesize;
+ btrfs_info(fs_info, "left=%llu, need=%llu, flags=%llu",
+ left, thresh, type);
+ btrfs_dump_space_info(fs_info, info, 0, 0);
+ }
- spin_lock(&block_rsv->lock);
- block_rsv->size = reserve_size;
- block_rsv->qgroup_rsv_size = qgroup_rsv_size;
- spin_unlock(&block_rsv->lock);
-}
+ if (left < thresh) {
+ u64 flags = btrfs_system_alloc_profile(fs_info);
-static void calc_inode_reservations(struct btrfs_fs_info *fs_info,
- u64 num_bytes, u64 *meta_reserve,
- u64 *qgroup_reserve)
-{
- u64 nr_extents = count_max_extents(num_bytes);
- u64 csum_leaves = btrfs_csum_bytes_to_leaves(fs_info, num_bytes);
+ /*
+ * Ignore failure to create system chunk. We might end up not
+ * needing it, as we might not need to COW all nodes/leafs from
+ * the paths we visit in the chunk tree (they were already COWed
+ * or created in the current transaction for example).
+ */
+ ret = btrfs_alloc_chunk(trans, flags);
+ }
- /* We add one for the inode update at finish ordered time */
- *meta_reserve = btrfs_calc_trans_metadata_size(fs_info,
- nr_extents + csum_leaves + 1);
- *qgroup_reserve = nr_extents * fs_info->nodesize;
+ if (!ret) {
+ ret = btrfs_block_rsv_add(fs_info->chunk_root,
+ &fs_info->chunk_block_rsv,
+ thresh, BTRFS_RESERVE_NO_FLUSH);
+ if (!ret)
+ trans->chunk_bytes_reserved += thresh;
+ }
}
-int btrfs_delalloc_reserve_metadata(struct btrfs_inode *inode, u64 num_bytes)
+/*
+ * If force is CHUNK_ALLOC_FORCE:
+ * - return 1 if it successfully allocates a chunk,
+ * - return errors including -ENOSPC otherwise.
+ * If force is NOT CHUNK_ALLOC_FORCE:
+ * - return 0 if it doesn't need to allocate a new chunk,
+ * - return 1 if it successfully allocates a chunk,
+ * - return errors including -ENOSPC otherwise.
+ */
+int btrfs_chunk_alloc(struct btrfs_trans_handle *trans, u64 flags,
+ enum btrfs_chunk_alloc_enum force)
{
- struct btrfs_root *root = inode->root;
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_block_rsv *block_rsv = &inode->block_rsv;
- u64 meta_reserve, qgroup_reserve;
- unsigned nr_extents;
- enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_FLUSH_ALL;
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+ struct btrfs_space_info *space_info;
+ bool wait_for_alloc = false;
+ bool should_alloc = false;
int ret = 0;
- bool delalloc_lock = true;
- /* If we are a free space inode we need to not flush since we will be in
- * the middle of a transaction commit. We also don't need the delalloc
- * mutex since we won't race with anybody. We need this mostly to make
- * lockdep shut its filthy mouth.
- *
- * If we have a transaction open (can happen if we call truncate_block
- * from truncate), then we need FLUSH_LIMIT so we don't deadlock.
- */
- if (btrfs_is_free_space_inode(inode)) {
- flush = BTRFS_RESERVE_NO_FLUSH;
- delalloc_lock = false;
- } else {
- if (current->journal_info)
- flush = BTRFS_RESERVE_FLUSH_LIMIT;
+ /* Don't re-enter if we're already allocating a chunk */
+ if (trans->allocating_chunk)
+ return -ENOSPC;
- if (btrfs_transaction_in_commit(fs_info))
- schedule_timeout(1);
- }
+ space_info = btrfs_find_space_info(fs_info, flags);
+ ASSERT(space_info);
+
+ do {
+ spin_lock(&space_info->lock);
+ if (force < space_info->force_alloc)
+ force = space_info->force_alloc;
+ should_alloc = should_alloc_chunk(fs_info, space_info, force);
+ if (space_info->full) {
+ /* No more free physical space */
+ if (should_alloc)
+ ret = -ENOSPC;
+ else
+ ret = 0;
+ spin_unlock(&space_info->lock);
+ return ret;
+ } else if (!should_alloc) {
+ spin_unlock(&space_info->lock);
+ return 0;
+ } else if (space_info->chunk_alloc) {
+ /*
+ * Someone is already allocating, so we need to block
+ * until this someone is finished and then loop to
+ * recheck if we should continue with our allocation
+ * attempt.
+ */
+ wait_for_alloc = true;
+ spin_unlock(&space_info->lock);
+ mutex_lock(&fs_info->chunk_mutex);
+ mutex_unlock(&fs_info->chunk_mutex);
+ } else {
+ /* Proceed with allocation */
+ space_info->chunk_alloc = 1;
+ wait_for_alloc = false;
+ spin_unlock(&space_info->lock);
+ }
- if (delalloc_lock)
- mutex_lock(&inode->delalloc_mutex);
+ cond_resched();
+ } while (wait_for_alloc);
- num_bytes = ALIGN(num_bytes, fs_info->sectorsize);
+ mutex_lock(&fs_info->chunk_mutex);
+ trans->allocating_chunk = true;
/*
- * We always want to do it this way, every other way is wrong and ends
- * in tears. Pre-reserving the amount we are going to add will always
- * be the right way, because otherwise if we have enough parallelism we
- * could end up with thousands of inodes all holding little bits of
- * reservations they were able to make previously and the only way to
- * reclaim that space is to ENOSPC out the operations and clear
- * everything out and try again, which is bad. This way we just
- * over-reserve slightly, and clean up the mess when we are done.
+ * If we have mixed data/metadata chunks we want to make sure we keep
+ * allocating mixed chunks instead of individual chunks.
*/
- calc_inode_reservations(fs_info, num_bytes, &meta_reserve,
- &qgroup_reserve);
- ret = btrfs_qgroup_reserve_meta_prealloc(root, qgroup_reserve, true);
- if (ret)
- goto out_fail;
- ret = reserve_metadata_bytes(root, block_rsv, meta_reserve, flush);
- if (ret)
- goto out_qgroup;
+ if (btrfs_mixed_space_info(space_info))
+ flags |= (BTRFS_BLOCK_GROUP_DATA | BTRFS_BLOCK_GROUP_METADATA);
/*
- * Now we need to update our outstanding extents and csum bytes _first_
- * and then add the reservation to the block_rsv. This keeps us from
- * racing with an ordered completion or some such that would think it
- * needs to free the reservation we just made.
+ * if we're doing a data chunk, go ahead and make sure that
+ * we keep a reasonable number of metadata chunks allocated in the
+ * FS as well.
*/
- spin_lock(&inode->lock);
- nr_extents = count_max_extents(num_bytes);
- btrfs_mod_outstanding_extents(inode, nr_extents);
- inode->csum_bytes += num_bytes;
- btrfs_calculate_inode_block_rsv_size(fs_info, inode);
- spin_unlock(&inode->lock);
-
- /* Now we can safely add our space to our block rsv */
- block_rsv_add_bytes(block_rsv, meta_reserve, false);
- trace_btrfs_space_reservation(root->fs_info, "delalloc",
- btrfs_ino(inode), meta_reserve, 1);
-
- spin_lock(&block_rsv->lock);
- block_rsv->qgroup_rsv_reserved += qgroup_reserve;
- spin_unlock(&block_rsv->lock);
-
- if (delalloc_lock)
- mutex_unlock(&inode->delalloc_mutex);
- return 0;
-out_qgroup:
- btrfs_qgroup_free_meta_prealloc(root, qgroup_reserve);
-out_fail:
- btrfs_inode_rsv_release(inode, true);
- if (delalloc_lock)
- mutex_unlock(&inode->delalloc_mutex);
- return ret;
-}
-
-/**
- * btrfs_delalloc_release_metadata - release a metadata reservation for an inode
- * @inode: the inode to release the reservation for.
- * @num_bytes: the number of bytes we are releasing.
- * @qgroup_free: free qgroup reservation or convert it to per-trans reservation
- *
- * This will release the metadata reservation for an inode. This can be called
- * once we complete IO for a given set of bytes to release their metadata
- * reservations, or on error for the same reason.
- */
-void btrfs_delalloc_release_metadata(struct btrfs_inode *inode, u64 num_bytes,
- bool qgroup_free)
-{
- struct btrfs_fs_info *fs_info = inode->root->fs_info;
-
- num_bytes = ALIGN(num_bytes, fs_info->sectorsize);
- spin_lock(&inode->lock);
- inode->csum_bytes -= num_bytes;
- btrfs_calculate_inode_block_rsv_size(fs_info, inode);
- spin_unlock(&inode->lock);
-
- if (btrfs_is_testing(fs_info))
- return;
-
- btrfs_inode_rsv_release(inode, qgroup_free);
-}
-
-/**
- * btrfs_delalloc_release_extents - release our outstanding_extents
- * @inode: the inode to balance the reservation for.
- * @num_bytes: the number of bytes we originally reserved with
- * @qgroup_free: do we need to free qgroup meta reservation or convert them.
- *
- * When we reserve space we increase outstanding_extents for the extents we may
- * add. Once we've set the range as delalloc or created our ordered extents we
- * have outstanding_extents to track the real usage, so we use this to free our
- * temporarily tracked outstanding_extents. This _must_ be used in conjunction
- * with btrfs_delalloc_reserve_metadata.
- */
-void btrfs_delalloc_release_extents(struct btrfs_inode *inode, u64 num_bytes,
- bool qgroup_free)
-{
- struct btrfs_fs_info *fs_info = inode->root->fs_info;
- unsigned num_extents;
+ if (flags & BTRFS_BLOCK_GROUP_DATA && fs_info->metadata_ratio) {
+ fs_info->data_chunk_allocations++;
+ if (!(fs_info->data_chunk_allocations %
+ fs_info->metadata_ratio))
+ force_metadata_allocation(fs_info);
+ }
- spin_lock(&inode->lock);
- num_extents = count_max_extents(num_bytes);
- btrfs_mod_outstanding_extents(inode, -num_extents);
- btrfs_calculate_inode_block_rsv_size(fs_info, inode);
- spin_unlock(&inode->lock);
+ /*
+ * Check if we have enough space in SYSTEM chunk because we may need
+ * to update devices.
+ */
+ check_system_chunk(trans, flags);
- if (btrfs_is_testing(fs_info))
- return;
+ ret = btrfs_alloc_chunk(trans, flags);
+ trans->allocating_chunk = false;
- btrfs_inode_rsv_release(inode, qgroup_free);
-}
+ spin_lock(&space_info->lock);
+ if (ret < 0) {
+ if (ret == -ENOSPC)
+ space_info->full = 1;
+ else
+ goto out;
+ } else {
+ ret = 1;
+ space_info->max_extent_size = 0;
+ }
-/**
- * btrfs_delalloc_reserve_space - reserve data and metadata space for
- * delalloc
- * @inode: inode we're writing to
- * @start: start range we are writing to
- * @len: how long the range we are writing to
- * @reserved: mandatory parameter, record actually reserved qgroup ranges of
- * current reservation.
- *
- * This will do the following things
- *
- * o reserve space in data space info for num bytes
- * and reserve precious corresponding qgroup space
- * (Done in check_data_free_space)
- *
- * o reserve space for metadata space, based on the number of outstanding
- * extents and how much csums will be needed
- * also reserve metadata space in a per root over-reserve method.
- * o add to the inodes->delalloc_bytes
- * o add it to the fs_info's delalloc inodes list.
- * (Above 3 all done in delalloc_reserve_metadata)
- *
- * Return 0 for success
- * Return <0 for error(-ENOSPC or -EQUOT)
- */
-int btrfs_delalloc_reserve_space(struct inode *inode,
- struct extent_changeset **reserved, u64 start, u64 len)
-{
- int ret;
+ space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
+out:
+ space_info->chunk_alloc = 0;
+ spin_unlock(&space_info->lock);
+ mutex_unlock(&fs_info->chunk_mutex);
+ /*
+ * When we allocate a new chunk we reserve space in the chunk block
+ * reserve to make sure we can COW nodes/leafs in the chunk tree or
+ * add new nodes/leafs to it if we end up needing to do it when
+ * inserting the chunk item and updating device items as part of the
+ * second phase of chunk allocation, performed by
+ * btrfs_finish_chunk_alloc(). So make sure we don't accumulate a
+ * large number of new block groups to create in our transaction
+ * handle's new_bgs list to avoid exhausting the chunk block reserve
+ * in extreme cases - like having a single transaction create many new
+ * block groups when starting to write out the free space caches of all
+ * the block groups that were made dirty during the lifetime of the
+ * transaction.
+ */
+ if (trans->chunk_bytes_reserved >= (u64)SZ_2M)
+ btrfs_create_pending_block_groups(trans);
- ret = btrfs_check_data_free_space(inode, reserved, start, len);
- if (ret < 0)
- return ret;
- ret = btrfs_delalloc_reserve_metadata(BTRFS_I(inode), len);
- if (ret < 0)
- btrfs_free_reserved_data_space(inode, *reserved, start, len);
return ret;
}
-/**
- * btrfs_delalloc_release_space - release data and metadata space for delalloc
- * @inode: inode we're releasing space for
- * @start: start position of the space already reserved
- * @len: the len of the space already reserved
- * @release_bytes: the len of the space we consumed or didn't use
- *
- * This function will release the metadata space that was not used and will
- * decrement ->delalloc_bytes and remove it from the fs_info delalloc_inodes
- * list if there are no delalloc bytes left.
- * Also it will handle the qgroup reserved space.
- */
-void btrfs_delalloc_release_space(struct inode *inode,
- struct extent_changeset *reserved,
- u64 start, u64 len, bool qgroup_free)
-{
- btrfs_delalloc_release_metadata(BTRFS_I(inode), len, qgroup_free);
- btrfs_free_reserved_data_space(inode, reserved, start, len);
-}
-
static int update_block_group(struct btrfs_trans_handle *trans,
u64 bytenr, u64 num_bytes, int alloc)
{
old_val -= num_bytes;
btrfs_set_block_group_used(&cache->item, old_val);
cache->pinned += num_bytes;
- update_bytes_pinned(cache->space_info, num_bytes);
+ btrfs_space_info_update_bytes_pinned(info,
+ cache->space_info, num_bytes);
cache->space_info->bytes_used -= num_bytes;
cache->space_info->disk_used -= num_bytes * factor;
spin_unlock(&cache->lock);
spin_lock(&cache->space_info->lock);
spin_lock(&cache->lock);
cache->pinned += num_bytes;
- update_bytes_pinned(cache->space_info, num_bytes);
+ btrfs_space_info_update_bytes_pinned(fs_info, cache->space_info,
+ num_bytes);
if (reserved) {
cache->reserved -= num_bytes;
cache->space_info->bytes_reserved -= num_bytes;
} else {
cache->reserved += num_bytes;
space_info->bytes_reserved += num_bytes;
- update_bytes_may_use(space_info, -ram_bytes);
+ btrfs_space_info_update_bytes_may_use(cache->fs_info,
+ space_info, -ram_bytes);
if (delalloc)
cache->delalloc_bytes += num_bytes;
}
up_write(&fs_info->commit_root_sem);
- update_global_block_rsv(fs_info);
+ btrfs_update_global_block_rsv(fs_info);
}
/*
spin_lock(&space_info->lock);
spin_lock(&cache->lock);
cache->pinned -= len;
- update_bytes_pinned(space_info, -len);
+ btrfs_space_info_update_bytes_pinned(fs_info, space_info, -len);
trace_btrfs_space_reservation(fs_info, "pinned",
space_info->flags, len, 0);
to_add = min(len, global_rsv->size -
global_rsv->reserved);
global_rsv->reserved += to_add;
- update_bytes_may_use(space_info, to_add);
+ btrfs_space_info_update_bytes_may_use(fs_info,
+ space_info, to_add);
if (global_rsv->reserved >= global_rsv->size)
global_rsv->full = 1;
trace_btrfs_space_reservation(fs_info,
spin_unlock(&global_rsv->lock);
/* Add to any tickets we may have */
if (len)
- space_info_add_new_bytes(fs_info, space_info,
- len);
+ btrfs_space_info_add_new_bytes(fs_info,
+ space_info, len);
}
spin_unlock(&space_info->lock);
}
}
out:
if (pin)
- add_pinned_bytes(fs_info, &generic_ref, 1);
+ add_pinned_bytes(fs_info, &generic_ref);
if (last_ref) {
/*
btrfs_ref_tree_mod(fs_info, ref);
if (ret == 0 && old_ref_mod >= 0 && new_ref_mod < 0)
- add_pinned_bytes(fs_info, ref, 1);
+ add_pinned_bytes(fs_info, ref);
return ret;
}
}
enum btrfs_loop_type {
- LOOP_CACHING_NOWAIT = 0,
- LOOP_CACHING_WAIT = 1,
- LOOP_ALLOC_CHUNK = 2,
- LOOP_NO_EMPTY_SIZE = 3,
+ LOOP_CACHING_NOWAIT,
+ LOOP_CACHING_WAIT,
+ LOOP_ALLOC_CHUNK,
+ LOOP_NO_EMPTY_SIZE,
};
static inline void
return ret;
}
- ret = do_chunk_alloc(trans, ffe_ctl->flags,
- CHUNK_ALLOC_FORCE);
+ ret = btrfs_chunk_alloc(trans, ffe_ctl->flags,
+ CHUNK_ALLOC_FORCE);
/*
* If we can't allocate a new chunk we've already looped
trace_find_free_extent(fs_info, num_bytes, empty_size, flags);
- space_info = __find_space_info(fs_info, flags);
+ space_info = btrfs_find_space_info(fs_info, flags);
if (!space_info) {
btrfs_err(fs_info, "No space info for %llu", flags);
return -ENOSPC;
*/
if (!block_group_bits(block_group, flags)) {
u64 extra = BTRFS_BLOCK_GROUP_DUP |
- BTRFS_BLOCK_GROUP_RAID1 |
- BTRFS_BLOCK_GROUP_RAID5 |
- BTRFS_BLOCK_GROUP_RAID6 |
+ BTRFS_BLOCK_GROUP_RAID1_MASK |
+ BTRFS_BLOCK_GROUP_RAID56_MASK |
BTRFS_BLOCK_GROUP_RAID10;
/*
return ret;
}
-#define DUMP_BLOCK_RSV(fs_info, rsv_name) \
-do { \
- struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \
- spin_lock(&__rsv->lock); \
- btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \
- __rsv->size, __rsv->reserved); \
- spin_unlock(&__rsv->lock); \
-} while (0)
-
-static void dump_space_info(struct btrfs_fs_info *fs_info,
- struct btrfs_space_info *info, u64 bytes,
- int dump_block_groups)
-{
- struct btrfs_block_group_cache *cache;
- int index = 0;
-
- spin_lock(&info->lock);
- btrfs_info(fs_info, "space_info %llu has %llu free, is %sfull",
- info->flags,
- info->total_bytes - btrfs_space_info_used(info, true),
- info->full ? "" : "not ");
- btrfs_info(fs_info,
- "space_info total=%llu, used=%llu, pinned=%llu, reserved=%llu, may_use=%llu, readonly=%llu",
- info->total_bytes, info->bytes_used, info->bytes_pinned,
- info->bytes_reserved, info->bytes_may_use,
- info->bytes_readonly);
- spin_unlock(&info->lock);
-
- DUMP_BLOCK_RSV(fs_info, global_block_rsv);
- DUMP_BLOCK_RSV(fs_info, trans_block_rsv);
- DUMP_BLOCK_RSV(fs_info, chunk_block_rsv);
- DUMP_BLOCK_RSV(fs_info, delayed_block_rsv);
- DUMP_BLOCK_RSV(fs_info, delayed_refs_rsv);
-
- if (!dump_block_groups)
- return;
-
- down_read(&info->groups_sem);
-again:
- list_for_each_entry(cache, &info->block_groups[index], list) {
- spin_lock(&cache->lock);
- btrfs_info(fs_info,
- "block group %llu has %llu bytes, %llu used %llu pinned %llu reserved %s",
- cache->key.objectid, cache->key.offset,
- btrfs_block_group_used(&cache->item), cache->pinned,
- cache->reserved, cache->ro ? "[readonly]" : "");
- btrfs_dump_free_space(cache, bytes);
- spin_unlock(&cache->lock);
- }
- if (++index < BTRFS_NR_RAID_TYPES)
- goto again;
- up_read(&info->groups_sem);
-}
-
/*
* btrfs_reserve_extent - entry point to the extent allocator. Tries to find a
* hole that is at least as big as @num_bytes.
} else if (btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
struct btrfs_space_info *sinfo;
- sinfo = __find_space_info(fs_info, flags);
+ sinfo = btrfs_find_space_info(fs_info, flags);
btrfs_err(fs_info,
"allocation failed flags %llu, wanted %llu",
flags, num_bytes);
if (sinfo)
- dump_space_info(fs_info, sinfo, num_bytes, 1);
+ btrfs_dump_space_info(fs_info, sinfo,
+ num_bytes, 1);
}
}
return buf;
}
-static struct btrfs_block_rsv *
-use_block_rsv(struct btrfs_trans_handle *trans,
- struct btrfs_root *root, u32 blocksize)
-{
- struct btrfs_fs_info *fs_info = root->fs_info;
- struct btrfs_block_rsv *block_rsv;
- struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
- int ret;
- bool global_updated = false;
-
- block_rsv = get_block_rsv(trans, root);
-
- if (unlikely(block_rsv->size == 0))
- goto try_reserve;
-again:
- ret = block_rsv_use_bytes(block_rsv, blocksize);
- if (!ret)
- return block_rsv;
-
- if (block_rsv->failfast)
- return ERR_PTR(ret);
-
- if (block_rsv->type == BTRFS_BLOCK_RSV_GLOBAL && !global_updated) {
- global_updated = true;
- update_global_block_rsv(fs_info);
- goto again;
- }
-
- /*
- * The global reserve still exists to save us from ourselves, so don't
- * warn_on if we are short on our delayed refs reserve.
- */
- if (block_rsv->type != BTRFS_BLOCK_RSV_DELREFS &&
- btrfs_test_opt(fs_info, ENOSPC_DEBUG)) {
- static DEFINE_RATELIMIT_STATE(_rs,
- DEFAULT_RATELIMIT_INTERVAL * 10,
- /*DEFAULT_RATELIMIT_BURST*/ 1);
- if (__ratelimit(&_rs))
- WARN(1, KERN_DEBUG
- "BTRFS: block rsv returned %d\n", ret);
- }
-try_reserve:
- ret = reserve_metadata_bytes(root, block_rsv, blocksize,
- BTRFS_RESERVE_NO_FLUSH);
- if (!ret)
- return block_rsv;
- /*
- * If we couldn't reserve metadata bytes try and use some from
- * the global reserve if its space type is the same as the global
- * reservation.
- */
- if (block_rsv->type != BTRFS_BLOCK_RSV_GLOBAL &&
- block_rsv->space_info == global_rsv->space_info) {
- ret = block_rsv_use_bytes(global_rsv, blocksize);
- if (!ret)
- return global_rsv;
- }
- return ERR_PTR(ret);
-}
-
-static void unuse_block_rsv(struct btrfs_fs_info *fs_info,
- struct btrfs_block_rsv *block_rsv, u32 blocksize)
-{
- block_rsv_add_bytes(block_rsv, blocksize, false);
- block_rsv_release_bytes(fs_info, block_rsv, NULL, 0, NULL);
-}
-
/*
* finds a free extent and does all the dirty work required for allocation
* returns the tree buffer or an ERR_PTR on error.
}
#endif
- block_rsv = use_block_rsv(trans, root, blocksize);
+ block_rsv = btrfs_use_block_rsv(trans, root, blocksize);
if (IS_ERR(block_rsv))
return ERR_CAST(block_rsv);
out_free_reserved:
btrfs_free_reserved_extent(fs_info, ins.objectid, ins.offset, 0);
out_unuse:
- unuse_block_rsv(fs_info, block_rsv, blocksize);
+ btrfs_unuse_block_rsv(fs_info, block_rsv, blocksize);
return ERR_PTR(ret);
}
num_devices = fs_info->fs_devices->rw_devices;
- stripped = BTRFS_BLOCK_GROUP_RAID0 |
- BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6 |
- BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID10;
+ stripped = BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID56_MASK |
+ BTRFS_BLOCK_GROUP_RAID1_MASK | BTRFS_BLOCK_GROUP_RAID10;
if (num_devices == 1) {
stripped |= BTRFS_BLOCK_GROUP_DUP;
return stripped;
/* turn mirroring into duplication */
- if (flags & (BTRFS_BLOCK_GROUP_RAID1 |
+ if (flags & (BTRFS_BLOCK_GROUP_RAID1_MASK |
BTRFS_BLOCK_GROUP_RAID10))
return stripped | BTRFS_BLOCK_GROUP_DUP;
} else {
btrfs_info(cache->fs_info,
"sinfo_used=%llu bg_num_bytes=%llu min_allocable=%llu",
sinfo_used, num_bytes, min_allocable_bytes);
- dump_space_info(cache->fs_info, cache->space_info, 0, 0);
+ btrfs_dump_space_info(cache->fs_info, cache->space_info, 0, 0);
}
return ret;
}
*/
alloc_flags = update_block_group_flags(fs_info, cache->flags);
if (alloc_flags != cache->flags) {
- ret = do_chunk_alloc(trans, alloc_flags,
- CHUNK_ALLOC_FORCE);
+ ret = btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
/*
* ENOSPC is allowed here, we may have enough space
* already allocated at the new raid level to
if (!ret)
goto out;
alloc_flags = get_alloc_profile(fs_info, cache->space_info->flags);
- ret = do_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
+ ret = btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
if (ret < 0)
goto out;
ret = inc_block_group_ro(cache, 0);
{
u64 alloc_flags = get_alloc_profile(trans->fs_info, type);
- return do_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
+ return btrfs_chunk_alloc(trans, alloc_flags, CHUNK_ALLOC_FORCE);
}
/*
struct extent_map_tree *em_tree;
struct extent_map *em;
- em_tree = &root->fs_info->mapping_tree.map_tree;
+ em_tree = &root->fs_info->mapping_tree;
read_lock(&em_tree->lock);
em = lookup_extent_mapping(em_tree, found_key.objectid,
found_key.offset);
*/
synchronize_rcu();
- release_global_block_rsv(info);
+ btrfs_release_global_block_rsv(info);
while (!list_empty(&info->space_info)) {
int i;
if (WARN_ON(space_info->bytes_pinned > 0 ||
space_info->bytes_reserved > 0 ||
space_info->bytes_may_use > 0))
- dump_space_info(info, space_info, 0, 0);
+ btrfs_dump_space_info(info, space_info, 0, 0);
list_del(&space_info->list);
for (i = 0; i < BTRFS_NR_RAID_TYPES; i++) {
struct kobject *kobj;
struct btrfs_space_info *space_info;
struct raid_kobject *rkobj;
LIST_HEAD(list);
- int index;
int ret = 0;
spin_lock(&fs_info->pending_raid_kobjs_lock);
spin_unlock(&fs_info->pending_raid_kobjs_lock);
list_for_each_entry(rkobj, &list, list) {
- space_info = __find_space_info(fs_info, rkobj->flags);
- index = btrfs_bg_flags_to_raid_index(rkobj->flags);
+ space_info = btrfs_find_space_info(fs_info, rkobj->flags);
ret = kobject_add(&rkobj->kobj, &space_info->kobj,
- "%s", get_raid_name(index));
+ "%s", btrfs_bg_type_to_raid_name(rkobj->flags));
if (ret) {
kobject_put(&rkobj->kobj);
break;
*/
static int check_chunk_block_group_mappings(struct btrfs_fs_info *fs_info)
{
- struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree;
+ struct extent_map_tree *map_tree = &fs_info->mapping_tree;
struct extent_map *em;
struct btrfs_block_group_cache *bg;
u64 start = 0;
int ret = 0;
while (1) {
- read_lock(&map_tree->map_tree.lock);
+ read_lock(&map_tree->lock);
/*
* lookup_extent_mapping will return the first extent map
* intersecting the range, so setting @len to 1 is enough to
* get the first chunk.
*/
- em = lookup_extent_mapping(&map_tree->map_tree, start, 1);
- read_unlock(&map_tree->map_tree.lock);
+ em = lookup_extent_mapping(map_tree, start, 1);
+ read_unlock(&map_tree->lock);
if (!em)
break;
}
trace_btrfs_add_block_group(info, cache, 0);
- update_space_info(info, cache->flags, found_key.offset,
- btrfs_block_group_used(&cache->item),
- cache->bytes_super, &space_info);
+ btrfs_update_space_info(info, cache->flags, found_key.offset,
+ btrfs_block_group_used(&cache->item),
+ cache->bytes_super, &space_info);
cache->space_info = space_info;
list_for_each_entry_rcu(space_info, &info->space_info, list) {
if (!(get_alloc_profile(info, space_info->flags) &
(BTRFS_BLOCK_GROUP_RAID10 |
- BTRFS_BLOCK_GROUP_RAID1 |
- BTRFS_BLOCK_GROUP_RAID5 |
- BTRFS_BLOCK_GROUP_RAID6 |
+ BTRFS_BLOCK_GROUP_RAID1_MASK |
+ BTRFS_BLOCK_GROUP_RAID56_MASK |
BTRFS_BLOCK_GROUP_DUP)))
continue;
/*
}
btrfs_add_raid_kobjects(info);
- init_global_block_rsv(info);
+ btrfs_init_global_block_rsv(info);
ret = check_chunk_block_group_mappings(info);
error:
btrfs_free_path(path);
* assigned to our block group. We want our bg to be added to the rbtree
* with its ->space_info set.
*/
- cache->space_info = __find_space_info(fs_info, cache->flags);
+ cache->space_info = btrfs_find_space_info(fs_info, cache->flags);
ASSERT(cache->space_info);
ret = btrfs_add_block_group_cache(fs_info, cache);
* the rbtree, update the space info's counters.
*/
trace_btrfs_add_block_group(fs_info, cache, 1);
- update_space_info(fs_info, cache->flags, size, bytes_used,
+ btrfs_update_space_info(fs_info, cache->flags, size, bytes_used,
cache->bytes_super, &cache->space_info);
- update_global_block_rsv(fs_info);
+ btrfs_update_global_block_rsv(fs_info);
link_block_group(cache);
write_sequnlock(&fs_info->profiles_lock);
}
+/*
+ * Clear incompat bits for the following feature(s):
+ *
+ * - RAID56 - in case there's neither RAID5 nor RAID6 profile block group
+ * in the whole filesystem
+ */
+static void clear_incompat_bg_bits(struct btrfs_fs_info *fs_info, u64 flags)
+{
+ if (flags & BTRFS_BLOCK_GROUP_RAID56_MASK) {
+ struct list_head *head = &fs_info->space_info;
+ struct btrfs_space_info *sinfo;
+
+ list_for_each_entry_rcu(sinfo, head, list) {
+ bool found = false;
+
+ down_read(&sinfo->groups_sem);
+ if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID5]))
+ found = true;
+ if (!list_empty(&sinfo->block_groups[BTRFS_RAID_RAID6]))
+ found = true;
+ up_read(&sinfo->groups_sem);
+
+ if (found)
+ return;
+ }
+ btrfs_clear_fs_incompat(fs_info, RAID56);
+ }
+}
+
int btrfs_remove_block_group(struct btrfs_trans_handle *trans,
u64 group_start, struct extent_map *em)
{
clear_avail_alloc_bits(fs_info, block_group->flags);
}
up_write(&block_group->space_info->groups_sem);
+ clear_incompat_bg_bits(fs_info, block_group->flags);
if (kobj) {
kobject_del(kobj);
kobject_put(kobj);
if (remove_em) {
struct extent_map_tree *em_tree;
- em_tree = &fs_info->mapping_tree.map_tree;
+ em_tree = &fs_info->mapping_tree;
write_lock(&em_tree->lock);
remove_extent_mapping(em_tree, em);
write_unlock(&em_tree->lock);
btrfs_start_trans_remove_block_group(struct btrfs_fs_info *fs_info,
const u64 chunk_offset)
{
- struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
+ struct extent_map_tree *em_tree = &fs_info->mapping_tree;
struct extent_map *em;
struct map_lookup *map;
unsigned int num_items;
spin_lock(&space_info->lock);
spin_lock(&block_group->lock);
- update_bytes_pinned(space_info, -block_group->pinned);
+ btrfs_space_info_update_bytes_pinned(fs_info, space_info,
+ -block_group->pinned);
space_info->bytes_readonly += block_group->pinned;
percpu_counter_add_batch(&space_info->total_bytes_pinned,
-block_group->pinned,
spin_unlock(&fs_info->unused_bgs_lock);
}
-int btrfs_init_space_info(struct btrfs_fs_info *fs_info)
-{
- struct btrfs_super_block *disk_super;
- u64 features;
- u64 flags;
- int mixed = 0;
- int ret;
-
- disk_super = fs_info->super_copy;
- if (!btrfs_super_root(disk_super))
- return -EINVAL;
-
- features = btrfs_super_incompat_flags(disk_super);
- if (features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)
- mixed = 1;
-
- flags = BTRFS_BLOCK_GROUP_SYSTEM;
- ret = create_space_info(fs_info, flags);
- if (ret)
- goto out;
-
- if (mixed) {
- flags = BTRFS_BLOCK_GROUP_METADATA | BTRFS_BLOCK_GROUP_DATA;
- ret = create_space_info(fs_info, flags);
- } else {
- flags = BTRFS_BLOCK_GROUP_METADATA;
- ret = create_space_info(fs_info, flags);
- if (ret)
- goto out;
-
- flags = BTRFS_BLOCK_GROUP_DATA;
- ret = create_space_info(fs_info, flags);
- }
-out:
- return ret;
-}
-
int btrfs_error_unpin_extent_range(struct btrfs_fs_info *fs_info,
u64 start, u64 end)
{
find_first_clear_extent_bit(&device->alloc_state, start,
&start, &end,
CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ /* Ensure we skip the reserved area in the first 1M */
+ start = max_t(u64, start, SZ_1M);
+
/*
* If find_first_clear_extent_bit find a range that spans the
* end of the device it will set end to -1, in this case it's up
* to the caller to trim the value to the size of the device.
*/
end = min(end, device->total_bytes - 1);
+
len = end - start + 1;
/* We didn't find any extents */
return NULL;
}
+/**
+ * __etree_search - searche @tree for an entry that contains @offset. Such
+ * entry would have entry->start <= offset && entry->end >= offset.
+ *
+ * @tree - the tree to search
+ * @offset - offset that should fall within an entry in @tree
+ * @next_ret - pointer to the first entry whose range ends after @offset
+ * @prev - pointer to the first entry whose range begins before @offset
+ * @p_ret - pointer where new node should be anchored (used when inserting an
+ * entry in the tree)
+ * @parent_ret - points to entry which would have been the parent of the entry,
+ * containing @offset
+ *
+ * This function returns a pointer to the entry that contains @offset byte
+ * address. If no such entry exists, then NULL is returned and the other
+ * pointer arguments to the function are filled, otherwise the found entry is
+ * returned and other pointers are left untouched.
+ */
static struct rb_node *__etree_search(struct extent_io_tree *tree, u64 offset,
struct rb_node **next_ret,
struct rb_node **prev_ret,
{
struct rb_node *node;
- if (end < start)
- WARN(1, KERN_ERR "BTRFS: end < start %llu %llu\n",
- end, start);
+ if (end < start) {
+ btrfs_err(tree->fs_info,
+ "insert state: end < start %llu %llu", end, start);
+ WARN_ON(1);
+ }
state->start = start;
state->end = end;
if (node) {
struct extent_state *found;
found = rb_entry(node, struct extent_state, rb_node);
- pr_err("BTRFS: found node %llu %llu on insert of %llu %llu\n",
+ btrfs_err(tree->fs_info,
+ "found node %llu %llu on insert of %llu %llu",
found->start, found->end, start, end);
return -EEXIST;
}
}
/**
- * find_first_clear_extent_bit - finds the first range that has @bits not set
- * and that starts after @start
+ * find_first_clear_extent_bit - find the first range that has @bits not set.
+ * This range could start before @start.
*
* @tree - the tree to search
* @start - the offset at/after which the found extent should start
goto out;
}
}
+ /*
+ * At this point 'node' either contains 'start' or start is
+ * before 'node'
+ */
state = rb_entry(node, struct extent_state, rb_node);
- if (in_range(start, state->start, state->end - state->start + 1) &&
- (state->state & bits)) {
- start = state->end + 1;
+
+ if (in_range(start, state->start, state->end - state->start + 1)) {
+ if (state->state & bits) {
+ /*
+ * |--range with bits sets--|
+ * |
+ * start
+ */
+ start = state->end + 1;
+ } else {
+ /*
+ * 'start' falls within a range that doesn't
+ * have the bits set, so take its start as
+ * the beginning of the desired range
+ *
+ * |--range with bits cleared----|
+ * |
+ * start
+ */
+ *start_ret = state->start;
+ break;
+ }
} else {
- *start_ret = start;
+ /*
+ * |---prev range---|---hole/unset---|---node range---|
+ * |
+ * start
+ *
+ * or
+ *
+ * |---hole/unset--||--first node--|
+ * 0 |
+ * start
+ */
+ if (prev) {
+ state = rb_entry(prev, struct extent_state,
+ rb_node);
+ *start_ret = state->end + 1;
+ } else {
+ *start_ret = 0;
+ }
break;
}
}
*/
EXPORT_FOR_TESTS
noinline_for_stack bool find_lock_delalloc_range(struct inode *inode,
- struct extent_io_tree *tree,
struct page *locked_page, u64 *start,
u64 *end)
{
+ struct extent_io_tree *tree = &BTRFS_I(inode)->io_tree;
u64 max_bytes = BTRFS_MAX_EXTENT_SIZE;
u64 delalloc_start;
u64 delalloc_end;
* never fail. We're returning a bio right now but you can call btrfs_io_bio
* for the appropriate container_of magic
*/
-struct bio *btrfs_bio_alloc(struct block_device *bdev, u64 first_byte)
+struct bio *btrfs_bio_alloc(u64 first_byte)
{
struct bio *bio;
bio = bio_alloc_bioset(GFP_NOFS, BIO_MAX_PAGES, &btrfs_bioset);
- bio_set_dev(bio, bdev);
bio->bi_iter.bi_sector = first_byte >> 9;
btrfs_io_bio_init(btrfs_io_bio(bio));
return bio;
}
}
- bio = btrfs_bio_alloc(bdev, offset);
+ bio = btrfs_bio_alloc(offset);
+ bio_set_dev(bio, bdev);
bio_add_page(bio, page, page_size, pg_offset);
bio->bi_end_io = end_io_func;
bio->bi_private = tree;
unsigned long *bio_flags,
u64 *prev_em_start)
{
- struct inode *inode;
- struct btrfs_ordered_extent *ordered;
+ struct btrfs_inode *inode = BTRFS_I(pages[0]->mapping->host);
int index;
- inode = pages[0]->mapping->host;
- while (1) {
- lock_extent(tree, start, end);
- ordered = btrfs_lookup_ordered_range(BTRFS_I(inode), start,
- end - start + 1);
- if (!ordered)
- break;
- unlock_extent(tree, start, end);
- btrfs_start_ordered_extent(inode, ordered, 1);
- btrfs_put_ordered_extent(ordered);
- }
+ btrfs_lock_and_flush_ordered_range(tree, inode, start, end, NULL);
for (index = 0; index < nr_pages; index++) {
__do_readpage(tree, pages[index], btrfs_get_extent, em_cached,
unsigned long *bio_flags,
unsigned int read_flags)
{
- struct inode *inode = page->mapping->host;
- struct btrfs_ordered_extent *ordered;
+ struct btrfs_inode *inode = BTRFS_I(page->mapping->host);
u64 start = page_offset(page);
u64 end = start + PAGE_SIZE - 1;
int ret;
- while (1) {
- lock_extent(tree, start, end);
- ordered = btrfs_lookup_ordered_range(BTRFS_I(inode), start,
- PAGE_SIZE);
- if (!ordered)
- break;
- unlock_extent(tree, start, end);
- btrfs_start_ordered_extent(inode, ordered, 1);
- btrfs_put_ordered_extent(ordered);
- }
+ btrfs_lock_and_flush_ordered_range(tree, inode, start, end, NULL);
ret = __do_readpage(tree, page, get_extent, NULL, bio, mirror_num,
bio_flags, read_flags, NULL);
struct page *page, struct writeback_control *wbc,
u64 delalloc_start, unsigned long *nr_written)
{
- struct extent_io_tree *tree = &BTRFS_I(inode)->io_tree;
u64 page_end = delalloc_start + PAGE_SIZE - 1;
bool found;
u64 delalloc_to_write = 0;
while (delalloc_end < page_end) {
- found = find_lock_delalloc_range(inode, tree,
- page,
+ found = find_lock_delalloc_range(inode, page,
&delalloc_start,
&delalloc_end);
if (!found) {
}
ret = btrfs_run_delalloc_range(inode, page, delalloc_start,
delalloc_end, &page_started, nr_written, wbc);
- /* File system has been set read-only */
if (ret) {
SetPageError(page);
/*
struct btrfs_path *path;
struct btrfs_root *root = BTRFS_I(inode)->root;
struct fiemap_cache cache = { 0 };
+ struct ulist *roots;
+ struct ulist *tmp_ulist;
int end = 0;
u64 em_start = 0;
u64 em_len = 0;
return -ENOMEM;
path->leave_spinning = 1;
+ roots = ulist_alloc(GFP_KERNEL);
+ tmp_ulist = ulist_alloc(GFP_KERNEL);
+ if (!roots || !tmp_ulist) {
+ ret = -ENOMEM;
+ goto out_free_ulist;
+ }
+
start = round_down(start, btrfs_inode_sectorsize(inode));
len = round_up(max, btrfs_inode_sectorsize(inode)) - start;
ret = btrfs_lookup_file_extent(NULL, root, path,
btrfs_ino(BTRFS_I(inode)), -1, 0);
if (ret < 0) {
- btrfs_free_path(path);
- return ret;
+ goto out_free_ulist;
} else {
WARN_ON(!ret);
if (ret == 1)
*/
ret = btrfs_check_shared(root,
btrfs_ino(BTRFS_I(inode)),
- bytenr);
+ bytenr, roots, tmp_ulist);
if (ret < 0)
goto out_free;
if (ret)
ret = emit_last_fiemap_cache(fieinfo, &cache);
free_extent_map(em);
out:
- btrfs_free_path(path);
unlock_extent_cached(&BTRFS_I(inode)->io_tree, start, start + len - 1,
&cached_state);
+
+out_free_ulist:
+ btrfs_free_path(path);
+ ulist_free(roots);
+ ulist_free(tmp_ulist);
return ret;
}
eb->bflags = 0;
rwlock_init(&eb->lock);
atomic_set(&eb->blocking_readers, 0);
- atomic_set(&eb->blocking_writers, 0);
+ eb->blocking_writers = 0;
eb->lock_nested = false;
init_waitqueue_head(&eb->write_lock_wq);
init_waitqueue_head(&eb->read_lock_wq);
BUG_ON(len > MAX_INLINE_EXTENT_BUFFER_SIZE);
#ifdef CONFIG_BTRFS_DEBUG
- atomic_set(&eb->spinning_writers, 0);
+ eb->spinning_writers = 0;
atomic_set(&eb->spinning_readers, 0);
atomic_set(&eb->read_locks, 0);
- atomic_set(&eb->write_locks, 0);
+ eb->write_locks = 0;
#endif
return eb;
struct rcu_head rcu_head;
pid_t lock_owner;
- atomic_t blocking_writers;
+ int blocking_writers;
atomic_t blocking_readers;
bool lock_nested;
/* >= 0 if eb belongs to a log tree, -1 otherwise */
wait_queue_head_t read_lock_wq;
struct page *pages[INLINE_EXTENT_BUFFER_PAGES];
#ifdef CONFIG_BTRFS_DEBUG
- atomic_t spinning_writers;
+ int spinning_writers;
atomic_t spinning_readers;
atomic_t read_locks;
- atomic_t write_locks;
+ int write_locks;
struct list_head leak_list;
#endif
};
u64 delalloc_end, struct page *locked_page,
unsigned bits_to_clear,
unsigned long page_ops);
-struct bio *btrfs_bio_alloc(struct block_device *bdev, u64 first_byte);
+struct bio *btrfs_bio_alloc(u64 first_byte);
struct bio *btrfs_io_bio_alloc(unsigned int nr_iovecs);
struct bio *btrfs_bio_clone(struct bio *bio);
struct bio *btrfs_bio_clone_partial(struct bio *orig, int offset, int size);
struct extent_io_tree *io_tree,
struct io_failure_record *rec);
#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
-bool find_lock_delalloc_range(struct inode *inode, struct extent_io_tree *tree,
+bool find_lock_delalloc_range(struct inode *inode,
struct page *locked_page, u64 *start,
u64 *end);
#endif
#include <linux/pagemap.h>
#include <linux/highmem.h>
#include <linux/sched/mm.h>
+#include <crypto/hash.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
#define MAX_CSUM_ITEMS(r, size) (min_t(u32, __MAX_CSUM_ITEMS(r, size), \
PAGE_SIZE))
-#define MAX_ORDERED_SUM_BYTES(fs_info) ((PAGE_SIZE - \
- sizeof(struct btrfs_ordered_sum)) / \
- sizeof(u32) * (fs_info)->sectorsize)
+static inline u32 max_ordered_sum_bytes(struct btrfs_fs_info *fs_info,
+ u16 csum_size)
+{
+ u32 ncsums = (PAGE_SIZE - sizeof(struct btrfs_ordered_sum)) / csum_size;
+
+ return ncsums * fs_info->sectorsize;
+}
int btrfs_insert_file_extent(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
}
static blk_status_t __btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio,
- u64 logical_offset, u32 *dst, int dio)
+ u64 logical_offset, u8 *dst, int dio)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
struct bio_vec bvec;
}
csum = btrfs_bio->csum;
} else {
- csum = (u8 *)dst;
+ csum = dst;
}
if (bio->bi_iter.bi_size > PAGE_SIZE * 8)
if (!dio)
offset = page_offset(bvec.bv_page) + bvec.bv_offset;
count = btrfs_find_ordered_sum(inode, offset, disk_bytenr,
- (u32 *)csum, nblocks);
+ csum, nblocks);
if (count)
goto found;
return 0;
}
-blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio, u32 *dst)
+blk_status_t btrfs_lookup_bio_sums(struct inode *inode, struct bio *bio,
+ u8 *dst)
{
return __btrfs_lookup_bio_sums(inode, bio, 0, dst, 0);
}
struct btrfs_csum_item);
while (start < csum_end) {
size = min_t(size_t, csum_end - start,
- MAX_ORDERED_SUM_BYTES(fs_info));
+ max_ordered_sum_bytes(fs_info, csum_size));
sums = kzalloc(btrfs_ordered_sum_size(fs_info, size),
GFP_NOFS);
if (!sums) {
u64 file_start, int contig)
{
struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
struct btrfs_ordered_sum *sums;
struct btrfs_ordered_extent *ordered = NULL;
char *data;
int i;
u64 offset;
unsigned nofs_flag;
+ const u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
nofs_flag = memalloc_nofs_save();
sums = kvzalloc(btrfs_ordered_sum_size(fs_info, bio->bi_iter.bi_size),
sums->bytenr = (u64)bio->bi_iter.bi_sector << 9;
index = 0;
+ shash->tfm = fs_info->csum_shash;
+
bio_for_each_segment(bvec, bio, iter) {
if (!contig)
offset = page_offset(bvec.bv_page) + bvec.bv_offset;
index = 0;
}
- sums->sums[index] = ~(u32)0;
+ crypto_shash_init(shash);
data = kmap_atomic(bvec.bv_page);
- sums->sums[index]
- = btrfs_csum_data(data + bvec.bv_offset
- + (i * fs_info->sectorsize),
- sums->sums[index],
- fs_info->sectorsize);
+ crypto_shash_update(shash, data + bvec.bv_offset
+ + (i * fs_info->sectorsize),
+ fs_info->sectorsize);
kunmap_atomic(data);
- btrfs_csum_final(sums->sums[index],
- (char *)(sums->sums + index));
- index++;
+ crypto_shash_final(shash, (char *)(sums->sums + index));
+ index += csum_size;
offset += fs_info->sectorsize;
this_sum_bytes += fs_info->sectorsize;
total_bytes += fs_info->sectorsize;
write_extent_buffer(leaf, sums->sums + index, (unsigned long)item,
ins_size);
+ index += ins_size;
ins_size /= csum_size;
total_bytes += ins_size * fs_info->sectorsize;
- index += ins_size;
btrfs_mark_buffer_dirty(path->nodes[0]);
if (total_bytes < sums->len) {
#include "volumes.h"
#include "qgroup.h"
#include "compression.h"
+#include "delalloc-space.h"
static struct kmem_cache *btrfs_inode_defrag_cachep;
/*
{
struct btrfs_fs_info *fs_info = inode->root->fs_info;
struct btrfs_root *root = inode->root;
- struct btrfs_ordered_extent *ordered;
u64 lockstart, lockend;
u64 num_bytes;
int ret;
ret = btrfs_start_write_no_snapshotting(root);
if (!ret)
- return -ENOSPC;
+ return -EAGAIN;
lockstart = round_down(pos, fs_info->sectorsize);
lockend = round_up(pos + *write_bytes,
fs_info->sectorsize) - 1;
- while (1) {
- lock_extent(&inode->io_tree, lockstart, lockend);
- ordered = btrfs_lookup_ordered_range(inode, lockstart,
- lockend - lockstart + 1);
- if (!ordered) {
- break;
- }
- unlock_extent(&inode->io_tree, lockstart, lockend);
- btrfs_start_ordered_extent(&inode->vfs_inode, ordered, 1);
- btrfs_put_ordered_extent(ordered);
- }
+ btrfs_lock_and_flush_ordered_range(&inode->io_tree, inode, lockstart,
+ lockend, NULL);
num_bytes = lockend - lockstart + 1;
ret = can_nocow_extent(&inode->vfs_inode, lockstart, &num_bytes,
* for detecting, at fsync time, if the inode isn't yet in the
* log tree or it's there but not up to date.
*/
+ struct timespec64 now = current_time(inode);
+
+ inode_inc_iversion(inode);
+ inode->i_mtime = now;
+ inode->i_ctime = now;
trans = btrfs_start_transaction(root, 1);
if (IS_ERR(trans)) {
err = PTR_ERR(trans);
}
enum {
- RANGE_BOUNDARY_WRITTEN_EXTENT = 0,
- RANGE_BOUNDARY_PREALLOC_EXTENT = 1,
- RANGE_BOUNDARY_HOLE = 2,
+ RANGE_BOUNDARY_WRITTEN_EXTENT,
+ RANGE_BOUNDARY_PREALLOC_EXTENT,
+ RANGE_BOUNDARY_HOLE,
};
static int btrfs_zero_range_check_range_boundary(struct inode *inode,
#include "extent_io.h"
#include "inode-map.h"
#include "volumes.h"
+#include "space-info.h"
+#include "delalloc-space.h"
#define BITS_PER_BITMAP (PAGE_SIZE * 8UL)
#define MAX_CACHE_BYTES_PER_GIG SZ_32K
if (index == 0)
offset = sizeof(u32) * io_ctl->num_pages;
- crc = btrfs_csum_data(io_ctl->orig + offset, crc,
- PAGE_SIZE - offset);
- btrfs_csum_final(crc, (u8 *)&crc);
+ crc = btrfs_crc32c(crc, io_ctl->orig + offset, PAGE_SIZE - offset);
+ btrfs_crc32c_final(crc, (u8 *)&crc);
io_ctl_unmap_page(io_ctl);
tmp = page_address(io_ctl->pages[0]);
tmp += index;
val = *tmp;
io_ctl_map_page(io_ctl, 0);
- crc = btrfs_csum_data(io_ctl->orig + offset, crc,
- PAGE_SIZE - offset);
- btrfs_csum_final(crc, (u8 *)&crc);
+ crc = btrfs_crc32c(crc, io_ctl->orig + offset, PAGE_SIZE - offset);
+ btrfs_crc32c_final(crc, (u8 *)&crc);
if (val != crc) {
btrfs_err_rl(io_ctl->fs_info,
"csum mismatch on free space cache");
space_info->bytes_readonly += reserved_bytes;
block_group->reserved -= reserved_bytes;
space_info->bytes_reserved -= reserved_bytes;
- spin_unlock(&space_info->lock);
spin_unlock(&block_group->lock);
+ spin_unlock(&space_info->lock);
}
return ret;
if (cleanup) {
mutex_lock(&fs_info->chunk_mutex);
- em_tree = &fs_info->mapping_tree.map_tree;
+ em_tree = &fs_info->mapping_tree;
write_lock(&em_tree->lock);
em = lookup_extent_mapping(em_tree, block_group->key.objectid,
1);
#include "free-space-cache.h"
#include "inode-map.h"
#include "transaction.h"
+#include "delalloc-space.h"
static int caching_kthread(void *data)
{
#include "props.h"
#include "qgroup.h"
#include "dedupe.h"
+#include "delalloc-space.h"
struct btrfs_iget_args {
struct btrfs_key *location;
u64 length = 0;
u64 map_length;
int ret;
+ struct btrfs_io_geometry geom;
if (bio_flags & EXTENT_BIO_COMPRESSED)
return 0;
length = bio->bi_iter.bi_size;
map_length = length;
- ret = btrfs_map_block(fs_info, btrfs_op(bio), logical, &map_length,
- NULL, 0);
+ ret = btrfs_get_io_geometry(fs_info, btrfs_op(bio), logical, map_length,
+ &geom);
if (ret < 0)
return ret;
- if (map_length < length + size)
+
+ if (geom.len < length + size)
return 1;
return 0;
}
int icsum, struct page *page,
int pgoff, u64 start, size_t len)
{
+ struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
char *kaddr;
- u32 csum_expected;
- u32 csum = ~(u32)0;
+ u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
+ u8 *csum_expected;
+ u8 csum[BTRFS_CSUM_SIZE];
- csum_expected = *(((u32 *)io_bio->csum) + icsum);
+ csum_expected = ((u8 *)io_bio->csum) + icsum * csum_size;
kaddr = kmap_atomic(page);
- csum = btrfs_csum_data(kaddr + pgoff, csum, len);
- btrfs_csum_final(csum, (u8 *)&csum);
- if (csum != csum_expected)
+ shash->tfm = fs_info->csum_shash;
+
+ crypto_shash_init(shash);
+ crypto_shash_update(shash, kaddr + pgoff, len);
+ crypto_shash_final(shash, csum);
+
+ if (memcmp(csum, csum_expected, csum_size))
goto zeroit;
kunmap_atomic(kaddr);
wake_up_process(fs_info->cleaner_kthread);
}
+static void run_delayed_iput_locked(struct btrfs_fs_info *fs_info,
+ struct btrfs_inode *inode)
+{
+ list_del_init(&inode->delayed_iput);
+ spin_unlock(&fs_info->delayed_iput_lock);
+ iput(&inode->vfs_inode);
+ if (atomic_dec_and_test(&fs_info->nr_delayed_iputs))
+ wake_up(&fs_info->delayed_iputs_wait);
+ spin_lock(&fs_info->delayed_iput_lock);
+}
+
+static void btrfs_run_delayed_iput(struct btrfs_fs_info *fs_info,
+ struct btrfs_inode *inode)
+{
+ if (!list_empty(&inode->delayed_iput)) {
+ spin_lock(&fs_info->delayed_iput_lock);
+ if (!list_empty(&inode->delayed_iput))
+ run_delayed_iput_locked(fs_info, inode);
+ spin_unlock(&fs_info->delayed_iput_lock);
+ }
+}
+
void btrfs_run_delayed_iputs(struct btrfs_fs_info *fs_info)
{
inode = list_first_entry(&fs_info->delayed_iputs,
struct btrfs_inode, delayed_iput);
- list_del_init(&inode->delayed_iput);
- spin_unlock(&fs_info->delayed_iput_lock);
- iput(&inode->vfs_inode);
- if (atomic_dec_and_test(&fs_info->nr_delayed_iputs))
- wake_up(&fs_info->delayed_iputs_wait);
- spin_lock(&fs_info->delayed_iput_lock);
+ run_delayed_iput_locked(fs_info, inode);
}
spin_unlock(&fs_info->delayed_iput_lock);
}
struct btrfs_fs_info *fs_info = root->fs_info;
struct btrfs_path *path;
int ret = 0;
- struct extent_buffer *leaf;
struct btrfs_dir_item *di;
- struct btrfs_key key;
u64 index;
u64 ino = btrfs_ino(inode);
u64 dir_ino = btrfs_ino(dir);
ret = di ? PTR_ERR(di) : -ENOENT;
goto err;
}
- leaf = path->nodes[0];
- btrfs_dir_item_key_to_cpu(leaf, di, &key);
ret = btrfs_delete_one_dir_name(trans, root, path, di);
if (ret)
goto err;
ret = 0;
else if (ret)
btrfs_abort_transaction(trans, ret);
+
+ /*
+ * If we have a pending delayed iput we could end up with the final iput
+ * being run in btrfs-cleaner context. If we have enough of these built
+ * up we can end up burning a lot of time in btrfs-cleaner without any
+ * way to throttle the unlinks. Since we're currently holding a ref on
+ * the inode we can run the delayed iput here without any issues as the
+ * final iput won't be done until after we drop the ref we're currently
+ * holding.
+ */
+ btrfs_run_delayed_iput(fs_info, inode);
err:
btrfs_free_path(path);
if (ret)
if (size <= hole_start)
return 0;
- while (1) {
- struct btrfs_ordered_extent *ordered;
-
- lock_extent_bits(io_tree, hole_start, block_end - 1,
- &cached_state);
- ordered = btrfs_lookup_ordered_range(BTRFS_I(inode), hole_start,
- block_end - hole_start);
- if (!ordered)
- break;
- unlock_extent_cached(io_tree, hole_start, block_end - 1,
- &cached_state);
- btrfs_start_ordered_extent(inode, ordered, 1);
- btrfs_put_ordered_extent(ordered);
- }
-
+ btrfs_lock_and_flush_ordered_range(io_tree, BTRFS_I(inode), hole_start,
+ block_end - 1, &cached_state);
cur_offset = hole_start;
while (1) {
em = btrfs_get_extent(BTRFS_I(inode), NULL, 0, cur_offset,
struct bio *orig_bio = dip->orig_bio;
u64 start_sector = orig_bio->bi_iter.bi_sector;
u64 file_offset = dip->logical_offset;
- u64 map_length;
int async_submit = 0;
u64 submit_len;
int clone_offset = 0;
int clone_len;
int ret;
blk_status_t status;
+ struct btrfs_io_geometry geom;
- map_length = orig_bio->bi_iter.bi_size;
- submit_len = map_length;
- ret = btrfs_map_block(fs_info, btrfs_op(orig_bio), start_sector << 9,
- &map_length, NULL, 0);
+ submit_len = orig_bio->bi_iter.bi_size;
+ ret = btrfs_get_io_geometry(fs_info, btrfs_op(orig_bio),
+ start_sector << 9, submit_len, &geom);
if (ret)
return -EIO;
- if (map_length >= submit_len) {
+ if (geom.len >= submit_len) {
bio = orig_bio;
dip->flags |= BTRFS_DIO_ORIG_BIO_SUBMITTED;
goto submit;
async_submit = 1;
/* bio split */
- ASSERT(map_length <= INT_MAX);
+ ASSERT(geom.len <= INT_MAX);
atomic_inc(&dip->pending_bios);
do {
- clone_len = min_t(int, submit_len, map_length);
+ clone_len = min_t(int, submit_len, geom.len);
/*
* This will never fail as it's passing GPF_NOFS and
start_sector += clone_len >> 9;
file_offset += clone_len;
- map_length = submit_len;
- ret = btrfs_map_block(fs_info, btrfs_op(orig_bio),
- start_sector << 9, &map_length, NULL, 0);
+ ret = btrfs_get_io_geometry(fs_info, btrfs_op(orig_bio),
+ start_sector << 9, submit_len, &geom);
if (ret)
goto out_err;
} while (submit_len > 0);
#include "qgroup.h"
#include "tree-log.h"
#include "compression.h"
+#include "space-info.h"
+#include "delalloc-space.h"
#ifdef CONFIG_64BIT
/* If we have a 32-bit userspace and 64-bit kernel, then the UAPI
if (!same_inode)
inode_dio_wait(inode_out);
+ /*
+ * Workaround to make sure NOCOW buffered write reach disk as NOCOW.
+ *
+ * Btrfs' back references do not have a block level granularity, they
+ * work at the whole extent level.
+ * NOCOW buffered write without data space reserved may not be able
+ * to fall back to CoW due to lack of data space, thus could cause
+ * data loss.
+ *
+ * Here we take a shortcut by flushing the whole inode, so that all
+ * nocow write should reach disk as nocow before we increase the
+ * reference of the extent. We could do better by only flushing NOCOW
+ * data, but that needs extra accounting.
+ *
+ * Also we don't need to check ASYNC_EXTENT, as async extent will be
+ * CoWed anyway, not affecting nocow part.
+ */
+ ret = filemap_flush(inode_in->i_mapping);
+ if (ret < 0)
+ return ret;
+
ret = btrfs_wait_ordered_range(inode_in, ALIGN_DOWN(pos_in, bs),
wb_len);
if (ret < 0)
#ifdef CONFIG_BTRFS_DEBUG
static void btrfs_assert_spinning_writers_get(struct extent_buffer *eb)
{
- WARN_ON(atomic_read(&eb->spinning_writers));
- atomic_inc(&eb->spinning_writers);
+ WARN_ON(eb->spinning_writers);
+ eb->spinning_writers++;
}
static void btrfs_assert_spinning_writers_put(struct extent_buffer *eb)
{
- WARN_ON(atomic_read(&eb->spinning_writers) != 1);
- atomic_dec(&eb->spinning_writers);
+ WARN_ON(eb->spinning_writers != 1);
+ eb->spinning_writers--;
}
static void btrfs_assert_no_spinning_writers(struct extent_buffer *eb)
{
- WARN_ON(atomic_read(&eb->spinning_writers));
+ WARN_ON(eb->spinning_writers);
}
static void btrfs_assert_spinning_readers_get(struct extent_buffer *eb)
static void btrfs_assert_tree_write_locks_get(struct extent_buffer *eb)
{
- atomic_inc(&eb->write_locks);
+ eb->write_locks++;
}
static void btrfs_assert_tree_write_locks_put(struct extent_buffer *eb)
{
- atomic_dec(&eb->write_locks);
+ eb->write_locks--;
}
void btrfs_assert_tree_locked(struct extent_buffer *eb)
{
- BUG_ON(!atomic_read(&eb->write_locks));
+ BUG_ON(!eb->write_locks);
}
#else
*/
if (eb->lock_nested && current->pid == eb->lock_owner)
return;
- if (atomic_read(&eb->blocking_writers) == 0) {
+ if (eb->blocking_writers == 0) {
btrfs_assert_spinning_writers_put(eb);
btrfs_assert_tree_locked(eb);
- atomic_inc(&eb->blocking_writers);
+ eb->blocking_writers++;
write_unlock(&eb->lock);
}
}
*/
if (eb->lock_nested && current->pid == eb->lock_owner)
return;
- BUG_ON(atomic_read(&eb->blocking_writers) != 1);
write_lock(&eb->lock);
+ BUG_ON(eb->blocking_writers != 1);
btrfs_assert_spinning_writers_get(eb);
- /* atomic_dec_and_test implies a barrier */
- if (atomic_dec_and_test(&eb->blocking_writers))
- cond_wake_up_nomb(&eb->write_lock_wq);
+ if (--eb->blocking_writers == 0)
+ cond_wake_up(&eb->write_lock_wq);
}
/*
if (trace_btrfs_tree_read_lock_enabled())
start_ns = ktime_get_ns();
again:
- BUG_ON(!atomic_read(&eb->blocking_writers) &&
- current->pid == eb->lock_owner);
-
read_lock(&eb->lock);
- if (atomic_read(&eb->blocking_writers) &&
- current->pid == eb->lock_owner) {
+ BUG_ON(eb->blocking_writers == 0 &&
+ current->pid == eb->lock_owner);
+ if (eb->blocking_writers && current->pid == eb->lock_owner) {
/*
* This extent is already write-locked by our thread. We allow
* an additional read lock to be added because it's for the same
trace_btrfs_tree_read_lock(eb, start_ns);
return;
}
- if (atomic_read(&eb->blocking_writers)) {
+ if (eb->blocking_writers) {
read_unlock(&eb->lock);
wait_event(eb->write_lock_wq,
- atomic_read(&eb->blocking_writers) == 0);
+ eb->blocking_writers == 0);
goto again;
}
btrfs_assert_tree_read_locks_get(eb);
*/
int btrfs_tree_read_lock_atomic(struct extent_buffer *eb)
{
- if (atomic_read(&eb->blocking_writers))
+ if (eb->blocking_writers)
return 0;
read_lock(&eb->lock);
- if (atomic_read(&eb->blocking_writers)) {
+ if (eb->blocking_writers) {
read_unlock(&eb->lock);
return 0;
}
*/
int btrfs_try_tree_read_lock(struct extent_buffer *eb)
{
- if (atomic_read(&eb->blocking_writers))
+ if (eb->blocking_writers)
return 0;
if (!read_trylock(&eb->lock))
return 0;
- if (atomic_read(&eb->blocking_writers)) {
+ if (eb->blocking_writers) {
read_unlock(&eb->lock);
return 0;
}
*/
int btrfs_try_tree_write_lock(struct extent_buffer *eb)
{
- if (atomic_read(&eb->blocking_writers) ||
- atomic_read(&eb->blocking_readers))
+ if (eb->blocking_writers || atomic_read(&eb->blocking_readers))
return 0;
write_lock(&eb->lock);
- if (atomic_read(&eb->blocking_writers) ||
- atomic_read(&eb->blocking_readers)) {
+ if (eb->blocking_writers || atomic_read(&eb->blocking_readers)) {
write_unlock(&eb->lock);
return 0;
}
WARN_ON(eb->lock_owner == current->pid);
again:
wait_event(eb->read_lock_wq, atomic_read(&eb->blocking_readers) == 0);
- wait_event(eb->write_lock_wq, atomic_read(&eb->blocking_writers) == 0);
+ wait_event(eb->write_lock_wq, eb->blocking_writers == 0);
write_lock(&eb->lock);
- if (atomic_read(&eb->blocking_readers) ||
- atomic_read(&eb->blocking_writers)) {
+ if (atomic_read(&eb->blocking_readers) || eb->blocking_writers) {
write_unlock(&eb->lock);
goto again;
}
*/
void btrfs_tree_unlock(struct extent_buffer *eb)
{
- int blockers = atomic_read(&eb->blocking_writers);
+ int blockers = eb->blocking_writers;
BUG_ON(blockers > 1);
if (blockers) {
btrfs_assert_no_spinning_writers(eb);
- atomic_dec(&eb->blocking_writers);
+ eb->blocking_writers--;
/* Use the lighter barrier after atomic */
smp_mb__after_atomic();
cond_wake_up_nomb(&eb->write_lock_wq);
#include "extent_io.h"
#include "disk-io.h"
#include "compression.h"
+#include "delalloc-space.h"
static struct kmem_cache *btrfs_ordered_extent_cache;
* be reclaimed before their checksum is actually put into the btree
*/
int btrfs_find_ordered_sum(struct inode *inode, u64 offset, u64 disk_bytenr,
- u32 *sum, int len)
+ u8 *sum, int len)
{
+ struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb);
struct btrfs_ordered_sum *ordered_sum;
struct btrfs_ordered_extent *ordered;
struct btrfs_ordered_inode_tree *tree = &BTRFS_I(inode)->ordered_tree;
unsigned long num_sectors;
unsigned long i;
u32 sectorsize = btrfs_inode_sectorsize(inode);
+ const u16 csum_size = btrfs_super_csum_size(fs_info->super_copy);
int index = 0;
ordered = btrfs_lookup_ordered_extent(inode, offset);
num_sectors = ordered_sum->len >>
inode->i_sb->s_blocksize_bits;
num_sectors = min_t(int, len - index, num_sectors - i);
- memcpy(sum + index, ordered_sum->sums + i,
- num_sectors);
+ memcpy(sum + index, ordered_sum->sums + i * csum_size,
+ num_sectors * csum_size);
- index += (int)num_sectors;
+ index += (int)num_sectors * csum_size;
if (index == len)
goto out;
disk_bytenr += num_sectors * sectorsize;
return index;
}
+/*
+ * btrfs_flush_ordered_range - Lock the passed range and ensures all pending
+ * ordered extents in it are run to completion.
+ *
+ * @tree: IO tree used for locking out other users of the range
+ * @inode: Inode whose ordered tree is to be searched
+ * @start: Beginning of range to flush
+ * @end: Last byte of range to lock
+ * @cached_state: If passed, will return the extent state responsible for the
+ * locked range. It's the caller's responsibility to free the cached state.
+ *
+ * This function always returns with the given range locked, ensuring after it's
+ * called no order extent can be pending.
+ */
+void btrfs_lock_and_flush_ordered_range(struct extent_io_tree *tree,
+ struct btrfs_inode *inode, u64 start,
+ u64 end,
+ struct extent_state **cached_state)
+{
+ struct btrfs_ordered_extent *ordered;
+ struct extent_state *cachedp = NULL;
+
+ if (cached_state)
+ cachedp = *cached_state;
+
+ while (1) {
+ lock_extent_bits(tree, start, end, &cachedp);
+ ordered = btrfs_lookup_ordered_range(inode, start,
+ end - start + 1);
+ if (!ordered) {
+ /*
+ * If no external cached_state has been passed then
+ * decrement the extra ref taken for cachedp since we
+ * aren't exposing it outside of this function
+ */
+ if (!cached_state)
+ refcount_dec(&cachedp->refs);
+ break;
+ }
+ unlock_extent_cached(tree, start, end, &cachedp);
+ btrfs_start_ordered_extent(&inode->vfs_inode, ordered, 1);
+ btrfs_put_ordered_extent(ordered);
+ }
+}
+
int __init ordered_data_init(void)
{
btrfs_ordered_extent_cache = kmem_cache_create("btrfs_ordered_extent",
int len;
struct list_head list;
/* last field is a variable length array of csums */
- u32 sums[];
+ u8 sums[];
};
/*
int btrfs_ordered_update_i_size(struct inode *inode, u64 offset,
struct btrfs_ordered_extent *ordered);
int btrfs_find_ordered_sum(struct inode *inode, u64 offset, u64 disk_bytenr,
- u32 *sum, int len);
+ u8 *sum, int len);
u64 btrfs_wait_ordered_extents(struct btrfs_root *root, u64 nr,
const u64 range_start, const u64 range_len);
u64 btrfs_wait_ordered_roots(struct btrfs_fs_info *fs_info, u64 nr,
const u64 range_start, const u64 range_len);
+void btrfs_lock_and_flush_ordered_range(struct extent_io_tree *tree,
+ struct btrfs_inode *inode, u64 start,
+ u64 end,
+ struct extent_state **cached_state);
int __init ordered_data_init(void);
void __cold ordered_data_exit(void);
#ifdef CONFIG_BTRFS_DEBUG
btrfs_info(eb->fs_info,
"refs %u lock (w:%d r:%d bw:%d br:%d sw:%d sr:%d) lock_owner %u current %u",
- atomic_read(&eb->refs), atomic_read(&eb->write_locks),
+ atomic_read(&eb->refs), eb->write_locks,
atomic_read(&eb->read_locks),
- atomic_read(&eb->blocking_writers),
+ eb->blocking_writers,
atomic_read(&eb->blocking_readers),
- atomic_read(&eb->spinning_writers),
+ eb->spinning_writers,
atomic_read(&eb->spinning_readers),
eb->lock_owner, current->pid);
#endif
if (!value)
return 0;
- if (!strncmp("lzo", value, 3))
- return 0;
- else if (!strncmp("zlib", value, 4))
- return 0;
- else if (!strncmp("zstd", value, 4))
+ if (btrfs_compress_is_valid_type(value, len))
return 0;
return -EINVAL;
for (i = 0; i < ARRAY_SIZE(prop_handlers); i++) {
const struct prop_handler *h = &prop_handlers[i];
const char *value;
- u64 num_bytes;
+ u64 num_bytes = 0;
if (!h->inheritable)
continue;
int ret = 0;
int i;
u64 *i_qgroups;
+ bool committing = false;
struct btrfs_fs_info *fs_info = trans->fs_info;
struct btrfs_root *quota_root;
struct btrfs_qgroup *srcgroup;
u32 level_size = 0;
u64 nums;
- mutex_lock(&fs_info->qgroup_ioctl_lock);
+ /*
+ * There are only two callers of this function.
+ *
+ * One in create_subvol() in the ioctl context, which needs to hold
+ * the qgroup_ioctl_lock.
+ *
+ * The other one in create_pending_snapshot() where no other qgroup
+ * code can modify the fs as they all need to either start a new trans
+ * or hold a trans handler, thus we don't need to hold
+ * qgroup_ioctl_lock.
+ * This would avoid long and complex lock chain and make lockdep happy.
+ */
+ spin_lock(&fs_info->trans_lock);
+ if (trans->transaction->state == TRANS_STATE_COMMIT_DOING)
+ committing = true;
+ spin_unlock(&fs_info->trans_lock);
+
+ if (!committing)
+ mutex_lock(&fs_info->qgroup_ioctl_lock);
if (!test_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags))
goto out;
unlock:
spin_unlock(&fs_info->qgroup_lock);
out:
- mutex_unlock(&fs_info->qgroup_ioctl_lock);
+ if (!committing)
+ mutex_unlock(&fs_info->qgroup_ioctl_lock);
return ret;
}
#ifndef BTRFS_RAID56_H
#define BTRFS_RAID56_H
-static inline int nr_parity_stripes(struct map_lookup *map)
+static inline int nr_parity_stripes(const struct map_lookup *map)
{
if (map->type & BTRFS_BLOCK_GROUP_RAID5)
return 1;
return 0;
}
-static inline int nr_data_stripes(struct map_lookup *map)
+static inline int nr_data_stripes(const struct map_lookup *map)
{
return map->num_stripes - nr_parity_stripes(map);
}
#include "inode-map.h"
#include "qgroup.h"
#include "print-tree.h"
+#include "delalloc-space.h"
/*
* backref_node, mapping_node and tree_block start with this
#include "transaction.h"
#include "disk-io.h"
#include "print-tree.h"
+#include "qgroup.h"
+#include "space-info.h"
/*
* Read a root item from the tree. In case we detect a root item smaller then
btrfs_set_stack_timespec_nsec(&item->ctime, ct.tv_nsec);
spin_unlock(&root->root_item_lock);
}
+
+/*
+ * btrfs_subvolume_reserve_metadata() - reserve space for subvolume operation
+ * root: the root of the parent directory
+ * rsv: block reservation
+ * items: the number of items that we need do reservation
+ * use_global_rsv: allow fallback to the global block reservation
+ *
+ * This function is used to reserve the space for snapshot/subvolume
+ * creation and deletion. Those operations are different with the
+ * common file/directory operations, they change two fs/file trees
+ * and root tree, the number of items that the qgroup reserves is
+ * different with the free space reservation. So we can not use
+ * the space reservation mechanism in start_transaction().
+ */
+int btrfs_subvolume_reserve_metadata(struct btrfs_root *root,
+ struct btrfs_block_rsv *rsv, int items,
+ bool use_global_rsv)
+{
+ u64 qgroup_num_bytes = 0;
+ u64 num_bytes;
+ int ret;
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+
+ if (test_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags)) {
+ /* One for parent inode, two for dir entries */
+ qgroup_num_bytes = 3 * fs_info->nodesize;
+ ret = btrfs_qgroup_reserve_meta_prealloc(root,
+ qgroup_num_bytes, true);
+ if (ret)
+ return ret;
+ }
+
+ num_bytes = btrfs_calc_trans_metadata_size(fs_info, items);
+ rsv->space_info = btrfs_find_space_info(fs_info,
+ BTRFS_BLOCK_GROUP_METADATA);
+ ret = btrfs_block_rsv_add(root, rsv, num_bytes,
+ BTRFS_RESERVE_FLUSH_ALL);
+
+ if (ret == -ENOSPC && use_global_rsv)
+ ret = btrfs_block_rsv_migrate(global_rsv, rsv, num_bytes, true);
+
+ if (ret && qgroup_num_bytes)
+ btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes);
+
+ return ret;
+}
+
+void btrfs_subvolume_release_metadata(struct btrfs_fs_info *fs_info,
+ struct btrfs_block_rsv *rsv)
+{
+ btrfs_block_rsv_release(fs_info, rsv, (u64)-1);
+}
#include <linux/blkdev.h>
#include <linux/ratelimit.h>
#include <linux/sched/mm.h>
+#include <crypto/hash.h>
#include "ctree.h"
#include "volumes.h"
#include "disk-io.h"
static int scrub_checksum_data(struct scrub_block *sblock)
{
struct scrub_ctx *sctx = sblock->sctx;
+ struct btrfs_fs_info *fs_info = sctx->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
u8 csum[BTRFS_CSUM_SIZE];
u8 *on_disk_csum;
struct page *page;
void *buffer;
- u32 crc = ~(u32)0;
u64 len;
int index;
if (!sblock->pagev[0]->have_csum)
return 0;
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
+
on_disk_csum = sblock->pagev[0]->csum;
page = sblock->pagev[0]->page;
buffer = kmap_atomic(page);
for (;;) {
u64 l = min_t(u64, len, PAGE_SIZE);
- crc = btrfs_csum_data(buffer, crc, l);
+ crypto_shash_update(shash, buffer, l);
kunmap_atomic(buffer);
len -= l;
if (len == 0)
buffer = kmap_atomic(page);
}
- btrfs_csum_final(crc, csum);
+ crypto_shash_final(shash, csum);
if (memcmp(csum, on_disk_csum, sctx->csum_size))
sblock->checksum_error = 1;
struct scrub_ctx *sctx = sblock->sctx;
struct btrfs_header *h;
struct btrfs_fs_info *fs_info = sctx->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
u8 calculated_csum[BTRFS_CSUM_SIZE];
u8 on_disk_csum[BTRFS_CSUM_SIZE];
struct page *page;
void *mapped_buffer;
u64 mapped_size;
void *p;
- u32 crc = ~(u32)0;
u64 len;
int index;
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
+
BUG_ON(sblock->page_count < 1);
page = sblock->pagev[0]->page;
mapped_buffer = kmap_atomic(page);
for (;;) {
u64 l = min_t(u64, len, mapped_size);
- crc = btrfs_csum_data(p, crc, l);
+ crypto_shash_update(shash, p, l);
kunmap_atomic(mapped_buffer);
len -= l;
if (len == 0)
p = mapped_buffer;
}
- btrfs_csum_final(crc, calculated_csum);
+ crypto_shash_final(shash, calculated_csum);
if (memcmp(calculated_csum, on_disk_csum, sctx->csum_size))
sblock->checksum_error = 1;
{
struct btrfs_super_block *s;
struct scrub_ctx *sctx = sblock->sctx;
+ struct btrfs_fs_info *fs_info = sctx->fs_info;
+ SHASH_DESC_ON_STACK(shash, fs_info->csum_shash);
u8 calculated_csum[BTRFS_CSUM_SIZE];
u8 on_disk_csum[BTRFS_CSUM_SIZE];
struct page *page;
void *mapped_buffer;
u64 mapped_size;
void *p;
- u32 crc = ~(u32)0;
int fail_gen = 0;
int fail_cor = 0;
u64 len;
int index;
+ shash->tfm = fs_info->csum_shash;
+ crypto_shash_init(shash);
+
BUG_ON(sblock->page_count < 1);
page = sblock->pagev[0]->page;
mapped_buffer = kmap_atomic(page);
for (;;) {
u64 l = min_t(u64, len, mapped_size);
- crc = btrfs_csum_data(p, crc, l);
+ crypto_shash_update(shash, p, l);
kunmap_atomic(mapped_buffer);
len -= l;
if (len == 0)
p = mapped_buffer;
}
- btrfs_csum_final(crc, calculated_csum);
+ crypto_shash_final(shash, calculated_csum);
if (memcmp(calculated_csum, on_disk_csum, sctx->csum_size))
++fail_cor;
ASSERT(index < UINT_MAX);
num_sectors = sum->len / sctx->fs_info->sectorsize;
- memcpy(csum, sum->sums + index, sctx->csum_size);
+ memcpy(csum, sum->sums + index * sctx->csum_size, sctx->csum_size);
if (index == num_sectors - 1) {
list_del(&sum->list);
kfree(sum);
u64 last_offset;
u32 stripe_index;
u32 rot;
+ const int data_stripes = nr_data_stripes(map);
- last_offset = (physical - map->stripes[num].physical) *
- nr_data_stripes(map);
+ last_offset = (physical - map->stripes[num].physical) * data_stripes;
if (stripe_start)
*stripe_start = last_offset;
*offset = last_offset;
- for (i = 0; i < nr_data_stripes(map); i++) {
+ for (i = 0; i < data_stripes; i++) {
*offset = last_offset + i * map->stripe_len;
stripe_nr = div64_u64(*offset, map->stripe_len);
- stripe_nr = div_u64(stripe_nr, nr_data_stripes(map));
+ stripe_nr = div_u64(stripe_nr, data_stripes);
/* Work out the disk rotation on this stripe-set */
stripe_nr = div_u64_rem(stripe_nr, map->num_stripes, &rot);
offset = map->stripe_len * (num / map->sub_stripes);
increment = map->stripe_len * factor;
mirror_num = num % map->sub_stripes + 1;
- } else if (map->type & BTRFS_BLOCK_GROUP_RAID1) {
+ } else if (map->type & BTRFS_BLOCK_GROUP_RAID1_MASK) {
increment = map->stripe_len;
mirror_num = num % map->num_stripes + 1;
} else if (map->type & BTRFS_BLOCK_GROUP_DUP) {
struct btrfs_block_group_cache *cache)
{
struct btrfs_fs_info *fs_info = sctx->fs_info;
- struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree;
+ struct extent_map_tree *map_tree = &fs_info->mapping_tree;
struct map_lookup *map;
struct extent_map *em;
int i;
int ret = 0;
- read_lock(&map_tree->map_tree.lock);
- em = lookup_extent_mapping(&map_tree->map_tree, chunk_offset, 1);
- read_unlock(&map_tree->map_tree.lock);
+ read_lock(&map_tree->lock);
+ em = lookup_extent_mapping(map_tree, chunk_offset, 1);
+ read_unlock(&map_tree->lock);
if (!em) {
/*
hdr->len = cpu_to_le32(sctx->send_size - sizeof(*hdr));
hdr->crc = 0;
- crc = crc32c(0, (unsigned char *)sctx->send_buf, sctx->send_size);
+ crc = btrfs_crc32c(0, (unsigned char *)sctx->send_buf, sctx->send_size);
hdr->crc = cpu_to_le32(crc);
ret = write_buf(sctx->send_filp, sctx->send_buf, sctx->send_size,
if (ret)
goto out;
+ mutex_lock(&fs_info->balance_mutex);
+ if (test_bit(BTRFS_FS_BALANCE_RUNNING, &fs_info->flags)) {
+ mutex_unlock(&fs_info->balance_mutex);
+ btrfs_warn_rl(fs_info,
+ "cannot run send because a balance operation is in progress");
+ ret = -EAGAIN;
+ goto out;
+ }
+ fs_info->send_in_progress++;
+ mutex_unlock(&fs_info->balance_mutex);
+
current->journal_info = BTRFS_SEND_TRANS_STUB;
ret = send_subvol(sctx);
current->journal_info = NULL;
+ mutex_lock(&fs_info->balance_mutex);
+ fs_info->send_in_progress--;
+ mutex_unlock(&fs_info->balance_mutex);
if (ret < 0)
goto out;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include "ctree.h"
+#include "space-info.h"
+#include "sysfs.h"
+#include "volumes.h"
+#include "free-space-cache.h"
+#include "ordered-data.h"
+#include "transaction.h"
+#include "math.h"
+
+u64 btrfs_space_info_used(struct btrfs_space_info *s_info,
+ bool may_use_included)
+{
+ ASSERT(s_info);
+ return s_info->bytes_used + s_info->bytes_reserved +
+ s_info->bytes_pinned + s_info->bytes_readonly +
+ (may_use_included ? s_info->bytes_may_use : 0);
+}
+
+/*
+ * after adding space to the filesystem, we need to clear the full flags
+ * on all the space infos.
+ */
+void btrfs_clear_space_info_full(struct btrfs_fs_info *info)
+{
+ struct list_head *head = &info->space_info;
+ struct btrfs_space_info *found;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(found, head, list)
+ found->full = 0;
+ rcu_read_unlock();
+}
+
+static const char *alloc_name(u64 flags)
+{
+ switch (flags) {
+ case BTRFS_BLOCK_GROUP_METADATA|BTRFS_BLOCK_GROUP_DATA:
+ return "mixed";
+ case BTRFS_BLOCK_GROUP_METADATA:
+ return "metadata";
+ case BTRFS_BLOCK_GROUP_DATA:
+ return "data";
+ case BTRFS_BLOCK_GROUP_SYSTEM:
+ return "system";
+ default:
+ WARN_ON(1);
+ return "invalid-combination";
+ };
+}
+
+static int create_space_info(struct btrfs_fs_info *info, u64 flags)
+{
+
+ struct btrfs_space_info *space_info;
+ int i;
+ int ret;
+
+ space_info = kzalloc(sizeof(*space_info), GFP_NOFS);
+ if (!space_info)
+ return -ENOMEM;
+
+ ret = percpu_counter_init(&space_info->total_bytes_pinned, 0,
+ GFP_KERNEL);
+ if (ret) {
+ kfree(space_info);
+ return ret;
+ }
+
+ for (i = 0; i < BTRFS_NR_RAID_TYPES; i++)
+ INIT_LIST_HEAD(&space_info->block_groups[i]);
+ init_rwsem(&space_info->groups_sem);
+ spin_lock_init(&space_info->lock);
+ space_info->flags = flags & BTRFS_BLOCK_GROUP_TYPE_MASK;
+ space_info->force_alloc = CHUNK_ALLOC_NO_FORCE;
+ init_waitqueue_head(&space_info->wait);
+ INIT_LIST_HEAD(&space_info->ro_bgs);
+ INIT_LIST_HEAD(&space_info->tickets);
+ INIT_LIST_HEAD(&space_info->priority_tickets);
+
+ ret = kobject_init_and_add(&space_info->kobj, &space_info_ktype,
+ info->space_info_kobj, "%s",
+ alloc_name(space_info->flags));
+ if (ret) {
+ kobject_put(&space_info->kobj);
+ return ret;
+ }
+
+ list_add_rcu(&space_info->list, &info->space_info);
+ if (flags & BTRFS_BLOCK_GROUP_DATA)
+ info->data_sinfo = space_info;
+
+ return ret;
+}
+
+int btrfs_init_space_info(struct btrfs_fs_info *fs_info)
+{
+ struct btrfs_super_block *disk_super;
+ u64 features;
+ u64 flags;
+ int mixed = 0;
+ int ret;
+
+ disk_super = fs_info->super_copy;
+ if (!btrfs_super_root(disk_super))
+ return -EINVAL;
+
+ features = btrfs_super_incompat_flags(disk_super);
+ if (features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)
+ mixed = 1;
+
+ flags = BTRFS_BLOCK_GROUP_SYSTEM;
+ ret = create_space_info(fs_info, flags);
+ if (ret)
+ goto out;
+
+ if (mixed) {
+ flags = BTRFS_BLOCK_GROUP_METADATA | BTRFS_BLOCK_GROUP_DATA;
+ ret = create_space_info(fs_info, flags);
+ } else {
+ flags = BTRFS_BLOCK_GROUP_METADATA;
+ ret = create_space_info(fs_info, flags);
+ if (ret)
+ goto out;
+
+ flags = BTRFS_BLOCK_GROUP_DATA;
+ ret = create_space_info(fs_info, flags);
+ }
+out:
+ return ret;
+}
+
+void btrfs_update_space_info(struct btrfs_fs_info *info, u64 flags,
+ u64 total_bytes, u64 bytes_used,
+ u64 bytes_readonly,
+ struct btrfs_space_info **space_info)
+{
+ struct btrfs_space_info *found;
+ int factor;
+
+ factor = btrfs_bg_type_to_factor(flags);
+
+ found = btrfs_find_space_info(info, flags);
+ ASSERT(found);
+ spin_lock(&found->lock);
+ found->total_bytes += total_bytes;
+ found->disk_total += total_bytes * factor;
+ found->bytes_used += bytes_used;
+ found->disk_used += bytes_used * factor;
+ found->bytes_readonly += bytes_readonly;
+ if (total_bytes > 0)
+ found->full = 0;
+ btrfs_space_info_add_new_bytes(info, found,
+ total_bytes - bytes_used -
+ bytes_readonly);
+ spin_unlock(&found->lock);
+ *space_info = found;
+}
+
+struct btrfs_space_info *btrfs_find_space_info(struct btrfs_fs_info *info,
+ u64 flags)
+{
+ struct list_head *head = &info->space_info;
+ struct btrfs_space_info *found;
+
+ flags &= BTRFS_BLOCK_GROUP_TYPE_MASK;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(found, head, list) {
+ if (found->flags & flags) {
+ rcu_read_unlock();
+ return found;
+ }
+ }
+ rcu_read_unlock();
+ return NULL;
+}
+
+static inline u64 calc_global_rsv_need_space(struct btrfs_block_rsv *global)
+{
+ return (global->size << 1);
+}
+
+static int can_overcommit(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info, u64 bytes,
+ enum btrfs_reserve_flush_enum flush,
+ bool system_chunk)
+{
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ u64 profile;
+ u64 space_size;
+ u64 avail;
+ u64 used;
+ int factor;
+
+ /* Don't overcommit when in mixed mode. */
+ if (space_info->flags & BTRFS_BLOCK_GROUP_DATA)
+ return 0;
+
+ if (system_chunk)
+ profile = btrfs_system_alloc_profile(fs_info);
+ else
+ profile = btrfs_metadata_alloc_profile(fs_info);
+
+ used = btrfs_space_info_used(space_info, false);
+
+ /*
+ * We only want to allow over committing if we have lots of actual space
+ * free, but if we don't have enough space to handle the global reserve
+ * space then we could end up having a real enospc problem when trying
+ * to allocate a chunk or some other such important allocation.
+ */
+ spin_lock(&global_rsv->lock);
+ space_size = calc_global_rsv_need_space(global_rsv);
+ spin_unlock(&global_rsv->lock);
+ if (used + space_size >= space_info->total_bytes)
+ return 0;
+
+ used += space_info->bytes_may_use;
+
+ avail = atomic64_read(&fs_info->free_chunk_space);
+
+ /*
+ * If we have dup, raid1 or raid10 then only half of the free
+ * space is actually usable. For raid56, the space info used
+ * doesn't include the parity drive, so we don't have to
+ * change the math
+ */
+ factor = btrfs_bg_type_to_factor(profile);
+ avail = div_u64(avail, factor);
+
+ /*
+ * If we aren't flushing all things, let us overcommit up to
+ * 1/2th of the space. If we can flush, don't let us overcommit
+ * too much, let it overcommit up to 1/8 of the space.
+ */
+ if (flush == BTRFS_RESERVE_FLUSH_ALL)
+ avail >>= 3;
+ else
+ avail >>= 1;
+
+ if (used + bytes < space_info->total_bytes + avail)
+ return 1;
+ return 0;
+}
+
+/*
+ * This is for space we already have accounted in space_info->bytes_may_use, so
+ * basically when we're returning space from block_rsv's.
+ */
+void btrfs_space_info_add_old_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 num_bytes)
+{
+ struct reserve_ticket *ticket;
+ struct list_head *head;
+ u64 used;
+ enum btrfs_reserve_flush_enum flush = BTRFS_RESERVE_NO_FLUSH;
+ bool check_overcommit = false;
+
+ spin_lock(&space_info->lock);
+ head = &space_info->priority_tickets;
+
+ /*
+ * If we are over our limit then we need to check and see if we can
+ * overcommit, and if we can't then we just need to free up our space
+ * and not satisfy any requests.
+ */
+ used = btrfs_space_info_used(space_info, true);
+ if (used - num_bytes >= space_info->total_bytes)
+ check_overcommit = true;
+again:
+ while (!list_empty(head) && num_bytes) {
+ ticket = list_first_entry(head, struct reserve_ticket,
+ list);
+ /*
+ * We use 0 bytes because this space is already reserved, so
+ * adding the ticket space would be a double count.
+ */
+ if (check_overcommit &&
+ !can_overcommit(fs_info, space_info, 0, flush, false))
+ break;
+ if (num_bytes >= ticket->bytes) {
+ list_del_init(&ticket->list);
+ num_bytes -= ticket->bytes;
+ ticket->bytes = 0;
+ space_info->tickets_id++;
+ wake_up(&ticket->wait);
+ } else {
+ ticket->bytes -= num_bytes;
+ num_bytes = 0;
+ }
+ }
+
+ if (num_bytes && head == &space_info->priority_tickets) {
+ head = &space_info->tickets;
+ flush = BTRFS_RESERVE_FLUSH_ALL;
+ goto again;
+ }
+ btrfs_space_info_update_bytes_may_use(fs_info, space_info, -num_bytes);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ space_info->flags, num_bytes, 0);
+ spin_unlock(&space_info->lock);
+}
+
+/*
+ * This is for newly allocated space that isn't accounted in
+ * space_info->bytes_may_use yet. So if we allocate a chunk or unpin an extent
+ * we use this helper.
+ */
+void btrfs_space_info_add_new_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 num_bytes)
+{
+ struct reserve_ticket *ticket;
+ struct list_head *head = &space_info->priority_tickets;
+
+again:
+ while (!list_empty(head) && num_bytes) {
+ ticket = list_first_entry(head, struct reserve_ticket,
+ list);
+ if (num_bytes >= ticket->bytes) {
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ space_info->flags,
+ ticket->bytes, 1);
+ list_del_init(&ticket->list);
+ num_bytes -= ticket->bytes;
+ btrfs_space_info_update_bytes_may_use(fs_info,
+ space_info,
+ ticket->bytes);
+ ticket->bytes = 0;
+ space_info->tickets_id++;
+ wake_up(&ticket->wait);
+ } else {
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ space_info->flags,
+ num_bytes, 1);
+ btrfs_space_info_update_bytes_may_use(fs_info,
+ space_info,
+ num_bytes);
+ ticket->bytes -= num_bytes;
+ num_bytes = 0;
+ }
+ }
+
+ if (num_bytes && head == &space_info->priority_tickets) {
+ head = &space_info->tickets;
+ goto again;
+ }
+}
+
+#define DUMP_BLOCK_RSV(fs_info, rsv_name) \
+do { \
+ struct btrfs_block_rsv *__rsv = &(fs_info)->rsv_name; \
+ spin_lock(&__rsv->lock); \
+ btrfs_info(fs_info, #rsv_name ": size %llu reserved %llu", \
+ __rsv->size, __rsv->reserved); \
+ spin_unlock(&__rsv->lock); \
+} while (0)
+
+void btrfs_dump_space_info(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *info, u64 bytes,
+ int dump_block_groups)
+{
+ struct btrfs_block_group_cache *cache;
+ int index = 0;
+
+ spin_lock(&info->lock);
+ btrfs_info(fs_info, "space_info %llu has %llu free, is %sfull",
+ info->flags,
+ info->total_bytes - btrfs_space_info_used(info, true),
+ info->full ? "" : "not ");
+ btrfs_info(fs_info,
+ "space_info total=%llu, used=%llu, pinned=%llu, reserved=%llu, may_use=%llu, readonly=%llu",
+ info->total_bytes, info->bytes_used, info->bytes_pinned,
+ info->bytes_reserved, info->bytes_may_use,
+ info->bytes_readonly);
+ spin_unlock(&info->lock);
+
+ DUMP_BLOCK_RSV(fs_info, global_block_rsv);
+ DUMP_BLOCK_RSV(fs_info, trans_block_rsv);
+ DUMP_BLOCK_RSV(fs_info, chunk_block_rsv);
+ DUMP_BLOCK_RSV(fs_info, delayed_block_rsv);
+ DUMP_BLOCK_RSV(fs_info, delayed_refs_rsv);
+
+ if (!dump_block_groups)
+ return;
+
+ down_read(&info->groups_sem);
+again:
+ list_for_each_entry(cache, &info->block_groups[index], list) {
+ spin_lock(&cache->lock);
+ btrfs_info(fs_info,
+ "block group %llu has %llu bytes, %llu used %llu pinned %llu reserved %s",
+ cache->key.objectid, cache->key.offset,
+ btrfs_block_group_used(&cache->item), cache->pinned,
+ cache->reserved, cache->ro ? "[readonly]" : "");
+ btrfs_dump_free_space(cache, bytes);
+ spin_unlock(&cache->lock);
+ }
+ if (++index < BTRFS_NR_RAID_TYPES)
+ goto again;
+ up_read(&info->groups_sem);
+}
+
+static void btrfs_writeback_inodes_sb_nr(struct btrfs_fs_info *fs_info,
+ unsigned long nr_pages, int nr_items)
+{
+ struct super_block *sb = fs_info->sb;
+
+ if (down_read_trylock(&sb->s_umount)) {
+ writeback_inodes_sb_nr(sb, nr_pages, WB_REASON_FS_FREE_SPACE);
+ up_read(&sb->s_umount);
+ } else {
+ /*
+ * We needn't worry the filesystem going from r/w to r/o though
+ * we don't acquire ->s_umount mutex, because the filesystem
+ * should guarantee the delalloc inodes list be empty after
+ * the filesystem is readonly(all dirty pages are written to
+ * the disk).
+ */
+ btrfs_start_delalloc_roots(fs_info, nr_items);
+ if (!current->journal_info)
+ btrfs_wait_ordered_roots(fs_info, nr_items, 0, (u64)-1);
+ }
+}
+
+static inline u64 calc_reclaim_items_nr(struct btrfs_fs_info *fs_info,
+ u64 to_reclaim)
+{
+ u64 bytes;
+ u64 nr;
+
+ bytes = btrfs_calc_trans_metadata_size(fs_info, 1);
+ nr = div64_u64(to_reclaim, bytes);
+ if (!nr)
+ nr = 1;
+ return nr;
+}
+
+#define EXTENT_SIZE_PER_ITEM SZ_256K
+
+/*
+ * shrink metadata reservation for delalloc
+ */
+static void shrink_delalloc(struct btrfs_fs_info *fs_info, u64 to_reclaim,
+ u64 orig, bool wait_ordered)
+{
+ struct btrfs_space_info *space_info;
+ struct btrfs_trans_handle *trans;
+ u64 delalloc_bytes;
+ u64 dio_bytes;
+ u64 async_pages;
+ u64 items;
+ long time_left;
+ unsigned long nr_pages;
+ int loops;
+
+ /* Calc the number of the pages we need flush for space reservation */
+ items = calc_reclaim_items_nr(fs_info, to_reclaim);
+ to_reclaim = items * EXTENT_SIZE_PER_ITEM;
+
+ trans = (struct btrfs_trans_handle *)current->journal_info;
+ space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
+
+ delalloc_bytes = percpu_counter_sum_positive(
+ &fs_info->delalloc_bytes);
+ dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes);
+ if (delalloc_bytes == 0 && dio_bytes == 0) {
+ if (trans)
+ return;
+ if (wait_ordered)
+ btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1);
+ return;
+ }
+
+ /*
+ * If we are doing more ordered than delalloc we need to just wait on
+ * ordered extents, otherwise we'll waste time trying to flush delalloc
+ * that likely won't give us the space back we need.
+ */
+ if (dio_bytes > delalloc_bytes)
+ wait_ordered = true;
+
+ loops = 0;
+ while ((delalloc_bytes || dio_bytes) && loops < 3) {
+ nr_pages = min(delalloc_bytes, to_reclaim) >> PAGE_SHIFT;
+
+ /*
+ * Triggers inode writeback for up to nr_pages. This will invoke
+ * ->writepages callback and trigger delalloc filling
+ * (btrfs_run_delalloc_range()).
+ */
+ btrfs_writeback_inodes_sb_nr(fs_info, nr_pages, items);
+
+ /*
+ * We need to wait for the compressed pages to start before
+ * we continue.
+ */
+ async_pages = atomic_read(&fs_info->async_delalloc_pages);
+ if (!async_pages)
+ goto skip_async;
+
+ /*
+ * Calculate how many compressed pages we want to be written
+ * before we continue. I.e if there are more async pages than we
+ * require wait_event will wait until nr_pages are written.
+ */
+ if (async_pages <= nr_pages)
+ async_pages = 0;
+ else
+ async_pages -= nr_pages;
+
+ wait_event(fs_info->async_submit_wait,
+ atomic_read(&fs_info->async_delalloc_pages) <=
+ (int)async_pages);
+skip_async:
+ spin_lock(&space_info->lock);
+ if (list_empty(&space_info->tickets) &&
+ list_empty(&space_info->priority_tickets)) {
+ spin_unlock(&space_info->lock);
+ break;
+ }
+ spin_unlock(&space_info->lock);
+
+ loops++;
+ if (wait_ordered && !trans) {
+ btrfs_wait_ordered_roots(fs_info, items, 0, (u64)-1);
+ } else {
+ time_left = schedule_timeout_killable(1);
+ if (time_left)
+ break;
+ }
+ delalloc_bytes = percpu_counter_sum_positive(
+ &fs_info->delalloc_bytes);
+ dio_bytes = percpu_counter_sum_positive(&fs_info->dio_bytes);
+ }
+}
+
+/**
+ * maybe_commit_transaction - possibly commit the transaction if its ok to
+ * @root - the root we're allocating for
+ * @bytes - the number of bytes we want to reserve
+ * @force - force the commit
+ *
+ * This will check to make sure that committing the transaction will actually
+ * get us somewhere and then commit the transaction if it does. Otherwise it
+ * will return -ENOSPC.
+ */
+static int may_commit_transaction(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info)
+{
+ struct reserve_ticket *ticket = NULL;
+ struct btrfs_block_rsv *delayed_rsv = &fs_info->delayed_block_rsv;
+ struct btrfs_block_rsv *delayed_refs_rsv = &fs_info->delayed_refs_rsv;
+ struct btrfs_trans_handle *trans;
+ u64 bytes_needed;
+ u64 reclaim_bytes = 0;
+
+ trans = (struct btrfs_trans_handle *)current->journal_info;
+ if (trans)
+ return -EAGAIN;
+
+ spin_lock(&space_info->lock);
+ if (!list_empty(&space_info->priority_tickets))
+ ticket = list_first_entry(&space_info->priority_tickets,
+ struct reserve_ticket, list);
+ else if (!list_empty(&space_info->tickets))
+ ticket = list_first_entry(&space_info->tickets,
+ struct reserve_ticket, list);
+ bytes_needed = (ticket) ? ticket->bytes : 0;
+ spin_unlock(&space_info->lock);
+
+ if (!bytes_needed)
+ return 0;
+
+ trans = btrfs_join_transaction(fs_info->extent_root);
+ if (IS_ERR(trans))
+ return PTR_ERR(trans);
+
+ /*
+ * See if there is enough pinned space to make this reservation, or if
+ * we have block groups that are going to be freed, allowing us to
+ * possibly do a chunk allocation the next loop through.
+ */
+ if (test_bit(BTRFS_TRANS_HAVE_FREE_BGS, &trans->transaction->flags) ||
+ __percpu_counter_compare(&space_info->total_bytes_pinned,
+ bytes_needed,
+ BTRFS_TOTAL_BYTES_PINNED_BATCH) >= 0)
+ goto commit;
+
+ /*
+ * See if there is some space in the delayed insertion reservation for
+ * this reservation.
+ */
+ if (space_info != delayed_rsv->space_info)
+ goto enospc;
+
+ spin_lock(&delayed_rsv->lock);
+ reclaim_bytes += delayed_rsv->reserved;
+ spin_unlock(&delayed_rsv->lock);
+
+ spin_lock(&delayed_refs_rsv->lock);
+ reclaim_bytes += delayed_refs_rsv->reserved;
+ spin_unlock(&delayed_refs_rsv->lock);
+ if (reclaim_bytes >= bytes_needed)
+ goto commit;
+ bytes_needed -= reclaim_bytes;
+
+ if (__percpu_counter_compare(&space_info->total_bytes_pinned,
+ bytes_needed,
+ BTRFS_TOTAL_BYTES_PINNED_BATCH) < 0)
+ goto enospc;
+
+commit:
+ return btrfs_commit_transaction(trans);
+enospc:
+ btrfs_end_transaction(trans);
+ return -ENOSPC;
+}
+
+/*
+ * Try to flush some data based on policy set by @state. This is only advisory
+ * and may fail for various reasons. The caller is supposed to examine the
+ * state of @space_info to detect the outcome.
+ */
+static void flush_space(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info, u64 num_bytes,
+ int state)
+{
+ struct btrfs_root *root = fs_info->extent_root;
+ struct btrfs_trans_handle *trans;
+ int nr;
+ int ret = 0;
+
+ switch (state) {
+ case FLUSH_DELAYED_ITEMS_NR:
+ case FLUSH_DELAYED_ITEMS:
+ if (state == FLUSH_DELAYED_ITEMS_NR)
+ nr = calc_reclaim_items_nr(fs_info, num_bytes) * 2;
+ else
+ nr = -1;
+
+ trans = btrfs_join_transaction(root);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ break;
+ }
+ ret = btrfs_run_delayed_items_nr(trans, nr);
+ btrfs_end_transaction(trans);
+ break;
+ case FLUSH_DELALLOC:
+ case FLUSH_DELALLOC_WAIT:
+ shrink_delalloc(fs_info, num_bytes * 2, num_bytes,
+ state == FLUSH_DELALLOC_WAIT);
+ break;
+ case FLUSH_DELAYED_REFS_NR:
+ case FLUSH_DELAYED_REFS:
+ trans = btrfs_join_transaction(root);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ break;
+ }
+ if (state == FLUSH_DELAYED_REFS_NR)
+ nr = calc_reclaim_items_nr(fs_info, num_bytes);
+ else
+ nr = 0;
+ btrfs_run_delayed_refs(trans, nr);
+ btrfs_end_transaction(trans);
+ break;
+ case ALLOC_CHUNK:
+ case ALLOC_CHUNK_FORCE:
+ trans = btrfs_join_transaction(root);
+ if (IS_ERR(trans)) {
+ ret = PTR_ERR(trans);
+ break;
+ }
+ ret = btrfs_chunk_alloc(trans,
+ btrfs_metadata_alloc_profile(fs_info),
+ (state == ALLOC_CHUNK) ? CHUNK_ALLOC_NO_FORCE :
+ CHUNK_ALLOC_FORCE);
+ btrfs_end_transaction(trans);
+ if (ret > 0 || ret == -ENOSPC)
+ ret = 0;
+ break;
+ case COMMIT_TRANS:
+ /*
+ * If we have pending delayed iputs then we could free up a
+ * bunch of pinned space, so make sure we run the iputs before
+ * we do our pinned bytes check below.
+ */
+ btrfs_run_delayed_iputs(fs_info);
+ btrfs_wait_on_delayed_iputs(fs_info);
+
+ ret = may_commit_transaction(fs_info, space_info);
+ break;
+ default:
+ ret = -ENOSPC;
+ break;
+ }
+
+ trace_btrfs_flush_space(fs_info, space_info->flags, num_bytes, state,
+ ret);
+ return;
+}
+
+static inline u64
+btrfs_calc_reclaim_metadata_size(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ bool system_chunk)
+{
+ struct reserve_ticket *ticket;
+ u64 used;
+ u64 expected;
+ u64 to_reclaim = 0;
+
+ list_for_each_entry(ticket, &space_info->tickets, list)
+ to_reclaim += ticket->bytes;
+ list_for_each_entry(ticket, &space_info->priority_tickets, list)
+ to_reclaim += ticket->bytes;
+ if (to_reclaim)
+ return to_reclaim;
+
+ to_reclaim = min_t(u64, num_online_cpus() * SZ_1M, SZ_16M);
+ if (can_overcommit(fs_info, space_info, to_reclaim,
+ BTRFS_RESERVE_FLUSH_ALL, system_chunk))
+ return 0;
+
+ used = btrfs_space_info_used(space_info, true);
+
+ if (can_overcommit(fs_info, space_info, SZ_1M,
+ BTRFS_RESERVE_FLUSH_ALL, system_chunk))
+ expected = div_factor_fine(space_info->total_bytes, 95);
+ else
+ expected = div_factor_fine(space_info->total_bytes, 90);
+
+ if (used > expected)
+ to_reclaim = used - expected;
+ else
+ to_reclaim = 0;
+ to_reclaim = min(to_reclaim, space_info->bytes_may_use +
+ space_info->bytes_reserved);
+ return to_reclaim;
+}
+
+static inline int need_do_async_reclaim(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 used, bool system_chunk)
+{
+ u64 thresh = div_factor_fine(space_info->total_bytes, 98);
+
+ /* If we're just plain full then async reclaim just slows us down. */
+ if ((space_info->bytes_used + space_info->bytes_reserved) >= thresh)
+ return 0;
+
+ if (!btrfs_calc_reclaim_metadata_size(fs_info, space_info,
+ system_chunk))
+ return 0;
+
+ return (used >= thresh && !btrfs_fs_closing(fs_info) &&
+ !test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state));
+}
+
+static bool wake_all_tickets(struct list_head *head)
+{
+ struct reserve_ticket *ticket;
+
+ while (!list_empty(head)) {
+ ticket = list_first_entry(head, struct reserve_ticket, list);
+ list_del_init(&ticket->list);
+ ticket->error = -ENOSPC;
+ wake_up(&ticket->wait);
+ if (ticket->bytes != ticket->orig_bytes)
+ return true;
+ }
+ return false;
+}
+
+/*
+ * This is for normal flushers, we can wait all goddamned day if we want to. We
+ * will loop and continuously try to flush as long as we are making progress.
+ * We count progress as clearing off tickets each time we have to loop.
+ */
+static void btrfs_async_reclaim_metadata_space(struct work_struct *work)
+{
+ struct btrfs_fs_info *fs_info;
+ struct btrfs_space_info *space_info;
+ u64 to_reclaim;
+ int flush_state;
+ int commit_cycles = 0;
+ u64 last_tickets_id;
+
+ fs_info = container_of(work, struct btrfs_fs_info, async_reclaim_work);
+ space_info = btrfs_find_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA);
+
+ spin_lock(&space_info->lock);
+ to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
+ false);
+ if (!to_reclaim) {
+ space_info->flush = 0;
+ spin_unlock(&space_info->lock);
+ return;
+ }
+ last_tickets_id = space_info->tickets_id;
+ spin_unlock(&space_info->lock);
+
+ flush_state = FLUSH_DELAYED_ITEMS_NR;
+ do {
+ flush_space(fs_info, space_info, to_reclaim, flush_state);
+ spin_lock(&space_info->lock);
+ if (list_empty(&space_info->tickets)) {
+ space_info->flush = 0;
+ spin_unlock(&space_info->lock);
+ return;
+ }
+ to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info,
+ space_info,
+ false);
+ if (last_tickets_id == space_info->tickets_id) {
+ flush_state++;
+ } else {
+ last_tickets_id = space_info->tickets_id;
+ flush_state = FLUSH_DELAYED_ITEMS_NR;
+ if (commit_cycles)
+ commit_cycles--;
+ }
+
+ /*
+ * We don't want to force a chunk allocation until we've tried
+ * pretty hard to reclaim space. Think of the case where we
+ * freed up a bunch of space and so have a lot of pinned space
+ * to reclaim. We would rather use that than possibly create a
+ * underutilized metadata chunk. So if this is our first run
+ * through the flushing state machine skip ALLOC_CHUNK_FORCE and
+ * commit the transaction. If nothing has changed the next go
+ * around then we can force a chunk allocation.
+ */
+ if (flush_state == ALLOC_CHUNK_FORCE && !commit_cycles)
+ flush_state++;
+
+ if (flush_state > COMMIT_TRANS) {
+ commit_cycles++;
+ if (commit_cycles > 2) {
+ if (wake_all_tickets(&space_info->tickets)) {
+ flush_state = FLUSH_DELAYED_ITEMS_NR;
+ commit_cycles--;
+ } else {
+ space_info->flush = 0;
+ }
+ } else {
+ flush_state = FLUSH_DELAYED_ITEMS_NR;
+ }
+ }
+ spin_unlock(&space_info->lock);
+ } while (flush_state <= COMMIT_TRANS);
+}
+
+void btrfs_init_async_reclaim_work(struct work_struct *work)
+{
+ INIT_WORK(work, btrfs_async_reclaim_metadata_space);
+}
+
+static const enum btrfs_flush_state priority_flush_states[] = {
+ FLUSH_DELAYED_ITEMS_NR,
+ FLUSH_DELAYED_ITEMS,
+ ALLOC_CHUNK,
+};
+
+static void priority_reclaim_metadata_space(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ struct reserve_ticket *ticket)
+{
+ u64 to_reclaim;
+ int flush_state;
+
+ spin_lock(&space_info->lock);
+ to_reclaim = btrfs_calc_reclaim_metadata_size(fs_info, space_info,
+ false);
+ if (!to_reclaim) {
+ spin_unlock(&space_info->lock);
+ return;
+ }
+ spin_unlock(&space_info->lock);
+
+ flush_state = 0;
+ do {
+ flush_space(fs_info, space_info, to_reclaim,
+ priority_flush_states[flush_state]);
+ flush_state++;
+ spin_lock(&space_info->lock);
+ if (ticket->bytes == 0) {
+ spin_unlock(&space_info->lock);
+ return;
+ }
+ spin_unlock(&space_info->lock);
+ } while (flush_state < ARRAY_SIZE(priority_flush_states));
+}
+
+static int wait_reserve_ticket(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ struct reserve_ticket *ticket)
+
+{
+ DEFINE_WAIT(wait);
+ u64 reclaim_bytes = 0;
+ int ret = 0;
+
+ spin_lock(&space_info->lock);
+ while (ticket->bytes > 0 && ticket->error == 0) {
+ ret = prepare_to_wait_event(&ticket->wait, &wait, TASK_KILLABLE);
+ if (ret) {
+ ret = -EINTR;
+ break;
+ }
+ spin_unlock(&space_info->lock);
+
+ schedule();
+
+ finish_wait(&ticket->wait, &wait);
+ spin_lock(&space_info->lock);
+ }
+ if (!ret)
+ ret = ticket->error;
+ if (!list_empty(&ticket->list))
+ list_del_init(&ticket->list);
+ if (ticket->bytes && ticket->bytes < ticket->orig_bytes)
+ reclaim_bytes = ticket->orig_bytes - ticket->bytes;
+ spin_unlock(&space_info->lock);
+
+ if (reclaim_bytes)
+ btrfs_space_info_add_old_bytes(fs_info, space_info,
+ reclaim_bytes);
+ return ret;
+}
+
+/**
+ * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
+ * @root - the root we're allocating for
+ * @space_info - the space info we want to allocate from
+ * @orig_bytes - the number of bytes we want
+ * @flush - whether or not we can flush to make our reservation
+ *
+ * This will reserve orig_bytes number of bytes from the space info associated
+ * with the block_rsv. If there is not enough space it will make an attempt to
+ * flush out space to make room. It will do this by flushing delalloc if
+ * possible or committing the transaction. If flush is 0 then no attempts to
+ * regain reservations will be made and this will fail if there is not enough
+ * space already.
+ */
+static int __reserve_metadata_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 orig_bytes,
+ enum btrfs_reserve_flush_enum flush,
+ bool system_chunk)
+{
+ struct reserve_ticket ticket;
+ u64 used;
+ u64 reclaim_bytes = 0;
+ int ret = 0;
+
+ ASSERT(orig_bytes);
+ ASSERT(!current->journal_info || flush != BTRFS_RESERVE_FLUSH_ALL);
+
+ spin_lock(&space_info->lock);
+ ret = -ENOSPC;
+ used = btrfs_space_info_used(space_info, true);
+
+ /*
+ * Carry on if we have enough space (short-circuit) OR call
+ * can_overcommit() to ensure we can overcommit to continue.
+ */
+ if ((used + orig_bytes <= space_info->total_bytes) ||
+ can_overcommit(fs_info, space_info, orig_bytes, flush,
+ system_chunk)) {
+ btrfs_space_info_update_bytes_may_use(fs_info, space_info,
+ orig_bytes);
+ trace_btrfs_space_reservation(fs_info, "space_info",
+ space_info->flags, orig_bytes, 1);
+ ret = 0;
+ }
+
+ /*
+ * If we couldn't make a reservation then setup our reservation ticket
+ * and kick the async worker if it's not already running.
+ *
+ * If we are a priority flusher then we just need to add our ticket to
+ * the list and we will do our own flushing further down.
+ */
+ if (ret && flush != BTRFS_RESERVE_NO_FLUSH) {
+ ticket.orig_bytes = orig_bytes;
+ ticket.bytes = orig_bytes;
+ ticket.error = 0;
+ init_waitqueue_head(&ticket.wait);
+ if (flush == BTRFS_RESERVE_FLUSH_ALL) {
+ list_add_tail(&ticket.list, &space_info->tickets);
+ if (!space_info->flush) {
+ space_info->flush = 1;
+ trace_btrfs_trigger_flush(fs_info,
+ space_info->flags,
+ orig_bytes, flush,
+ "enospc");
+ queue_work(system_unbound_wq,
+ &fs_info->async_reclaim_work);
+ }
+ } else {
+ list_add_tail(&ticket.list,
+ &space_info->priority_tickets);
+ }
+ } else if (!ret && space_info->flags & BTRFS_BLOCK_GROUP_METADATA) {
+ used += orig_bytes;
+ /*
+ * We will do the space reservation dance during log replay,
+ * which means we won't have fs_info->fs_root set, so don't do
+ * the async reclaim as we will panic.
+ */
+ if (!test_bit(BTRFS_FS_LOG_RECOVERING, &fs_info->flags) &&
+ need_do_async_reclaim(fs_info, space_info,
+ used, system_chunk) &&
+ !work_busy(&fs_info->async_reclaim_work)) {
+ trace_btrfs_trigger_flush(fs_info, space_info->flags,
+ orig_bytes, flush, "preempt");
+ queue_work(system_unbound_wq,
+ &fs_info->async_reclaim_work);
+ }
+ }
+ spin_unlock(&space_info->lock);
+ if (!ret || flush == BTRFS_RESERVE_NO_FLUSH)
+ return ret;
+
+ if (flush == BTRFS_RESERVE_FLUSH_ALL)
+ return wait_reserve_ticket(fs_info, space_info, &ticket);
+
+ ret = 0;
+ priority_reclaim_metadata_space(fs_info, space_info, &ticket);
+ spin_lock(&space_info->lock);
+ if (ticket.bytes) {
+ if (ticket.bytes < orig_bytes)
+ reclaim_bytes = orig_bytes - ticket.bytes;
+ list_del_init(&ticket.list);
+ ret = -ENOSPC;
+ }
+ spin_unlock(&space_info->lock);
+
+ if (reclaim_bytes)
+ btrfs_space_info_add_old_bytes(fs_info, space_info,
+ reclaim_bytes);
+ ASSERT(list_empty(&ticket.list));
+ return ret;
+}
+
+/**
+ * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
+ * @root - the root we're allocating for
+ * @block_rsv - the block_rsv we're allocating for
+ * @orig_bytes - the number of bytes we want
+ * @flush - whether or not we can flush to make our reservation
+ *
+ * This will reserve orig_bytes number of bytes from the space info associated
+ * with the block_rsv. If there is not enough space it will make an attempt to
+ * flush out space to make room. It will do this by flushing delalloc if
+ * possible or committing the transaction. If flush is 0 then no attempts to
+ * regain reservations will be made and this will fail if there is not enough
+ * space already.
+ */
+int btrfs_reserve_metadata_bytes(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv,
+ u64 orig_bytes,
+ enum btrfs_reserve_flush_enum flush)
+{
+ struct btrfs_fs_info *fs_info = root->fs_info;
+ struct btrfs_block_rsv *global_rsv = &fs_info->global_block_rsv;
+ int ret;
+ bool system_chunk = (root == fs_info->chunk_root);
+
+ ret = __reserve_metadata_bytes(fs_info, block_rsv->space_info,
+ orig_bytes, flush, system_chunk);
+ if (ret == -ENOSPC &&
+ unlikely(root->orphan_cleanup_state == ORPHAN_CLEANUP_STARTED)) {
+ if (block_rsv != global_rsv &&
+ !btrfs_block_rsv_use_bytes(global_rsv, orig_bytes))
+ ret = 0;
+ }
+ if (ret == -ENOSPC) {
+ trace_btrfs_space_reservation(fs_info, "space_info:enospc",
+ block_rsv->space_info->flags,
+ orig_bytes, 1);
+
+ if (btrfs_test_opt(fs_info, ENOSPC_DEBUG))
+ btrfs_dump_space_info(fs_info, block_rsv->space_info,
+ orig_bytes, 0);
+ }
+ return ret;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef BTRFS_SPACE_INFO_H
+#define BTRFS_SPACE_INFO_H
+
+struct btrfs_space_info {
+ spinlock_t lock;
+
+ u64 total_bytes; /* total bytes in the space,
+ this doesn't take mirrors into account */
+ u64 bytes_used; /* total bytes used,
+ this doesn't take mirrors into account */
+ u64 bytes_pinned; /* total bytes pinned, will be freed when the
+ transaction finishes */
+ u64 bytes_reserved; /* total bytes the allocator has reserved for
+ current allocations */
+ u64 bytes_may_use; /* number of bytes that may be used for
+ delalloc/allocations */
+ u64 bytes_readonly; /* total bytes that are read only */
+
+ u64 max_extent_size; /* This will hold the maximum extent size of
+ the space info if we had an ENOSPC in the
+ allocator. */
+
+ unsigned int full:1; /* indicates that we cannot allocate any more
+ chunks for this space */
+ unsigned int chunk_alloc:1; /* set if we are allocating a chunk */
+
+ unsigned int flush:1; /* set if we are trying to make space */
+
+ unsigned int force_alloc; /* set if we need to force a chunk
+ alloc for this space */
+
+ u64 disk_used; /* total bytes used on disk */
+ u64 disk_total; /* total bytes on disk, takes mirrors into
+ account */
+
+ u64 flags;
+
+ /*
+ * bytes_pinned is kept in line with what is actually pinned, as in
+ * we've called update_block_group and dropped the bytes_used counter
+ * and increased the bytes_pinned counter. However this means that
+ * bytes_pinned does not reflect the bytes that will be pinned once the
+ * delayed refs are flushed, so this counter is inc'ed every time we
+ * call btrfs_free_extent so it is a realtime count of what will be
+ * freed once the transaction is committed. It will be zeroed every
+ * time the transaction commits.
+ */
+ struct percpu_counter total_bytes_pinned;
+
+ struct list_head list;
+ /* Protected by the spinlock 'lock'. */
+ struct list_head ro_bgs;
+ struct list_head priority_tickets;
+ struct list_head tickets;
+ /*
+ * tickets_id just indicates the next ticket will be handled, so note
+ * it's not stored per ticket.
+ */
+ u64 tickets_id;
+
+ struct rw_semaphore groups_sem;
+ /* for block groups in our same type */
+ struct list_head block_groups[BTRFS_NR_RAID_TYPES];
+ wait_queue_head_t wait;
+
+ struct kobject kobj;
+ struct kobject *block_group_kobjs[BTRFS_NR_RAID_TYPES];
+};
+
+struct reserve_ticket {
+ u64 orig_bytes;
+ u64 bytes;
+ int error;
+ struct list_head list;
+ wait_queue_head_t wait;
+};
+
+static inline bool btrfs_mixed_space_info(struct btrfs_space_info *space_info)
+{
+ return ((space_info->flags & BTRFS_BLOCK_GROUP_METADATA) &&
+ (space_info->flags & BTRFS_BLOCK_GROUP_DATA));
+}
+
+/*
+ *
+ * Declare a helper function to detect underflow of various space info members
+ */
+#define DECLARE_SPACE_INFO_UPDATE(name) \
+static inline void \
+btrfs_space_info_update_##name(struct btrfs_fs_info *fs_info, \
+ struct btrfs_space_info *sinfo, \
+ s64 bytes) \
+{ \
+ lockdep_assert_held(&sinfo->lock); \
+ trace_update_##name(fs_info, sinfo, sinfo->name, bytes); \
+ if (bytes < 0 && sinfo->name < -bytes) { \
+ WARN_ON(1); \
+ sinfo->name = 0; \
+ return; \
+ } \
+ sinfo->name += bytes; \
+}
+
+DECLARE_SPACE_INFO_UPDATE(bytes_may_use);
+DECLARE_SPACE_INFO_UPDATE(bytes_pinned);
+
+void btrfs_space_info_add_new_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 num_bytes);
+void btrfs_space_info_add_old_bytes(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *space_info,
+ u64 num_bytes);
+int btrfs_init_space_info(struct btrfs_fs_info *fs_info);
+void btrfs_update_space_info(struct btrfs_fs_info *info, u64 flags,
+ u64 total_bytes, u64 bytes_used,
+ u64 bytes_readonly,
+ struct btrfs_space_info **space_info);
+struct btrfs_space_info *btrfs_find_space_info(struct btrfs_fs_info *info,
+ u64 flags);
+u64 btrfs_space_info_used(struct btrfs_space_info *s_info,
+ bool may_use_included);
+void btrfs_clear_space_info_full(struct btrfs_fs_info *info);
+void btrfs_dump_space_info(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *info, u64 bytes,
+ int dump_block_groups);
+int btrfs_reserve_metadata_bytes(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv,
+ u64 orig_bytes,
+ enum btrfs_reserve_flush_enum flush);
+
+#endif /* BTRFS_SPACE_INFO_H */
#include "dev-replace.h"
#include "free-space-cache.h"
#include "backref.h"
+#include "space-info.h"
#include "tests/btrfs-tests.h"
#include "qgroup.h"
} else {
snprintf(s->s_id, sizeof(s->s_id), "%pg", bdev);
btrfs_sb(s)->bdev_holder = fs_type;
+ if (!strstr(crc32c_impl(), "generic"))
+ set_bit(BTRFS_FS_CSUM_IMPL_FAST, &fs_info->flags);
error = btrfs_fill_super(s, fs_devices, data);
}
if (!error)
{
struct vfsmount *mnt_root;
struct dentry *root;
- fmode_t mode = FMODE_READ;
char *subvol_name = NULL;
u64 subvol_objectid = 0;
int error = 0;
- if (!(flags & SB_RDONLY))
- mode |= FMODE_WRITE;
-
error = btrfs_parse_subvol_options(data, &subvol_name,
&subvol_objectid);
if (error) {
u64 type;
u64 avail_space;
u64 min_stripe_size;
- int min_stripes = 1, num_stripes = 1;
+ int min_stripes, num_stripes = 1;
int i = 0, nr_devices;
+ const struct btrfs_raid_attr *rattr;
/*
* We aren't under the device list lock, so this is racy-ish, but good
/* calc min stripe number for data space allocation */
type = btrfs_data_alloc_profile(fs_info);
- if (type & BTRFS_BLOCK_GROUP_RAID0) {
- min_stripes = 2;
+ rattr = &btrfs_raid_array[btrfs_bg_flags_to_raid_index(type)];
+ min_stripes = rattr->devs_min;
+
+ if (type & BTRFS_BLOCK_GROUP_RAID0)
num_stripes = nr_devices;
- } else if (type & BTRFS_BLOCK_GROUP_RAID1) {
- min_stripes = 2;
+ else if (type & BTRFS_BLOCK_GROUP_RAID1)
num_stripes = 2;
- } else if (type & BTRFS_BLOCK_GROUP_RAID10) {
- min_stripes = 4;
+ else if (type & BTRFS_BLOCK_GROUP_RAID10)
num_stripes = 4;
- }
- if (type & BTRFS_BLOCK_GROUP_DUP)
- min_stripe_size = 2 * BTRFS_STRIPE_LEN;
- else
- min_stripe_size = BTRFS_STRIPE_LEN;
+ /* Adjust for more than 1 stripe per device */
+ min_stripe_size = rattr->dev_stripes * BTRFS_STRIPE_LEN;
rcu_read_lock();
list_for_each_entry_rcu(device, &fs_devices->devices, dev_list) {
module_exit(exit_btrfs_fs)
MODULE_LICENSE("GPL");
+MODULE_SOFTDEP("pre: crc32c");
#include "transaction.h"
#include "sysfs.h"
#include "volumes.h"
+#include "space-info.h"
static inline struct btrfs_fs_info *to_fs_info(struct kobject *kobj);
static inline struct btrfs_fs_devices *to_fs_devs(struct kobject *kobj);
#include <linux/fs.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/magic.h>
#include "btrfs-tests.h"
#include "../ctree.h"
.destroy_inode = btrfs_test_destroy_inode,
};
-static struct dentry *btrfs_test_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name,
- void *data)
+
+static int btrfs_test_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "btrfs_test:", &btrfs_test_super_ops,
- NULL, BTRFS_TEST_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, BTRFS_TEST_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->ops = &btrfs_test_super_ops;
+ return 0;
}
static struct file_system_type test_type = {
.name = "btrfs_test_fs",
- .mount = btrfs_test_mount,
+ .init_fs_context = btrfs_test_init_fs_context,
.kill_sb = kill_anon_super,
};
#include "btrfs-tests.h"
#include "../ctree.h"
#include "../extent_io.h"
+#include "../btrfs_inode.h"
#define PROCESS_UNLOCK (1 << 0)
#define PROCESS_RELEASE (1 << 1)
static int test_find_delalloc(u32 sectorsize)
{
struct inode *inode;
- struct extent_io_tree tmp;
+ struct extent_io_tree *tmp;
struct page *page;
struct page *locked_page = NULL;
unsigned long index = 0;
test_std_err(TEST_ALLOC_INODE);
return -ENOMEM;
}
+ tmp = &BTRFS_I(inode)->io_tree;
/*
* Passing NULL as we don't have fs_info but tracepoints are not used
* at this point
*/
- extent_io_tree_init(NULL, &tmp, IO_TREE_SELFTEST, NULL);
+ extent_io_tree_init(NULL, tmp, IO_TREE_SELFTEST, NULL);
/*
* First go through and create and mark all of our pages dirty, we pin
* |--- delalloc ---|
* |--- search ---|
*/
- set_extent_delalloc(&tmp, 0, sectorsize - 1, 0, NULL);
+ set_extent_delalloc(tmp, 0, sectorsize - 1, 0, NULL);
start = 0;
end = 0;
- found = find_lock_delalloc_range(inode, &tmp, locked_page, &start,
+ found = find_lock_delalloc_range(inode, locked_page, &start,
&end);
if (!found) {
test_err("should have found at least one delalloc");
sectorsize - 1, start, end);
goto out_bits;
}
- unlock_extent(&tmp, start, end);
+ unlock_extent(tmp, start, end);
unlock_page(locked_page);
put_page(locked_page);
test_err("couldn't find the locked page");
goto out_bits;
}
- set_extent_delalloc(&tmp, sectorsize, max_bytes - 1, 0, NULL);
+ set_extent_delalloc(tmp, sectorsize, max_bytes - 1, 0, NULL);
start = test_start;
end = 0;
- found = find_lock_delalloc_range(inode, &tmp, locked_page, &start,
+ found = find_lock_delalloc_range(inode, locked_page, &start,
&end);
if (!found) {
test_err("couldn't find delalloc in our range");
test_err("there were unlocked pages in the range");
goto out_bits;
}
- unlock_extent(&tmp, start, end);
+ unlock_extent(tmp, start, end);
/* locked_page was unlocked above */
put_page(locked_page);
}
start = test_start;
end = 0;
- found = find_lock_delalloc_range(inode, &tmp, locked_page, &start,
+ found = find_lock_delalloc_range(inode, locked_page, &start,
&end);
if (found) {
test_err("found range when we shouldn't have");
*
* We are re-using our test_start from above since it works out well.
*/
- set_extent_delalloc(&tmp, max_bytes, total_dirty - 1, 0, NULL);
+ set_extent_delalloc(tmp, max_bytes, total_dirty - 1, 0, NULL);
start = test_start;
end = 0;
- found = find_lock_delalloc_range(inode, &tmp, locked_page, &start,
+ found = find_lock_delalloc_range(inode, locked_page, &start,
&end);
if (!found) {
test_err("didn't find our range");
test_err("pages in range were not all locked");
goto out_bits;
}
- unlock_extent(&tmp, start, end);
+ unlock_extent(tmp, start, end);
/*
* Now to test where we run into a page that is no longer dirty in the
* this changes at any point in the future we will need to fix this
* tests expected behavior.
*/
- found = find_lock_delalloc_range(inode, &tmp, locked_page, &start,
+ found = find_lock_delalloc_range(inode, locked_page, &start,
&end);
if (!found) {
test_err("didn't find our range");
}
ret = 0;
out_bits:
- clear_extent_bits(&tmp, 0, total_dirty - 1, (unsigned)-1);
+ clear_extent_bits(tmp, 0, total_dirty - 1, (unsigned)-1);
out:
if (locked_page)
put_page(locked_page);
return ret;
}
+static int test_find_first_clear_extent_bit(void)
+{
+ struct extent_io_tree tree;
+ u64 start, end;
+
+ test_msg("running find_first_clear_extent_bit test");
+ extent_io_tree_init(NULL, &tree, IO_TREE_SELFTEST, NULL);
+
+ /*
+ * Set 1M-4M alloc/discard and 32M-64M thus leaving a hole between
+ * 4M-32M
+ */
+ set_extent_bits(&tree, SZ_1M, SZ_4M - 1,
+ CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ find_first_clear_extent_bit(&tree, SZ_512K, &start, &end,
+ CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ if (start != 0 || end != SZ_1M -1)
+ test_err("error finding beginning range: start %llu end %llu",
+ start, end);
+
+ /* Now add 32M-64M so that we have a hole between 4M-32M */
+ set_extent_bits(&tree, SZ_32M, SZ_64M - 1,
+ CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ /*
+ * Request first hole starting at 12M, we should get 4M-32M
+ */
+ find_first_clear_extent_bit(&tree, 12 * SZ_1M, &start, &end,
+ CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ if (start != SZ_4M || end != SZ_32M - 1)
+ test_err("error finding trimmed range: start %llu end %llu",
+ start, end);
+
+ /*
+ * Search in the middle of allocated range, should get the next one
+ * available, which happens to be unallocated -> 4M-32M
+ */
+ find_first_clear_extent_bit(&tree, SZ_2M, &start, &end,
+ CHUNK_TRIMMED | CHUNK_ALLOCATED);
+
+ if (start != SZ_4M || end != SZ_32M -1)
+ test_err("error finding next unalloc range: start %llu end %llu",
+ start, end);
+
+ /*
+ * Set 64M-72M with CHUNK_ALLOC flag, then search for CHUNK_TRIMMED flag
+ * being unset in this range, we should get the entry in range 64M-72M
+ */
+ set_extent_bits(&tree, SZ_64M, SZ_64M + SZ_8M - 1, CHUNK_ALLOCATED);
+ find_first_clear_extent_bit(&tree, SZ_64M + SZ_1M, &start, &end,
+ CHUNK_TRIMMED);
+
+ if (start != SZ_64M || end != SZ_64M + SZ_8M - 1)
+ test_err("error finding exact range: start %llu end %llu",
+ start, end);
+
+ find_first_clear_extent_bit(&tree, SZ_64M - SZ_8M, &start, &end,
+ CHUNK_TRIMMED);
+
+ /*
+ * Search in the middle of set range whose immediate neighbour doesn't
+ * have the bits set so it must be returned
+ */
+ if (start != SZ_64M || end != SZ_64M + SZ_8M - 1)
+ test_err("error finding next alloc range: start %llu end %llu",
+ start, end);
+
+ /*
+ * Search beyond any known range, shall return after last known range
+ * and end should be -1
+ */
+ find_first_clear_extent_bit(&tree, -1, &start, &end, CHUNK_TRIMMED);
+ if (start != SZ_64M + SZ_8M || end != -1)
+ test_err(
+ "error handling beyond end of range search: start %llu end %llu",
+ start, end);
+
+ return 0;
+}
+
int btrfs_test_extent_io(u32 sectorsize, u32 nodesize)
{
int ret;
if (ret)
goto out;
+ ret = test_find_first_clear_extent_bit();
+ if (ret)
+ goto out;
+
ret = test_eb_bitmaps(sectorsize, nodesize);
out:
return ret;
em->len = SZ_16K;
em->block_start = 0;
em->block_len = SZ_16K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [0, 16K)");
goto out;
em->len = SZ_4K;
em->block_start = SZ_32K; /* avoid merging */
em->block_len = SZ_4K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [16K, 20K)");
goto out;
em->len = len;
em->block_start = start;
em->block_len = len;
+ write_lock(&em_tree->lock);
ret = btrfs_add_extent_mapping(fs_info, em_tree, &em, em->start, em->len);
+ write_unlock(&em_tree->lock);
if (ret) {
test_err("case1 [%llu %llu]: ret %d", start, start + len, ret);
goto out;
em->len = SZ_1K;
em->block_start = EXTENT_MAP_INLINE;
em->block_len = (u64)-1;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [0, 1K)");
goto out;
em->len = SZ_4K;
em->block_start = SZ_4K;
em->block_len = SZ_4K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [4K, 8K)");
goto out;
em->len = SZ_1K;
em->block_start = EXTENT_MAP_INLINE;
em->block_len = (u64)-1;
+ write_lock(&em_tree->lock);
ret = btrfs_add_extent_mapping(fs_info, em_tree, &em, em->start, em->len);
+ write_unlock(&em_tree->lock);
if (ret) {
test_err("case2 [0 1K]: ret %d", ret);
goto out;
em->len = SZ_4K;
em->block_start = SZ_4K;
em->block_len = SZ_4K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [4K, 8K)");
goto out;
em->len = SZ_16K;
em->block_start = 0;
em->block_len = SZ_16K;
+ write_lock(&em_tree->lock);
ret = btrfs_add_extent_mapping(fs_info, em_tree, &em, start, len);
+ write_unlock(&em_tree->lock);
if (ret) {
test_err("case3 [0x%llx 0x%llx): ret %d",
start, start + len, ret);
em->len = SZ_8K;
em->block_start = 0;
em->block_len = SZ_8K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [0, 8K)");
goto out;
em->len = 24 * SZ_1K;
em->block_start = SZ_16K; /* avoid merging */
em->block_len = 24 * SZ_1K;
+ write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
+ write_unlock(&em_tree->lock);
if (ret < 0) {
test_err("cannot add extent range [8K, 32K)");
goto out;
em->len = SZ_32K;
em->block_start = 0;
em->block_len = SZ_32K;
+ write_lock(&em_tree->lock);
ret = btrfs_add_extent_mapping(fs_info, em_tree, &em, start, len);
+ write_unlock(&em_tree->lock);
if (ret) {
test_err("case4 [0x%llx 0x%llx): ret %d",
start, len, ret);
}
/*
+ * To be called after all the new block groups attached to the transaction
+ * handle have been created (btrfs_create_pending_block_groups()).
+ */
+void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans)
+{
+ struct btrfs_fs_info *fs_info = trans->fs_info;
+
+ if (!trans->chunk_bytes_reserved)
+ return;
+
+ WARN_ON_ONCE(!list_empty(&trans->new_bgs));
+
+ btrfs_block_rsv_release(fs_info, &fs_info->chunk_block_rsv,
+ trans->chunk_bytes_reserved);
+ trans->chunk_bytes_reserved = 0;
+}
+
+/*
* either allocate a new transaction or hop into the existing one
*/
static noinline int join_transaction(struct btrfs_fs_info *fs_info,
void btrfs_apply_pending_changes(struct btrfs_fs_info *fs_info);
void btrfs_add_dropped_root(struct btrfs_trans_handle *trans,
struct btrfs_root *root);
+void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans);
#endif
struct btrfs_file_extent_item *fi;
u32 sectorsize = fs_info->sectorsize;
u32 item_size = btrfs_item_size_nr(leaf, slot);
+ u64 extent_end;
if (!IS_ALIGNED(key->offset, sectorsize)) {
file_extent_err(leaf, slot,
CHECK_FE_ALIGNED(leaf, slot, fi, num_bytes, sectorsize))
return -EUCLEAN;
+ /* Catch extent end overflow */
+ if (check_add_overflow(btrfs_file_extent_num_bytes(leaf, fi),
+ key->offset, &extent_end)) {
+ file_extent_err(leaf, slot,
+ "extent end overflow, have file offset %llu extent num bytes %llu",
+ key->offset,
+ btrfs_file_extent_num_bytes(leaf, fi));
+ return -EUCLEAN;
+ }
+
/*
* Check that no two consecutive file extent items, in the same leaf,
* present ranges that overlap each other.
}
/*
+ * Check if an inode was logged in the current transaction. We can't always rely
+ * on an inode's logged_trans value, because it's an in-memory only field and
+ * therefore not persisted. This means that its value is lost if the inode gets
+ * evicted and loaded again from disk (in which case it has a value of 0, and
+ * certainly it is smaller then any possible transaction ID), when that happens
+ * the full_sync flag is set in the inode's runtime flags, so on that case we
+ * assume eviction happened and ignore the logged_trans value, assuming the
+ * worst case, that the inode was logged before in the current transaction.
+ */
+static bool inode_logged(struct btrfs_trans_handle *trans,
+ struct btrfs_inode *inode)
+{
+ if (inode->logged_trans == trans->transid)
+ return true;
+
+ if (inode->last_trans == trans->transid &&
+ test_bit(BTRFS_INODE_NEEDS_FULL_SYNC, &inode->runtime_flags) &&
+ !test_bit(BTRFS_FS_LOG_RECOVERING, &trans->fs_info->flags))
+ return true;
+
+ return false;
+}
+
+/*
* If both a file and directory are logged, and unlinks or renames are
* mixed in, we have a few interesting corners:
*
int bytes_del = 0;
u64 dir_ino = btrfs_ino(dir);
- if (dir->logged_trans < trans->transid)
+ if (!inode_logged(trans, dir))
return 0;
ret = join_running_log_trans(root);
u64 index;
int ret;
- if (inode->logged_trans < trans->transid)
+ if (!inode_logged(trans, inode))
return 0;
ret = join_running_log_trans(root);
}
}
+ /*
+ * Don't update last_log_commit if we logged that an inode exists after
+ * it was loaded to memory (full_sync bit set).
+ * This is to prevent data loss when we do a write to the inode, then
+ * the inode gets evicted after all delalloc was flushed, then we log
+ * it exists (due to a rename for example) and then fsync it. This last
+ * fsync would do nothing (not logging the extents previously written).
+ */
spin_lock(&inode->lock);
inode->logged_trans = trans->transid;
- inode->last_log_commit = inode->last_sub_trans;
+ if (inode_only != LOG_INODE_EXISTS ||
+ !test_bit(BTRFS_INODE_NEEDS_FULL_SYNC, &inode->runtime_flags))
+ inode->last_log_commit = inode->last_sub_trans;
spin_unlock(&inode->lock);
out_unlock:
mutex_unlock(&inode->log_mutex);
#include "dev-replace.h"
#include "sysfs.h"
#include "tree-checker.h"
+#include "space-info.h"
const struct btrfs_raid_attr btrfs_raid_array[BTRFS_NR_RAID_TYPES] = {
[BTRFS_RAID_RAID10] = {
},
};
-const char *get_raid_name(enum btrfs_raid_types type)
+const char *btrfs_bg_type_to_raid_name(u64 flags)
{
- if (type >= BTRFS_NR_RAID_TYPES)
+ const int index = btrfs_bg_flags_to_raid_index(flags);
+
+ if (index >= BTRFS_NR_RAID_TYPES)
return NULL;
- return btrfs_raid_array[type].raid_name;
+ return btrfs_raid_array[index].raid_name;
}
/*
* chunk_mutex
* -----------
* protects chunks, adding or removing during allocation, trim or when a new
- * device is added/removed
+ * device is added/removed. Additionally it also protects post_commit_list of
+ * individual devices, since they can be added to the transaction's
+ * post_commit_list only with chunk_mutex held.
*
* cleaner_mutex
* -------------
struct rb_node *n;
u64 ret = 0;
- em_tree = &fs_info->mapping_tree.map_tree;
+ em_tree = &fs_info->mapping_tree;
read_lock(&em_tree->lock);
n = rb_last(&em_tree->map.rb_root);
if (n) {
struct extent_map_tree *em_tree;
struct extent_map *em;
- em_tree = &fs_info->mapping_tree.map_tree;
+ em_tree = &fs_info->mapping_tree;
read_lock(&em_tree->lock);
em = lookup_extent_mapping(em_tree, logical, length);
read_unlock(&em_tree->lock);
return 1;
}
+static u64 calc_data_stripes(u64 type, int num_stripes)
+{
+ const int index = btrfs_bg_flags_to_raid_index(type);
+ const int ncopies = btrfs_raid_array[index].ncopies;
+ const int nparity = btrfs_raid_array[index].nparity;
+
+ if (nparity)
+ return num_stripes - nparity;
+ else
+ return num_stripes / ncopies;
+}
+
/* [pstart, pend) */
static int chunk_drange_filter(struct extent_buffer *leaf,
struct btrfs_chunk *chunk,
int num_stripes = btrfs_chunk_num_stripes(leaf, chunk);
u64 stripe_offset;
u64 stripe_length;
+ u64 type;
int factor;
int i;
if (!(bargs->flags & BTRFS_BALANCE_ARGS_DEVID))
return 0;
- if (btrfs_chunk_type(leaf, chunk) & (BTRFS_BLOCK_GROUP_DUP |
- BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID10)) {
- factor = num_stripes / 2;
- } else if (btrfs_chunk_type(leaf, chunk) & BTRFS_BLOCK_GROUP_RAID5) {
- factor = num_stripes - 1;
- } else if (btrfs_chunk_type(leaf, chunk) & BTRFS_BLOCK_GROUP_RAID6) {
- factor = num_stripes - 2;
- } else {
- factor = num_stripes;
- }
+ type = btrfs_chunk_type(leaf, chunk);
+ factor = calc_data_stripes(type, num_stripes);
for (i = 0; i < num_stripes; i++) {
stripe = btrfs_stripe_nr(chunk, i);
bp += ret; \
} while (0)
- if (flags & BTRFS_BALANCE_ARGS_CONVERT) {
- int index = btrfs_bg_flags_to_raid_index(bargs->target);
-
- CHECK_APPEND_1ARG("convert=%s,", get_raid_name(index));
- }
+ if (flags & BTRFS_BALANCE_ARGS_CONVERT)
+ CHECK_APPEND_1ARG("convert=%s,",
+ btrfs_bg_type_to_raid_name(bargs->target));
if (flags & BTRFS_BALANCE_ARGS_SOFT)
CHECK_APPEND_NOARG("soft,");
u64 num_devices;
unsigned seq;
bool reducing_integrity;
+ int i;
if (btrfs_fs_closing(fs_info) ||
atomic_read(&fs_info->balance_pause_req) ||
}
num_devices = btrfs_num_devices(fs_info);
+ allowed = 0;
+ for (i = 0; i < ARRAY_SIZE(btrfs_raid_array); i++)
+ if (num_devices >= btrfs_raid_array[i].devs_min)
+ allowed |= btrfs_raid_array[i].bg_flag;
- allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE | BTRFS_BLOCK_GROUP_DUP;
- if (num_devices > 1)
- allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1);
- if (num_devices > 2)
- allowed |= BTRFS_BLOCK_GROUP_RAID5;
- if (num_devices > 3)
- allowed |= (BTRFS_BLOCK_GROUP_RAID10 |
- BTRFS_BLOCK_GROUP_RAID6);
if (validate_convert_profile(&bctl->data, allowed)) {
- int index = btrfs_bg_flags_to_raid_index(bctl->data.target);
-
btrfs_err(fs_info,
"balance: invalid convert data profile %s",
- get_raid_name(index));
+ btrfs_bg_type_to_raid_name(bctl->data.target));
ret = -EINVAL;
goto out;
}
if (validate_convert_profile(&bctl->meta, allowed)) {
- int index = btrfs_bg_flags_to_raid_index(bctl->meta.target);
-
btrfs_err(fs_info,
"balance: invalid convert metadata profile %s",
- get_raid_name(index));
+ btrfs_bg_type_to_raid_name(bctl->meta.target));
ret = -EINVAL;
goto out;
}
if (validate_convert_profile(&bctl->sys, allowed)) {
- int index = btrfs_bg_flags_to_raid_index(bctl->sys.target);
-
btrfs_err(fs_info,
"balance: invalid convert system profile %s",
- get_raid_name(index));
+ btrfs_bg_type_to_raid_name(bctl->sys.target));
ret = -EINVAL;
goto out;
}
- /* allow to reduce meta or sys integrity only if force set */
- allowed = BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1 |
- BTRFS_BLOCK_GROUP_RAID10 |
- BTRFS_BLOCK_GROUP_RAID5 |
- BTRFS_BLOCK_GROUP_RAID6;
+ /*
+ * Allow to reduce metadata or system integrity only if force set for
+ * profiles with redundancy (copies, parity)
+ */
+ allowed = 0;
+ for (i = 0; i < ARRAY_SIZE(btrfs_raid_array); i++) {
+ if (btrfs_raid_array[i].ncopies >= 2 ||
+ btrfs_raid_array[i].tolerated_failures >= 1)
+ allowed |= btrfs_raid_array[i].bg_flag;
+ }
do {
seq = read_seqbegin(&fs_info->profiles_lock);
if (btrfs_get_num_tolerated_disk_barrier_failures(meta_target) <
btrfs_get_num_tolerated_disk_barrier_failures(data_target)) {
- int meta_index = btrfs_bg_flags_to_raid_index(meta_target);
- int data_index = btrfs_bg_flags_to_raid_index(data_target);
-
btrfs_warn(fs_info,
"balance: metadata profile %s has lower redundancy than data profile %s",
- get_raid_name(meta_index), get_raid_name(data_index));
+ btrfs_bg_type_to_raid_name(meta_target),
+ btrfs_bg_type_to_raid_name(data_target));
+ }
+
+ if (fs_info->send_in_progress) {
+ btrfs_warn_rl(fs_info,
+"cannot run balance while send operations are in progress (%d in progress)",
+ fs_info->send_in_progress);
+ ret = -EAGAIN;
+ goto out;
}
ret = insert_balance_item(fs_info, bctl);
sub_stripes = btrfs_raid_array[index].sub_stripes;
dev_stripes = btrfs_raid_array[index].dev_stripes;
devs_max = btrfs_raid_array[index].devs_max;
+ if (!devs_max)
+ devs_max = BTRFS_MAX_DEVS(info);
devs_min = btrfs_raid_array[index].devs_min;
devs_increment = btrfs_raid_array[index].devs_increment;
ncopies = btrfs_raid_array[index].ncopies;
if (type & BTRFS_BLOCK_GROUP_DATA) {
max_stripe_size = SZ_1G;
max_chunk_size = BTRFS_MAX_DATA_CHUNK_SIZE;
- if (!devs_max)
- devs_max = BTRFS_MAX_DEVS(info);
} else if (type & BTRFS_BLOCK_GROUP_METADATA) {
/* for larger filesystems, use larger metadata chunks */
if (fs_devices->total_rw_bytes > 50ULL * SZ_1G)
else
max_stripe_size = SZ_256M;
max_chunk_size = max_stripe_size;
- if (!devs_max)
- devs_max = BTRFS_MAX_DEVS(info);
} else if (type & BTRFS_BLOCK_GROUP_SYSTEM) {
max_stripe_size = SZ_32M;
max_chunk_size = 2 * max_stripe_size;
- if (!devs_max)
- devs_max = BTRFS_MAX_DEVS_SYS_CHUNK;
} else {
btrfs_err(info, "invalid chunk type 0x%llx requested",
type);
em->block_len = em->len;
em->orig_block_len = stripe_size;
- em_tree = &info->mapping_tree.map_tree;
+ em_tree = &info->mapping_tree;
write_lock(&em_tree->lock);
ret = add_extent_mapping(em_tree, em, 0);
if (ret) {
static inline int btrfs_chunk_max_errors(struct map_lookup *map)
{
- int max_errors;
+ const int index = btrfs_bg_flags_to_raid_index(map->type);
- if (map->type & (BTRFS_BLOCK_GROUP_RAID1 |
- BTRFS_BLOCK_GROUP_RAID10 |
- BTRFS_BLOCK_GROUP_RAID5 |
- BTRFS_BLOCK_GROUP_DUP)) {
- max_errors = 1;
- } else if (map->type & BTRFS_BLOCK_GROUP_RAID6) {
- max_errors = 2;
- } else {
- max_errors = 0;
- }
-
- return max_errors;
+ return btrfs_raid_array[index].tolerated_failures;
}
int btrfs_chunk_readonly(struct btrfs_fs_info *fs_info, u64 chunk_offset)
return readonly;
}
-void btrfs_mapping_init(struct btrfs_mapping_tree *tree)
-{
- extent_map_tree_init(&tree->map_tree);
-}
-
-void btrfs_mapping_tree_free(struct btrfs_mapping_tree *tree)
+void btrfs_mapping_tree_free(struct extent_map_tree *tree)
{
struct extent_map *em;
while (1) {
- write_lock(&tree->map_tree.lock);
- em = lookup_extent_mapping(&tree->map_tree, 0, (u64)-1);
+ write_lock(&tree->lock);
+ em = lookup_extent_mapping(tree, 0, (u64)-1);
if (em)
- remove_extent_mapping(&tree->map_tree, em);
- write_unlock(&tree->map_tree.lock);
+ remove_extent_mapping(tree, em);
+ write_unlock(&tree->lock);
if (!em)
break;
/* once for us */
return 1;
map = em->map_lookup;
- if (map->type & (BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1))
+ if (map->type & (BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1_MASK))
ret = map->num_stripes;
else if (map->type & BTRFS_BLOCK_GROUP_RAID10)
ret = map->sub_stripes;
struct btrfs_device *srcdev;
ASSERT((map->type &
- (BTRFS_BLOCK_GROUP_RAID1 | BTRFS_BLOCK_GROUP_RAID10)));
+ (BTRFS_BLOCK_GROUP_RAID1_MASK | BTRFS_BLOCK_GROUP_RAID10)));
if (map->type & BTRFS_BLOCK_GROUP_RAID10)
num_stripes = map->sub_stripes;
&remaining_stripes);
div_u64_rem(stripe_nr_end - 1, factor, &last_stripe);
last_stripe *= sub_stripes;
- } else if (map->type & (BTRFS_BLOCK_GROUP_RAID1 |
+ } else if (map->type & (BTRFS_BLOCK_GROUP_RAID1_MASK |
BTRFS_BLOCK_GROUP_DUP)) {
num_stripes = map->num_stripes;
} else {
return (op == BTRFS_MAP_WRITE || op == BTRFS_MAP_GET_READ_MIRRORS);
}
+/*
+ * btrfs_get_io_geometry - calculates the geomery of a particular (address, len)
+ * tuple. This information is used to calculate how big a
+ * particular bio can get before it straddles a stripe.
+ *
+ * @fs_info - the filesystem
+ * @logical - address that we want to figure out the geometry of
+ * @len - the length of IO we are going to perform, starting at @logical
+ * @op - type of operation - write or read
+ * @io_geom - pointer used to return values
+ *
+ * Returns < 0 in case a chunk for the given logical address cannot be found,
+ * usually shouldn't happen unless @logical is corrupted, 0 otherwise.
+ */
+int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
+ u64 logical, u64 len, struct btrfs_io_geometry *io_geom)
+{
+ struct extent_map *em;
+ struct map_lookup *map;
+ u64 offset;
+ u64 stripe_offset;
+ u64 stripe_nr;
+ u64 stripe_len;
+ u64 raid56_full_stripe_start = (u64)-1;
+ int data_stripes;
+
+ ASSERT(op != BTRFS_MAP_DISCARD);
+
+ em = btrfs_get_chunk_map(fs_info, logical, len);
+ if (IS_ERR(em))
+ return PTR_ERR(em);
+
+ map = em->map_lookup;
+ /* Offset of this logical address in the chunk */
+ offset = logical - em->start;
+ /* Len of a stripe in a chunk */
+ stripe_len = map->stripe_len;
+ /* Stripe wher this block falls in */
+ stripe_nr = div64_u64(offset, stripe_len);
+ /* Offset of stripe in the chunk */
+ stripe_offset = stripe_nr * stripe_len;
+ if (offset < stripe_offset) {
+ btrfs_crit(fs_info,
+"stripe math has gone wrong, stripe_offset=%llu offset=%llu start=%llu logical=%llu stripe_len=%llu",
+ stripe_offset, offset, em->start, logical, stripe_len);
+ free_extent_map(em);
+ return -EINVAL;
+ }
+
+ /* stripe_offset is the offset of this block in its stripe */
+ stripe_offset = offset - stripe_offset;
+ data_stripes = nr_data_stripes(map);
+
+ if (map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK) {
+ u64 max_len = stripe_len - stripe_offset;
+
+ /*
+ * In case of raid56, we need to know the stripe aligned start
+ */
+ if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
+ unsigned long full_stripe_len = stripe_len * data_stripes;
+ raid56_full_stripe_start = offset;
+
+ /*
+ * Allow a write of a full stripe, but make sure we
+ * don't allow straddling of stripes
+ */
+ raid56_full_stripe_start = div64_u64(raid56_full_stripe_start,
+ full_stripe_len);
+ raid56_full_stripe_start *= full_stripe_len;
+
+ /*
+ * For writes to RAID[56], allow a full stripeset across
+ * all disks. For other RAID types and for RAID[56]
+ * reads, just allow a single stripe (on a single disk).
+ */
+ if (op == BTRFS_MAP_WRITE) {
+ max_len = stripe_len * data_stripes -
+ (offset - raid56_full_stripe_start);
+ }
+ }
+ len = min_t(u64, em->len - offset, max_len);
+ } else {
+ len = em->len - offset;
+ }
+
+ io_geom->len = len;
+ io_geom->offset = offset;
+ io_geom->stripe_len = stripe_len;
+ io_geom->stripe_nr = stripe_nr;
+ io_geom->stripe_offset = stripe_offset;
+ io_geom->raid56_stripe_offset = raid56_full_stripe_start;
+
+ return 0;
+}
+
static int __btrfs_map_block(struct btrfs_fs_info *fs_info,
enum btrfs_map_op op,
u64 logical, u64 *length,
u64 stripe_nr;
u64 stripe_len;
u32 stripe_index;
+ int data_stripes;
int i;
int ret = 0;
int num_stripes;
int patch_the_first_stripe_for_dev_replace = 0;
u64 physical_to_patch_in_first_stripe = 0;
u64 raid56_full_stripe_start = (u64)-1;
+ struct btrfs_io_geometry geom;
+
+ ASSERT(bbio_ret);
if (op == BTRFS_MAP_DISCARD)
return __btrfs_map_block_for_discard(fs_info, logical,
*length, bbio_ret);
- em = btrfs_get_chunk_map(fs_info, logical, *length);
- if (IS_ERR(em))
- return PTR_ERR(em);
+ ret = btrfs_get_io_geometry(fs_info, op, logical, *length, &geom);
+ if (ret < 0)
+ return ret;
+ em = btrfs_get_chunk_map(fs_info, logical, *length);
+ ASSERT(em);
map = em->map_lookup;
- offset = logical - em->start;
-
- stripe_len = map->stripe_len;
- stripe_nr = offset;
- /*
- * stripe_nr counts the total number of stripes we have to stride
- * to get to this block
- */
- stripe_nr = div64_u64(stripe_nr, stripe_len);
-
- stripe_offset = stripe_nr * stripe_len;
- if (offset < stripe_offset) {
- btrfs_crit(fs_info,
- "stripe math has gone wrong, stripe_offset=%llu, offset=%llu, start=%llu, logical=%llu, stripe_len=%llu",
- stripe_offset, offset, em->start, logical,
- stripe_len);
- free_extent_map(em);
- return -EINVAL;
- }
-
- /* stripe_offset is the offset of this block in its stripe*/
- stripe_offset = offset - stripe_offset;
-
- /* if we're here for raid56, we need to know the stripe aligned start */
- if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
- unsigned long full_stripe_len = stripe_len * nr_data_stripes(map);
- raid56_full_stripe_start = offset;
- /* allow a write of a full stripe, but make sure we don't
- * allow straddling of stripes
- */
- raid56_full_stripe_start = div64_u64(raid56_full_stripe_start,
- full_stripe_len);
- raid56_full_stripe_start *= full_stripe_len;
- }
-
- if (map->type & BTRFS_BLOCK_GROUP_PROFILE_MASK) {
- u64 max_len;
- /* For writes to RAID[56], allow a full stripeset across all disks.
- For other RAID types and for RAID[56] reads, just allow a single
- stripe (on a single disk). */
- if ((map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) &&
- (op == BTRFS_MAP_WRITE)) {
- max_len = stripe_len * nr_data_stripes(map) -
- (offset - raid56_full_stripe_start);
- } else {
- /* we limit the length of each bio to what fits in a stripe */
- max_len = stripe_len - stripe_offset;
- }
- *length = min_t(u64, em->len - offset, max_len);
- } else {
- *length = em->len - offset;
- }
-
- /*
- * This is for when we're called from btrfs_bio_fits_in_stripe and all
- * it cares about is the length
- */
- if (!bbio_ret)
- goto out;
+ *length = geom.len;
+ offset = geom.offset;
+ stripe_len = geom.stripe_len;
+ stripe_nr = geom.stripe_nr;
+ stripe_offset = geom.stripe_offset;
+ raid56_full_stripe_start = geom.raid56_stripe_offset;
+ data_stripes = nr_data_stripes(map);
down_read(&dev_replace->rwsem);
dev_replace_is_ongoing = btrfs_dev_replace_is_ongoing(dev_replace);
&stripe_index);
if (!need_full_stripe(op))
mirror_num = 1;
- } else if (map->type & BTRFS_BLOCK_GROUP_RAID1) {
+ } else if (map->type & BTRFS_BLOCK_GROUP_RAID1_MASK) {
if (need_full_stripe(op))
num_stripes = map->num_stripes;
else if (mirror_num)
if (need_raid_map && (need_full_stripe(op) || mirror_num > 1)) {
/* push stripe_nr back to the start of the full stripe */
stripe_nr = div64_u64(raid56_full_stripe_start,
- stripe_len * nr_data_stripes(map));
+ stripe_len * data_stripes);
/* RAID[56] write or recovery. Return all stripes */
num_stripes = map->num_stripes;
* Mirror #3 is RAID6 Q block.
*/
stripe_nr = div_u64_rem(stripe_nr,
- nr_data_stripes(map), &stripe_index);
+ data_stripes, &stripe_index);
if (mirror_num > 1)
- stripe_index = nr_data_stripes(map) +
- mirror_num - 2;
+ stripe_index = data_stripes + mirror_num - 2;
/* We distribute the parity blocks across stripes */
div_u64_rem(stripe_nr + stripe_index, map->num_stripes,
div_u64_rem(stripe_nr, num_stripes, &rot);
/* Fill in the logical address of each stripe */
- tmp = stripe_nr * nr_data_stripes(map);
- for (i = 0; i < nr_data_stripes(map); i++)
+ tmp = stripe_nr * data_stripes;
+ for (i = 0; i < data_stripes; i++)
bbio->raid_map[(i+rot) % num_stripes] =
em->start + (tmp + i) * map->stripe_len;
struct btrfs_chunk *chunk)
{
struct btrfs_fs_info *fs_info = leaf->fs_info;
- struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree;
+ struct extent_map_tree *map_tree = &fs_info->mapping_tree;
struct map_lookup *map;
struct extent_map *em;
u64 logical;
return ret;
}
- read_lock(&map_tree->map_tree.lock);
- em = lookup_extent_mapping(&map_tree->map_tree, logical, 1);
- read_unlock(&map_tree->map_tree.lock);
+ read_lock(&map_tree->lock);
+ em = lookup_extent_mapping(map_tree, logical, 1);
+ read_unlock(&map_tree->lock);
/* already mapped? */
if (em && em->start <= logical && em->start + em->len > logical) {
}
- write_lock(&map_tree->map_tree.lock);
- ret = add_extent_mapping(&map_tree->map_tree, em, 0);
- write_unlock(&map_tree->map_tree.lock);
+ write_lock(&map_tree->lock);
+ ret = add_extent_mapping(map_tree, em, 0);
+ write_unlock(&map_tree->lock);
if (ret < 0) {
btrfs_err(fs_info,
"failed to add chunk map, start=%llu len=%llu: %d",
bool btrfs_check_rw_degradable(struct btrfs_fs_info *fs_info,
struct btrfs_device *failing_dev)
{
- struct btrfs_mapping_tree *map_tree = &fs_info->mapping_tree;
+ struct extent_map_tree *map_tree = &fs_info->mapping_tree;
struct extent_map *em;
u64 next_start = 0;
bool ret = true;
- read_lock(&map_tree->map_tree.lock);
- em = lookup_extent_mapping(&map_tree->map_tree, 0, (u64)-1);
- read_unlock(&map_tree->map_tree.lock);
+ read_lock(&map_tree->lock);
+ em = lookup_extent_mapping(map_tree, 0, (u64)-1);
+ read_unlock(&map_tree->lock);
/* No chunk at all? Return false anyway */
if (!em) {
ret = false;
next_start = extent_map_end(em);
free_extent_map(em);
- read_lock(&map_tree->map_tree.lock);
- em = lookup_extent_mapping(&map_tree->map_tree, next_start,
+ read_lock(&map_tree->lock);
+ em = lookup_extent_mapping(map_tree, next_start,
(u64)(-1) - next_start);
- read_unlock(&map_tree->map_tree.lock);
+ read_unlock(&map_tree->lock);
}
out:
return ret;
*/
int btrfs_bg_type_to_factor(u64 flags)
{
- if (flags & (BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1 |
- BTRFS_BLOCK_GROUP_RAID10))
- return 2;
- return 1;
+ const int index = btrfs_bg_flags_to_raid_index(flags);
+
+ return btrfs_raid_array[index].ncopies;
}
u64 chunk_offset, u64 devid,
u64 physical_offset, u64 physical_len)
{
- struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
+ struct extent_map_tree *em_tree = &fs_info->mapping_tree;
struct extent_map *em;
struct map_lookup *map;
struct btrfs_device *dev;
static int verify_chunk_dev_extent_mapping(struct btrfs_fs_info *fs_info)
{
- struct extent_map_tree *em_tree = &fs_info->mapping_tree.map_tree;
+ struct extent_map_tree *em_tree = &fs_info->mapping_tree;
struct extent_map *em;
struct rb_node *node;
int ret = 0;
struct bio *tail;
};
+struct btrfs_io_geometry {
+ /* remaining bytes before crossing a stripe */
+ u64 len;
+ /* offset of logical address in chunk */
+ u64 offset;
+ /* length of single IO stripe */
+ u64 stripe_len;
+ /* number of stripe where address falls */
+ u64 stripe_nr;
+ /* offset of address in stripe */
+ u64 stripe_offset;
+ /* offset of raid56 stripe into the chunk */
+ u64 raid56_stripe_offset;
+};
+
/*
* Use sequence counter to get consistent device stat data on
* 32-bit processors.
#define BTRFS_DEV_STATE_FLUSH_SENT (4)
struct btrfs_device {
- struct list_head dev_list;
- struct list_head dev_alloc_list;
+ struct list_head dev_list; /* device_list_mutex */
+ struct list_head dev_alloc_list; /* chunk mutex */
struct list_head post_commit_list; /* chunk mutex */
struct btrfs_fs_devices *fs_devices;
struct btrfs_fs_info *fs_info;
* this mutex lock.
*/
struct mutex device_list_mutex;
+
+ /* List of all devices, protected by device_list_mutex */
struct list_head devices;
- /* devices not currently being allocated */
+ /*
+ * Devices which can satisfy space allocation. Protected by
+ * chunk_mutex
+ */
struct list_head alloc_list;
struct btrfs_fs_devices *seed;
};
struct btrfs_raid_attr {
- int sub_stripes; /* sub_stripes info for map */
- int dev_stripes; /* stripes per dev */
- int devs_max; /* max devs to use */
- int devs_min; /* min devs needed */
- int tolerated_failures; /* max tolerated fail devs */
- int devs_increment; /* ndevs has to be a multiple of this */
- int ncopies; /* how many copies to data has */
- int nparity; /* number of stripes worth of bytes to store
+ u8 sub_stripes; /* sub_stripes info for map */
+ u8 dev_stripes; /* stripes per dev */
+ u8 devs_max; /* max devs to use */
+ u8 devs_min; /* min devs needed */
+ u8 tolerated_failures; /* max tolerated fail devs */
+ u8 devs_increment; /* ndevs has to be a multiple of this */
+ u8 ncopies; /* how many copies to data has */
+ u8 nparity; /* number of stripes worth of bytes to store
* parity information */
- int mindev_error; /* error code if min devs requisite is unmet */
+ u8 mindev_error; /* error code if min devs requisite is unmet */
const char raid_name[8]; /* name of the raid */
u64 bg_flag; /* block group flag of the raid */
};
int btrfs_map_sblock(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
u64 logical, u64 *length,
struct btrfs_bio **bbio_ret);
+int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
+ u64 logical, u64 len, struct btrfs_io_geometry *io_geom);
int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
u64 physical, u64 **logical, int *naddrs, int *stripe_len);
int btrfs_read_sys_array(struct btrfs_fs_info *fs_info);
int btrfs_read_chunk_tree(struct btrfs_fs_info *fs_info);
int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, u64 type);
-void btrfs_mapping_init(struct btrfs_mapping_tree *tree);
-void btrfs_mapping_tree_free(struct btrfs_mapping_tree *tree);
+void btrfs_mapping_tree_free(struct extent_map_tree *tree);
blk_status_t btrfs_map_bio(struct btrfs_fs_info *fs_info, struct bio *bio,
int mirror_num, int async_submit);
int btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
return BTRFS_RAID_SINGLE; /* BTRFS_BLOCK_GROUP_SINGLE */
}
-const char *get_raid_name(enum btrfs_raid_types type);
-
void btrfs_commit_device_sizes(struct btrfs_transaction *trans);
struct list_head *btrfs_get_fs_uuids(void);
struct btrfs_device *failing_dev);
int btrfs_bg_type_to_factor(u64 flags);
+const char *btrfs_bg_type_to_raid_name(u64 flags);
int btrfs_verify_dev_extents(struct btrfs_fs_info *fs_info);
#endif
groups beyond the owner/group/world scheme.
If you don't know what Access Control Lists are, say N
+
+config CEPH_FS_SECURITY_LABEL
+ bool "CephFS Security Labels"
+ depends on CEPH_FS && SECURITY
+ help
+ Security labels support alternative access control models
+ implemented by security modules like SELinux. This option
+ enables an extended attribute handler for file security
+ labels in the Ceph filesystem.
+
+ If you are not using a security module that requires using
+ extended attributes for file security labels, say N.
}
int ceph_pre_init_acls(struct inode *dir, umode_t *mode,
- struct ceph_acls_info *info)
+ struct ceph_acl_sec_ctx *as_ctx)
{
struct posix_acl *acl, *default_acl;
size_t val_size1 = 0, val_size2 = 0;
kfree(tmp_buf);
- info->acl = acl;
- info->default_acl = default_acl;
- info->pagelist = pagelist;
+ as_ctx->acl = acl;
+ as_ctx->default_acl = default_acl;
+ as_ctx->pagelist = pagelist;
return 0;
out_err:
return err;
}
-void ceph_init_inode_acls(struct inode* inode, struct ceph_acls_info *info)
+void ceph_init_inode_acls(struct inode *inode, struct ceph_acl_sec_ctx *as_ctx)
{
if (!inode)
return;
- ceph_set_cached_acl(inode, ACL_TYPE_ACCESS, info->acl);
- ceph_set_cached_acl(inode, ACL_TYPE_DEFAULT, info->default_acl);
-}
-
-void ceph_release_acls_info(struct ceph_acls_info *info)
-{
- posix_acl_release(info->acl);
- posix_acl_release(info->default_acl);
- if (info->pagelist)
- ceph_pagelist_release(info->pagelist);
+ ceph_set_cached_acl(inode, ACL_TYPE_ACCESS, as_ctx->acl);
+ ceph_set_cached_acl(inode, ACL_TYPE_DEFAULT, as_ctx->default_acl);
}
#include <linux/pagevec.h>
#include <linux/task_io_accounting_ops.h>
#include <linux/signal.h>
+#include <linux/iversion.h>
#include "super.h"
#include "mds_client.h"
/* Update time before taking page lock */
file_update_time(vma->vm_file);
+ inode_inc_iversion_raw(inode);
do {
lock_page(page);
#include <linux/vmalloc.h>
#include <linux/wait.h>
#include <linux/writeback.h>
+#include <linux/iversion.h>
#include "super.h"
#include "mds_client.h"
u64 ino, cid, follows;
u64 flush_tid, oldest_flush_tid, size, max_size;
u64 xattr_version;
+ u64 change_attr;
struct ceph_buffer *xattr_buf;
- struct timespec64 atime, mtime, ctime;
+ struct timespec64 atime, mtime, ctime, btime;
int op, caps, wanted, dirty;
u32 seq, issue_seq, mseq, time_warp_seq;
u32 flags;
struct ceph_msg *msg;
void *p;
size_t extra_len;
- struct timespec64 zerotime = {0};
struct ceph_osd_client *osdc = &arg->session->s_mdsc->fsc->client->osdc;
dout("send_cap_msg %s %llx %llx caps %s wanted %s dirty %s"
/* pool namespace (version 8) (mds always ignores this) */
ceph_encode_32(&p, 0);
- /*
- * btime and change_attr (version 9)
- *
- * We just zero these out for now, as the MDS ignores them unless
- * the requisite feature flags are set (which we don't do yet).
- */
- ceph_encode_timespec64(p, &zerotime);
+ /* btime and change_attr (version 9) */
+ ceph_encode_timespec64(p, &arg->btime);
p += sizeof(struct ceph_timespec);
- ceph_encode_64(&p, 0);
+ ceph_encode_64(&p, arg->change_attr);
/* Advisory flags (version 10) */
ceph_encode_32(&p, arg->flags);
}
/*
- * Queue cap releases when an inode is dropped from our cache. Since
- * inode is about to be destroyed, there is no need for i_ceph_lock.
+ * Queue cap releases when an inode is dropped from our cache.
*/
-void __ceph_remove_caps(struct inode *inode)
+void __ceph_remove_caps(struct ceph_inode_info *ci)
{
- struct ceph_inode_info *ci = ceph_inode(inode);
struct rb_node *p;
+ /* lock i_ceph_lock, because ceph_d_revalidate(..., LOOKUP_RCU)
+ * may call __ceph_caps_issued_mask() on a freeing inode. */
+ spin_lock(&ci->i_ceph_lock);
p = rb_first(&ci->i_caps);
while (p) {
struct ceph_cap *cap = rb_entry(p, struct ceph_cap, ci_node);
p = rb_next(p);
__ceph_remove_cap(cap, true);
}
+ spin_unlock(&ci->i_ceph_lock);
}
/*
* caller should hold snap_rwsem (read), s_mutex.
*/
static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
- int op, bool sync, int used, int want, int retain,
+ int op, int flags, int used, int want, int retain,
int flushing, u64 flush_tid, u64 oldest_flush_tid)
__releases(cap->ci->i_ceph_lock)
{
arg.mtime = inode->i_mtime;
arg.atime = inode->i_atime;
arg.ctime = inode->i_ctime;
+ arg.btime = ci->i_btime;
+ arg.change_attr = inode_peek_iversion_raw(inode);
arg.op = op;
arg.caps = cap->implemented;
arg.mode = inode->i_mode;
arg.inline_data = ci->i_inline_version != CEPH_INLINE_NONE;
- if (list_empty(&ci->i_cap_snaps))
- arg.flags = CEPH_CLIENT_CAPS_NO_CAPSNAP;
- else
- arg.flags = CEPH_CLIENT_CAPS_PENDING_CAPSNAP;
- if (sync)
- arg.flags |= CEPH_CLIENT_CAPS_SYNC;
+ if (!(flags & CEPH_CLIENT_CAPS_PENDING_CAPSNAP) &&
+ !list_empty(&ci->i_cap_snaps)) {
+ struct ceph_cap_snap *capsnap;
+ list_for_each_entry_reverse(capsnap, &ci->i_cap_snaps, ci_item) {
+ if (capsnap->cap_flush.tid)
+ break;
+ if (capsnap->need_flush) {
+ flags |= CEPH_CLIENT_CAPS_PENDING_CAPSNAP;
+ break;
+ }
+ }
+ }
+ arg.flags = flags;
spin_unlock(&ci->i_ceph_lock);
arg.atime = capsnap->atime;
arg.mtime = capsnap->mtime;
arg.ctime = capsnap->ctime;
+ arg.btime = capsnap->btime;
+ arg.change_attr = capsnap->change_attr;
arg.op = CEPH_CAP_OP_FLUSHSNAP;
arg.caps = capsnap->issued;
}
// make sure flushsnap messages are sent in proper order.
- if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH) {
+ if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH)
__kick_flushing_caps(mdsc, session, ci, 0);
- ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
- }
__ceph_flush_snaps(ci, session);
out:
if (cap == ci->i_auth_cap &&
(ci->i_ceph_flags &
(CEPH_I_KICK_FLUSH | CEPH_I_FLUSH_SNAPS))) {
- if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH) {
+ if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH)
__kick_flushing_caps(mdsc, session, ci, 0);
- ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
- }
if (ci->i_ceph_flags & CEPH_I_FLUSH_SNAPS)
__ceph_flush_snaps(ci, session);
sent++;
/* __send_cap drops i_ceph_lock */
- delayed += __send_cap(mdsc, cap, CEPH_CAP_OP_UPDATE, false,
+ delayed += __send_cap(mdsc, cap, CEPH_CAP_OP_UPDATE, 0,
cap_used, want, retain, flushing,
flush_tid, oldest_flush_tid);
goto retry; /* retake i_ceph_lock and restart our cap scan. */
retry:
spin_lock(&ci->i_ceph_lock);
+retry_locked:
if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
spin_unlock(&ci->i_ceph_lock);
dout("try_flush_caps skipping %p I_NOFLUSH set\n", inode);
}
if (ci->i_dirty_caps && ci->i_auth_cap) {
struct ceph_cap *cap = ci->i_auth_cap;
- int used = __ceph_caps_used(ci);
- int want = __ceph_caps_wanted(ci);
int delayed;
if (!session || session != cap->session) {
goto out;
}
+ if (ci->i_ceph_flags &
+ (CEPH_I_KICK_FLUSH | CEPH_I_FLUSH_SNAPS)) {
+ if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH)
+ __kick_flushing_caps(mdsc, session, ci, 0);
+ if (ci->i_ceph_flags & CEPH_I_FLUSH_SNAPS)
+ __ceph_flush_snaps(ci, session);
+ goto retry_locked;
+ }
+
flushing = __mark_caps_flushing(inode, session, true,
&flush_tid, &oldest_flush_tid);
/* __send_cap drops i_ceph_lock */
- delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH, true,
- used, want, (cap->issued | cap->implemented),
- flushing, flush_tid, oldest_flush_tid);
+ delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH,
+ CEPH_CLIENT_CAPS_SYNC,
+ __ceph_caps_used(ci),
+ __ceph_caps_wanted(ci),
+ (cap->issued | cap->implemented),
+ flushing, flush_tid, oldest_flush_tid);
if (delayed) {
spin_lock(&ci->i_ceph_lock);
struct ceph_cap_flush *cf;
int ret;
u64 first_tid = 0;
+ u64 last_snap_flush = 0;
+
+ ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
+
+ list_for_each_entry_reverse(cf, &ci->i_cap_flush_list, i_list) {
+ if (!cf->caps) {
+ last_snap_flush = cf->tid;
+ break;
+ }
+ }
list_for_each_entry(cf, &ci->i_cap_flush_list, i_list) {
if (cf->tid < first_tid)
dout("kick_flushing_caps %p cap %p tid %llu %s\n",
inode, cap, cf->tid, ceph_cap_string(cf->caps));
ci->i_ceph_flags |= CEPH_I_NODELAY;
+
ret = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH,
- false, __ceph_caps_used(ci),
+ (cf->tid < last_snap_flush ?
+ CEPH_CLIENT_CAPS_PENDING_CAPSNAP : 0),
+ __ceph_caps_used(ci),
__ceph_caps_wanted(ci),
- cap->issued | cap->implemented,
+ (cap->issued | cap->implemented),
cf->caps, cf->tid, oldest_flush_tid);
if (ret) {
pr_err("kick_flushing_caps: error sending "
*/
if ((cap->issued & ci->i_flushing_caps) !=
ci->i_flushing_caps) {
- ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
/* encode_caps_cb() also will reset these sequence
* numbers. make sure sequence numbers in cap flush
* message match later reconnect message */
continue;
}
if (ci->i_ceph_flags & CEPH_I_KICK_FLUSH) {
- ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
__kick_flushing_caps(mdsc, session, ci,
oldest_flush_tid);
}
oldest_flush_tid = __get_oldest_flush_tid(mdsc);
spin_unlock(&mdsc->cap_dirty_lock);
- ci->i_ceph_flags &= ~CEPH_I_KICK_FLUSH;
__kick_flushing_caps(mdsc, session, ci, oldest_flush_tid);
spin_unlock(&ci->i_ceph_lock);
} else {
bool dirstat_valid;
u64 nfiles;
u64 nsubdirs;
+ u64 change_attr;
/* currently issued */
int issued;
+ struct timespec64 btime;
};
/*
__check_cap_issue(ci, cap, newcaps);
+ inode_set_max_iversion_raw(inode, extra_info->change_attr);
+
if ((newcaps & CEPH_CAP_AUTH_SHARED) &&
(extra_info->issued & CEPH_CAP_AUTH_EXCL) == 0) {
inode->i_mode = le32_to_cpu(grant->mode);
inode->i_uid = make_kuid(&init_user_ns, le32_to_cpu(grant->uid));
inode->i_gid = make_kgid(&init_user_ns, le32_to_cpu(grant->gid));
+ ci->i_btime = extra_info->btime;
dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
from_kuid(&init_user_ns, inode->i_uid),
from_kgid(&init_user_ns, inode->i_gid));
ci->i_xattrs.blob = ceph_buffer_get(xattr_buf);
ci->i_xattrs.version = version;
ceph_forget_all_cached_acls(inode);
+ ceph_security_invalidate_secctx(inode);
}
}
}
}
- if (msg_version >= 11) {
+ if (msg_version >= 9) {
struct ceph_timespec *btime;
- u64 change_attr;
- u32 flags;
- /* version >= 9 */
if (p + sizeof(*btime) > end)
goto bad;
btime = p;
+ ceph_decode_timespec64(&extra_info.btime, btime);
p += sizeof(*btime);
- ceph_decode_64_safe(&p, end, change_attr, bad);
+ ceph_decode_64_safe(&p, end, extra_info.change_attr, bad);
+ }
+
+ if (msg_version >= 11) {
+ u32 flags;
/* version >= 10 */
ceph_decode_32_safe(&p, end, flags, bad);
/* version >= 11 */
struct ceph_mds_client *mdsc = fsc->mdsc;
struct ceph_mds_request *req;
struct rb_node *rp;
- int pathlen;
+ int pathlen = 0;
u64 pathbase;
char *path;
struct ceph_fs_client *fsc = ceph_sb_to_client(dir->i_sb);
struct ceph_mds_client *mdsc = fsc->mdsc;
struct ceph_mds_request *req;
- struct ceph_acls_info acls = {};
+ struct ceph_acl_sec_ctx as_ctx = {};
int err;
if (ceph_snap(dir) != CEPH_NOSNAP)
goto out;
}
- err = ceph_pre_init_acls(dir, &mode, &acls);
+ err = ceph_pre_init_acls(dir, &mode, &as_ctx);
+ if (err < 0)
+ goto out;
+ err = ceph_security_init_secctx(dentry, mode, &as_ctx);
if (err < 0)
goto out;
req->r_args.mknod.rdev = cpu_to_le32(rdev);
req->r_dentry_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_AUTH_EXCL;
req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
- if (acls.pagelist) {
- req->r_pagelist = acls.pagelist;
- acls.pagelist = NULL;
+ if (as_ctx.pagelist) {
+ req->r_pagelist = as_ctx.pagelist;
+ as_ctx.pagelist = NULL;
}
err = ceph_mdsc_do_request(mdsc, dir, req);
if (!err && !req->r_reply_info.head->is_dentry)
ceph_mdsc_put_request(req);
out:
if (!err)
- ceph_init_inode_acls(d_inode(dentry), &acls);
+ ceph_init_inode_acls(d_inode(dentry), &as_ctx);
else
d_drop(dentry);
- ceph_release_acls_info(&acls);
+ ceph_release_acl_sec_ctx(&as_ctx);
return err;
}
struct ceph_fs_client *fsc = ceph_sb_to_client(dir->i_sb);
struct ceph_mds_client *mdsc = fsc->mdsc;
struct ceph_mds_request *req;
+ struct ceph_acl_sec_ctx as_ctx = {};
int err;
if (ceph_snap(dir) != CEPH_NOSNAP)
goto out;
}
+ err = ceph_security_init_secctx(dentry, S_IFLNK | 0777, &as_ctx);
+ if (err < 0)
+ goto out;
+
dout("symlink in dir %p dentry %p to '%s'\n", dir, dentry, dest);
req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SYMLINK, USE_AUTH_MDS);
if (IS_ERR(req)) {
out:
if (err)
d_drop(dentry);
+ ceph_release_acl_sec_ctx(&as_ctx);
return err;
}
struct ceph_fs_client *fsc = ceph_sb_to_client(dir->i_sb);
struct ceph_mds_client *mdsc = fsc->mdsc;
struct ceph_mds_request *req;
- struct ceph_acls_info acls = {};
+ struct ceph_acl_sec_ctx as_ctx = {};
int err = -EROFS;
int op;
}
mode |= S_IFDIR;
- err = ceph_pre_init_acls(dir, &mode, &acls);
+ err = ceph_pre_init_acls(dir, &mode, &as_ctx);
+ if (err < 0)
+ goto out;
+ err = ceph_security_init_secctx(dentry, mode, &as_ctx);
if (err < 0)
goto out;
req->r_args.mkdir.mode = cpu_to_le32(mode);
req->r_dentry_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_AUTH_EXCL;
req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
- if (acls.pagelist) {
- req->r_pagelist = acls.pagelist;
- acls.pagelist = NULL;
+ if (as_ctx.pagelist) {
+ req->r_pagelist = as_ctx.pagelist;
+ as_ctx.pagelist = NULL;
}
err = ceph_mdsc_do_request(mdsc, dir, req);
if (!err &&
ceph_mdsc_put_request(req);
out:
if (!err)
- ceph_init_inode_acls(d_inode(dentry), &acls);
+ ceph_init_inode_acls(d_inode(dentry), &as_ctx);
else
d_drop(dentry);
- ceph_release_acls_info(&acls);
+ ceph_release_acl_sec_ctx(&as_ctx);
return err;
}
return false;
}
-static int dentry_lease_is_valid(struct dentry *dentry, unsigned int flags,
- struct inode *dir)
+static int dentry_lease_is_valid(struct dentry *dentry, unsigned int flags)
{
struct ceph_dentry_info *di;
struct ceph_mds_session *session = NULL;
spin_unlock(&dentry->d_lock);
if (session) {
- ceph_mdsc_lease_send_msg(session, dir, dentry,
+ ceph_mdsc_lease_send_msg(session, dentry,
CEPH_MDS_LEASE_RENEW, seq);
ceph_put_mds_session(session);
}
static int dir_lease_is_valid(struct inode *dir, struct dentry *dentry)
{
struct ceph_inode_info *ci = ceph_inode(dir);
- struct ceph_dentry_info *di = ceph_dentry(dentry);
- int valid = 0;
+ int valid;
+ int shared_gen;
spin_lock(&ci->i_ceph_lock);
- if (atomic_read(&ci->i_shared_gen) == di->lease_shared_gen)
- valid = __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1);
+ valid = __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1);
+ shared_gen = atomic_read(&ci->i_shared_gen);
spin_unlock(&ci->i_ceph_lock);
- if (valid)
- __ceph_dentry_dir_lease_touch(di);
- dout("dir_lease_is_valid dir %p v%u dentry %p v%u = %d\n",
- dir, (unsigned)atomic_read(&ci->i_shared_gen),
- dentry, (unsigned)di->lease_shared_gen, valid);
+ if (valid) {
+ struct ceph_dentry_info *di;
+ spin_lock(&dentry->d_lock);
+ di = ceph_dentry(dentry);
+ if (dir == d_inode(dentry->d_parent) &&
+ di && di->lease_shared_gen == shared_gen)
+ __ceph_dentry_dir_lease_touch(di);
+ else
+ valid = 0;
+ spin_unlock(&dentry->d_lock);
+ }
+ dout("dir_lease_is_valid dir %p v%u dentry %p = %d\n",
+ dir, (unsigned)atomic_read(&ci->i_shared_gen), dentry, valid);
return valid;
}
ceph_snap(d_inode(dentry)) == CEPH_SNAPDIR) {
valid = 1;
} else {
- valid = dentry_lease_is_valid(dentry, flags, dir);
+ valid = dentry_lease_is_valid(dentry, flags);
if (valid == -ECHILD)
return valid;
if (valid || dir_lease_is_valid(dir, dentry)) {
}
out:
dout("get_parent %p ino %llx.%llx err=%ld\n",
- child, ceph_vinop(inode), (IS_ERR(dn) ? PTR_ERR(dn) : 0));
+ child, ceph_vinop(inode), (long)PTR_ERR_OR_ZERO(dn));
return dn;
}
#include <linux/namei.h>
#include <linux/writeback.h>
#include <linux/falloc.h>
+#include <linux/iversion.h>
#include "super.h"
#include "mds_client.h"
struct ceph_mds_client *mdsc = fsc->mdsc;
struct ceph_mds_request *req;
struct dentry *dn;
- struct ceph_acls_info acls = {};
+ struct ceph_acl_sec_ctx as_ctx = {};
int mask;
int err;
if (flags & O_CREAT) {
if (ceph_quota_is_max_files_exceeded(dir))
return -EDQUOT;
- err = ceph_pre_init_acls(dir, &mode, &acls);
+ err = ceph_pre_init_acls(dir, &mode, &as_ctx);
if (err < 0)
return err;
+ err = ceph_security_init_secctx(dentry, mode, &as_ctx);
+ if (err < 0)
+ goto out_ctx;
}
/* do the open */
req = prepare_open_request(dir->i_sb, flags, mode);
if (IS_ERR(req)) {
err = PTR_ERR(req);
- goto out_acl;
+ goto out_ctx;
}
req->r_dentry = dget(dentry);
req->r_num_caps = 2;
if (flags & O_CREAT) {
req->r_dentry_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_AUTH_EXCL;
req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
- if (acls.pagelist) {
- req->r_pagelist = acls.pagelist;
- acls.pagelist = NULL;
+ if (as_ctx.pagelist) {
+ req->r_pagelist = as_ctx.pagelist;
+ as_ctx.pagelist = NULL;
}
}
} else {
dout("atomic_open finish_open on dn %p\n", dn);
if (req->r_op == CEPH_MDS_OP_CREATE && req->r_reply_info.has_create_ino) {
- ceph_init_inode_acls(d_inode(dentry), &acls);
+ ceph_init_inode_acls(d_inode(dentry), &as_ctx);
file->f_mode |= FMODE_CREATED;
}
err = finish_open(file, dentry, ceph_open);
if (!req->r_err && req->r_target_inode)
ceph_put_fmode(ceph_inode(req->r_target_inode), req->r_fmode);
ceph_mdsc_put_request(req);
-out_acl:
- ceph_release_acls_info(&acls);
+out_ctx:
+ ceph_release_acl_sec_ctx(&as_ctx);
dout("atomic_open result=%d\n", err);
return err;
}
* may block.
*/
truncate_inode_pages_range(inode->i_mapping, pos,
- (pos+len) | (PAGE_SIZE - 1));
+ PAGE_ALIGN(pos + len) - 1);
req->r_mtime = mtime;
}
req->r_callback = ceph_aio_complete_req;
req->r_inode = inode;
req->r_priv = aio_req;
- list_add_tail(&req->r_unsafe_item, &aio_req->osd_reqs);
+ list_add_tail(&req->r_private_item, &aio_req->osd_reqs);
pos += len;
continue;
while (!list_empty(&osd_reqs)) {
req = list_first_entry(&osd_reqs,
struct ceph_osd_request,
- r_unsafe_item);
- list_del_init(&req->r_unsafe_item);
+ r_private_item);
+ list_del_init(&req->r_private_item);
if (ret >= 0)
ret = ceph_osdc_start_request(req->r_osdc,
req, false);
if (err)
goto out;
+ inode_inc_iversion_raw(inode);
+
if (ci->i_inline_version != CEPH_INLINE_NONE) {
err = ceph_uninline_data(file, NULL);
if (err < 0)
do_final_copy = true;
file_update_time(dst_file);
+ inode_inc_iversion_raw(dst_inode);
+
if (endoff > size) {
int caps_flags = 0;
#include <linux/posix_acl.h>
#include <linux/random.h>
#include <linux/sort.h>
+#include <linux/iversion.h>
#include "super.h"
#include "mds_client.h"
{
ceph_inode(inode)->i_vino = *(struct ceph_vino *)data;
inode->i_ino = ceph_vino_to_ino(*(struct ceph_vino *)data);
+ inode_set_iversion_raw(inode, 0);
return 0;
}
INIT_WORK(&ci->i_work, ceph_inode_work);
ci->i_work_mask = 0;
+ memset(&ci->i_btime, '\0', sizeof(ci->i_btime));
ceph_fscache_inode_init(ci);
kmem_cache_free(ceph_inode_cachep, ci);
}
-void ceph_destroy_inode(struct inode *inode)
+void ceph_evict_inode(struct inode *inode)
{
struct ceph_inode_info *ci = ceph_inode(inode);
struct ceph_inode_frag *frag;
struct rb_node *n;
- dout("destroy_inode %p ino %llx.%llx\n", inode, ceph_vinop(inode));
+ dout("evict_inode %p ino %llx.%llx\n", inode, ceph_vinop(inode));
+
+ truncate_inode_pages_final(&inode->i_data);
+ clear_inode(inode);
ceph_fscache_unregister_inode_cookie(ci);
- __ceph_remove_caps(inode);
+ __ceph_remove_caps(ci);
if (__ceph_has_any_quota(ci))
ceph_adjust_quota_realms_count(inode, false);
ceph_put_string(rcu_dereference_raw(ci->i_layout.pool_ns));
}
-int ceph_drop_inode(struct inode *inode)
-{
- /*
- * Positve dentry and corresponding inode are always accompanied
- * in MDS reply. So no need to keep inode in the cache after
- * dropping all its aliases.
- */
- return 1;
-}
-
static inline blkcnt_t calc_inode_blocks(u64 size)
{
return (size + (1<<9) - 1) >> 9;
le64_to_cpu(info->version) > (ci->i_version & ~1)))
new_version = true;
+ /* Update change_attribute */
+ inode_set_max_iversion_raw(inode, iinfo->change_attr);
+
__ceph_caps_issued(ci, &issued);
issued |= __ceph_caps_dirty(ci);
new_issued = ~issued & info_caps;
dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
from_kuid(&init_user_ns, inode->i_uid),
from_kgid(&init_user_ns, inode->i_gid));
+ ceph_decode_timespec64(&ci->i_btime, &iinfo->btime);
+ ceph_decode_timespec64(&ci->i_snap_btime, &iinfo->snap_btime);
}
if ((new_version || (new_issued & CEPH_CAP_LINK_SHARED)) &&
iinfo->xattr_data, iinfo->xattr_len);
ci->i_xattrs.version = le64_to_cpu(info->xattr_version);
ceph_forget_all_cached_acls(inode);
+ ceph_security_invalidate_secctx(inode);
xattr_blob = NULL;
}
}
/*
- * caller should hold session s_mutex.
+ * caller should hold session s_mutex and dentry->d_lock.
*/
-static void update_dentry_lease(struct dentry *dentry,
- struct ceph_mds_reply_lease *lease,
- struct ceph_mds_session *session,
- unsigned long from_time,
- struct ceph_vino *tgt_vino,
- struct ceph_vino *dir_vino)
+static void __update_dentry_lease(struct inode *dir, struct dentry *dentry,
+ struct ceph_mds_reply_lease *lease,
+ struct ceph_mds_session *session,
+ unsigned long from_time,
+ struct ceph_mds_session **old_lease_session)
{
struct ceph_dentry_info *di = ceph_dentry(dentry);
long unsigned duration = le32_to_cpu(lease->duration_ms);
long unsigned ttl = from_time + (duration * HZ) / 1000;
long unsigned half_ttl = from_time + (duration * HZ / 2) / 1000;
- struct inode *dir;
- struct ceph_mds_session *old_lease_session = NULL;
- /*
- * Make sure dentry's inode matches tgt_vino. NULL tgt_vino means that
- * we expect a negative dentry.
- */
- if (!tgt_vino && d_really_is_positive(dentry))
- return;
-
- if (tgt_vino && (d_really_is_negative(dentry) ||
- !ceph_ino_compare(d_inode(dentry), tgt_vino)))
- return;
-
- spin_lock(&dentry->d_lock);
dout("update_dentry_lease %p duration %lu ms ttl %lu\n",
dentry, duration, ttl);
- dir = d_inode(dentry->d_parent);
-
- /* make sure parent matches dir_vino */
- if (!ceph_ino_compare(dir, dir_vino))
- goto out_unlock;
-
/* only track leases on regular dentries */
if (ceph_snap(dir) != CEPH_NOSNAP)
- goto out_unlock;
+ return;
di->lease_shared_gen = atomic_read(&ceph_inode(dir)->i_shared_gen);
if (duration == 0) {
__ceph_dentry_dir_lease_touch(di);
- goto out_unlock;
+ return;
}
if (di->lease_gen == session->s_cap_gen &&
time_before(ttl, di->time))
- goto out_unlock; /* we already have a newer lease. */
+ return; /* we already have a newer lease. */
if (di->lease_session && di->lease_session != session) {
- old_lease_session = di->lease_session;
+ *old_lease_session = di->lease_session;
di->lease_session = NULL;
}
di->time = ttl;
__ceph_dentry_lease_touch(di);
+}
+
+static inline void update_dentry_lease(struct inode *dir, struct dentry *dentry,
+ struct ceph_mds_reply_lease *lease,
+ struct ceph_mds_session *session,
+ unsigned long from_time)
+{
+ struct ceph_mds_session *old_lease_session = NULL;
+ spin_lock(&dentry->d_lock);
+ __update_dentry_lease(dir, dentry, lease, session, from_time,
+ &old_lease_session);
+ spin_unlock(&dentry->d_lock);
+ if (old_lease_session)
+ ceph_put_mds_session(old_lease_session);
+}
+
+/*
+ * update dentry lease without having parent inode locked
+ */
+static void update_dentry_lease_careful(struct dentry *dentry,
+ struct ceph_mds_reply_lease *lease,
+ struct ceph_mds_session *session,
+ unsigned long from_time,
+ char *dname, u32 dname_len,
+ struct ceph_vino *pdvino,
+ struct ceph_vino *ptvino)
+
+{
+ struct inode *dir;
+ struct ceph_mds_session *old_lease_session = NULL;
+
+ spin_lock(&dentry->d_lock);
+ /* make sure dentry's name matches target */
+ if (dentry->d_name.len != dname_len ||
+ memcmp(dentry->d_name.name, dname, dname_len))
+ goto out_unlock;
+
+ dir = d_inode(dentry->d_parent);
+ /* make sure parent matches dvino */
+ if (!ceph_ino_compare(dir, pdvino))
+ goto out_unlock;
+
+ /* make sure dentry's inode matches target. NULL ptvino means that
+ * we expect a negative dentry */
+ if (ptvino) {
+ if (d_really_is_negative(dentry))
+ goto out_unlock;
+ if (!ceph_ino_compare(d_inode(dentry), ptvino))
+ goto out_unlock;
+ } else {
+ if (d_really_is_positive(dentry))
+ goto out_unlock;
+ }
+
+ __update_dentry_lease(dir, dentry, lease, session,
+ from_time, &old_lease_session);
out_unlock:
spin_unlock(&dentry->d_lock);
if (old_lease_session)
return 0;
}
-static int d_name_cmp(struct dentry *dentry, const char *name, size_t len)
-{
- int ret;
-
- /* take d_lock to ensure dentry->d_name stability */
- spin_lock(&dentry->d_lock);
- ret = dentry->d_name.len - len;
- if (!ret)
- ret = memcmp(dentry->d_name.name, name, len);
- spin_unlock(&dentry->d_lock);
- return ret;
-}
-
/*
* Incorporate results into the local cache. This is either just
* one inode, or a directory, dentry, and possibly linked-to inode (e.g.,
} else if (have_lease) {
if (d_unhashed(dn))
d_add(dn, NULL);
- update_dentry_lease(dn, rinfo->dlease,
- session,
- req->r_request_started,
- NULL, &dvino);
+ update_dentry_lease(dir, dn,
+ rinfo->dlease, session,
+ req->r_request_started);
}
goto done;
}
}
if (have_lease) {
- tvino.ino = le64_to_cpu(rinfo->targeti.in->ino);
- tvino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
- update_dentry_lease(dn, rinfo->dlease, session,
- req->r_request_started,
- &tvino, &dvino);
+ update_dentry_lease(dir, dn,
+ rinfo->dlease, session,
+ req->r_request_started);
}
dout(" final dn %p\n", dn);
} else if ((req->r_op == CEPH_MDS_OP_LOOKUPSNAP ||
err = splice_dentry(&req->r_dentry, in);
if (err < 0)
goto done;
- } else if (rinfo->head->is_dentry &&
- !d_name_cmp(req->r_dentry, rinfo->dname, rinfo->dname_len)) {
+ } else if (rinfo->head->is_dentry && req->r_dentry) {
+ /* parent inode is not locked, be carefull */
struct ceph_vino *ptvino = NULL;
-
- if ((le32_to_cpu(rinfo->diri.in->cap.caps) & CEPH_CAP_FILE_SHARED) ||
- le32_to_cpu(rinfo->dlease->duration_ms)) {
- dvino.ino = le64_to_cpu(rinfo->diri.in->ino);
- dvino.snap = le64_to_cpu(rinfo->diri.in->snapid);
-
- if (rinfo->head->is_target) {
- tvino.ino = le64_to_cpu(rinfo->targeti.in->ino);
- tvino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
- ptvino = &tvino;
- }
-
- update_dentry_lease(req->r_dentry, rinfo->dlease,
- session, req->r_request_started, ptvino,
- &dvino);
- } else {
- dout("%s: no dentry lease or dir cap\n", __func__);
+ dvino.ino = le64_to_cpu(rinfo->diri.in->ino);
+ dvino.snap = le64_to_cpu(rinfo->diri.in->snapid);
+ if (rinfo->head->is_target) {
+ tvino.ino = le64_to_cpu(rinfo->targeti.in->ino);
+ tvino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
+ ptvino = &tvino;
}
+ update_dentry_lease_careful(req->r_dentry, rinfo->dlease,
+ session, req->r_request_started,
+ rinfo->dname, rinfo->dname_len,
+ &dvino, ptvino);
}
done:
dout("fill_trace done err=%d\n", err);
/* FIXME: release caps/leases if error occurs */
for (i = 0; i < rinfo->dir_nr; i++) {
struct ceph_mds_reply_dir_entry *rde = rinfo->dir_entries + i;
- struct ceph_vino tvino, dvino;
+ struct ceph_vino tvino;
dname.name = rde->name;
dname.len = rde->name_len;
ceph_dentry(dn)->offset = rde->offset;
- dvino = ceph_vino(d_inode(parent));
- update_dentry_lease(dn, rde->lease, req->r_session,
- req->r_request_started, &tvino, &dvino);
+ update_dentry_lease(d_inode(parent), dn,
+ rde->lease, req->r_session,
+ req->r_request_started);
if (err == 0 && skipped == 0 && cache_ctl.index >= 0) {
ret = fill_readdir_cache(d_inode(parent), dn,
{
int mask = 0;
- if (want & (STATX_MODE|STATX_UID|STATX_GID|STATX_CTIME))
+ if (want & (STATX_MODE|STATX_UID|STATX_GID|STATX_CTIME|STATX_BTIME))
mask |= CEPH_CAP_AUTH_SHARED;
if (want & (STATX_NLINK|STATX_CTIME))
{
struct inode *inode = d_inode(path->dentry);
struct ceph_inode_info *ci = ceph_inode(inode);
+ u32 valid_mask = STATX_BASIC_STATS;
int err = 0;
/* Skip the getattr altogether if we're asked not to sync */
generic_fillattr(inode, stat);
stat->ino = ceph_translate_ino(inode->i_sb, inode->i_ino);
+
+ /*
+ * btime on newly-allocated inodes is 0, so if this is still set to
+ * that, then assume that it's not valid.
+ */
+ if (ci->i_btime.tv_sec || ci->i_btime.tv_nsec) {
+ stat->btime = ci->i_btime;
+ valid_mask |= STATX_BTIME;
+ }
+
if (ceph_snap(inode) == CEPH_NOSNAP)
stat->dev = inode->i_sb->s_dev;
else
stat->nlink = 1 + 1 + ci->i_subdirs;
}
- /* Mask off any higher bits (e.g. btime) until we have support */
- stat->result_mask = request_mask & STATX_BASIC_STATS;
+ stat->result_mask = request_mask & valid_mask;
return err;
}
info->pool_ns_data = *p;
*p += info->pool_ns_len;
}
- /* btime, change_attr */
- {
- struct ceph_timespec btime;
- u64 change_attr;
- ceph_decode_need(p, end, sizeof(btime), bad);
- ceph_decode_copy(p, &btime, sizeof(btime));
- ceph_decode_64_safe(p, end, change_attr, bad);
- }
+
+ /* btime */
+ ceph_decode_need(p, end, sizeof(info->btime), bad);
+ ceph_decode_copy(p, &info->btime, sizeof(info->btime));
+
+ /* change attribute */
+ ceph_decode_64_safe(p, end, info->change_attr, bad);
/* dir pin */
if (struct_v >= 2) {
info->dir_pin = -ENODATA;
}
+ /* snapshot birth time, remains zero for v<=2 */
+ if (struct_v >= 3) {
+ ceph_decode_need(p, end, sizeof(info->snap_btime), bad);
+ ceph_decode_copy(p, &info->snap_btime,
+ sizeof(info->snap_btime));
+ } else {
+ memset(&info->snap_btime, 0, sizeof(info->snap_btime));
+ }
+
*p = end;
} else {
if (features & CEPH_FEATURE_MDS_INLINE_DATA) {
}
}
+ if (features & CEPH_FEATURE_FS_BTIME) {
+ ceph_decode_need(p, end, sizeof(info->btime), bad);
+ ceph_decode_copy(p, &info->btime, sizeof(info->btime));
+ ceph_decode_64_safe(p, end, info->change_attr, bad);
+ }
+
info->dir_pin = -ENODATA;
+ /* info->snap_btime remains zero */
}
return 0;
bad:
ceph_pagelist_release(req->r_pagelist);
put_request_session(req);
ceph_unreserve_caps(req->r_mdsc, &req->r_caps_reservation);
+ WARN_ON_ONCE(!list_empty(&req->r_wait));
kfree(req);
}
struct inode *dir;
rcu_read_lock();
- parent = req->r_dentry->d_parent;
+ parent = READ_ONCE(req->r_dentry->d_parent);
dir = req->r_parent ? : d_inode_rcu(parent);
if (!dir || dir->i_sb != mdsc->fsc->sb) {
memcpy(path + pos, temp->d_name.name, temp->d_name.len);
}
spin_unlock(&temp->d_lock);
- temp = temp->d_parent;
+ temp = READ_ONCE(temp->d_parent);
/* Are we at the root? */
if (IS_ROOT(temp))
ceph_mdsmap_is_laggy(newmap, i) ? " (laggy)" : "",
ceph_session_state_name(s->s_state));
- if (i >= newmap->m_num_mds ||
- memcmp(ceph_mdsmap_get_addr(oldmap, i),
- ceph_mdsmap_get_addr(newmap, i),
- sizeof(struct ceph_entity_addr))) {
- if (s->s_state == CEPH_MDS_SESSION_OPENING) {
- /* the session never opened, just close it
- * out now */
- get_session(s);
- __unregister_session(mdsc, s);
- __wake_requests(mdsc, &s->s_waiting);
- ceph_put_mds_session(s);
- } else if (i >= newmap->m_num_mds) {
- /* force close session for stopped mds */
- get_session(s);
- __unregister_session(mdsc, s);
- __wake_requests(mdsc, &s->s_waiting);
- kick_requests(mdsc, i);
- mutex_unlock(&mdsc->mutex);
+ if (i >= newmap->m_num_mds) {
+ /* force close session for stopped mds */
+ get_session(s);
+ __unregister_session(mdsc, s);
+ __wake_requests(mdsc, &s->s_waiting);
+ mutex_unlock(&mdsc->mutex);
- mutex_lock(&s->s_mutex);
- cleanup_session_requests(mdsc, s);
- remove_session_caps(s);
- mutex_unlock(&s->s_mutex);
+ mutex_lock(&s->s_mutex);
+ cleanup_session_requests(mdsc, s);
+ remove_session_caps(s);
+ mutex_unlock(&s->s_mutex);
- ceph_put_mds_session(s);
+ ceph_put_mds_session(s);
- mutex_lock(&mdsc->mutex);
- } else {
- /* just close it */
- mutex_unlock(&mdsc->mutex);
- mutex_lock(&s->s_mutex);
- mutex_lock(&mdsc->mutex);
- ceph_con_close(&s->s_con);
- mutex_unlock(&s->s_mutex);
- s->s_state = CEPH_MDS_SESSION_RESTARTING;
- }
+ mutex_lock(&mdsc->mutex);
+ kick_requests(mdsc, i);
+ continue;
+ }
+
+ if (memcmp(ceph_mdsmap_get_addr(oldmap, i),
+ ceph_mdsmap_get_addr(newmap, i),
+ sizeof(struct ceph_entity_addr))) {
+ /* just close it */
+ mutex_unlock(&mdsc->mutex);
+ mutex_lock(&s->s_mutex);
+ mutex_lock(&mdsc->mutex);
+ ceph_con_close(&s->s_con);
+ mutex_unlock(&s->s_mutex);
+ s->s_state = CEPH_MDS_SESSION_RESTARTING;
} else if (oldstate == newstate) {
continue; /* nothing new with this mds */
}
}
void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
- struct inode *inode,
struct dentry *dentry, char action,
u32 seq)
{
struct ceph_msg *msg;
struct ceph_mds_lease *lease;
- int len = sizeof(*lease) + sizeof(u32);
- int dnamelen = 0;
+ struct inode *dir;
+ int len = sizeof(*lease) + sizeof(u32) + NAME_MAX;
- dout("lease_send_msg inode %p dentry %p %s to mds%d\n",
- inode, dentry, ceph_lease_op_name(action), session->s_mds);
- dnamelen = dentry->d_name.len;
- len += dnamelen;
+ dout("lease_send_msg identry %p %s to mds%d\n",
+ dentry, ceph_lease_op_name(action), session->s_mds);
msg = ceph_msg_new(CEPH_MSG_CLIENT_LEASE, len, GFP_NOFS, false);
if (!msg)
return;
lease = msg->front.iov_base;
lease->action = action;
- lease->ino = cpu_to_le64(ceph_vino(inode).ino);
- lease->first = lease->last = cpu_to_le64(ceph_vino(inode).snap);
lease->seq = cpu_to_le32(seq);
- put_unaligned_le32(dnamelen, lease + 1);
- memcpy((void *)(lease + 1) + 4, dentry->d_name.name, dnamelen);
+ spin_lock(&dentry->d_lock);
+ dir = d_inode(dentry->d_parent);
+ lease->ino = cpu_to_le64(ceph_ino(dir));
+ lease->first = lease->last = cpu_to_le64(ceph_snap(dir));
+
+ put_unaligned_le32(dentry->d_name.len, lease + 1);
+ memcpy((void *)(lease + 1) + 4,
+ dentry->d_name.name, dentry->d_name.len);
+ spin_unlock(&dentry->d_lock);
/*
* if this is a preemptive lease RELEASE, no need to
* flush request stream, since the actual request will
while ((req = __get_oldest_req(mdsc))) {
dout("wait_requests timed out on tid %llu\n",
req->r_tid);
+ list_del_init(&req->r_wait);
__unregister_request(mdsc, req);
}
}
u64 max_bytes;
u64 max_files;
s32 dir_pin;
+ struct ceph_timespec btime;
+ struct ceph_timespec snap_btime;
+ u64 change_attr;
};
struct ceph_mds_reply_dir_entry {
extern void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry);
extern void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
- struct inode *inode,
struct dentry *dentry, char action,
u32 seq);
struct ceph_mdsmap *m;
const void *start = *p;
int i, j, n;
- int err = -EINVAL;
+ int err;
u8 mdsmap_v, mdsmap_cv;
u16 mdsmap_ev;
inc = ceph_decode_32(p);
state = ceph_decode_32(p);
state_seq = ceph_decode_64(p);
- ceph_decode_copy(p, &addr, sizeof(addr));
- ceph_decode_addr(&addr);
+ err = ceph_decode_entity_addr(p, end, &addr);
+ if (err)
+ goto corrupt;
ceph_decode_copy(p, &laggy_since, sizeof(laggy_since));
*p += sizeof(u32);
ceph_decode_32_safe(p, end, namelen, bad);
nomem:
err = -ENOMEM;
goto out_err;
-bad:
+corrupt:
pr_err("corrupt mdsmap\n");
print_hex_dump(KERN_DEBUG, "mdsmap: ",
DUMP_PREFIX_OFFSET, 16, 1,
out_err:
ceph_mdsmap_destroy(m);
return ERR_PTR(err);
+bad:
+ err = -EINVAL;
+ goto corrupt;
}
void ceph_mdsmap_destroy(struct ceph_mdsmap *m)
return NULL;
mutex_lock(&qri->mutex);
- if (qri->inode) {
+ if (qri->inode && ceph_is_any_caps(qri->inode)) {
/* A request has already returned the inode */
mutex_unlock(&qri->mutex);
return qri->inode;
mutex_unlock(&qri->mutex);
return NULL;
}
- in = ceph_lookup_inode(sb, realm->ino);
+ if (qri->inode) {
+ /* get caps */
+ int ret = __ceph_do_getattr(qri->inode, NULL,
+ CEPH_STAT_CAP_INODE, true);
+ if (ret >= 0)
+ in = qri->inode;
+ else
+ in = ERR_PTR(ret);
+ } else {
+ in = ceph_lookup_inode(sb, realm->ino);
+ }
+
if (IS_ERR(in)) {
pr_warn("Can't lookup inode %llx (err: %ld)\n",
realm->ino, PTR_ERR(in));
#include <linux/sort.h>
#include <linux/slab.h>
+#include <linux/iversion.h>
#include "super.h"
#include "mds_client.h"
#include <linux/ceph/decode.h>
capsnap->mtime = inode->i_mtime;
capsnap->atime = inode->i_atime;
capsnap->ctime = inode->i_ctime;
+ capsnap->btime = ci->i_btime;
+ capsnap->change_attr = inode_peek_iversion_raw(inode);
capsnap->time_warp_seq = ci->i_time_warp_seq;
capsnap->truncate_size = ci->i_truncate_size;
capsnap->truncate_seq = ci->i_truncate_seq;
static const struct super_operations ceph_super_ops = {
.alloc_inode = ceph_alloc_inode,
- .destroy_inode = ceph_destroy_inode,
.free_inode = ceph_free_inode,
.write_inode = ceph_write_inode,
- .drop_inode = ceph_drop_inode,
+ .drop_inode = generic_delete_inode,
+ .evict_inode = ceph_evict_inode,
.sync_fs = ceph_sync_fs,
.put_super = ceph_put_super,
.remount_fs = ceph_remount,
s->s_d_op = &ceph_dentry_ops;
s->s_export_op = &ceph_export_ops;
- s->s_time_gran = 1000; /* 1000 ns == 1 us */
+ s->s_time_gran = 1;
ret = set_anon_super(s, NULL); /* what is that second arg for? */
if (ret != 0)
goto out;
ceph_flock_init();
- ceph_xattr_init();
ret = register_filesystem(&ceph_fs_type);
if (ret)
- goto out_xattr;
+ goto out_caches;
pr_info("loaded (mds proto %d)\n", CEPH_MDSC_PROTOCOL);
return 0;
-out_xattr:
- ceph_xattr_exit();
+out_caches:
destroy_caches();
out:
return ret;
{
dout("exit_ceph\n");
unregister_filesystem(&ceph_fs_type);
- ceph_xattr_exit();
destroy_caches();
}
u64 xattr_version;
u64 size;
- struct timespec64 mtime, atime, ctime;
+ u64 change_attr;
+ struct timespec64 mtime, atime, ctime, btime;
u64 time_warp_seq;
u64 truncate_size;
u32 truncate_seq;
int i_snap_realm_counter; /* snap realm (if caps) */
struct list_head i_snap_realm_item;
struct list_head i_snap_flush_item;
+ struct timespec64 i_btime;
+ struct timespec64 i_snap_btime;
struct work_struct i_work;
unsigned long i_work_mask;
long long release_count,
long long ordered_count)
{
- smp_mb__before_atomic();
+ /*
+ * Makes sure operations that setup readdir cache (update page
+ * cache and i_size) are strongly ordered w.r.t. the following
+ * atomic64_set() operations.
+ */
+ smp_mb();
atomic64_set(&ci->i_complete_seq[0], release_count);
atomic64_set(&ci->i_complete_seq[1], ordered_count);
}
extern const struct inode_operations ceph_file_iops;
extern struct inode *ceph_alloc_inode(struct super_block *sb);
-extern void ceph_destroy_inode(struct inode *inode);
+extern void ceph_evict_inode(struct inode *inode);
extern void ceph_free_inode(struct inode *inode);
-extern int ceph_drop_inode(struct inode *inode);
extern struct inode *ceph_get_inode(struct super_block *sb,
struct ceph_vino vino);
extern ssize_t ceph_listxattr(struct dentry *, char *, size_t);
extern void __ceph_build_xattrs_blob(struct ceph_inode_info *ci);
extern void __ceph_destroy_xattrs(struct ceph_inode_info *ci);
-extern void __init ceph_xattr_init(void);
-extern void ceph_xattr_exit(void);
extern const struct xattr_handler *ceph_xattr_handlers[];
+struct ceph_acl_sec_ctx {
+#ifdef CONFIG_CEPH_FS_POSIX_ACL
+ void *default_acl;
+ void *acl;
+#endif
+#ifdef CONFIG_CEPH_FS_SECURITY_LABEL
+ void *sec_ctx;
+ u32 sec_ctxlen;
+#endif
+ struct ceph_pagelist *pagelist;
+};
+
#ifdef CONFIG_SECURITY
extern bool ceph_security_xattr_deadlock(struct inode *in);
extern bool ceph_security_xattr_wanted(struct inode *in);
}
#endif
-/* acl.c */
-struct ceph_acls_info {
- void *default_acl;
- void *acl;
- struct ceph_pagelist *pagelist;
-};
+#ifdef CONFIG_CEPH_FS_SECURITY_LABEL
+extern int ceph_security_init_secctx(struct dentry *dentry, umode_t mode,
+ struct ceph_acl_sec_ctx *ctx);
+extern void ceph_security_invalidate_secctx(struct inode *inode);
+#else
+static inline int ceph_security_init_secctx(struct dentry *dentry, umode_t mode,
+ struct ceph_acl_sec_ctx *ctx)
+{
+ return 0;
+}
+static inline void ceph_security_invalidate_secctx(struct inode *inode)
+{
+}
+#endif
+
+void ceph_release_acl_sec_ctx(struct ceph_acl_sec_ctx *as_ctx);
+/* acl.c */
#ifdef CONFIG_CEPH_FS_POSIX_ACL
struct posix_acl *ceph_get_acl(struct inode *, int);
int ceph_set_acl(struct inode *inode, struct posix_acl *acl, int type);
int ceph_pre_init_acls(struct inode *dir, umode_t *mode,
- struct ceph_acls_info *info);
-void ceph_init_inode_acls(struct inode *inode, struct ceph_acls_info *info);
-void ceph_release_acls_info(struct ceph_acls_info *info);
+ struct ceph_acl_sec_ctx *as_ctx);
+void ceph_init_inode_acls(struct inode *inode,
+ struct ceph_acl_sec_ctx *as_ctx);
static inline void ceph_forget_all_cached_acls(struct inode *inode)
{
#define ceph_set_acl NULL
static inline int ceph_pre_init_acls(struct inode *dir, umode_t *mode,
- struct ceph_acls_info *info)
+ struct ceph_acl_sec_ctx *as_ctx)
{
return 0;
}
static inline void ceph_init_inode_acls(struct inode *inode,
- struct ceph_acls_info *info)
-{
-}
-static inline void ceph_release_acls_info(struct ceph_acls_info *info)
+ struct ceph_acl_sec_ctx *as_ctx)
{
}
static inline int ceph_acl_chmod(struct dentry *dentry, struct inode *inode)
unsigned cap, unsigned seq, u64 realmino, int flags,
struct ceph_cap **new_cap);
extern void __ceph_remove_cap(struct ceph_cap *cap, bool queue_release);
-extern void __ceph_remove_caps(struct inode* inode);
+extern void __ceph_remove_caps(struct ceph_inode_info *ci);
extern void ceph_put_cap(struct ceph_mds_client *mdsc,
struct ceph_cap *cap);
extern int ceph_is_any_caps(struct inode *inode);
#include <linux/ceph/decode.h>
#include <linux/xattr.h>
+#include <linux/security.h>
#include <linux/posix_acl_xattr.h>
#include <linux/slab.h>
static int __remove_xattr(struct ceph_inode_info *ci,
struct ceph_inode_xattr *xattr);
-static const struct xattr_handler ceph_other_xattr_handler;
-
-/*
- * List of handlers for synthetic system.* attributes. Other
- * attributes are handled directly.
- */
-const struct xattr_handler *ceph_xattr_handlers[] = {
-#ifdef CONFIG_CEPH_FS_POSIX_ACL
- &posix_acl_access_xattr_handler,
- &posix_acl_default_xattr_handler,
-#endif
- &ceph_other_xattr_handler,
- NULL,
-};
-
static bool ceph_is_valid_xattr(const char *name)
{
return !strncmp(name, XATTR_CEPH_PREFIX, XATTR_CEPH_PREFIX_LEN) ||
- !strncmp(name, XATTR_SECURITY_PREFIX,
- XATTR_SECURITY_PREFIX_LEN) ||
!strncmp(name, XATTR_TRUSTED_PREFIX, XATTR_TRUSTED_PREFIX_LEN) ||
!strncmp(name, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN);
}
struct ceph_vxattr {
char *name;
size_t name_size; /* strlen(name) + 1 (for '\0') */
- size_t (*getxattr_cb)(struct ceph_inode_info *ci, char *val,
- size_t size);
+ ssize_t (*getxattr_cb)(struct ceph_inode_info *ci, char *val,
+ size_t size);
bool (*exists_cb)(struct ceph_inode_info *ci);
unsigned int flags;
};
rcu_dereference_raw(fl->pool_ns) != NULL);
}
-static size_t ceph_vxattrcb_layout(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_layout(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
struct ceph_fs_client *fsc = ceph_sb_to_client(ci->vfs_inode.i_sb);
struct ceph_osd_client *osdc = &fsc->client->osdc;
const char *ns_field = " pool_namespace=";
char buf[128];
size_t len, total_len = 0;
- int ret;
+ ssize_t ret;
pool_ns = ceph_try_get_string(ci->i_layout.pool_ns);
len = snprintf(buf, sizeof(buf),
"stripe_unit=%u stripe_count=%u object_size=%u pool=%lld",
ci->i_layout.stripe_unit, ci->i_layout.stripe_count,
- ci->i_layout.object_size, (unsigned long long)pool);
+ ci->i_layout.object_size, pool);
total_len = len;
}
if (pool_ns)
total_len += strlen(ns_field) + pool_ns->len;
- if (!size) {
- ret = total_len;
- } else if (total_len > size) {
- ret = -ERANGE;
- } else {
+ ret = total_len;
+ if (size >= total_len) {
memcpy(val, buf, len);
ret = len;
if (pool_name) {
return ret;
}
-static size_t ceph_vxattrcb_layout_stripe_unit(struct ceph_inode_info *ci,
- char *val, size_t size)
+/*
+ * The convention with strings in xattrs is that they should not be NULL
+ * terminated, since we're returning the length with them. snprintf always
+ * NULL terminates however, so call it on a temporary buffer and then memcpy
+ * the result into place.
+ */
+static int ceph_fmt_xattr(char *val, size_t size, const char *fmt, ...)
{
- return snprintf(val, size, "%u", ci->i_layout.stripe_unit);
+ int ret;
+ va_list args;
+ char buf[96]; /* NB: reevaluate size if new vxattrs are added */
+
+ va_start(args, fmt);
+ ret = vsnprintf(buf, size ? sizeof(buf) : 0, fmt, args);
+ va_end(args);
+
+ /* Sanity check */
+ if (size && ret + 1 > sizeof(buf)) {
+ WARN_ONCE(true, "Returned length too big (%d)", ret);
+ return -E2BIG;
+ }
+
+ if (ret <= size)
+ memcpy(val, buf, ret);
+ return ret;
}
-static size_t ceph_vxattrcb_layout_stripe_count(struct ceph_inode_info *ci,
+static ssize_t ceph_vxattrcb_layout_stripe_unit(struct ceph_inode_info *ci,
char *val, size_t size)
{
- return snprintf(val, size, "%u", ci->i_layout.stripe_count);
+ return ceph_fmt_xattr(val, size, "%u", ci->i_layout.stripe_unit);
+}
+
+static ssize_t ceph_vxattrcb_layout_stripe_count(struct ceph_inode_info *ci,
+ char *val, size_t size)
+{
+ return ceph_fmt_xattr(val, size, "%u", ci->i_layout.stripe_count);
}
-static size_t ceph_vxattrcb_layout_object_size(struct ceph_inode_info *ci,
- char *val, size_t size)
+static ssize_t ceph_vxattrcb_layout_object_size(struct ceph_inode_info *ci,
+ char *val, size_t size)
{
- return snprintf(val, size, "%u", ci->i_layout.object_size);
+ return ceph_fmt_xattr(val, size, "%u", ci->i_layout.object_size);
}
-static size_t ceph_vxattrcb_layout_pool(struct ceph_inode_info *ci,
- char *val, size_t size)
+static ssize_t ceph_vxattrcb_layout_pool(struct ceph_inode_info *ci,
+ char *val, size_t size)
{
- int ret;
+ ssize_t ret;
struct ceph_fs_client *fsc = ceph_sb_to_client(ci->vfs_inode.i_sb);
struct ceph_osd_client *osdc = &fsc->client->osdc;
s64 pool = ci->i_layout.pool_id;
down_read(&osdc->lock);
pool_name = ceph_pg_pool_name_by_id(osdc->osdmap, pool);
- if (pool_name)
- ret = snprintf(val, size, "%s", pool_name);
- else
- ret = snprintf(val, size, "%lld", (unsigned long long)pool);
+ if (pool_name) {
+ ret = strlen(pool_name);
+ if (ret <= size)
+ memcpy(val, pool_name, ret);
+ } else {
+ ret = ceph_fmt_xattr(val, size, "%lld", pool);
+ }
up_read(&osdc->lock);
return ret;
}
-static size_t ceph_vxattrcb_layout_pool_namespace(struct ceph_inode_info *ci,
- char *val, size_t size)
+static ssize_t ceph_vxattrcb_layout_pool_namespace(struct ceph_inode_info *ci,
+ char *val, size_t size)
{
- int ret = 0;
+ ssize_t ret = 0;
struct ceph_string *ns = ceph_try_get_string(ci->i_layout.pool_ns);
+
if (ns) {
- ret = snprintf(val, size, "%.*s", (int)ns->len, ns->str);
+ ret = ns->len;
+ if (ret <= size)
+ memcpy(val, ns->str, ret);
ceph_put_string(ns);
}
return ret;
/* directories */
-static size_t ceph_vxattrcb_dir_entries(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_entries(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_files + ci->i_subdirs);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_files + ci->i_subdirs);
}
-static size_t ceph_vxattrcb_dir_files(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_files(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_files);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_files);
}
-static size_t ceph_vxattrcb_dir_subdirs(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_subdirs(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_subdirs);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_subdirs);
}
-static size_t ceph_vxattrcb_dir_rentries(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_rentries(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_rfiles + ci->i_rsubdirs);
+ return ceph_fmt_xattr(val, size, "%lld",
+ ci->i_rfiles + ci->i_rsubdirs);
}
-static size_t ceph_vxattrcb_dir_rfiles(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_rfiles(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_rfiles);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_rfiles);
}
-static size_t ceph_vxattrcb_dir_rsubdirs(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_rsubdirs(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_rsubdirs);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_rsubdirs);
}
-static size_t ceph_vxattrcb_dir_rbytes(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_rbytes(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld", ci->i_rbytes);
+ return ceph_fmt_xattr(val, size, "%lld", ci->i_rbytes);
}
-static size_t ceph_vxattrcb_dir_rctime(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_rctime(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%lld.09%ld", ci->i_rctime.tv_sec,
- ci->i_rctime.tv_nsec);
+ return ceph_fmt_xattr(val, size, "%lld.%09ld", ci->i_rctime.tv_sec,
+ ci->i_rctime.tv_nsec);
}
/* dir pin */
return ci->i_dir_pin != -ENODATA;
}
-static size_t ceph_vxattrcb_dir_pin(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_dir_pin(struct ceph_inode_info *ci, char *val,
+ size_t size)
{
- return snprintf(val, size, "%d", (int)ci->i_dir_pin);
+ return ceph_fmt_xattr(val, size, "%d", (int)ci->i_dir_pin);
}
/* quotas */
return ret;
}
-static size_t ceph_vxattrcb_quota(struct ceph_inode_info *ci, char *val,
- size_t size)
+static ssize_t ceph_vxattrcb_quota(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return ceph_fmt_xattr(val, size, "max_bytes=%llu max_files=%llu",
+ ci->i_max_bytes, ci->i_max_files);
+}
+
+static ssize_t ceph_vxattrcb_quota_max_bytes(struct ceph_inode_info *ci,
+ char *val, size_t size)
{
- return snprintf(val, size, "max_bytes=%llu max_files=%llu",
- ci->i_max_bytes, ci->i_max_files);
+ return ceph_fmt_xattr(val, size, "%llu", ci->i_max_bytes);
}
-static size_t ceph_vxattrcb_quota_max_bytes(struct ceph_inode_info *ci,
- char *val, size_t size)
+static ssize_t ceph_vxattrcb_quota_max_files(struct ceph_inode_info *ci,
+ char *val, size_t size)
{
- return snprintf(val, size, "%llu", ci->i_max_bytes);
+ return ceph_fmt_xattr(val, size, "%llu", ci->i_max_files);
}
-static size_t ceph_vxattrcb_quota_max_files(struct ceph_inode_info *ci,
- char *val, size_t size)
+/* snapshots */
+static bool ceph_vxattrcb_snap_btime_exists(struct ceph_inode_info *ci)
{
- return snprintf(val, size, "%llu", ci->i_max_files);
+ return (ci->i_snap_btime.tv_sec != 0 || ci->i_snap_btime.tv_nsec != 0);
+}
+
+static ssize_t ceph_vxattrcb_snap_btime(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return ceph_fmt_xattr(val, size, "%lld.%09ld", ci->i_snap_btime.tv_sec,
+ ci->i_snap_btime.tv_nsec);
}
#define CEPH_XATTR_NAME(_type, _name) XATTR_CEPH_PREFIX #_type "." #_name
XATTR_RSTAT_FIELD(dir, rctime),
{
.name = "ceph.dir.pin",
- .name_size = sizeof("ceph.dir_pin"),
+ .name_size = sizeof("ceph.dir.pin"),
.getxattr_cb = ceph_vxattrcb_dir_pin,
.exists_cb = ceph_vxattrcb_dir_pin_exists,
.flags = VXATTR_FLAG_HIDDEN,
},
XATTR_QUOTA_FIELD(quota, max_bytes),
XATTR_QUOTA_FIELD(quota, max_files),
+ {
+ .name = "ceph.snap.btime",
+ .name_size = sizeof("ceph.snap.btime"),
+ .getxattr_cb = ceph_vxattrcb_snap_btime,
+ .exists_cb = ceph_vxattrcb_snap_btime_exists,
+ .flags = VXATTR_FLAG_READONLY,
+ },
{ .name = NULL, 0 } /* Required table terminator */
};
-static size_t ceph_dir_vxattrs_name_size; /* total size of all names */
/* files */
XATTR_LAYOUT_FIELD(file, layout, object_size),
XATTR_LAYOUT_FIELD(file, layout, pool),
XATTR_LAYOUT_FIELD(file, layout, pool_namespace),
+ {
+ .name = "ceph.snap.btime",
+ .name_size = sizeof("ceph.snap.btime"),
+ .getxattr_cb = ceph_vxattrcb_snap_btime,
+ .exists_cb = ceph_vxattrcb_snap_btime_exists,
+ .flags = VXATTR_FLAG_READONLY,
+ },
{ .name = NULL, 0 } /* Required table terminator */
};
-static size_t ceph_file_vxattrs_name_size; /* total size of all names */
static struct ceph_vxattr *ceph_inode_vxattrs(struct inode *inode)
{
return NULL;
}
-static size_t ceph_vxattrs_name_size(struct ceph_vxattr *vxattrs)
-{
- if (vxattrs == ceph_dir_vxattrs)
- return ceph_dir_vxattrs_name_size;
- if (vxattrs == ceph_file_vxattrs)
- return ceph_file_vxattrs_name_size;
- BUG_ON(vxattrs);
- return 0;
-}
-
-/*
- * Compute the aggregate size (including terminating '\0') of all
- * virtual extended attribute names in the given vxattr table.
- */
-static size_t __init vxattrs_name_size(struct ceph_vxattr *vxattrs)
-{
- struct ceph_vxattr *vxattr;
- size_t size = 0;
-
- for (vxattr = vxattrs; vxattr->name; vxattr++) {
- if (!(vxattr->flags & VXATTR_FLAG_HIDDEN))
- size += vxattr->name_size;
- }
-
- return size;
-}
-
-/* Routines called at initialization and exit time */
-
-void __init ceph_xattr_init(void)
-{
- ceph_dir_vxattrs_name_size = vxattrs_name_size(ceph_dir_vxattrs);
- ceph_file_vxattrs_name_size = vxattrs_name_size(ceph_file_vxattrs);
-}
-
-void ceph_xattr_exit(void)
-{
- ceph_dir_vxattrs_name_size = 0;
- ceph_file_vxattrs_name_size = 0;
-}
-
static struct ceph_vxattr *ceph_match_vxattr(struct inode *inode,
const char *name)
{
dout("__set_xattr_val p=%p\n", p);
}
- dout("__set_xattr_val added %llx.%llx xattr %p %s=%.*s\n",
- ceph_vinop(&ci->vfs_inode), xattr, name, val_len, val);
+ dout("__set_xattr_val added %llx.%llx xattr %p %.*s=%.*s\n",
+ ceph_vinop(&ci->vfs_inode), xattr, name_len, name, val_len, val);
return 0;
}
struct ceph_inode_xattr *xattr;
struct ceph_vxattr *vxattr = NULL;
int req_mask;
- int err;
+ ssize_t err;
/* let's see if a virtual xattr was requested */
vxattr = ceph_match_vxattr(inode, name);
if (err)
return err;
err = -ENODATA;
- if (!(vxattr->exists_cb && !vxattr->exists_cb(ci)))
+ if (!(vxattr->exists_cb && !vxattr->exists_cb(ci))) {
err = vxattr->getxattr_cb(ci, value, size);
+ if (size && size < err)
+ err = -ERANGE;
+ }
return err;
}
struct inode *inode = d_inode(dentry);
struct ceph_inode_info *ci = ceph_inode(inode);
struct ceph_vxattr *vxattrs = ceph_inode_vxattrs(inode);
- u32 vir_namelen = 0;
+ bool len_only = (size == 0);
u32 namelen;
int err;
- u32 len;
int i;
spin_lock(&ci->i_ceph_lock);
err = __build_xattrs(inode);
if (err < 0)
goto out;
- /*
- * Start with virtual dir xattr names (if any) (including
- * terminating '\0' characters for each).
- */
- vir_namelen = ceph_vxattrs_name_size(vxattrs);
- /* adding 1 byte per each variable due to the null termination */
+ /* add 1 byte for each xattr due to the null termination */
namelen = ci->i_xattrs.names_size + ci->i_xattrs.count;
- err = -ERANGE;
- if (size && vir_namelen + namelen > size)
- goto out;
-
- err = namelen + vir_namelen;
- if (size == 0)
- goto out;
+ if (!len_only) {
+ if (namelen > size) {
+ err = -ERANGE;
+ goto out;
+ }
+ names = __copy_xattr_names(ci, names);
+ size -= namelen;
+ }
- names = __copy_xattr_names(ci, names);
/* virtual xattr names, too */
- err = namelen;
if (vxattrs) {
for (i = 0; vxattrs[i].name; i++) {
- if (!(vxattrs[i].flags & VXATTR_FLAG_HIDDEN) &&
- !(vxattrs[i].exists_cb &&
- !vxattrs[i].exists_cb(ci))) {
- len = sprintf(names, "%s", vxattrs[i].name);
- names += len + 1;
- err += len + 1;
+ size_t this_len;
+
+ if (vxattrs[i].flags & VXATTR_FLAG_HIDDEN)
+ continue;
+ if (vxattrs[i].exists_cb && !vxattrs[i].exists_cb(ci))
+ continue;
+
+ this_len = strlen(vxattrs[i].name) + 1;
+ namelen += this_len;
+ if (len_only)
+ continue;
+
+ if (this_len > size) {
+ err = -ERANGE;
+ goto out;
}
+
+ memcpy(names, vxattrs[i].name, this_len);
+ names += this_len;
+ size -= this_len;
}
}
-
+ err = namelen;
out:
spin_unlock(&ci->i_ceph_lock);
return err;
spin_unlock(&ci->i_ceph_lock);
return ret;
}
+
+#ifdef CONFIG_CEPH_FS_SECURITY_LABEL
+int ceph_security_init_secctx(struct dentry *dentry, umode_t mode,
+ struct ceph_acl_sec_ctx *as_ctx)
+{
+ struct ceph_pagelist *pagelist = as_ctx->pagelist;
+ const char *name;
+ size_t name_len;
+ int err;
+
+ err = security_dentry_init_security(dentry, mode, &dentry->d_name,
+ &as_ctx->sec_ctx,
+ &as_ctx->sec_ctxlen);
+ if (err < 0) {
+ WARN_ON_ONCE(err != -EOPNOTSUPP);
+ err = 0; /* do nothing */
+ goto out;
+ }
+
+ err = -ENOMEM;
+ if (!pagelist) {
+ pagelist = ceph_pagelist_alloc(GFP_KERNEL);
+ if (!pagelist)
+ goto out;
+ err = ceph_pagelist_reserve(pagelist, PAGE_SIZE);
+ if (err)
+ goto out;
+ ceph_pagelist_encode_32(pagelist, 1);
+ }
+
+ /*
+ * FIXME: Make security_dentry_init_security() generic. Currently
+ * It only supports single security module and only selinux has
+ * dentry_init_security hook.
+ */
+ name = XATTR_NAME_SELINUX;
+ name_len = strlen(name);
+ err = ceph_pagelist_reserve(pagelist,
+ 4 * 2 + name_len + as_ctx->sec_ctxlen);
+ if (err)
+ goto out;
+
+ if (as_ctx->pagelist) {
+ /* update count of KV pairs */
+ BUG_ON(pagelist->length <= sizeof(__le32));
+ if (list_is_singular(&pagelist->head)) {
+ le32_add_cpu((__le32*)pagelist->mapped_tail, 1);
+ } else {
+ struct page *page = list_first_entry(&pagelist->head,
+ struct page, lru);
+ void *addr = kmap_atomic(page);
+ le32_add_cpu((__le32*)addr, 1);
+ kunmap_atomic(addr);
+ }
+ } else {
+ as_ctx->pagelist = pagelist;
+ }
+
+ ceph_pagelist_encode_32(pagelist, name_len);
+ ceph_pagelist_append(pagelist, name, name_len);
+
+ ceph_pagelist_encode_32(pagelist, as_ctx->sec_ctxlen);
+ ceph_pagelist_append(pagelist, as_ctx->sec_ctx, as_ctx->sec_ctxlen);
+
+ err = 0;
+out:
+ if (pagelist && !as_ctx->pagelist)
+ ceph_pagelist_release(pagelist);
+ return err;
+}
+
+void ceph_security_invalidate_secctx(struct inode *inode)
+{
+ security_inode_invalidate_secctx(inode);
+}
+
+static int ceph_xattr_set_security_label(const struct xattr_handler *handler,
+ struct dentry *unused, struct inode *inode,
+ const char *key, const void *buf,
+ size_t buflen, int flags)
+{
+ if (security_ismaclabel(key)) {
+ const char *name = xattr_full_name(handler, key);
+ return __ceph_setxattr(inode, name, buf, buflen, flags);
+ }
+ return -EOPNOTSUPP;
+}
+
+static int ceph_xattr_get_security_label(const struct xattr_handler *handler,
+ struct dentry *unused, struct inode *inode,
+ const char *key, void *buf, size_t buflen)
+{
+ if (security_ismaclabel(key)) {
+ const char *name = xattr_full_name(handler, key);
+ return __ceph_getxattr(inode, name, buf, buflen);
+ }
+ return -EOPNOTSUPP;
+}
+
+static const struct xattr_handler ceph_security_label_handler = {
+ .prefix = XATTR_SECURITY_PREFIX,
+ .get = ceph_xattr_get_security_label,
+ .set = ceph_xattr_set_security_label,
+};
+#endif
#endif
+
+void ceph_release_acl_sec_ctx(struct ceph_acl_sec_ctx *as_ctx)
+{
+#ifdef CONFIG_CEPH_FS_POSIX_ACL
+ posix_acl_release(as_ctx->acl);
+ posix_acl_release(as_ctx->default_acl);
+#endif
+#ifdef CONFIG_CEPH_FS_SECURITY_LABEL
+ security_release_secctx(as_ctx->sec_ctx, as_ctx->sec_ctxlen);
+#endif
+ if (as_ctx->pagelist)
+ ceph_pagelist_release(as_ctx->pagelist);
+}
+
+/*
+ * List of handlers for synthetic system.* attributes. Other
+ * attributes are handled directly.
+ */
+const struct xattr_handler *ceph_xattr_handlers[] = {
+#ifdef CONFIG_CEPH_FS_POSIX_ACL
+ &posix_acl_access_xattr_handler,
+ &posix_acl_default_xattr_handler,
+#endif
+#ifdef CONFIG_CEPH_FS_SECURITY_LABEL
+ &ceph_security_label_handler,
+#endif
+ &ceph_other_xattr_handler,
+ NULL,
+};
select CRYPTO_LIB_ARC4
select CRYPTO_AEAD2
select CRYPTO_CCM
+ select CRYPTO_GCM
select CRYPTO_ECB
select CRYPTO_AES
select CRYPTO_DES
+ select KEYS
help
This is the client VFS module for the SMB3 family of NAS protocols,
(including support for the most recent, most secure dialect SMB3.1.1)
config CIFS_UPCALL
bool "Kerberos/SPNEGO advanced session setup"
- depends on CIFS && KEYS
+ depends on CIFS
select DNS_RESOLVER
help
Enables an upcall mechanism for CIFS which accesses userspace helper
(such as Samba 3.10 and later) which can negotiate
CIFS POSIX ACL support. If unsure, say N.
-config CIFS_ACL
- bool "Provide CIFS ACL support"
- depends on CIFS_XATTR && KEYS
- help
- Allows fetching CIFS/NTFS ACL from the server. The DACL blob
- is handed over to the application/caller. See the man
- page for getcifsacl for more information. If unsure, say Y.
-
config CIFS_DEBUG
bool "Enable CIFS debugging routines"
default y
config CIFS_DFS_UPCALL
bool "DFS feature support"
- depends on CIFS && KEYS
+ depends on CIFS
select DNS_RESOLVER
help
Distributed File System (DFS) support is used to access shares
Allows NFS server to export a CIFS mounted share (nfsd over cifs)
config CIFS_SMB_DIRECT
- bool "SMB Direct support (Experimental)"
+ bool "SMB Direct support"
depends on CIFS=m && INFINIBAND && INFINIBAND_ADDR_TRANS || CIFS=y && INFINIBAND=y && INFINIBAND_ADDR_TRANS=y
help
- Enables SMB Direct experimental support for SMB 3.0, 3.02 and 3.1.1.
+ Enables SMB Direct support for SMB 3.0, 3.02 and 3.1.1.
SMB Direct allows transferring SMB packets over RDMA. If unsure,
say N.
cifs_unicode.o nterr.o cifsencrypt.o \
readdir.o ioctl.o sess.o export.o smb1ops.o winucase.o \
smb2ops.o smb2maperror.o smb2transport.o \
- smb2misc.o smb2pdu.o smb2inode.o smb2file.o
+ smb2misc.o smb2pdu.o smb2inode.o smb2file.o cifsacl.o
cifs-$(CONFIG_CIFS_XATTR) += xattr.o
-cifs-$(CONFIG_CIFS_ACL) += cifsacl.o
cifs-$(CONFIG_CIFS_UPCALL) += cifs_spnego.o
#ifdef CONFIG_CIFS_XATTR
seq_printf(m, ",XATTR");
#endif
-#ifdef CONFIG_CIFS_ACL
seq_printf(m, ",ACL");
-#endif
seq_putc(m, '\n');
seq_printf(m, "CIFSMaxBufSize: %d\n", CIFSMaxBufSize);
seq_printf(m, "Active VFS Requests: %d\n", GlobalTotalActiveXid);
#define CIFS_MOUNT_UID_FROM_ACL 0x2000000 /* try to get UID via special SID */
#define CIFS_MOUNT_NO_HANDLE_CACHE 0x4000000 /* disable caching dir handles */
#define CIFS_MOUNT_NO_DFS 0x8000000 /* disable DFS resolving */
+#define CIFS_MOUNT_MODE_FROM_SID 0x10000000 /* retrieve mode from special ACE */
struct cifs_sb_info {
struct rb_root tlink_tree;
* failover properly.
*/
char *origin_fullpath; /* \\HOST\SHARE\[OPTIONAL PATH] */
+ /*
+ * Indicate whether serverino option was turned off later
+ * (cifs_autodisable_serverino) in order to match new mounts.
+ */
+ bool mnt_cifs_serverino_autodisabled;
};
#endif /* _CIFS_FS_SB_H */
seq_puts(s, ",nobrl");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NO_HANDLE_CACHE)
seq_puts(s, ",nohandlecache");
+ if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_MODE_FROM_SID)
+ seq_puts(s, ",modefromsid");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL)
seq_puts(s, ",cifsacl");
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_DYNPERM)
seq_printf(s, ",bsize=%u", cifs_sb->bsize);
seq_printf(s, ",echo_interval=%lu",
tcon->ses->server->echo_interval / HZ);
+
+ /* Only display max_credits if it was overridden on mount */
+ if (tcon->ses->server->max_credits != SMB2_MAX_CREDITS_AVAILABLE)
+ seq_printf(s, ",max_credits=%u", tcon->ses->server->max_credits);
+
if (tcon->snapshot_time)
seq_printf(s, ",snapshot=%llu", tcon->snapshot_time);
if (tcon->handle_timeout)
goto out_destroy_dfs_cache;
#endif /* CONFIG_CIFS_UPCALL */
-#ifdef CONFIG_CIFS_ACL
rc = init_cifs_idmap();
if (rc)
goto out_register_key_type;
-#endif /* CONFIG_CIFS_ACL */
rc = register_filesystem(&cifs_fs_type);
if (rc)
return 0;
out_init_cifs_idmap:
-#ifdef CONFIG_CIFS_ACL
exit_cifs_idmap();
out_register_key_type:
-#endif
#ifdef CONFIG_CIFS_UPCALL
exit_cifs_spnego();
out_destroy_dfs_cache:
unregister_filesystem(&cifs_fs_type);
unregister_filesystem(&smb3_fs_type);
cifs_dfs_release_automount_timer();
-#ifdef CONFIG_CIFS_ACL
exit_cifs_idmap();
-#endif
#ifdef CONFIG_CIFS_UPCALL
exit_cifs_spnego();
#endif
MODULE_SOFTDEP("pre: sha512");
MODULE_SOFTDEP("pre: aead2");
MODULE_SOFTDEP("pre: ccm");
+MODULE_SOFTDEP("pre: gcm");
module_init(init_cifs)
module_exit(exit_cifs)
bool override_gid:1;
bool dynperm:1;
bool noperm:1;
+ bool mode_ace:1;
bool no_psx_acl:1; /* set if posix acl support should be disabled */
bool cifs_acl:1;
bool backupuid_specified; /* mount option backupuid is specified */
__u64 snapshot_time; /* needed for timewarp tokens */
__u32 handle_timeout; /* persistent and durable handle timeout in ms */
unsigned int max_credits; /* smb3 max_credits 10 < credits < 60000 */
+ __u16 compression; /* compression algorithm 0xFFFF default 0=disabled */
};
/**
CIFS_MOUNT_FSCACHE | CIFS_MOUNT_MF_SYMLINKS | \
CIFS_MOUNT_MULTIUSER | CIFS_MOUNT_STRICT_IO | \
CIFS_MOUNT_CIFS_BACKUPUID | CIFS_MOUNT_CIFS_BACKUPGID | \
- CIFS_MOUNT_NO_DFS)
+ CIFS_MOUNT_UID_FROM_ACL | CIFS_MOUNT_NO_HANDLE_CACHE | \
+ CIFS_MOUNT_NO_DFS | CIFS_MOUNT_MODE_FROM_SID)
/**
* Generic VFS superblock mount flags (s_flags) to consider when
extern unsigned int cifs_max_pending; /* MAX requests at once to server*/
extern bool disable_legacy_dialects; /* forbid vers=1.0 and vers=2.0 mounts */
-#ifdef CONFIG_CIFS_ACL
GLOBAL_EXTERN struct rb_root uidtree;
GLOBAL_EXTERN struct rb_root gidtree;
GLOBAL_EXTERN spinlock_t siduidlock;
GLOBAL_EXTERN struct rb_root sidgidtree;
GLOBAL_EXTERN spinlock_t uidsidlock;
GLOBAL_EXTERN spinlock_t gidsidlock;
-#endif /* CONFIG_CIFS_ACL */
void cifs_oplock_break(struct work_struct *work);
void cifs_queue_oplock_break(struct cifsFileInfo *cfile);
return size;
}
-static __u16 convert_ace_to_cifs_ace(struct cifs_posix_ace *cifs_ace,
+static void convert_ace_to_cifs_ace(struct cifs_posix_ace *cifs_ace,
const struct posix_acl_xattr_entry *local_ace)
{
- __u16 rc = 0; /* 0 = ACL converted ok */
-
cifs_ace->cifs_e_perm = le16_to_cpu(local_ace->e_perm);
cifs_ace->cifs_e_tag = le16_to_cpu(local_ace->e_tag);
/* BB is there a better way to handle the large uid? */
cifs_dbg(FYI, "perm %d tag %d id %d\n",
ace->e_perm, ace->e_tag, ace->e_id);
*/
- return rc;
}
/* Convert ACL from local Linux POSIX xattr to CIFS POSIX ACL wire format */
cifs_dbg(FYI, "unknown ACL type %d\n", acl_type);
return 0;
}
- for (i = 0; i < count; i++) {
- rc = convert_ace_to_cifs_ace(&cifs_acl->ace_array[i], &ace[i]);
- if (rc != 0) {
- /* ACE not converted */
- break;
- }
- }
+ for (i = 0; i < count; i++)
+ convert_ace_to_cifs_ace(&cifs_acl->ace_array[i], &ace[i]);
if (rc == 0) {
rc = (__u16)(count * sizeof(struct cifs_posix_ace));
rc += sizeof(struct cifs_posix_acl);
#endif /* CONFIG_POSIX */
-#ifdef CONFIG_CIFS_ACL
/*
* Initialize NT TRANSACT SMB into small smb request buffer. This assumes that
* all NT TRANSACTS that we init here have total parm and data under about 400
return (rc);
}
-#endif /* CONFIG_CIFS_ACL */
/* Legacy Query Path Information call for lookup to old servers such
as Win9x/WinME */
Opt_multiuser, Opt_sloppy, Opt_nosharesock,
Opt_persistent, Opt_nopersistent,
Opt_resilient, Opt_noresilient,
- Opt_domainauto, Opt_rdma,
+ Opt_domainauto, Opt_rdma, Opt_modesid,
+ Opt_compress,
/* Mount options which take numeric value */
Opt_backupuid, Opt_backupgid, Opt_uid,
{ Opt_serverino, "serverino" },
{ Opt_noserverino, "noserverino" },
{ Opt_rwpidforward, "rwpidforward" },
+ { Opt_modesid, "modefromsid" },
{ Opt_cifsacl, "cifsacl" },
{ Opt_nocifsacl, "nocifsacl" },
{ Opt_acl, "acl" },
{ Opt_echo_interval, "echo_interval=%s" },
{ Opt_max_credits, "max_credits=%s" },
{ Opt_snapshot, "snapshot=%s" },
+ { Opt_compress, "compress=%s" },
{ Opt_blank_user, "user=" },
{ Opt_blank_user, "username=" },
server_unresponsive(struct TCP_Server_Info *server)
{
/*
- * We need to wait 2 echo intervals to make sure we handle such
+ * We need to wait 3 echo intervals to make sure we handle such
* situations right:
* 1s client sends a normal SMB request
- * 2s client gets a response
+ * 3s client gets a response
* 30s echo workqueue job pops, and decides we got a response recently
* and don't need to send another
* ...
*/
if ((server->tcpStatus == CifsGood ||
server->tcpStatus == CifsNeedNegotiate) &&
- time_after(jiffies, server->lstrp + 2 * server->echo_interval)) {
+ time_after(jiffies, server->lstrp + 3 * server->echo_interval)) {
cifs_dbg(VFS, "Server %s has not responded in %lu seconds. Reconnecting...\n",
- server->hostname, (2 * server->echo_interval) / HZ);
+ server->hostname, (3 * server->echo_interval) / HZ);
cifs_reconnect(server);
wake_up(&server->response_q);
return true;
atomic_read(&midCount));
cifs_dump_mem("Received Data is: ", bufs[i],
HEADER_SIZE(server));
+ smb2_add_credits_from_hdr(bufs[i], server);
#ifdef CONFIG_CIFS_DEBUG2
if (server->ops->dump_detail)
server->ops->dump_detail(bufs[i],
server);
- smb2_add_credits_from_hdr(bufs[i], server);
cifs_dump_mids(server);
#endif /* CIFS_DEBUG2 */
}
case Opt_rwpidforward:
vol->rwpidforward = 1;
break;
+ case Opt_modesid:
+ vol->mode_ace = 1;
+ break;
case Opt_cifsacl:
vol->cifs_acl = 1;
break;
case Opt_rdma:
vol->rdma = true;
break;
+ case Opt_compress:
+ vol->compression = UNKNOWN_TYPE;
+ cifs_dbg(VFS,
+ "SMB3 compression support is experimental\n");
+ break;
/* Numeric Values */
case Opt_backupuid:
if (vol->nosharesock)
return 0;
- /* BB update this for smb3any and default case */
- if ((server->vals != vol->vals) || (server->ops != vol->ops))
+ /* If multidialect negotiation see if existing sessions match one */
+ if (strcmp(vol->vals->version_string, SMB3ANY_VERSION_STRING) == 0) {
+ if (server->vals->protocol_id < SMB30_PROT_ID)
+ return 0;
+ } else if (strcmp(vol->vals->version_string,
+ SMBDEFAULT_VERSION_STRING) == 0) {
+ if (server->vals->protocol_id < SMB21_PROT_ID)
+ return 0;
+ } else if ((server->vals != vol->vals) || (server->ops != vol->ops))
return 0;
if (!net_eq(cifs_net_ns(server), current->nsproxy->net_ns))
tcp_ses->sequence_number = 0;
tcp_ses->reconnect_instance = 1;
tcp_ses->lstrp = jiffies;
+ tcp_ses->compress_algorithm = cpu_to_le16(volume_info->compression);
spin_lock_init(&tcp_ses->req_lock);
INIT_LIST_HEAD(&tcp_ses->tcp_ses_list);
INIT_LIST_HEAD(&tcp_ses->smb_ses_list);
{
struct cifs_sb_info *old = CIFS_SB(sb);
struct cifs_sb_info *new = mnt_data->cifs_sb;
+ unsigned int oldflags = old->mnt_cifs_flags & CIFS_MOUNT_MASK;
+ unsigned int newflags = new->mnt_cifs_flags & CIFS_MOUNT_MASK;
if ((sb->s_flags & CIFS_MS_MASK) != (mnt_data->flags & CIFS_MS_MASK))
return 0;
- if ((old->mnt_cifs_flags & CIFS_MOUNT_MASK) !=
- (new->mnt_cifs_flags & CIFS_MOUNT_MASK))
+ if (old->mnt_cifs_serverino_autodisabled)
+ newflags &= ~CIFS_MOUNT_SERVER_INUM;
+
+ if (oldflags != newflags)
return 0;
/*
cifs_sb->mnt_cifs_flags |= CIFS_MOUNT_NOPOSIXBRL;
if (pvolume_info->rwpidforward)
cifs_sb->mnt_cifs_flags |= CIFS_MOUNT_RWPIDFORWARD;
+ if (pvolume_info->mode_ace)
+ cifs_sb->mnt_cifs_flags |= CIFS_MOUNT_MODE_FROM_SID;
if (pvolume_info->cifs_acl)
cifs_sb->mnt_cifs_flags |= CIFS_MOUNT_CIFS_ACL;
if (pvolume_info->backupuid_specified) {
unsigned int xid,
struct cifs_tcon *tcon,
struct cifs_sb_info *cifs_sb,
- char *full_path)
+ char *full_path,
+ int added_treename)
{
int rc;
char *s;
char sep, tmp;
+ int skip = added_treename ? 1 : 0;
sep = CIFS_DIR_SEP(cifs_sb);
s = full_path;
/* next separator */
while (*s && *s != sep)
s++;
-
+ /*
+ * if the treename is added, we then have to skip the first
+ * part within the separators
+ */
+ if (skip) {
+ skip = 0;
+ continue;
+ }
/*
* temporarily null-terminate the path at the end of
* the current component
if (rc != -EREMOTE) {
rc = cifs_are_all_path_components_accessible(server, xid, tcon,
- cifs_sb,
- full_path);
+ cifs_sb, full_path, tcon->Flags & SMB_SHARE_IS_IN_DFS);
if (rc != 0) {
cifs_dbg(VFS, "cannot query dirs between root and final path, "
"enabling CIFS_MOUNT_USE_PREFIX_PATH\n");
#ifdef CONFIG_CIFS_DEBUG2
char *name = get_tgt_name(ce);
- if (unlikely(IS_ERR(name))) {
+ if (IS_ERR(name)) {
rcu_read_unlock();
return ERR_CAST(name);
}
cifs_dbg(FYI, "cifs_sfu_type failed: %d\n", tmprc);
}
-#ifdef CONFIG_CIFS_ACL
/* fill in 0777 bits from ACL */
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL) {
rc = cifs_acl_to_fattr(cifs_sb, &fattr, *inode, full_path, fid);
goto cgii_exit;
}
}
-#endif /* CONFIG_CIFS_ACL */
/* fill in remaining high mode bits e.g. SUID, VTX */
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_UNX_EMUL)
xid = get_xid();
- cifs_dbg(FYI, "setattr on file %pd attrs->iavalid 0x%x\n",
+ cifs_dbg(FYI, "setattr on file %pd attrs->ia_valid 0x%x\n",
direntry, attrs->ia_valid);
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_NO_PERM)
if (attrs->ia_valid & ATTR_GID)
gid = attrs->ia_gid;
-#ifdef CONFIG_CIFS_ACL
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL) {
if (uid_valid(uid) || gid_valid(gid)) {
rc = id_mode_to_cifs_acl(inode, full_path, NO_CHANGE_64,
}
}
} else
-#endif /* CONFIG_CIFS_ACL */
if (!(cifs_sb->mnt_cifs_flags & CIFS_MOUNT_SET_UID))
attrs->ia_valid &= ~(ATTR_UID | ATTR_GID);
if (attrs->ia_valid & ATTR_MODE) {
mode = attrs->ia_mode;
rc = 0;
-#ifdef CONFIG_CIFS_ACL
if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_CIFS_ACL) {
rc = id_mode_to_cifs_acl(inode, full_path, mode,
INVALID_UID, INVALID_GID);
goto cifs_setattr_exit;
}
} else
-#endif /* CONFIG_CIFS_ACL */
if (((mode & S_IWUGO) == 0) &&
(cifsInode->cifsAttrs & ATTR_READONLY) == 0) {
tcon = cifs_sb_master_tcon(cifs_sb);
cifs_sb->mnt_cifs_flags &= ~CIFS_MOUNT_SERVER_INUM;
+ cifs_sb->mnt_cifs_serverino_autodisabled = true;
cifs_dbg(VFS, "Autodisabling the use of server inode numbers on %s.\n",
tcon ? tcon->treeName : "new server");
cifs_dbg(VFS, "The server doesn't seem to support them properly or the files might be on different servers (DFS).\n");
.query_all_EAs = CIFSSMBQAllEAs,
.set_EA = CIFSSMBSetEA,
#endif /* CIFS_XATTR */
-#ifdef CONFIG_CIFS_ACL
.get_acl = get_cifs_acl,
.get_acl_by_fid = get_cifs_acl_by_fid,
.set_acl = set_cifs_acl,
-#endif /* CIFS_ACL */
.make_node = cifs_make_node,
};
struct smb_version_values smb1_values = {
.version_string = SMB1_VERSION_STRING,
+ .protocol_id = SMB10_PROT_ID,
.large_lock_type = LOCKING_ANDX_LARGE_FILES,
.exclusive_lock_type = 0,
.shared_lock_type = LOCKING_ANDX_SHARED_LOCK,
SMB2_O_INFO_FILE, 0,
sizeof(struct smb2_file_all_info) +
PATH_MAX * 2, 0, NULL);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_query_info_compound_enter(xid, ses->Suid, tcon->tid,
COMPOUND_FID, current->tgid,
FILE_DISPOSITION_INFORMATION,
SMB2_O_INFO_FILE, 0, data, size);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_rmdir_enter(xid, ses->Suid, tcon->tid, full_path);
COMPOUND_FID, current->tgid,
FILE_END_OF_FILE_INFORMATION,
SMB2_O_INFO_FILE, 0, data, size);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_set_eof_enter(xid, ses->Suid, tcon->tid, full_path);
COMPOUND_FID, current->tgid,
FILE_BASIC_INFORMATION,
SMB2_O_INFO_FILE, 0, data, size);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_set_info_compound_enter(xid, ses->Suid, tcon->tid,
COMPOUND_FID, current->tgid,
FILE_RENAME_INFORMATION,
SMB2_O_INFO_FILE, 0, data, size);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_rename_enter(xid, ses->Suid, tcon->tid, full_path);
COMPOUND_FID, current->tgid,
FILE_LINK_INFORMATION,
SMB2_O_INFO_FILE, 0, data, size);
+ if (rc)
+ goto finished;
smb2_set_next_command(tcon, &rqst[num_rqst]);
smb2_set_related(&rqst[num_rqst++]);
trace_smb3_hardlink_enter(xid, ses->Suid, tcon->tid, full_path);
struct smb2_sync_hdr *shdr;
shdr = (struct smb2_sync_hdr *)(rqst->rq_iov[0].iov_base);
+ if (shdr == NULL) {
+ cifs_dbg(FYI, "shdr NULL in smb2_set_related\n");
+ return;
+ }
shdr->Flags |= SMB2_FLAGS_RELATED_OPERATIONS;
}
unsigned long len = smb_rqst_len(server, rqst);
int i, num_padding;
+ shdr = (struct smb2_sync_hdr *)(rqst->rq_iov[0].iov_base);
+ if (shdr == NULL) {
+ cifs_dbg(FYI, "shdr NULL in smb2_set_next_command\n");
+ return;
+ }
+
/* SMB headers in a compound are 8 byte aligned. */
/* No padding needed */
}
finished:
- shdr = (struct smb2_sync_hdr *)(rqst->rq_iov[0].iov_base);
shdr->NextCommand = cpu_to_le32(len);
}
}
static int
+parse_reparse_posix(struct reparse_posix_data *symlink_buf,
+ u32 plen, char **target_path,
+ struct cifs_sb_info *cifs_sb)
+{
+ unsigned int len;
+
+ /* See MS-FSCC 2.1.2.6 for the 'NFS' style reparse tags */
+ len = le16_to_cpu(symlink_buf->ReparseDataLength);
+
+ if (le64_to_cpu(symlink_buf->InodeType) != NFS_SPECFILE_LNK) {
+ cifs_dbg(VFS, "%lld not a supported symlink type\n",
+ le64_to_cpu(symlink_buf->InodeType));
+ return -EOPNOTSUPP;
+ }
+
+ *target_path = cifs_strndup_from_utf16(
+ symlink_buf->PathBuffer,
+ len, true, cifs_sb->local_nls);
+ if (!(*target_path))
+ return -ENOMEM;
+
+ convert_delimiter(*target_path, '/');
+ cifs_dbg(FYI, "%s: target path: %s\n", __func__, *target_path);
+
+ return 0;
+}
+
+static int
parse_reparse_symlink(struct reparse_symlink_data_buffer *symlink_buf,
u32 plen, char **target_path,
struct cifs_sb_info *cifs_sb)
unsigned int sub_len;
unsigned int sub_offset;
- /* We only handle Symbolic Link : MS-FSCC 2.1.2.4 */
- if (le32_to_cpu(symlink_buf->ReparseTag) != IO_REPARSE_TAG_SYMLINK) {
- cifs_dbg(VFS, "srv returned invalid symlink buffer\n");
- return -EIO;
- }
+ /* We handle Symbolic Link reparse tag here. See: MS-FSCC 2.1.2.4 */
sub_offset = le16_to_cpu(symlink_buf->SubstituteNameOffset);
sub_len = le16_to_cpu(symlink_buf->SubstituteNameLength);
return 0;
}
+static int
+parse_reparse_point(struct reparse_data_buffer *buf,
+ u32 plen, char **target_path,
+ struct cifs_sb_info *cifs_sb)
+{
+ if (plen < sizeof(struct reparse_data_buffer)) {
+ cifs_dbg(VFS, "reparse buffer is too small. Must be "
+ "at least 8 bytes but was %d\n", plen);
+ return -EIO;
+ }
+
+ if (plen < le16_to_cpu(buf->ReparseDataLength) +
+ sizeof(struct reparse_data_buffer)) {
+ cifs_dbg(VFS, "srv returned invalid reparse buf "
+ "length: %d\n", plen);
+ return -EIO;
+ }
+
+ /* See MS-FSCC 2.1.2 */
+ switch (le32_to_cpu(buf->ReparseTag)) {
+ case IO_REPARSE_TAG_NFS:
+ return parse_reparse_posix(
+ (struct reparse_posix_data *)buf,
+ plen, target_path, cifs_sb);
+ case IO_REPARSE_TAG_SYMLINK:
+ return parse_reparse_symlink(
+ (struct reparse_symlink_data_buffer *)buf,
+ plen, target_path, cifs_sb);
+ default:
+ cifs_dbg(VFS, "srv returned unknown symlink buffer "
+ "tag:0x%08x\n", le32_to_cpu(buf->ReparseTag));
+ return -EOPNOTSUPP;
+ }
+}
+
#define SMB2_SYMLINK_STRUCT_SIZE \
(sizeof(struct smb2_err_rsp) - 1 + sizeof(struct smb2_symlink_err_rsp))
goto querty_exit;
}
- if (plen < 8) {
- cifs_dbg(VFS, "reparse buffer is too small. Must be "
- "at least 8 bytes but was %d\n", plen);
- rc = -EIO;
- goto querty_exit;
- }
-
- if (plen < le16_to_cpu(reparse_buf->ReparseDataLength) + 8) {
- cifs_dbg(VFS, "srv returned invalid reparse buf "
- "length: %d\n", plen);
- rc = -EIO;
- goto querty_exit;
- }
-
- rc = parse_reparse_symlink(
- (struct reparse_symlink_data_buffer *)reparse_buf,
- plen, target_path, cifs_sb);
+ rc = parse_reparse_point(reparse_buf, plen, target_path,
+ cifs_sb);
goto querty_exit;
}
err_buf = err_iov.iov_base;
if (le32_to_cpu(err_buf->ByteCount) < sizeof(struct smb2_symlink_err_rsp) ||
err_iov.iov_len < SMB2_SYMLINK_STRUCT_SIZE) {
- rc = -ENOENT;
+ rc = -EINVAL;
+ goto querty_exit;
+ }
+
+ symlink = (struct smb2_symlink_err_rsp *)err_buf->ErrorData;
+ if (le32_to_cpu(symlink->SymLinkErrorTag) != SYMLINK_ERROR_TAG ||
+ le32_to_cpu(symlink->ReparseTag) != IO_REPARSE_TAG_SYMLINK) {
+ rc = -EINVAL;
goto querty_exit;
}
/* open must fail on symlink - reset rc */
rc = 0;
- symlink = (struct smb2_symlink_err_rsp *)err_buf->ErrorData;
sub_len = le16_to_cpu(symlink->SubstituteNameLength);
sub_offset = le16_to_cpu(symlink->SubstituteNameOffset);
print_len = le16_to_cpu(symlink->PrintNameLength);
print_offset = le16_to_cpu(symlink->PrintNameOffset);
if (err_iov.iov_len < SMB2_SYMLINK_STRUCT_SIZE + sub_offset + sub_len) {
- rc = -ENOENT;
+ rc = -EINVAL;
goto querty_exit;
}
if (err_iov.iov_len <
SMB2_SYMLINK_STRUCT_SIZE + print_offset + print_len) {
- rc = -ENOENT;
+ rc = -EINVAL;
goto querty_exit;
}
return rc;
}
-#ifdef CONFIG_CIFS_ACL
static struct cifs_ntsd *
get_smb2_acl_by_fid(struct cifs_sb_info *cifs_sb,
const struct cifs_fid *cifsfid, u32 *pacllen)
return pntsd;
}
-#ifdef CONFIG_CIFS_ACL
static int
set_smb2_acl(struct cifs_ntsd *pnntsd, __u32 acllen,
struct inode *inode, const char *path, int aclflag)
free_xid(xid);
return rc;
}
-#endif /* CIFS_ACL */
/* Retrieve an ACL from the server */
static struct cifs_ntsd *
cifsFileInfo_put(open_file);
return pntsd;
}
-#endif
static long smb3_zero_range(struct file *file, struct cifs_tcon *tcon,
loff_t offset, loff_t len, bool keep_size)
static void
fill_transform_hdr(struct smb2_transform_hdr *tr_hdr, unsigned int orig_len,
- struct smb_rqst *old_rq)
+ struct smb_rqst *old_rq, __le16 cipher_type)
{
struct smb2_sync_hdr *shdr =
(struct smb2_sync_hdr *)old_rq->rq_iov[0].iov_base;
tr_hdr->ProtocolId = SMB2_TRANSFORM_PROTO_NUM;
tr_hdr->OriginalMessageSize = cpu_to_le32(orig_len);
tr_hdr->Flags = cpu_to_le16(0x01);
- get_random_bytes(&tr_hdr->Nonce, SMB3_AES128CMM_NONCE);
+ if (cipher_type == SMB2_ENCRYPTION_AES128_GCM)
+ get_random_bytes(&tr_hdr->Nonce, SMB3_AES128GCM_NONCE);
+ else
+ get_random_bytes(&tr_hdr->Nonce, SMB3_AES128CCM_NONCE);
memcpy(&tr_hdr->SessionId, &shdr->SessionId, 8);
}
rc = -ENOMEM;
goto free_sg;
}
- iv[0] = 3;
- memcpy(iv + 1, (char *)tr_hdr->Nonce, SMB3_AES128CMM_NONCE);
+
+ if (server->cipher_type == SMB2_ENCRYPTION_AES128_GCM)
+ memcpy(iv, (char *)tr_hdr->Nonce, SMB3_AES128GCM_NONCE);
+ else {
+ iv[0] = 3;
+ memcpy(iv + 1, (char *)tr_hdr->Nonce, SMB3_AES128CCM_NONCE);
+ }
aead_request_set_crypt(req, sg, sg, crypt_len, iv);
aead_request_set_ad(req, assoc_data_len);
}
/* fill the 1st iov with a transform header */
- fill_transform_hdr(tr_hdr, orig_len, old_rq);
+ fill_transform_hdr(tr_hdr, orig_len, old_rq, server->cipher_type);
rc = crypt_message(server, num_rqst, new_rq, 1);
cifs_dbg(FYI, "Encrypt message returned %d\n", rc);
.query_all_EAs = smb2_query_eas,
.set_EA = smb2_set_ea,
#endif /* CIFS_XATTR */
-#ifdef CONFIG_CIFS_ACL
.get_acl = get_smb2_acl,
.get_acl_by_fid = get_smb2_acl_by_fid,
.set_acl = set_smb2_acl,
-#endif /* CIFS_ACL */
.next_header = smb2_next_header,
.ioctl_query_info = smb2_ioctl_query_info,
.make_node = smb2_make_node,
.query_all_EAs = smb2_query_eas,
.set_EA = smb2_set_ea,
#endif /* CIFS_XATTR */
-#ifdef CONFIG_CIFS_ACL
.get_acl = get_smb2_acl,
.get_acl_by_fid = get_smb2_acl_by_fid,
.set_acl = set_smb2_acl,
-#endif /* CIFS_ACL */
.next_header = smb2_next_header,
.ioctl_query_info = smb2_ioctl_query_info,
.make_node = smb2_make_node,
.query_all_EAs = smb2_query_eas,
.set_EA = smb2_set_ea,
#endif /* CIFS_XATTR */
-#ifdef CONFIG_CIFS_ACL
.get_acl = get_smb2_acl,
.get_acl_by_fid = get_smb2_acl_by_fid,
.set_acl = set_smb2_acl,
-#endif /* CIFS_ACL */
.next_header = smb2_next_header,
.ioctl_query_info = smb2_ioctl_query_info,
.make_node = smb2_make_node,
.query_all_EAs = smb2_query_eas,
.set_EA = smb2_set_ea,
#endif /* CIFS_XATTR */
-#ifdef CONFIG_CIFS_ACL
.get_acl = get_smb2_acl,
.get_acl_by_fid = get_smb2_acl_by_fid,
.set_acl = set_smb2_acl,
-#endif /* CIFS_ACL */
.next_header = smb2_next_header,
.ioctl_query_info = smb2_ioctl_query_info,
.make_node = smb2_make_node,
build_encrypt_ctxt(struct smb2_encryption_neg_context *pneg_ctxt)
{
pneg_ctxt->ContextType = SMB2_ENCRYPTION_CAPABILITIES;
- pneg_ctxt->DataLength = cpu_to_le16(4); /* Cipher Count + le16 cipher */
- pneg_ctxt->CipherCount = cpu_to_le16(1);
-/* pneg_ctxt->Ciphers[0] = SMB2_ENCRYPTION_AES128_GCM;*/ /* not supported yet */
- pneg_ctxt->Ciphers[0] = SMB2_ENCRYPTION_AES128_CCM;
+ pneg_ctxt->DataLength = cpu_to_le16(6); /* Cipher Count + two ciphers */
+ pneg_ctxt->CipherCount = cpu_to_le16(2);
+ pneg_ctxt->Ciphers[0] = SMB2_ENCRYPTION_AES128_GCM;
+ pneg_ctxt->Ciphers[1] = SMB2_ENCRYPTION_AES128_CCM;
+}
+
+static unsigned int
+build_netname_ctxt(struct smb2_netname_neg_context *pneg_ctxt, char *hostname)
+{
+ struct nls_table *cp = load_nls_default();
+
+ pneg_ctxt->ContextType = SMB2_NETNAME_NEGOTIATE_CONTEXT_ID;
+
+ /* copy up to max of first 100 bytes of server name to NetName field */
+ pneg_ctxt->DataLength = cpu_to_le16(2 +
+ (2 * cifs_strtoUTF16(pneg_ctxt->NetName, hostname, 100, cp)));
+ /* context size is DataLength + minimal smb2_neg_context */
+ return DIV_ROUND_UP(le16_to_cpu(pneg_ctxt->DataLength) +
+ sizeof(struct smb2_neg_context), 8) * 8;
}
static void
static void
assemble_neg_contexts(struct smb2_negotiate_req *req,
- unsigned int *total_len)
+ struct TCP_Server_Info *server, unsigned int *total_len)
{
char *pneg_ctxt = (char *)req;
unsigned int ctxt_len;
*total_len += ctxt_len;
pneg_ctxt += ctxt_len;
- build_compression_ctxt((struct smb2_compression_capabilities_context *)
+ if (server->compress_algorithm) {
+ build_compression_ctxt((struct smb2_compression_capabilities_context *)
pneg_ctxt);
- ctxt_len = DIV_ROUND_UP(
- sizeof(struct smb2_compression_capabilities_context), 8) * 8;
+ ctxt_len = DIV_ROUND_UP(
+ sizeof(struct smb2_compression_capabilities_context),
+ 8) * 8;
+ *total_len += ctxt_len;
+ pneg_ctxt += ctxt_len;
+ req->NegotiateContextCount = cpu_to_le16(5);
+ } else
+ req->NegotiateContextCount = cpu_to_le16(4);
+
+ ctxt_len = build_netname_ctxt((struct smb2_netname_neg_context *)pneg_ctxt,
+ server->hostname);
*total_len += ctxt_len;
pneg_ctxt += ctxt_len;
build_posix_ctxt((struct smb2_posix_neg_context *)pneg_ctxt);
*total_len += sizeof(struct smb2_posix_neg_context);
-
- req->NegotiateContextCount = cpu_to_le16(4);
}
static void decode_preauth_context(struct smb2_preauth_neg_context *ctxt)
if ((ses->server->vals->protocol_id == SMB311_PROT_ID) ||
(strcmp(ses->server->vals->version_string,
SMBDEFAULT_VERSION_STRING) == 0))
- assemble_neg_contexts(req, &total_len);
+ assemble_neg_contexts(req, server, &total_len);
}
iov[0].iov_base = (char *)req;
iov[0].iov_len = total_len;
return 0;
}
+static struct crt_query_id_ctxt *
+create_query_id_buf(void)
+{
+ struct crt_query_id_ctxt *buf;
+
+ buf = kzalloc(sizeof(struct crt_query_id_ctxt), GFP_KERNEL);
+ if (!buf)
+ return NULL;
+
+ buf->ccontext.DataOffset = cpu_to_le16(0);
+ buf->ccontext.DataLength = cpu_to_le32(0);
+ buf->ccontext.NameOffset = cpu_to_le16(offsetof
+ (struct crt_query_id_ctxt, Name));
+ buf->ccontext.NameLength = cpu_to_le16(4);
+ /* SMB2_CREATE_QUERY_ON_DISK_ID is "QFid" */
+ buf->Name[0] = 'Q';
+ buf->Name[1] = 'F';
+ buf->Name[2] = 'i';
+ buf->Name[3] = 'd';
+ return buf;
+}
+
+/* See MS-SMB2 2.2.13.2.9 */
+static int
+add_query_id_context(struct kvec *iov, unsigned int *num_iovec)
+{
+ struct smb2_create_req *req = iov[0].iov_base;
+ unsigned int num = *num_iovec;
+
+ iov[num].iov_base = create_query_id_buf();
+ if (iov[num].iov_base == NULL)
+ return -ENOMEM;
+ iov[num].iov_len = sizeof(struct crt_query_id_ctxt);
+ if (!req->CreateContextsOffset)
+ req->CreateContextsOffset = cpu_to_le32(
+ sizeof(struct smb2_create_req) +
+ iov[num - 1].iov_len);
+ le32_add_cpu(&req->CreateContextsLength, sizeof(struct crt_query_id_ctxt));
+ *num_iovec = num + 1;
+ return 0;
+}
+
static int
alloc_path_with_tree_prefix(__le16 **out_path, int *out_size, int *out_len,
const char *treename, const __le16 *path)
return rc;
}
+ if (n_iov > 2) {
+ struct create_context *ccontext =
+ (struct create_context *)iov[n_iov-1].iov_base;
+ ccontext->Next = cpu_to_le32(iov[n_iov-1].iov_len);
+ }
+ add_query_id_context(iov, &n_iov);
rqst->rq_nvec = n_iov;
return 0;
* indatalen is usually small at a couple of bytes max, so
* just allocate through generic pool
*/
- in_data_buf = kmalloc(indatalen, GFP_NOFS);
+ in_data_buf = kmemdup(in_data, indatalen, GFP_NOFS);
if (!in_data_buf) {
cifs_small_buf_release(req);
return -ENOMEM;
}
- memcpy(in_data_buf, in_data, indatalen);
}
req->CtlCode = cpu_to_le32(opcode);
__le16 StructureSize2; /* size of wct area (varies, request specific) */
} __packed;
-#define SMB3_AES128CMM_NONCE 11
+#define SMB3_AES128CCM_NONCE 11
#define SMB3_AES128GCM_NONCE 12
struct smb2_transform_hdr {
__u8 ErrorData[1]; /* variable length */
} __packed;
+#define SYMLINK_ERROR_TAG 0x4c4d5953
+
struct smb2_symlink_err_rsp {
__le32 SymLinkLength;
__le32 SymLinkErrorTag;
} __packed;
/* Dialects */
+#define SMB10_PROT_ID 0x0000 /* local only, not sent on wire w/CIFS negprot */
#define SMB20_PROT_ID 0x0202
#define SMB21_PROT_ID 0x0210
#define SMB30_PROT_ID 0x0300
__le16 DataLength;
__le32 Reserved;
__le16 CipherCount; /* AES-128-GCM and AES-128-CCM */
- __le16 Ciphers[1]; /* Ciphers[0] since only one used now */
+ __le16 Ciphers[2];
} __packed;
/* See MS-SMB2 2.2.3.1.3 */
* For smb2_netname_negotiate_context_id See MS-SMB2 2.2.3.1.4.
* Its struct simply contains NetName, an array of Unicode characters
*/
+struct smb2_netname_neg_context {
+ __le16 ContextType; /* 0x100 */
+ __le16 DataLength;
+ __le32 Reserved;
+ __le16 NetName[0]; /* hostname of target converted to UCS-2 */
+} __packed;
#define POSIX_CTXT_DATA_LEN 16
struct smb2_posix_neg_context {
#define SMB2_CREATE_DURABLE_HANDLE_REQUEST_V2 "DH2Q"
#define SMB2_CREATE_DURABLE_HANDLE_RECONNECT_V2 "DH2C"
#define SMB2_CREATE_APP_INSTANCE_ID 0x45BCA66AEFA7F74A9008FA462E144D74
+#define SMB2_CREATE_APP_INSTANCE_VERSION 0xB982D0B73B56074FA07B524A8116A010
#define SVHDX_OPEN_DEVICE_CONTEX 0x9CCBCF9E04C1E643980E158DA1F6EC83
#define SMB2_CREATE_TAG_POSIX 0x93AD25509CB411E7B42383DE968BCD7C
* [3] : durable context
* [4] : posix context
* [5] : time warp context
- * [6] : compound padding
+ * [6] : query id context
+ * [7] : compound padding
*/
-#define SMB2_CREATE_IOV_SIZE 7
+#define SMB2_CREATE_IOV_SIZE 8
struct smb2_create_req {
struct smb2_sync_hdr sync_hdr;
/*
* Maximum size of a SMB2_CREATE response is 64 (smb2 header) +
- * 88 (fixed part of create response) + 520 (path) + 150 (contexts) +
+ * 88 (fixed part of create response) + 520 (path) + 208 (contexts) +
* 2 bytes of padding.
*/
-#define MAX_SMB2_CREATE_RESPONSE_SIZE 824
+#define MAX_SMB2_CREATE_RESPONSE_SIZE 880
struct smb2_create_rsp {
struct smb2_sync_hdr sync_hdr;
__le32 Flags; /* see above DHANDLE_FLAG_PERSISTENT */
} __packed;
+/* See MS-SMB2 2.2.14.2.9 */
+struct on_disk_id {
+ __le64 DiskFileId;
+ __le64 VolumeId;
+ __u32 Reserved[4];
+} __packed;
+
/* See MS-SMB2 2.2.14.2.12 */
struct durable_reconnect_context_v2_rsp {
__le32 Timeout;
} __packed;
+/* See MS-SMB2 2.2.13.2.9 */
+struct crt_query_id_ctxt {
+ struct create_context ccontext;
+ __u8 Name[8];
+} __packed;
+
#define COPY_CHUNK_RES_KEY_SIZE 24
struct resume_key_req {
char ResumeKey[COPY_CHUNK_RES_KEY_SIZE];
struct crypto_aead *tfm;
if (!server->secmech.ccmaesencrypt) {
- tfm = crypto_alloc_aead("ccm(aes)", 0, 0);
+ if (server->cipher_type == SMB2_ENCRYPTION_AES128_GCM)
+ tfm = crypto_alloc_aead("gcm(aes)", 0, 0);
+ else
+ tfm = crypto_alloc_aead("ccm(aes)", 0, 0);
if (IS_ERR(tfm)) {
cifs_dbg(VFS, "%s: Failed to alloc encrypt aead\n",
__func__);
}
if (!server->secmech.ccmaesdecrypt) {
- tfm = crypto_alloc_aead("ccm(aes)", 0, 0);
+ if (server->cipher_type == SMB2_ENCRYPTION_AES128_GCM)
+ tfm = crypto_alloc_aead("gcm(aes)", 0, 0);
+ else
+ tfm = crypto_alloc_aead("ccm(aes)", 0, 0);
if (IS_ERR(tfm)) {
crypto_free_aead(server->secmech.ccmaesencrypt);
server->secmech.ccmaesencrypt = NULL;
};
unsigned int instance;
char *buf;
+ struct TCP_Server_Info *server;
optype = flags & CIFS_OP_MASK;
return -EIO;
}
- if (ses->server->tcpStatus == CifsExiting)
+ server = ses->server;
+ if (server->tcpStatus == CifsExiting)
return -ENOENT;
/*
* other requests.
* This can be handled by the eventual session reconnect.
*/
- rc = wait_for_compound_request(ses->server, num_rqst, flags,
+ rc = wait_for_compound_request(server, num_rqst, flags,
&instance);
if (rc)
return rc;
* of smb data.
*/
- mutex_lock(&ses->server->srv_mutex);
+ mutex_lock(&server->srv_mutex);
/*
* All the parts of the compound chain belong obtained credits from the
* we obtained credits and return -EAGAIN in such cases to let callers
* handle it.
*/
- if (instance != ses->server->reconnect_instance) {
- mutex_unlock(&ses->server->srv_mutex);
+ if (instance != server->reconnect_instance) {
+ mutex_unlock(&server->srv_mutex);
for (j = 0; j < num_rqst; j++)
- add_credits(ses->server, &credits[j], optype);
+ add_credits(server, &credits[j], optype);
return -EAGAIN;
}
for (i = 0; i < num_rqst; i++) {
- midQ[i] = ses->server->ops->setup_request(ses, &rqst[i]);
+ midQ[i] = server->ops->setup_request(ses, &rqst[i]);
if (IS_ERR(midQ[i])) {
- revert_current_mid(ses->server, i);
+ revert_current_mid(server, i);
for (j = 0; j < i; j++)
cifs_delete_mid(midQ[j]);
- mutex_unlock(&ses->server->srv_mutex);
+ mutex_unlock(&server->srv_mutex);
/* Update # of requests on wire to server */
for (j = 0; j < num_rqst; j++)
- add_credits(ses->server, &credits[j], optype);
+ add_credits(server, &credits[j], optype);
return PTR_ERR(midQ[i]);
}
else
midQ[i]->callback = cifs_compound_last_callback;
}
- cifs_in_send_inc(ses->server);
- rc = smb_send_rqst(ses->server, num_rqst, rqst, flags);
- cifs_in_send_dec(ses->server);
+ cifs_in_send_inc(server);
+ rc = smb_send_rqst(server, num_rqst, rqst, flags);
+ cifs_in_send_dec(server);
for (i = 0; i < num_rqst; i++)
cifs_save_when_sent(midQ[i]);
if (rc < 0) {
- revert_current_mid(ses->server, num_rqst);
- ses->server->sequence_number -= 2;
+ revert_current_mid(server, num_rqst);
+ server->sequence_number -= 2;
}
- mutex_unlock(&ses->server->srv_mutex);
+ mutex_unlock(&server->srv_mutex);
/*
* If sending failed for some reason or it is an oplock break that we
*/
if (rc < 0 || (flags & CIFS_NO_SRV_RSP)) {
for (i = 0; i < num_rqst; i++)
- add_credits(ses->server, &credits[i], optype);
+ add_credits(server, &credits[i], optype);
goto out;
}
rqst[0].rq_nvec);
for (i = 0; i < num_rqst; i++) {
- rc = wait_for_response(ses->server, midQ[i]);
+ rc = wait_for_response(server, midQ[i]);
if (rc != 0)
break;
}
for (; i < num_rqst; i++) {
cifs_dbg(VFS, "Cancelling wait for mid %llu cmd: %d\n",
midQ[i]->mid, le16_to_cpu(midQ[i]->command));
- send_cancel(ses->server, &rqst[i], midQ[i]);
+ send_cancel(server, &rqst[i], midQ[i]);
spin_lock(&GlobalMid_Lock);
if (midQ[i]->mid_state == MID_REQUEST_SUBMITTED) {
midQ[i]->mid_flags |= MID_WAIT_CANCELLED;
if (rc < 0)
goto out;
- rc = cifs_sync_mid_result(midQ[i], ses->server);
+ rc = cifs_sync_mid_result(midQ[i], server);
if (rc != 0) {
/* mark this mid as cancelled to not free it below */
cancelled_mid[i] = true;
buf = (char *)midQ[i]->resp_buf;
resp_iov[i].iov_base = buf;
resp_iov[i].iov_len = midQ[i]->resp_buf_size +
- ses->server->vals->header_preamble_size;
+ server->vals->header_preamble_size;
if (midQ[i]->large_buf)
resp_buf_type[i] = CIFS_LARGE_BUFFER;
else
resp_buf_type[i] = CIFS_SMALL_BUFFER;
- rc = ses->server->ops->check_receive(midQ[i], ses->server,
+ rc = server->ops->check_receive(midQ[i], server,
flags & CIFS_LOG_ERROR);
/* mark it so buf will not be freed by cifs_delete_mid */
break;
case XATTR_CIFS_ACL: {
-#ifdef CONFIG_CIFS_ACL
struct cifs_ntsd *pacl;
if (!value)
CIFS_I(inode)->time = 0;
kfree(pacl);
}
-#endif /* CONFIG_CIFS_ACL */
break;
}
break;
case XATTR_CIFS_ACL: {
-#ifdef CONFIG_CIFS_ACL
u32 acllen;
struct cifs_ntsd *pacl;
rc = acllen;
kfree(pacl);
}
-#endif /* CONFIG_CIFS_ACL */
break;
}
obj-$(CONFIG_CODA_FS) += coda.o
coda-objs := psdev.o cache.o cnode.o inode.o dir.o file.o upcall.o \
- coda_linux.o symlink.o pioctl.o sysctl.o
+ coda_linux.o symlink.o pioctl.o
+coda-$(CONFIG_SYSCTL) += sysctl.o
# If you want debugging output, please uncomment the following line.
#include <linux/spinlock.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_cache.h"
#include <linux/time.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
#include <linux/pagemap.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
static inline int coda_fideq(struct CodaFid *fid1, struct CodaFid *fid2)
struct inode *inode;
unsigned long hash = coda_f2i(fid);
- if ( !sb ) {
- pr_warn("%s: no sb!\n", __func__);
- return NULL;
- }
-
inode = ilookup5(sb, hash, coda_test_inode, fid);
if ( !inode )
return NULL;
return inode;
}
+struct coda_file_info *coda_ftoc(struct file *file)
+{
+ struct coda_file_info *cfi = file->private_data;
+
+ BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+
+ return cfi;
+
+}
+
/* the CONTROL inode is made without asking attributes from Venus */
struct inode *coda_cnode_makectl(struct super_block *sb)
{
int cfi_magic; /* magic number */
struct file *cfi_container; /* container file for this cnode */
unsigned int cfi_mapcount; /* nr of times this file is mapped */
+ bool cfi_access_intent; /* is access intent supported */
};
-#define CODA_FTOC(file) ((struct coda_file_info *)((file)->private_data))
-
/* flags */
#define C_VATTR 0x1 /* Validity of vattr in inode */
#define C_FLUSH 0x2 /* used after a flush */
struct inode *coda_iget(struct super_block *sb, struct CodaFid *fid, struct coda_vattr *attr);
struct inode *coda_cnode_makectl(struct super_block *sb);
struct inode *coda_fid_to_inode(struct CodaFid *fid, struct super_block *sb);
+struct coda_file_info *coda_ftoc(struct file *file);
void coda_replace_fid(struct inode *, struct CodaFid *, struct CodaFid *);
#endif
void coda_destroy_inodecache(void);
int __init coda_init_inodecache(void);
int coda_fsync(struct file *coda_file, loff_t start, loff_t end, int datasync);
+
+#ifdef CONFIG_SYSCTL
void coda_sysctl_init(void);
void coda_sysctl_clean(void);
+#else
+static inline void coda_sysctl_init(void)
+{
+}
+static inline void coda_sysctl_clean(void)
+{
+}
+#endif
#endif /* _CODA_INT_ */
#include <linux/string.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
/* initialize the debugging variables */
return coda_flags;
}
+static struct timespec64 coda_to_timespec64(struct coda_timespec ts)
+{
+ struct timespec64 ts64 = {
+ .tv_sec = ts.tv_sec,
+ .tv_nsec = ts.tv_nsec,
+ };
+
+ return ts64;
+}
+
+static struct coda_timespec timespec64_to_coda(struct timespec64 ts64)
+{
+ struct coda_timespec ts = {
+ .tv_sec = ts64.tv_sec,
+ .tv_nsec = ts64.tv_nsec,
+ };
+
+ return ts;
+}
/* utility functions below */
void coda_vattr_to_iattr(struct inode *inode, struct coda_vattr *attr)
if (attr->va_size != -1)
inode->i_blocks = (attr->va_size + 511) >> 9;
if (attr->va_atime.tv_sec != -1)
- inode->i_atime = timespec_to_timespec64(attr->va_atime);
+ inode->i_atime = coda_to_timespec64(attr->va_atime);
if (attr->va_mtime.tv_sec != -1)
- inode->i_mtime = timespec_to_timespec64(attr->va_mtime);
+ inode->i_mtime = coda_to_timespec64(attr->va_mtime);
if (attr->va_ctime.tv_sec != -1)
- inode->i_ctime = timespec_to_timespec64(attr->va_ctime);
+ inode->i_ctime = coda_to_timespec64(attr->va_ctime);
}
vattr->va_uid = (vuid_t) -1;
vattr->va_gid = (vgid_t) -1;
vattr->va_size = (off_t) -1;
- vattr->va_atime.tv_sec = (time_t) -1;
- vattr->va_atime.tv_nsec = (time_t) -1;
- vattr->va_mtime.tv_sec = (time_t) -1;
- vattr->va_mtime.tv_nsec = (time_t) -1;
- vattr->va_ctime.tv_sec = (time_t) -1;
- vattr->va_ctime.tv_nsec = (time_t) -1;
+ vattr->va_atime.tv_sec = (int64_t) -1;
+ vattr->va_atime.tv_nsec = (long) -1;
+ vattr->va_mtime.tv_sec = (int64_t) -1;
+ vattr->va_mtime.tv_nsec = (long) -1;
+ vattr->va_ctime.tv_sec = (int64_t) -1;
+ vattr->va_ctime.tv_nsec = (long) -1;
vattr->va_type = C_VNON;
vattr->va_fileid = -1;
vattr->va_gen = -1;
vattr->va_size = iattr->ia_size;
}
if ( valid & ATTR_ATIME ) {
- vattr->va_atime = timespec64_to_timespec(iattr->ia_atime);
+ vattr->va_atime = timespec64_to_coda(iattr->ia_atime);
}
if ( valid & ATTR_MTIME ) {
- vattr->va_mtime = timespec64_to_timespec(iattr->ia_mtime);
+ vattr->va_mtime = timespec64_to_coda(iattr->ia_mtime);
}
if ( valid & ATTR_CTIME ) {
- vattr->va_ctime = timespec64_to_timespec(iattr->ia_ctime);
+ vattr->va_ctime = timespec64_to_coda(iattr->ia_ctime);
}
}
void coda_iattr_to_vattr(struct iattr *, struct coda_vattr *);
unsigned short coda_flags_to_cflags(unsigned short);
-/* sysctl.h */
-void coda_sysctl_init(void);
-void coda_sysctl_clean(void);
-
-#define CODA_ALLOC(ptr, cast, size) do { \
- if (size < PAGE_SIZE) \
- ptr = kzalloc((unsigned long) size, GFP_KERNEL); \
- else \
- ptr = (cast)vzalloc((unsigned long) size); \
- if (!ptr) \
- pr_warn("kernel malloc returns 0 at %s:%d\n", __FILE__, __LINE__); \
-} while (0)
-
-
-#define CODA_FREE(ptr, size) kvfree((ptr))
-
/* inode to cnode access functions */
static inline struct coda_inode_info *ITOC(struct inode *inode)
#define __CODA_PSDEV_H
#include <linux/backing-dev.h>
+#include <linux/magic.h>
#include <linux/mutex.h>
-#include <uapi/linux/coda_psdev.h>
+
+#define CODA_PSDEV_MAJOR 67
+#define MAX_CODADEVS 5 /* how many do we allow */
struct kstatfs;
+/* messages between coda filesystem in kernel and Venus */
+struct upc_req {
+ struct list_head uc_chain;
+ caddr_t uc_data;
+ u_short uc_flags;
+ u_short uc_inSize; /* Size is at most 5000 bytes */
+ u_short uc_outSize;
+ u_short uc_opcode; /* copied from data to save lookup */
+ int uc_unique;
+ wait_queue_head_t uc_sleep; /* process' wait queue */
+};
+
+#define CODA_REQ_ASYNC 0x1
+#define CODA_REQ_READ 0x2
+#define CODA_REQ_WRITE 0x4
+#define CODA_REQ_ABORT 0x8
+
/* communication pending/processing queues */
struct venus_comm {
u_long vc_seq;
struct mutex vc_mutex;
};
-
static inline struct venus_comm *coda_vcp(struct super_block *sb)
{
return (struct venus_comm *)((sb)->s_fs_info);
int venus_getattr(struct super_block *sb, struct CodaFid *fid,
struct coda_vattr *attr);
int venus_setattr(struct super_block *, struct CodaFid *, struct coda_vattr *);
-int venus_lookup(struct super_block *sb, struct CodaFid *fid,
- const char *name, int length, int *type,
+int venus_lookup(struct super_block *sb, struct CodaFid *fid,
+ const char *name, int length, int *type,
struct CodaFid *resfid);
int venus_close(struct super_block *sb, struct CodaFid *fid, int flags,
kuid_t uid);
int venus_open(struct super_block *sb, struct CodaFid *fid, int flags,
struct file **f);
-int venus_mkdir(struct super_block *sb, struct CodaFid *dirfid,
- const char *name, int length,
+int venus_mkdir(struct super_block *sb, struct CodaFid *dirfid,
+ const char *name, int length,
struct CodaFid *newfid, struct coda_vattr *attrs);
-int venus_create(struct super_block *sb, struct CodaFid *dirfid,
+int venus_create(struct super_block *sb, struct CodaFid *dirfid,
const char *name, int length, int excl, int mode,
- struct CodaFid *newfid, struct coda_vattr *attrs) ;
-int venus_rmdir(struct super_block *sb, struct CodaFid *dirfid,
+ struct CodaFid *newfid, struct coda_vattr *attrs);
+int venus_rmdir(struct super_block *sb, struct CodaFid *dirfid,
const char *name, int length);
-int venus_remove(struct super_block *sb, struct CodaFid *dirfid,
+int venus_remove(struct super_block *sb, struct CodaFid *dirfid,
const char *name, int length);
-int venus_readlink(struct super_block *sb, struct CodaFid *fid,
+int venus_readlink(struct super_block *sb, struct CodaFid *fid,
char *buffer, int *length);
-int venus_rename(struct super_block *, struct CodaFid *new_fid,
- struct CodaFid *old_fid, size_t old_length,
- size_t new_length, const char *old_name,
+int venus_rename(struct super_block *sb, struct CodaFid *new_fid,
+ struct CodaFid *old_fid, size_t old_length,
+ size_t new_length, const char *old_name,
const char *new_name);
-int venus_link(struct super_block *sb, struct CodaFid *fid,
+int venus_link(struct super_block *sb, struct CodaFid *fid,
struct CodaFid *dirfid, const char *name, int len );
int venus_symlink(struct super_block *sb, struct CodaFid *fid,
const char *name, int len, const char *symname, int symlen);
int venus_access(struct super_block *sb, struct CodaFid *fid, int mask);
int venus_pioctl(struct super_block *sb, struct CodaFid *fid,
unsigned int cmd, struct PioctlData *data);
-int coda_downcall(struct venus_comm *vcp, int opcode, union outputArgs *out);
+int coda_downcall(struct venus_comm *vcp, int opcode, union outputArgs *out,
+ size_t nbytes);
int venus_fsync(struct super_block *sb, struct CodaFid *fid);
int venus_statfs(struct dentry *dentry, struct kstatfs *sfs);
+int venus_access_intent(struct super_block *sb, struct CodaFid *fid,
+ bool *access_intent_supported,
+ size_t count, loff_t ppos, int type);
/*
* Statistics
#include <linux/uaccess.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_cache.h"
int type = 0;
if (length > CODA_MAXNAMLEN) {
- pr_err("name too long: lookup, %s (%*s)\n",
- coda_i2s(dir), (int)length, name);
+ pr_err("name too long: lookup, %s %zu\n",
+ coda_i2s(dir), length);
return ERR_PTR(-ENAMETOOLONG);
}
ino_t ino;
int ret;
- cfi = CODA_FTOC(coda_file);
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ cfi = coda_ftoc(coda_file);
host_file = cfi->cfi_container;
cii = ITOC(file_inode(coda_file));
struct file *host_file;
int ret;
- cfi = CODA_FTOC(coda_file);
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ cfi = coda_ftoc(coda_file);
host_file = cfi->cfi_container;
if (host_file->f_op->iterate || host_file->f_op->iterate_shared) {
#include <linux/string.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
+#include <linux/uio.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
-
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_int.h"
+struct coda_vm_ops {
+ atomic_t refcnt;
+ struct file *coda_file;
+ const struct vm_operations_struct *host_vm_ops;
+ struct vm_operations_struct vm_ops;
+};
+
static ssize_t
coda_file_read_iter(struct kiocb *iocb, struct iov_iter *to)
{
struct file *coda_file = iocb->ki_filp;
- struct coda_file_info *cfi = CODA_FTOC(coda_file);
+ struct inode *coda_inode = file_inode(coda_file);
+ struct coda_file_info *cfi = coda_ftoc(coda_file);
+ loff_t ki_pos = iocb->ki_pos;
+ size_t count = iov_iter_count(to);
+ ssize_t ret;
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode),
+ &cfi->cfi_access_intent,
+ count, ki_pos, CODA_ACCESS_TYPE_READ);
+ if (ret)
+ goto finish_read;
- return vfs_iter_read(cfi->cfi_container, to, &iocb->ki_pos, 0);
+ ret = vfs_iter_read(cfi->cfi_container, to, &iocb->ki_pos, 0);
+
+finish_read:
+ venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode),
+ &cfi->cfi_access_intent,
+ count, ki_pos, CODA_ACCESS_TYPE_READ_FINISH);
+ return ret;
}
static ssize_t
{
struct file *coda_file = iocb->ki_filp;
struct inode *coda_inode = file_inode(coda_file);
- struct coda_file_info *cfi = CODA_FTOC(coda_file);
- struct file *host_file;
+ struct coda_file_info *cfi = coda_ftoc(coda_file);
+ struct file *host_file = cfi->cfi_container;
+ loff_t ki_pos = iocb->ki_pos;
+ size_t count = iov_iter_count(to);
ssize_t ret;
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode),
+ &cfi->cfi_access_intent,
+ count, ki_pos, CODA_ACCESS_TYPE_WRITE);
+ if (ret)
+ goto finish_write;
- host_file = cfi->cfi_container;
file_start_write(host_file);
inode_lock(coda_inode);
ret = vfs_iter_write(cfi->cfi_container, to, &iocb->ki_pos, 0);
coda_inode->i_mtime = coda_inode->i_ctime = current_time(coda_inode);
inode_unlock(coda_inode);
file_end_write(host_file);
+
+finish_write:
+ venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode),
+ &cfi->cfi_access_intent,
+ count, ki_pos, CODA_ACCESS_TYPE_WRITE_FINISH);
return ret;
}
+static void
+coda_vm_open(struct vm_area_struct *vma)
+{
+ struct coda_vm_ops *cvm_ops =
+ container_of(vma->vm_ops, struct coda_vm_ops, vm_ops);
+
+ atomic_inc(&cvm_ops->refcnt);
+
+ if (cvm_ops->host_vm_ops && cvm_ops->host_vm_ops->open)
+ cvm_ops->host_vm_ops->open(vma);
+}
+
+static void
+coda_vm_close(struct vm_area_struct *vma)
+{
+ struct coda_vm_ops *cvm_ops =
+ container_of(vma->vm_ops, struct coda_vm_ops, vm_ops);
+
+ if (cvm_ops->host_vm_ops && cvm_ops->host_vm_ops->close)
+ cvm_ops->host_vm_ops->close(vma);
+
+ if (atomic_dec_and_test(&cvm_ops->refcnt)) {
+ vma->vm_ops = cvm_ops->host_vm_ops;
+ fput(cvm_ops->coda_file);
+ kfree(cvm_ops);
+ }
+}
+
static int
coda_file_mmap(struct file *coda_file, struct vm_area_struct *vma)
{
- struct coda_file_info *cfi;
+ struct inode *coda_inode = file_inode(coda_file);
+ struct coda_file_info *cfi = coda_ftoc(coda_file);
+ struct file *host_file = cfi->cfi_container;
+ struct inode *host_inode = file_inode(host_file);
struct coda_inode_info *cii;
- struct file *host_file;
- struct inode *coda_inode, *host_inode;
-
- cfi = CODA_FTOC(coda_file);
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
- host_file = cfi->cfi_container;
+ struct coda_vm_ops *cvm_ops;
+ loff_t ppos;
+ size_t count;
+ int ret;
if (!host_file->f_op->mmap)
return -ENODEV;
- coda_inode = file_inode(coda_file);
- host_inode = file_inode(host_file);
+ if (WARN_ON(coda_file != vma->vm_file))
+ return -EIO;
+
+ count = vma->vm_end - vma->vm_start;
+ ppos = vma->vm_pgoff * PAGE_SIZE;
+
+ ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode),
+ &cfi->cfi_access_intent,
+ count, ppos, CODA_ACCESS_TYPE_MMAP);
+ if (ret)
+ return ret;
+
+ cvm_ops = kmalloc(sizeof(struct coda_vm_ops), GFP_KERNEL);
+ if (!cvm_ops)
+ return -ENOMEM;
cii = ITOC(coda_inode);
spin_lock(&cii->c_lock);
* the container file on us! */
else if (coda_inode->i_mapping != host_inode->i_mapping) {
spin_unlock(&cii->c_lock);
+ kfree(cvm_ops);
return -EBUSY;
}
cfi->cfi_mapcount++;
spin_unlock(&cii->c_lock);
- return call_mmap(host_file, vma);
+ vma->vm_file = get_file(host_file);
+ ret = call_mmap(vma->vm_file, vma);
+
+ if (ret) {
+ /* if call_mmap fails, our caller will put coda_file so we
+ * should drop the reference to the host_file that we got.
+ */
+ fput(host_file);
+ kfree(cvm_ops);
+ } else {
+ /* here we add redirects for the open/close vm_operations */
+ cvm_ops->host_vm_ops = vma->vm_ops;
+ if (vma->vm_ops)
+ cvm_ops->vm_ops = *vma->vm_ops;
+
+ cvm_ops->vm_ops.open = coda_vm_open;
+ cvm_ops->vm_ops.close = coda_vm_close;
+ cvm_ops->coda_file = coda_file;
+ atomic_set(&cvm_ops->refcnt, 1);
+
+ vma->vm_ops = &cvm_ops->vm_ops;
+ }
+ return ret;
}
int coda_open(struct inode *coda_inode, struct file *coda_file)
cfi->cfi_magic = CODA_MAGIC;
cfi->cfi_mapcount = 0;
cfi->cfi_container = host_file;
+ /* assume access intents are supported unless we hear otherwise */
+ cfi->cfi_access_intent = true;
BUG_ON(coda_file->private_data != NULL);
coda_file->private_data = cfi;
struct inode *host_inode;
int err;
- cfi = CODA_FTOC(coda_file);
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ cfi = coda_ftoc(coda_file);
err = venus_close(coda_inode->i_sb, coda_i2f(coda_inode),
coda_flags, coda_file->f_cred->fsuid);
return err;
inode_lock(coda_inode);
- cfi = CODA_FTOC(coda_file);
- BUG_ON(!cfi || cfi->cfi_magic != CODA_MAGIC);
+ cfi = coda_ftoc(coda_file);
host_file = cfi->cfi_container;
err = vfs_fsync(host_file, datasync);
.fsync = coda_fsync,
.splice_read = generic_file_splice_read,
};
-
#include <linux/vmalloc.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_cache.h"
vcp->vc_sb = NULL;
sb->s_fs_info = NULL;
mutex_unlock(&vcp->vc_mutex);
+ mutex_destroy(&vcp->vc_mutex);
pr_info("Bye bye.\n");
}
#include <linux/uaccess.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
-
+#include "coda_psdev.h"
#include "coda_linux.h"
/* pioctl ops */
#include <linux/uaccess.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
-
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_int.h"
ssize_t retval = 0, count = 0;
int error;
+ /* make sure there is enough to copy out the (opcode, unique) values */
+ if (nbytes < (2 * sizeof(u_int32_t)))
+ return -EINVAL;
+
/* Peek at the opcode, uniquefier */
- if (copy_from_user(&hdr, buf, 2 * sizeof(u_long)))
+ if (copy_from_user(&hdr, buf, 2 * sizeof(u_int32_t)))
return -EFAULT;
if (DOWNCALL(hdr.opcode)) {
hdr.opcode, hdr.unique);
nbytes = size;
}
- CODA_ALLOC(dcbuf, union outputArgs *, nbytes);
+ dcbuf = kvmalloc(nbytes, GFP_KERNEL);
+ if (!dcbuf) {
+ retval = -ENOMEM;
+ goto out;
+ }
if (copy_from_user(dcbuf, buf, nbytes)) {
- CODA_FREE(dcbuf, nbytes);
+ kvfree(dcbuf);
retval = -EFAULT;
goto out;
}
/* what downcall errors does Venus handle ? */
- error = coda_downcall(vcp, hdr.opcode, dcbuf);
+ error = coda_downcall(vcp, hdr.opcode, dcbuf, nbytes);
- CODA_FREE(dcbuf, nbytes);
+ kvfree(dcbuf);
if (error) {
pr_warn("%s: coda_downcall error: %d\n",
__func__, error);
if (req->uc_opcode == CODA_OPEN_BY_FD) {
struct coda_open_by_fd_out *outp =
(struct coda_open_by_fd_out *)req->uc_data;
- if (!outp->oh.result)
+ if (!outp->oh.result) {
outp->fh = fget(outp->fd);
+ if (!outp->fh)
+ return -EBADF;
+ }
}
wake_up(&req->uc_sleep);
goto out;
}
- CODA_FREE(req->uc_data, sizeof(struct coda_in_hdr));
+ kvfree(req->uc_data);
kfree(req);
out:
mutex_unlock(&vcp->vc_mutex);
/* Async requests need to be freed here */
if (req->uc_flags & CODA_REQ_ASYNC) {
- CODA_FREE(req->uc_data, sizeof(struct coda_in_hdr));
+ kvfree(req->uc_data);
kfree(req);
continue;
}
.llseek = noop_llseek,
};
-static int init_coda_psdev(void)
+static int __init init_coda_psdev(void)
{
int i, err = 0;
if (register_chrdev(CODA_PSDEV_MAJOR, "coda", &coda_psdev_fops)) {
pr_err("%s: unable to get major %d\n",
__func__, CODA_PSDEV_MAJOR);
- return -EIO;
+ return -EIO;
}
coda_psdev_class = class_create(THIS_MODULE, "coda");
if (IS_ERR(coda_psdev_class)) {
MODULE_DESCRIPTION("Coda Distributed File System VFS interface");
MODULE_ALIAS_CHARDEV_MAJOR(CODA_PSDEV_MAJOR);
MODULE_LICENSE("GPL");
-MODULE_VERSION("6.6");
+MODULE_VERSION("7.0");
static int __init init_coda(void)
{
#include <linux/pagemap.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
-
+#include "coda_psdev.h"
#include "coda_linux.h"
static int coda_symlink_filler(struct file *file, struct page *page)
#include "coda_int.h"
-#ifdef CONFIG_SYSCTL
static struct ctl_table_header *fs_table_header;
static struct ctl_table coda_table[] = {
fs_table_header = NULL;
}
}
-
-#else
-void coda_sysctl_init(void)
-{
-}
-
-void coda_sysctl_clean(void)
-{
-}
-#endif
#include <linux/vfs.h>
#include <linux/coda.h>
-#include <linux/coda_psdev.h>
+#include "coda_psdev.h"
#include "coda_linux.h"
#include "coda_cache.h"
{
union inputArgs *inp;
- CODA_ALLOC(inp, union inputArgs *, size);
+ inp = kvzalloc(size, GFP_KERNEL);
if (!inp)
return ERR_PTR(-ENOMEM);
if (!error)
*fidp = outp->coda_root.VFid;
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
if (!error)
*attr = outp->coda_getattr.attr;
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
*type = outp->coda_lookup.vtype;
}
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
if (!error)
*fh = outp->coda_open_by_fd.fh;
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
*newfid = outp->coda_mkdir.VFid;
}
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
*newfid = outp->coda_create.VFid;
}
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
*(buffer + retlen) = '\0';
}
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
inp->coda_fsync.VFid = *fid;
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
error = coda_upcall(coda_vcp(sb), insize, &outsize, inp);
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
}
exit:
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
union outputArgs *outp;
int insize, outsize, error;
- insize = max_t(unsigned int, INSIZE(statfs), OUTSIZE(statfs));
+ insize = SIZE(statfs);
UPARG(CODA_STATFS);
error = coda_upcall(coda_vcp(dentry->d_sb), insize, &outsize, inp);
sfs->f_ffree = outp->coda_statfs.stat.f_ffree;
}
- CODA_FREE(inp, insize);
+ kvfree(inp);
return error;
}
+int venus_access_intent(struct super_block *sb, struct CodaFid *fid,
+ bool *access_intent_supported,
+ size_t count, loff_t ppos, int type)
+{
+ union inputArgs *inp;
+ union outputArgs *outp;
+ int insize, outsize, error;
+ bool finalizer =
+ type == CODA_ACCESS_TYPE_READ_FINISH ||
+ type == CODA_ACCESS_TYPE_WRITE_FINISH;
+
+ if (!*access_intent_supported && !finalizer)
+ return 0;
+
+ insize = SIZE(access_intent);
+ UPARG(CODA_ACCESS_INTENT);
+
+ inp->coda_access_intent.VFid = *fid;
+ inp->coda_access_intent.count = count;
+ inp->coda_access_intent.pos = ppos;
+ inp->coda_access_intent.type = type;
+
+ error = coda_upcall(coda_vcp(sb), insize,
+ finalizer ? NULL : &outsize, inp);
+
+ /*
+ * we have to free the request buffer for synchronous upcalls
+ * or when asynchronous upcalls fail, but not when asynchronous
+ * upcalls succeed
+ */
+ if (!finalizer || error)
+ kvfree(inp);
+
+ /* Chunked access is not supported or an old Coda client */
+ if (error == -EOPNOTSUPP) {
+ *access_intent_supported = false;
+ error = 0;
+ }
+ return error;
+}
+
/*
* coda_upcall and coda_downcall routines.
*/
* has seen them,
* - CODA_CLOSE or CODA_RELEASE upcall (to avoid reference count problems)
* - CODA_STORE (to avoid data loss)
+ * - CODA_ACCESS_INTENT (to avoid reference count problems)
*/
#define CODA_INTERRUPTIBLE(r) (!coda_hard && \
(((r)->uc_opcode != CODA_CLOSE && \
(r)->uc_opcode != CODA_STORE && \
+ (r)->uc_opcode != CODA_ACCESS_INTENT && \
(r)->uc_opcode != CODA_RELEASE) || \
(r)->uc_flags & CODA_REQ_READ))
goto exit;
}
+ buffer->ih.unique = ++vcp->vc_seq;
+
req->uc_data = (void *)buffer;
- req->uc_flags = 0;
+ req->uc_flags = outSize ? 0 : CODA_REQ_ASYNC;
req->uc_inSize = inSize;
- req->uc_outSize = *outSize ? *outSize : inSize;
- req->uc_opcode = ((union inputArgs *)buffer)->ih.opcode;
- req->uc_unique = ++vcp->vc_seq;
+ req->uc_outSize = (outSize && *outSize) ? *outSize : inSize;
+ req->uc_opcode = buffer->ih.opcode;
+ req->uc_unique = buffer->ih.unique;
init_waitqueue_head(&req->uc_sleep);
- /* Fill in the common input args. */
- ((union inputArgs *)buffer)->ih.unique = req->uc_unique;
-
/* Append msg to pending queue and poke Venus. */
list_add_tail(&req->uc_chain, &vcp->vc_pending);
-
wake_up_interruptible(&vcp->vc_waitq);
+
+ if (req->uc_flags & CODA_REQ_ASYNC) {
+ mutex_unlock(&vcp->vc_mutex);
+ return 0;
+ }
+
/* We can be interrupted while we wait for Venus to process
* our request. If the interrupt occurs before Venus has read
* the request, we dequeue and return. If it occurs after the
sig_req = kmalloc(sizeof(struct upc_req), GFP_KERNEL);
if (!sig_req) goto exit;
- CODA_ALLOC((sig_req->uc_data), char *, sizeof(struct coda_in_hdr));
- if (!sig_req->uc_data) {
+ sig_inputArgs = kvzalloc(sizeof(struct coda_in_hdr), GFP_KERNEL);
+ if (!sig_inputArgs) {
kfree(sig_req);
goto exit;
}
error = -EINTR;
- sig_inputArgs = (union inputArgs *)sig_req->uc_data;
sig_inputArgs->ih.opcode = CODA_SIGNAL;
sig_inputArgs->ih.unique = req->uc_unique;
sig_req->uc_flags = CODA_REQ_ASYNC;
sig_req->uc_opcode = sig_inputArgs->ih.opcode;
sig_req->uc_unique = sig_inputArgs->ih.unique;
+ sig_req->uc_data = (void *)sig_inputArgs;
sig_req->uc_inSize = sizeof(struct coda_in_hdr);
sig_req->uc_outSize = sizeof(struct coda_in_hdr);
*
* CODA_REPLACE -- replace one CodaFid with another throughout the name cache */
-int coda_downcall(struct venus_comm *vcp, int opcode, union outputArgs *out)
+int coda_downcall(struct venus_comm *vcp, int opcode, union outputArgs *out,
+ size_t nbytes)
{
struct inode *inode = NULL;
struct CodaFid *fid = NULL, *newfid;
struct super_block *sb;
+ /*
+ * Make sure we have received enough data from the cache
+ * manager to populate the necessary fields in the buffer
+ */
+ switch (opcode) {
+ case CODA_PURGEUSER:
+ if (nbytes < sizeof(struct coda_purgeuser_out))
+ return -EINVAL;
+ break;
+
+ case CODA_ZAPDIR:
+ if (nbytes < sizeof(struct coda_zapdir_out))
+ return -EINVAL;
+ break;
+
+ case CODA_ZAPFILE:
+ if (nbytes < sizeof(struct coda_zapfile_out))
+ return -EINVAL;
+ break;
+
+ case CODA_PURGEFID:
+ if (nbytes < sizeof(struct coda_purgefid_out))
+ return -EINVAL;
+ break;
+
+ case CODA_REPLACE:
+ if (nbytes < sizeof(struct coda_replace_out))
+ return -EINVAL;
+ break;
+ }
+
/* Handle invalidation requests. */
mutex_lock(&vcp->vc_mutex);
sb = vcp->vc_sb;
iput(inode);
return 0;
}
-
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/mount.h>
+#include <linux/fs_context.h>
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/slab.h>
.s_iattr = NULL,
};
-static int configfs_fill_super(struct super_block *sb, void *data, int silent)
+static int configfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *inode;
struct dentry *root;
return 0;
}
-static struct dentry *configfs_do_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int configfs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, configfs_fill_super);
+ return get_tree_single(fc, configfs_fill_super);
+}
+
+static const struct fs_context_operations configfs_context_ops = {
+ .get_tree = configfs_get_tree,
+};
+
+static int configfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &configfs_context_ops;
+ return 0;
}
static struct file_system_type configfs_fs_type = {
.owner = THIS_MODULE,
.name = "configfs",
- .mount = configfs_do_mount,
+ .init_fs_context = configfs_init_fs_context,
.kill_sb = kill_litter_super,
};
MODULE_ALIAS_FS("configfs");
end = ERR_PTR(-ENAMETOOLONG);
return end;
}
-EXPORT_SYMBOL(simple_dname);
/*
* Write full pathname from the root of the filesystem into the buffer.
#include <linux/mmu_notifier.h>
#include <linux/iomap.h>
#include <asm/pgalloc.h>
-#include "internal.h"
#define CREATE_TRACE_POINTS
#include <trace/events/fs_dax.h>
}
/*
+ * true if the entry that was found is of a smaller order than the entry
+ * we were looking for
+ */
+static bool dax_is_conflict(void *entry)
+{
+ return entry == XA_RETRY_ENTRY;
+}
+
+/*
* DAX page cache entry locking
*/
struct exceptional_entry_key {
* Look up entry in page cache, wait for it to become unlocked if it
* is a DAX entry and return it. The caller must subsequently call
* put_unlocked_entry() if it did not lock the entry or dax_unlock_entry()
- * if it did.
+ * if it did. The entry returned may have a larger order than @order.
+ * If @order is larger than the order of the entry found in i_pages, this
+ * function returns a dax_is_conflict entry.
*
* Must be called with the i_pages lock held.
*/
-static void *get_unlocked_entry(struct xa_state *xas)
+static void *get_unlocked_entry(struct xa_state *xas, unsigned int order)
{
void *entry;
struct wait_exceptional_entry_queue ewait;
for (;;) {
entry = xas_find_conflict(xas);
+ if (dax_entry_order(entry) < order)
+ return XA_RETRY_ENTRY;
if (!entry || WARN_ON_ONCE(!xa_is_value(entry)) ||
!dax_is_locked(entry))
return entry;
static void put_unlocked_entry(struct xa_state *xas, void *entry)
{
/* If we were the only waiter woken, wake the next one */
- if (entry)
+ if (entry && dax_is_conflict(entry))
dax_wake_entry(xas, entry, false);
}
* overlap with xarray value entries.
*/
static void *grab_mapping_entry(struct xa_state *xas,
- struct address_space *mapping, unsigned long size_flag)
+ struct address_space *mapping, unsigned int order)
{
unsigned long index = xas->xa_index;
bool pmd_downgrade = false; /* splitting PMD entry into PTE entries? */
retry:
xas_lock_irq(xas);
- entry = get_unlocked_entry(xas);
+ entry = get_unlocked_entry(xas, order);
if (entry) {
+ if (dax_is_conflict(entry))
+ goto fallback;
if (!xa_is_value(entry)) {
xas_set_err(xas, EIO);
goto out_unlock;
}
- if (size_flag & DAX_PMD) {
- if (dax_is_pte_entry(entry)) {
- put_unlocked_entry(xas, entry);
- goto fallback;
- }
- } else { /* trying to grab a PTE entry */
+ if (order == 0) {
if (dax_is_pmd_entry(entry) &&
(dax_is_zero_entry(entry) ||
dax_is_empty_entry(entry))) {
if (entry) {
dax_lock_entry(xas, entry);
} else {
- entry = dax_make_entry(pfn_to_pfn_t(0), size_flag | DAX_EMPTY);
+ unsigned long flags = DAX_EMPTY;
+
+ if (order > 0)
+ flags |= DAX_PMD;
+ entry = dax_make_entry(pfn_to_pfn_t(0), flags);
dax_lock_entry(xas, entry);
if (xas_error(xas))
goto out_unlock;
if (WARN_ON_ONCE(!xa_is_value(entry)))
continue;
if (unlikely(dax_is_locked(entry)))
- entry = get_unlocked_entry(&xas);
+ entry = get_unlocked_entry(&xas, 0);
if (entry)
page = dax_busy_page(entry);
put_unlocked_entry(&xas, entry);
void *entry;
xas_lock_irq(&xas);
- entry = get_unlocked_entry(&xas);
+ entry = get_unlocked_entry(&xas, 0);
if (!entry || WARN_ON_ONCE(!xa_is_value(entry)))
goto out;
if (!trunc &&
if (unlikely(dax_is_locked(entry))) {
void *old_entry = entry;
- entry = get_unlocked_entry(xas);
+ entry = get_unlocked_entry(xas, 0);
/* Entry got punched out / reallocated? */
if (!entry || WARN_ON_ONCE(!xa_is_value(entry)))
* entry is already in the array, for instance), it will return
* VM_FAULT_FALLBACK.
*/
- entry = grab_mapping_entry(&xas, mapping, DAX_PMD);
+ entry = grab_mapping_entry(&xas, mapping, PMD_ORDER);
if (xa_is_internal(entry)) {
result = xa_to_internal(entry);
goto fallback;
vm_fault_t ret;
xas_lock_irq(&xas);
- entry = get_unlocked_entry(&xas);
+ entry = get_unlocked_entry(&xas, order);
/* Did we race with someone splitting entry or so? */
- if (!entry ||
- (order == 0 && !dax_is_pte_entry(entry)) ||
- (order == PMD_ORDER && !dax_is_pmd_entry(entry))) {
+ if (!entry || dax_is_conflict(entry) ||
+ (order == 0 && !dax_is_pte_entry(entry))) {
put_unlocked_entry(&xas, entry);
xas_unlock_irq(&xas);
trace_dax_insert_pfn_mkwrite_no_entry(mapping->host, vmf,
#include <linux/ctype.h>
#include <linux/efi.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/module.h>
#include <linux/pagemap.h>
#include <linux/ucs2_string.h>
.evict_inode = efivarfs_evict_inode,
};
-static struct super_block *efivarfs_sb;
-
/*
* Compare two efivarfs file names.
*
return 0;
}
-static int efivarfs_fill_super(struct super_block *sb, void *data, int silent)
+static int efivarfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *inode = NULL;
struct dentry *root;
int err;
- efivarfs_sb = sb;
-
sb->s_maxbytes = MAX_LFS_FILESIZE;
sb->s_blocksize = PAGE_SIZE;
sb->s_blocksize_bits = PAGE_SHIFT;
return err;
}
-static struct dentry *efivarfs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int efivarfs_get_tree(struct fs_context *fc)
+{
+ return get_tree_single(fc, efivarfs_fill_super);
+}
+
+static const struct fs_context_operations efivarfs_context_ops = {
+ .get_tree = efivarfs_get_tree,
+};
+
+static int efivarfs_init_fs_context(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, efivarfs_fill_super);
+ fc->ops = &efivarfs_context_ops;
+ return 0;
}
static void efivarfs_kill_sb(struct super_block *sb)
{
kill_litter_super(sb);
- efivarfs_sb = NULL;
/* Remove all entries and destroy */
__efivar_entry_iter(efivarfs_destroy, &efivarfs_list, NULL, NULL);
static struct file_system_type efivarfs_type = {
.owner = THIS_MODULE,
.name = "efivarfs",
- .mount = efivarfs_mount,
+ .init_fs_context = efivarfs_init_fs_context,
.kill_sb = efivarfs_kill_sb,
};
#include <linux/sysctl.h>
-static long zero;
+static long long_zero;
static long long_max = LONG_MAX;
struct ctl_table epoll_table[] = {
.maxlen = sizeof(max_user_watches),
.mode = 0644,
.proc_handler = proc_doulongvec_minmax,
- .extra1 = &zero,
+ .extra1 = &long_zero,
.extra2 = &long_max,
},
{ }
size_t, sigsetsize)
{
int error;
- sigset_t ksigmask, sigsaved;
/*
* If the caller wants a certain signal mask to be set during the wait,
* we apply it here.
*/
- error = set_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ error = set_user_sigmask(sigmask, sigsetsize);
if (error)
return error;
error = do_epoll_wait(epfd, events, maxevents, timeout);
-
- restore_user_sigmask(sigmask, &sigsaved, error == -EINTR);
+ restore_saved_sigmask_unless(error == -EINTR);
return error;
}
compat_size_t, sigsetsize)
{
long err;
- sigset_t ksigmask, sigsaved;
/*
* If the caller wants a certain signal mask to be set during the wait,
* we apply it here.
*/
- err = set_compat_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ err = set_compat_user_sigmask(sigmask, sigsetsize);
if (err)
return err;
err = do_epoll_wait(epfd, events, maxevents, timeout);
-
- restore_user_sigmask(sigmask, &sigsaved, err == -EINTR);
+ restore_saved_sigmask_unless(err == -EINTR);
return err;
}
static int ext4_file_mmap(struct file *file, struct vm_area_struct *vma)
{
struct inode *inode = file->f_mapping->host;
+ struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb);
+ struct dax_device *dax_dev = sbi->s_daxdev;
- if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb))))
+ if (unlikely(ext4_forced_shutdown(sbi)))
return -EIO;
/*
- * We don't support synchronous mappings for non-DAX files. At least
- * until someone comes with a sensible use case.
+ * We don't support synchronous mappings for non-DAX files and
+ * for DAX files if underneath dax_device is not synchronous.
*/
- if (!IS_DAX(file_inode(file)) && (vma->vm_flags & VM_SYNC))
+ if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
file_accessed(file);
/* one extra reference was held for atomic_write page */
extra_count = atomic_written ? 1 : 0;
rc = migrate_page_move_mapping(mapping, newpage,
- page, mode, extra_count);
+ page, extra_count);
if (rc != MIGRATEPAGE_SUCCESS) {
if (atomic_written)
mutex_unlock(&fi->inmem_lock);
return invalf(fc, "%s: not usable as path", param->key);
}
+ f->refcnt++; /* filename_lookup() drops our ref. */
ret = filename_lookup(param->dirfd, f, flags, _path, NULL);
if (ret < 0) {
errorf(fc, "%s: Lookup failure for '%s'", param->key, f->name);
case FSCONFIG_CMD_CREATE:
if (fc->phase != FS_CONTEXT_CREATE_PARAMS)
return -EBUSY;
+ if (!mount_capable(fc))
+ return -EPERM;
fc->phase = FS_CONTEXT_CREATING;
ret = vfs_get_tree(fc);
if (ret)
static int fuse_ctl_get_tree(struct fs_context *fc)
{
- return vfs_get_super(fc, vfs_get_single_super, fuse_ctl_fill_super);
+ return get_tree_single(fc, fuse_ctl_fill_super);
}
static const struct fs_context_operations fuse_ctl_context_ops = {
int offset = 0;
if (!is_known_namespace(xattr_name)) {
- strncpy(buffer, XATTR_MAC_OSX_PREFIX, XATTR_MAC_OSX_PREFIX_LEN);
+ memcpy(buffer, XATTR_MAC_OSX_PREFIX, XATTR_MAC_OSX_PREFIX_LEN);
offset += XATTR_MAC_OSX_PREFIX_LEN;
len += XATTR_MAC_OSX_PREFIX_LEN;
}
int err = hugetlbfs_validate(fc);
if (err)
return err;
- return vfs_get_super(fc, vfs_get_independent_super, hugetlbfs_fill_super);
+ return get_tree_nodev(fc, hugetlbfs_fill_super);
}
static void hugetlbfs_fs_context_free(struct fs_context *fc)
struct mount;
struct shrink_control;
struct fs_context;
+struct user_namespace;
/*
* block_dev.c
extern int reconfigure_super(struct fs_context *);
extern bool trylock_super(struct super_block *sb);
extern struct super_block *user_get_super(dev_t);
+extern bool mount_capable(struct fs_context *);
/*
* open.c
extern long prune_dcache_sb(struct super_block *sb, struct shrink_control *sc);
extern struct dentry *d_alloc_cursor(struct dentry *);
extern struct dentry * d_alloc_pseudo(struct super_block *, const struct qstr *);
+extern char *simple_dname(struct dentry *, char *, int);
/*
* read_write.c
extern int do_vfs_ioctl(struct file *file, unsigned int fd, unsigned int cmd,
unsigned long arg);
-/*
- * iomap support:
- */
-typedef loff_t (*iomap_actor_t)(struct inode *inode, loff_t pos, loff_t len,
- void *data, struct iomap *iomap);
-
-loff_t iomap_apply(struct inode *inode, loff_t pos, loff_t length,
- unsigned flags, const struct iomap_ops *ops, void *data,
- iomap_actor_t actor);
-
/* direct-io.c: */
int sb_init_dio_done_wq(struct super_block *sb);
const sigset_t __user *sig, size_t sigsz)
{
struct io_cq_ring *ring = ctx->cq_ring;
- sigset_t ksigmask, sigsaved;
int ret;
if (io_cqring_events(ring) >= min_events)
#ifdef CONFIG_COMPAT
if (in_compat_syscall())
ret = set_compat_user_sigmask((const compat_sigset_t __user *)sig,
- &ksigmask, &sigsaved, sigsz);
+ sigsz);
else
#endif
- ret = set_user_sigmask(sig, &ksigmask,
- &sigsaved, sigsz);
+ ret = set_user_sigmask(sig, sigsz);
if (ret)
return ret;
}
ret = wait_event_interruptible(ctx->wait, io_cqring_events(ring) >= min_events);
-
- if (sig)
- restore_user_sigmask(sig, &sigsaved, ret == -ERESTARTSYS);
-
+ restore_saved_sigmask_unless(ret == -ERESTARTSYS);
if (ret == -ERESTARTSYS)
ret = -EINTR;
+++ /dev/null
-// SPDX-License-Identifier: GPL-2.0
-/*
- * Copyright (C) 2010 Red Hat, Inc.
- * Copyright (c) 2016-2018 Christoph Hellwig.
- */
-#include <linux/module.h>
-#include <linux/compiler.h>
-#include <linux/fs.h>
-#include <linux/iomap.h>
-#include <linux/uaccess.h>
-#include <linux/gfp.h>
-#include <linux/migrate.h>
-#include <linux/mm.h>
-#include <linux/mm_inline.h>
-#include <linux/swap.h>
-#include <linux/pagemap.h>
-#include <linux/pagevec.h>
-#include <linux/file.h>
-#include <linux/uio.h>
-#include <linux/backing-dev.h>
-#include <linux/buffer_head.h>
-#include <linux/task_io_accounting_ops.h>
-#include <linux/dax.h>
-#include <linux/sched/signal.h>
-
-#include "internal.h"
-
-/*
- * Execute a iomap write on a segment of the mapping that spans a
- * contiguous range of pages that have identical block mapping state.
- *
- * This avoids the need to map pages individually, do individual allocations
- * for each page and most importantly avoid the need for filesystem specific
- * locking per page. Instead, all the operations are amortised over the entire
- * range of pages. It is assumed that the filesystems will lock whatever
- * resources they require in the iomap_begin call, and release them in the
- * iomap_end call.
- */
-loff_t
-iomap_apply(struct inode *inode, loff_t pos, loff_t length, unsigned flags,
- const struct iomap_ops *ops, void *data, iomap_actor_t actor)
-{
- struct iomap iomap = { 0 };
- loff_t written = 0, ret;
-
- /*
- * Need to map a range from start position for length bytes. This can
- * span multiple pages - it is only guaranteed to return a range of a
- * single type of pages (e.g. all into a hole, all mapped or all
- * unwritten). Failure at this point has nothing to undo.
- *
- * If allocation is required for this range, reserve the space now so
- * that the allocation is guaranteed to succeed later on. Once we copy
- * the data into the page cache pages, then we cannot fail otherwise we
- * expose transient stale data. If the reserve fails, we can safely
- * back out at this point as there is nothing to undo.
- */
- ret = ops->iomap_begin(inode, pos, length, flags, &iomap);
- if (ret)
- return ret;
- if (WARN_ON(iomap.offset > pos))
- return -EIO;
- if (WARN_ON(iomap.length == 0))
- return -EIO;
-
- /*
- * Cut down the length to the one actually provided by the filesystem,
- * as it might not be able to give us the whole size that we requested.
- */
- if (iomap.offset + iomap.length < pos + length)
- length = iomap.offset + iomap.length - pos;
-
- /*
- * Now that we have guaranteed that the space allocation will succeed.
- * we can do the copy-in page by page without having to worry about
- * failures exposing transient data.
- */
- written = actor(inode, pos, length, data, &iomap);
-
- /*
- * Now the data has been copied, commit the range we've copied. This
- * should not fail unless the filesystem has had a fatal error.
- */
- if (ops->iomap_end) {
- ret = ops->iomap_end(inode, pos, length,
- written > 0 ? written : 0,
- flags, &iomap);
- }
-
- return written ? written : ret;
-}
-
-static sector_t
-iomap_sector(struct iomap *iomap, loff_t pos)
-{
- return (iomap->addr + pos - iomap->offset) >> SECTOR_SHIFT;
-}
-
-static struct iomap_page *
-iomap_page_create(struct inode *inode, struct page *page)
-{
- struct iomap_page *iop = to_iomap_page(page);
-
- if (iop || i_blocksize(inode) == PAGE_SIZE)
- return iop;
-
- iop = kmalloc(sizeof(*iop), GFP_NOFS | __GFP_NOFAIL);
- atomic_set(&iop->read_count, 0);
- atomic_set(&iop->write_count, 0);
- bitmap_zero(iop->uptodate, PAGE_SIZE / SECTOR_SIZE);
-
- /*
- * migrate_page_move_mapping() assumes that pages with private data have
- * their count elevated by 1.
- */
- get_page(page);
- set_page_private(page, (unsigned long)iop);
- SetPagePrivate(page);
- return iop;
-}
-
-static void
-iomap_page_release(struct page *page)
-{
- struct iomap_page *iop = to_iomap_page(page);
-
- if (!iop)
- return;
- WARN_ON_ONCE(atomic_read(&iop->read_count));
- WARN_ON_ONCE(atomic_read(&iop->write_count));
- ClearPagePrivate(page);
- set_page_private(page, 0);
- put_page(page);
- kfree(iop);
-}
-
-/*
- * Calculate the range inside the page that we actually need to read.
- */
-static void
-iomap_adjust_read_range(struct inode *inode, struct iomap_page *iop,
- loff_t *pos, loff_t length, unsigned *offp, unsigned *lenp)
-{
- loff_t orig_pos = *pos;
- loff_t isize = i_size_read(inode);
- unsigned block_bits = inode->i_blkbits;
- unsigned block_size = (1 << block_bits);
- unsigned poff = offset_in_page(*pos);
- unsigned plen = min_t(loff_t, PAGE_SIZE - poff, length);
- unsigned first = poff >> block_bits;
- unsigned last = (poff + plen - 1) >> block_bits;
-
- /*
- * If the block size is smaller than the page size we need to check the
- * per-block uptodate status and adjust the offset and length if needed
- * to avoid reading in already uptodate ranges.
- */
- if (iop) {
- unsigned int i;
-
- /* move forward for each leading block marked uptodate */
- for (i = first; i <= last; i++) {
- if (!test_bit(i, iop->uptodate))
- break;
- *pos += block_size;
- poff += block_size;
- plen -= block_size;
- first++;
- }
-
- /* truncate len if we find any trailing uptodate block(s) */
- for ( ; i <= last; i++) {
- if (test_bit(i, iop->uptodate)) {
- plen -= (last - i + 1) * block_size;
- last = i - 1;
- break;
- }
- }
- }
-
- /*
- * If the extent spans the block that contains the i_size we need to
- * handle both halves separately so that we properly zero data in the
- * page cache for blocks that are entirely outside of i_size.
- */
- if (orig_pos <= isize && orig_pos + length > isize) {
- unsigned end = offset_in_page(isize - 1) >> block_bits;
-
- if (first <= end && last > end)
- plen -= (last - end) * block_size;
- }
-
- *offp = poff;
- *lenp = plen;
-}
-
-static void
-iomap_set_range_uptodate(struct page *page, unsigned off, unsigned len)
-{
- struct iomap_page *iop = to_iomap_page(page);
- struct inode *inode = page->mapping->host;
- unsigned first = off >> inode->i_blkbits;
- unsigned last = (off + len - 1) >> inode->i_blkbits;
- unsigned int i;
- bool uptodate = true;
-
- if (iop) {
- for (i = 0; i < PAGE_SIZE / i_blocksize(inode); i++) {
- if (i >= first && i <= last)
- set_bit(i, iop->uptodate);
- else if (!test_bit(i, iop->uptodate))
- uptodate = false;
- }
- }
-
- if (uptodate && !PageError(page))
- SetPageUptodate(page);
-}
-
-static void
-iomap_read_finish(struct iomap_page *iop, struct page *page)
-{
- if (!iop || atomic_dec_and_test(&iop->read_count))
- unlock_page(page);
-}
-
-static void
-iomap_read_page_end_io(struct bio_vec *bvec, int error)
-{
- struct page *page = bvec->bv_page;
- struct iomap_page *iop = to_iomap_page(page);
-
- if (unlikely(error)) {
- ClearPageUptodate(page);
- SetPageError(page);
- } else {
- iomap_set_range_uptodate(page, bvec->bv_offset, bvec->bv_len);
- }
-
- iomap_read_finish(iop, page);
-}
-
-static void
-iomap_read_end_io(struct bio *bio)
-{
- int error = blk_status_to_errno(bio->bi_status);
- struct bio_vec *bvec;
- struct bvec_iter_all iter_all;
-
- bio_for_each_segment_all(bvec, bio, iter_all)
- iomap_read_page_end_io(bvec, error);
- bio_put(bio);
-}
-
-struct iomap_readpage_ctx {
- struct page *cur_page;
- bool cur_page_in_bio;
- bool is_readahead;
- struct bio *bio;
- struct list_head *pages;
-};
-
-static void
-iomap_read_inline_data(struct inode *inode, struct page *page,
- struct iomap *iomap)
-{
- size_t size = i_size_read(inode);
- void *addr;
-
- if (PageUptodate(page))
- return;
-
- BUG_ON(page->index);
- BUG_ON(size > PAGE_SIZE - offset_in_page(iomap->inline_data));
-
- addr = kmap_atomic(page);
- memcpy(addr, iomap->inline_data, size);
- memset(addr + size, 0, PAGE_SIZE - size);
- kunmap_atomic(addr);
- SetPageUptodate(page);
-}
-
-static loff_t
-iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
-{
- struct iomap_readpage_ctx *ctx = data;
- struct page *page = ctx->cur_page;
- struct iomap_page *iop = iomap_page_create(inode, page);
- bool same_page = false, is_contig = false;
- loff_t orig_pos = pos;
- unsigned poff, plen;
- sector_t sector;
-
- if (iomap->type == IOMAP_INLINE) {
- WARN_ON_ONCE(pos);
- iomap_read_inline_data(inode, page, iomap);
- return PAGE_SIZE;
- }
-
- /* zero post-eof blocks as the page may be mapped */
- iomap_adjust_read_range(inode, iop, &pos, length, &poff, &plen);
- if (plen == 0)
- goto done;
-
- if (iomap->type != IOMAP_MAPPED || pos >= i_size_read(inode)) {
- zero_user(page, poff, plen);
- iomap_set_range_uptodate(page, poff, plen);
- goto done;
- }
-
- ctx->cur_page_in_bio = true;
-
- /*
- * Try to merge into a previous segment if we can.
- */
- sector = iomap_sector(iomap, pos);
- if (ctx->bio && bio_end_sector(ctx->bio) == sector)
- is_contig = true;
-
- if (is_contig &&
- __bio_try_merge_page(ctx->bio, page, plen, poff, &same_page)) {
- if (!same_page && iop)
- atomic_inc(&iop->read_count);
- goto done;
- }
-
- /*
- * If we start a new segment we need to increase the read count, and we
- * need to do so before submitting any previous full bio to make sure
- * that we don't prematurely unlock the page.
- */
- if (iop)
- atomic_inc(&iop->read_count);
-
- if (!ctx->bio || !is_contig || bio_full(ctx->bio, plen)) {
- gfp_t gfp = mapping_gfp_constraint(page->mapping, GFP_KERNEL);
- int nr_vecs = (length + PAGE_SIZE - 1) >> PAGE_SHIFT;
-
- if (ctx->bio)
- submit_bio(ctx->bio);
-
- if (ctx->is_readahead) /* same as readahead_gfp_mask */
- gfp |= __GFP_NORETRY | __GFP_NOWARN;
- ctx->bio = bio_alloc(gfp, min(BIO_MAX_PAGES, nr_vecs));
- ctx->bio->bi_opf = REQ_OP_READ;
- if (ctx->is_readahead)
- ctx->bio->bi_opf |= REQ_RAHEAD;
- ctx->bio->bi_iter.bi_sector = sector;
- bio_set_dev(ctx->bio, iomap->bdev);
- ctx->bio->bi_end_io = iomap_read_end_io;
- }
-
- bio_add_page(ctx->bio, page, plen, poff);
-done:
- /*
- * Move the caller beyond our range so that it keeps making progress.
- * For that we have to include any leading non-uptodate ranges, but
- * we can skip trailing ones as they will be handled in the next
- * iteration.
- */
- return pos - orig_pos + plen;
-}
-
-int
-iomap_readpage(struct page *page, const struct iomap_ops *ops)
-{
- struct iomap_readpage_ctx ctx = { .cur_page = page };
- struct inode *inode = page->mapping->host;
- unsigned poff;
- loff_t ret;
-
- for (poff = 0; poff < PAGE_SIZE; poff += ret) {
- ret = iomap_apply(inode, page_offset(page) + poff,
- PAGE_SIZE - poff, 0, ops, &ctx,
- iomap_readpage_actor);
- if (ret <= 0) {
- WARN_ON_ONCE(ret == 0);
- SetPageError(page);
- break;
- }
- }
-
- if (ctx.bio) {
- submit_bio(ctx.bio);
- WARN_ON_ONCE(!ctx.cur_page_in_bio);
- } else {
- WARN_ON_ONCE(ctx.cur_page_in_bio);
- unlock_page(page);
- }
-
- /*
- * Just like mpage_readpages and block_read_full_page we always
- * return 0 and just mark the page as PageError on errors. This
- * should be cleaned up all through the stack eventually.
- */
- return 0;
-}
-EXPORT_SYMBOL_GPL(iomap_readpage);
-
-static struct page *
-iomap_next_page(struct inode *inode, struct list_head *pages, loff_t pos,
- loff_t length, loff_t *done)
-{
- while (!list_empty(pages)) {
- struct page *page = lru_to_page(pages);
-
- if (page_offset(page) >= (u64)pos + length)
- break;
-
- list_del(&page->lru);
- if (!add_to_page_cache_lru(page, inode->i_mapping, page->index,
- GFP_NOFS))
- return page;
-
- /*
- * If we already have a page in the page cache at index we are
- * done. Upper layers don't care if it is uptodate after the
- * readpages call itself as every page gets checked again once
- * actually needed.
- */
- *done += PAGE_SIZE;
- put_page(page);
- }
-
- return NULL;
-}
-
-static loff_t
-iomap_readpages_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
-{
- struct iomap_readpage_ctx *ctx = data;
- loff_t done, ret;
-
- for (done = 0; done < length; done += ret) {
- if (ctx->cur_page && offset_in_page(pos + done) == 0) {
- if (!ctx->cur_page_in_bio)
- unlock_page(ctx->cur_page);
- put_page(ctx->cur_page);
- ctx->cur_page = NULL;
- }
- if (!ctx->cur_page) {
- ctx->cur_page = iomap_next_page(inode, ctx->pages,
- pos, length, &done);
- if (!ctx->cur_page)
- break;
- ctx->cur_page_in_bio = false;
- }
- ret = iomap_readpage_actor(inode, pos + done, length - done,
- ctx, iomap);
- }
-
- return done;
-}
-
-int
-iomap_readpages(struct address_space *mapping, struct list_head *pages,
- unsigned nr_pages, const struct iomap_ops *ops)
-{
- struct iomap_readpage_ctx ctx = {
- .pages = pages,
- .is_readahead = true,
- };
- loff_t pos = page_offset(list_entry(pages->prev, struct page, lru));
- loff_t last = page_offset(list_entry(pages->next, struct page, lru));
- loff_t length = last - pos + PAGE_SIZE, ret = 0;
-
- while (length > 0) {
- ret = iomap_apply(mapping->host, pos, length, 0, ops,
- &ctx, iomap_readpages_actor);
- if (ret <= 0) {
- WARN_ON_ONCE(ret == 0);
- goto done;
- }
- pos += ret;
- length -= ret;
- }
- ret = 0;
-done:
- if (ctx.bio)
- submit_bio(ctx.bio);
- if (ctx.cur_page) {
- if (!ctx.cur_page_in_bio)
- unlock_page(ctx.cur_page);
- put_page(ctx.cur_page);
- }
-
- /*
- * Check that we didn't lose a page due to the arcance calling
- * conventions..
- */
- WARN_ON_ONCE(!ret && !list_empty(ctx.pages));
- return ret;
-}
-EXPORT_SYMBOL_GPL(iomap_readpages);
-
-/*
- * iomap_is_partially_uptodate checks whether blocks within a page are
- * uptodate or not.
- *
- * Returns true if all blocks which correspond to a file portion
- * we want to read within the page are uptodate.
- */
-int
-iomap_is_partially_uptodate(struct page *page, unsigned long from,
- unsigned long count)
-{
- struct iomap_page *iop = to_iomap_page(page);
- struct inode *inode = page->mapping->host;
- unsigned len, first, last;
- unsigned i;
-
- /* Limit range to one page */
- len = min_t(unsigned, PAGE_SIZE - from, count);
-
- /* First and last blocks in range within page */
- first = from >> inode->i_blkbits;
- last = (from + len - 1) >> inode->i_blkbits;
-
- if (iop) {
- for (i = first; i <= last; i++)
- if (!test_bit(i, iop->uptodate))
- return 0;
- return 1;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate);
-
-int
-iomap_releasepage(struct page *page, gfp_t gfp_mask)
-{
- /*
- * mm accommodates an old ext3 case where clean pages might not have had
- * the dirty bit cleared. Thus, it can send actual dirty pages to
- * ->releasepage() via shrink_active_list(), skip those here.
- */
- if (PageDirty(page) || PageWriteback(page))
- return 0;
- iomap_page_release(page);
- return 1;
-}
-EXPORT_SYMBOL_GPL(iomap_releasepage);
-
-void
-iomap_invalidatepage(struct page *page, unsigned int offset, unsigned int len)
-{
- /*
- * If we are invalidating the entire page, clear the dirty state from it
- * and release it to avoid unnecessary buildup of the LRU.
- */
- if (offset == 0 && len == PAGE_SIZE) {
- WARN_ON_ONCE(PageWriteback(page));
- cancel_dirty_page(page);
- iomap_page_release(page);
- }
-}
-EXPORT_SYMBOL_GPL(iomap_invalidatepage);
-
-#ifdef CONFIG_MIGRATION
-int
-iomap_migrate_page(struct address_space *mapping, struct page *newpage,
- struct page *page, enum migrate_mode mode)
-{
- int ret;
-
- ret = migrate_page_move_mapping(mapping, newpage, page, mode, 0);
- if (ret != MIGRATEPAGE_SUCCESS)
- return ret;
-
- if (page_has_private(page)) {
- ClearPagePrivate(page);
- get_page(newpage);
- set_page_private(newpage, page_private(page));
- set_page_private(page, 0);
- put_page(page);
- SetPagePrivate(newpage);
- }
-
- if (mode != MIGRATE_SYNC_NO_COPY)
- migrate_page_copy(newpage, page);
- else
- migrate_page_states(newpage, page);
- return MIGRATEPAGE_SUCCESS;
-}
-EXPORT_SYMBOL_GPL(iomap_migrate_page);
-#endif /* CONFIG_MIGRATION */
-
-static void
-iomap_write_failed(struct inode *inode, loff_t pos, unsigned len)
-{
- loff_t i_size = i_size_read(inode);
-
- /*
- * Only truncate newly allocated pages beyoned EOF, even if the
- * write started inside the existing inode size.
- */
- if (pos + len > i_size)
- truncate_pagecache_range(inode, max(pos, i_size), pos + len);
-}
-
-static int
-iomap_read_page_sync(struct inode *inode, loff_t block_start, struct page *page,
- unsigned poff, unsigned plen, unsigned from, unsigned to,
- struct iomap *iomap)
-{
- struct bio_vec bvec;
- struct bio bio;
-
- if (iomap->type != IOMAP_MAPPED || block_start >= i_size_read(inode)) {
- zero_user_segments(page, poff, from, to, poff + plen);
- iomap_set_range_uptodate(page, poff, plen);
- return 0;
- }
-
- bio_init(&bio, &bvec, 1);
- bio.bi_opf = REQ_OP_READ;
- bio.bi_iter.bi_sector = iomap_sector(iomap, block_start);
- bio_set_dev(&bio, iomap->bdev);
- __bio_add_page(&bio, page, plen, poff);
- return submit_bio_wait(&bio);
-}
-
-static int
-__iomap_write_begin(struct inode *inode, loff_t pos, unsigned len,
- struct page *page, struct iomap *iomap)
-{
- struct iomap_page *iop = iomap_page_create(inode, page);
- loff_t block_size = i_blocksize(inode);
- loff_t block_start = pos & ~(block_size - 1);
- loff_t block_end = (pos + len + block_size - 1) & ~(block_size - 1);
- unsigned from = offset_in_page(pos), to = from + len, poff, plen;
- int status = 0;
-
- if (PageUptodate(page))
- return 0;
-
- do {
- iomap_adjust_read_range(inode, iop, &block_start,
- block_end - block_start, &poff, &plen);
- if (plen == 0)
- break;
-
- if ((from > poff && from < poff + plen) ||
- (to > poff && to < poff + plen)) {
- status = iomap_read_page_sync(inode, block_start, page,
- poff, plen, from, to, iomap);
- if (status)
- break;
- }
-
- } while ((block_start += plen) < block_end);
-
- return status;
-}
-
-static int
-iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags,
- struct page **pagep, struct iomap *iomap)
-{
- const struct iomap_page_ops *page_ops = iomap->page_ops;
- pgoff_t index = pos >> PAGE_SHIFT;
- struct page *page;
- int status = 0;
-
- BUG_ON(pos + len > iomap->offset + iomap->length);
-
- if (fatal_signal_pending(current))
- return -EINTR;
-
- if (page_ops && page_ops->page_prepare) {
- status = page_ops->page_prepare(inode, pos, len, iomap);
- if (status)
- return status;
- }
-
- page = grab_cache_page_write_begin(inode->i_mapping, index, flags);
- if (!page) {
- status = -ENOMEM;
- goto out_no_page;
- }
-
- if (iomap->type == IOMAP_INLINE)
- iomap_read_inline_data(inode, page, iomap);
- else if (iomap->flags & IOMAP_F_BUFFER_HEAD)
- status = __block_write_begin_int(page, pos, len, NULL, iomap);
- else
- status = __iomap_write_begin(inode, pos, len, page, iomap);
-
- if (unlikely(status))
- goto out_unlock;
-
- *pagep = page;
- return 0;
-
-out_unlock:
- unlock_page(page);
- put_page(page);
- iomap_write_failed(inode, pos, len);
-
-out_no_page:
- if (page_ops && page_ops->page_done)
- page_ops->page_done(inode, pos, 0, NULL, iomap);
- return status;
-}
-
-int
-iomap_set_page_dirty(struct page *page)
-{
- struct address_space *mapping = page_mapping(page);
- int newly_dirty;
-
- if (unlikely(!mapping))
- return !TestSetPageDirty(page);
-
- /*
- * Lock out page->mem_cgroup migration to keep PageDirty
- * synchronized with per-memcg dirty page counters.
- */
- lock_page_memcg(page);
- newly_dirty = !TestSetPageDirty(page);
- if (newly_dirty)
- __set_page_dirty(page, mapping, 0);
- unlock_page_memcg(page);
-
- if (newly_dirty)
- __mark_inode_dirty(mapping->host, I_DIRTY_PAGES);
- return newly_dirty;
-}
-EXPORT_SYMBOL_GPL(iomap_set_page_dirty);
-
-static int
-__iomap_write_end(struct inode *inode, loff_t pos, unsigned len,
- unsigned copied, struct page *page, struct iomap *iomap)
-{
- flush_dcache_page(page);
-
- /*
- * The blocks that were entirely written will now be uptodate, so we
- * don't have to worry about a readpage reading them and overwriting a
- * partial write. However if we have encountered a short write and only
- * partially written into a block, it will not be marked uptodate, so a
- * readpage might come in and destroy our partial write.
- *
- * Do the simplest thing, and just treat any short write to a non
- * uptodate page as a zero-length write, and force the caller to redo
- * the whole thing.
- */
- if (unlikely(copied < len && !PageUptodate(page)))
- return 0;
- iomap_set_range_uptodate(page, offset_in_page(pos), len);
- iomap_set_page_dirty(page);
- return copied;
-}
-
-static int
-iomap_write_end_inline(struct inode *inode, struct page *page,
- struct iomap *iomap, loff_t pos, unsigned copied)
-{
- void *addr;
-
- WARN_ON_ONCE(!PageUptodate(page));
- BUG_ON(pos + copied > PAGE_SIZE - offset_in_page(iomap->inline_data));
-
- addr = kmap_atomic(page);
- memcpy(iomap->inline_data + pos, addr + pos, copied);
- kunmap_atomic(addr);
-
- mark_inode_dirty(inode);
- return copied;
-}
-
-static int
-iomap_write_end(struct inode *inode, loff_t pos, unsigned len,
- unsigned copied, struct page *page, struct iomap *iomap)
-{
- const struct iomap_page_ops *page_ops = iomap->page_ops;
- loff_t old_size = inode->i_size;
- int ret;
-
- if (iomap->type == IOMAP_INLINE) {
- ret = iomap_write_end_inline(inode, page, iomap, pos, copied);
- } else if (iomap->flags & IOMAP_F_BUFFER_HEAD) {
- ret = block_write_end(NULL, inode->i_mapping, pos, len, copied,
- page, NULL);
- } else {
- ret = __iomap_write_end(inode, pos, len, copied, page, iomap);
- }
-
- /*
- * Update the in-memory inode size after copying the data into the page
- * cache. It's up to the file system to write the updated size to disk,
- * preferably after I/O completion so that no stale data is exposed.
- */
- if (pos + ret > old_size) {
- i_size_write(inode, pos + ret);
- iomap->flags |= IOMAP_F_SIZE_CHANGED;
- }
- unlock_page(page);
-
- if (old_size < pos)
- pagecache_isize_extended(inode, old_size, pos);
- if (page_ops && page_ops->page_done)
- page_ops->page_done(inode, pos, ret, page, iomap);
- put_page(page);
-
- if (ret < len)
- iomap_write_failed(inode, pos, len);
- return ret;
-}
-
-static loff_t
-iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
-{
- struct iov_iter *i = data;
- long status = 0;
- ssize_t written = 0;
- unsigned int flags = AOP_FLAG_NOFS;
-
- do {
- struct page *page;
- unsigned long offset; /* Offset into pagecache page */
- unsigned long bytes; /* Bytes to write to page */
- size_t copied; /* Bytes copied from user */
-
- offset = offset_in_page(pos);
- bytes = min_t(unsigned long, PAGE_SIZE - offset,
- iov_iter_count(i));
-again:
- if (bytes > length)
- bytes = length;
-
- /*
- * Bring in the user page that we will copy from _first_.
- * Otherwise there's a nasty deadlock on copying from the
- * same page as we're writing to, without it being marked
- * up-to-date.
- *
- * Not only is this an optimisation, but it is also required
- * to check that the address is actually valid, when atomic
- * usercopies are used, below.
- */
- if (unlikely(iov_iter_fault_in_readable(i, bytes))) {
- status = -EFAULT;
- break;
- }
-
- status = iomap_write_begin(inode, pos, bytes, flags, &page,
- iomap);
- if (unlikely(status))
- break;
-
- if (mapping_writably_mapped(inode->i_mapping))
- flush_dcache_page(page);
-
- copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
-
- flush_dcache_page(page);
-
- status = iomap_write_end(inode, pos, bytes, copied, page,
- iomap);
- if (unlikely(status < 0))
- break;
- copied = status;
-
- cond_resched();
-
- iov_iter_advance(i, copied);
- if (unlikely(copied == 0)) {
- /*
- * If we were unable to copy any data at all, we must
- * fall back to a single segment length write.
- *
- * If we didn't fallback here, we could livelock
- * because not all segments in the iov can be copied at
- * once without a pagefault.
- */
- bytes = min_t(unsigned long, PAGE_SIZE - offset,
- iov_iter_single_seg_count(i));
- goto again;
- }
- pos += copied;
- written += copied;
- length -= copied;
-
- balance_dirty_pages_ratelimited(inode->i_mapping);
- } while (iov_iter_count(i) && length);
-
- return written ? written : status;
-}
-
-ssize_t
-iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *iter,
- const struct iomap_ops *ops)
-{
- struct inode *inode = iocb->ki_filp->f_mapping->host;
- loff_t pos = iocb->ki_pos, ret = 0, written = 0;
-
- while (iov_iter_count(iter)) {
- ret = iomap_apply(inode, pos, iov_iter_count(iter),
- IOMAP_WRITE, ops, iter, iomap_write_actor);
- if (ret <= 0)
- break;
- pos += ret;
- written += ret;
- }
-
- return written ? written : ret;
-}
-EXPORT_SYMBOL_GPL(iomap_file_buffered_write);
-
-static struct page *
-__iomap_read_page(struct inode *inode, loff_t offset)
-{
- struct address_space *mapping = inode->i_mapping;
- struct page *page;
-
- page = read_mapping_page(mapping, offset >> PAGE_SHIFT, NULL);
- if (IS_ERR(page))
- return page;
- if (!PageUptodate(page)) {
- put_page(page);
- return ERR_PTR(-EIO);
- }
- return page;
-}
-
-static loff_t
-iomap_dirty_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
-{
- long status = 0;
- ssize_t written = 0;
-
- do {
- struct page *page, *rpage;
- unsigned long offset; /* Offset into pagecache page */
- unsigned long bytes; /* Bytes to write to page */
-
- offset = offset_in_page(pos);
- bytes = min_t(loff_t, PAGE_SIZE - offset, length);
-
- rpage = __iomap_read_page(inode, pos);
- if (IS_ERR(rpage))
- return PTR_ERR(rpage);
-
- status = iomap_write_begin(inode, pos, bytes,
- AOP_FLAG_NOFS, &page, iomap);
- put_page(rpage);
- if (unlikely(status))
- return status;
-
- WARN_ON_ONCE(!PageUptodate(page));
-
- status = iomap_write_end(inode, pos, bytes, bytes, page, iomap);
- if (unlikely(status <= 0)) {
- if (WARN_ON_ONCE(status == 0))
- return -EIO;
- return status;
- }
-
- cond_resched();
-
- pos += status;
- written += status;
- length -= status;
-
- balance_dirty_pages_ratelimited(inode->i_mapping);
- } while (length);
-
- return written;
-}
-
-int
-iomap_file_dirty(struct inode *inode, loff_t pos, loff_t len,
- const struct iomap_ops *ops)
-{
- loff_t ret;
-
- while (len) {
- ret = iomap_apply(inode, pos, len, IOMAP_WRITE, ops, NULL,
- iomap_dirty_actor);
- if (ret <= 0)
- return ret;
- pos += ret;
- len -= ret;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(iomap_file_dirty);
-
-static int iomap_zero(struct inode *inode, loff_t pos, unsigned offset,
- unsigned bytes, struct iomap *iomap)
-{
- struct page *page;
- int status;
-
- status = iomap_write_begin(inode, pos, bytes, AOP_FLAG_NOFS, &page,
- iomap);
- if (status)
- return status;
-
- zero_user(page, offset, bytes);
- mark_page_accessed(page);
-
- return iomap_write_end(inode, pos, bytes, bytes, page, iomap);
-}
-
-static int iomap_dax_zero(loff_t pos, unsigned offset, unsigned bytes,
- struct iomap *iomap)
-{
- return __dax_zero_page_range(iomap->bdev, iomap->dax_dev,
- iomap_sector(iomap, pos & PAGE_MASK), offset, bytes);
-}
-
-static loff_t
-iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count,
- void *data, struct iomap *iomap)
-{
- bool *did_zero = data;
- loff_t written = 0;
- int status;
-
- /* already zeroed? we're done. */
- if (iomap->type == IOMAP_HOLE || iomap->type == IOMAP_UNWRITTEN)
- return count;
-
- do {
- unsigned offset, bytes;
-
- offset = offset_in_page(pos);
- bytes = min_t(loff_t, PAGE_SIZE - offset, count);
-
- if (IS_DAX(inode))
- status = iomap_dax_zero(pos, offset, bytes, iomap);
- else
- status = iomap_zero(inode, pos, offset, bytes, iomap);
- if (status < 0)
- return status;
-
- pos += bytes;
- count -= bytes;
- written += bytes;
- if (did_zero)
- *did_zero = true;
- } while (count > 0);
-
- return written;
-}
-
-int
-iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero,
- const struct iomap_ops *ops)
-{
- loff_t ret;
-
- while (len > 0) {
- ret = iomap_apply(inode, pos, len, IOMAP_ZERO,
- ops, did_zero, iomap_zero_range_actor);
- if (ret <= 0)
- return ret;
-
- pos += ret;
- len -= ret;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(iomap_zero_range);
-
-int
-iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,
- const struct iomap_ops *ops)
-{
- unsigned int blocksize = i_blocksize(inode);
- unsigned int off = pos & (blocksize - 1);
-
- /* Block boundary? Nothing to do */
- if (!off)
- return 0;
- return iomap_zero_range(inode, pos, blocksize - off, did_zero, ops);
-}
-EXPORT_SYMBOL_GPL(iomap_truncate_page);
-
-static loff_t
-iomap_page_mkwrite_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
-{
- struct page *page = data;
- int ret;
-
- if (iomap->flags & IOMAP_F_BUFFER_HEAD) {
- ret = __block_write_begin_int(page, pos, length, NULL, iomap);
- if (ret)
- return ret;
- block_commit_write(page, 0, length);
- } else {
- WARN_ON_ONCE(!PageUptodate(page));
- iomap_page_create(inode, page);
- set_page_dirty(page);
- }
-
- return length;
-}
-
-vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops)
-{
- struct page *page = vmf->page;
- struct inode *inode = file_inode(vmf->vma->vm_file);
- unsigned long length;
- loff_t offset, size;
- ssize_t ret;
-
- lock_page(page);
- size = i_size_read(inode);
- if ((page->mapping != inode->i_mapping) ||
- (page_offset(page) > size)) {
- /* We overload EFAULT to mean page got truncated */
- ret = -EFAULT;
- goto out_unlock;
- }
-
- /* page is wholly or partially inside EOF */
- if (((page->index + 1) << PAGE_SHIFT) > size)
- length = offset_in_page(size);
- else
- length = PAGE_SIZE;
-
- offset = page_offset(page);
- while (length > 0) {
- ret = iomap_apply(inode, offset, length,
- IOMAP_WRITE | IOMAP_FAULT, ops, page,
- iomap_page_mkwrite_actor);
- if (unlikely(ret <= 0))
- goto out_unlock;
- offset += ret;
- length -= ret;
- }
-
- wait_for_stable_page(page);
- return VM_FAULT_LOCKED;
-out_unlock:
- unlock_page(page);
- return block_page_mkwrite_return(ret);
-}
-EXPORT_SYMBOL_GPL(iomap_page_mkwrite);
-
-struct fiemap_ctx {
- struct fiemap_extent_info *fi;
- struct iomap prev;
-};
-
-static int iomap_to_fiemap(struct fiemap_extent_info *fi,
- struct iomap *iomap, u32 flags)
-{
- switch (iomap->type) {
- case IOMAP_HOLE:
- /* skip holes */
- return 0;
- case IOMAP_DELALLOC:
- flags |= FIEMAP_EXTENT_DELALLOC | FIEMAP_EXTENT_UNKNOWN;
- break;
- case IOMAP_MAPPED:
- break;
- case IOMAP_UNWRITTEN:
- flags |= FIEMAP_EXTENT_UNWRITTEN;
- break;
- case IOMAP_INLINE:
- flags |= FIEMAP_EXTENT_DATA_INLINE;
- break;
- }
-
- if (iomap->flags & IOMAP_F_MERGED)
- flags |= FIEMAP_EXTENT_MERGED;
- if (iomap->flags & IOMAP_F_SHARED)
- flags |= FIEMAP_EXTENT_SHARED;
-
- return fiemap_fill_next_extent(fi, iomap->offset,
- iomap->addr != IOMAP_NULL_ADDR ? iomap->addr : 0,
- iomap->length, flags);
-}
-
-static loff_t
-iomap_fiemap_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
- struct iomap *iomap)
-{
- struct fiemap_ctx *ctx = data;
- loff_t ret = length;
-
- if (iomap->type == IOMAP_HOLE)
- return length;
-
- ret = iomap_to_fiemap(ctx->fi, &ctx->prev, 0);
- ctx->prev = *iomap;
- switch (ret) {
- case 0: /* success */
- return length;
- case 1: /* extent array full */
- return 0;
- default:
- return ret;
- }
-}
-
-int iomap_fiemap(struct inode *inode, struct fiemap_extent_info *fi,
- loff_t start, loff_t len, const struct iomap_ops *ops)
-{
- struct fiemap_ctx ctx;
- loff_t ret;
-
- memset(&ctx, 0, sizeof(ctx));
- ctx.fi = fi;
- ctx.prev.type = IOMAP_HOLE;
-
- ret = fiemap_check_flags(fi, FIEMAP_FLAG_SYNC);
- if (ret)
- return ret;
-
- if (fi->fi_flags & FIEMAP_FLAG_SYNC) {
- ret = filemap_write_and_wait(inode->i_mapping);
- if (ret)
- return ret;
- }
-
- while (len > 0) {
- ret = iomap_apply(inode, start, len, IOMAP_REPORT, ops, &ctx,
- iomap_fiemap_actor);
- /* inode with no (attribute) mapping will give ENOENT */
- if (ret == -ENOENT)
- break;
- if (ret < 0)
- return ret;
- if (ret == 0)
- break;
-
- start += ret;
- len -= ret;
- }
-
- if (ctx.prev.type != IOMAP_HOLE) {
- ret = iomap_to_fiemap(fi, &ctx.prev, FIEMAP_EXTENT_LAST);
- if (ret < 0)
- return ret;
- }
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(iomap_fiemap);
-
-/*
- * Seek for SEEK_DATA / SEEK_HOLE within @page, starting at @lastoff.
- * Returns true if found and updates @lastoff to the offset in file.
- */
-static bool
-page_seek_hole_data(struct inode *inode, struct page *page, loff_t *lastoff,
- int whence)
-{
- const struct address_space_operations *ops = inode->i_mapping->a_ops;
- unsigned int bsize = i_blocksize(inode), off;
- bool seek_data = whence == SEEK_DATA;
- loff_t poff = page_offset(page);
-
- if (WARN_ON_ONCE(*lastoff >= poff + PAGE_SIZE))
- return false;
-
- if (*lastoff < poff) {
- /*
- * Last offset smaller than the start of the page means we found
- * a hole:
- */
- if (whence == SEEK_HOLE)
- return true;
- *lastoff = poff;
- }
-
- /*
- * Just check the page unless we can and should check block ranges:
- */
- if (bsize == PAGE_SIZE || !ops->is_partially_uptodate)
- return PageUptodate(page) == seek_data;
-
- lock_page(page);
- if (unlikely(page->mapping != inode->i_mapping))
- goto out_unlock_not_found;
-
- for (off = 0; off < PAGE_SIZE; off += bsize) {
- if (offset_in_page(*lastoff) >= off + bsize)
- continue;
- if (ops->is_partially_uptodate(page, off, bsize) == seek_data) {
- unlock_page(page);
- return true;
- }
- *lastoff = poff + off + bsize;
- }
-
-out_unlock_not_found:
- unlock_page(page);
- return false;
-}
-
-/*
- * Seek for SEEK_DATA / SEEK_HOLE in the page cache.
- *
- * Within unwritten extents, the page cache determines which parts are holes
- * and which are data: uptodate buffer heads count as data; everything else
- * counts as a hole.
- *
- * Returns the resulting offset on successs, and -ENOENT otherwise.
- */
-static loff_t
-page_cache_seek_hole_data(struct inode *inode, loff_t offset, loff_t length,
- int whence)
-{
- pgoff_t index = offset >> PAGE_SHIFT;
- pgoff_t end = DIV_ROUND_UP(offset + length, PAGE_SIZE);
- loff_t lastoff = offset;
- struct pagevec pvec;
-
- if (length <= 0)
- return -ENOENT;
-
- pagevec_init(&pvec);
-
- do {
- unsigned nr_pages, i;
-
- nr_pages = pagevec_lookup_range(&pvec, inode->i_mapping, &index,
- end - 1);
- if (nr_pages == 0)
- break;
-
- for (i = 0; i < nr_pages; i++) {
- struct page *page = pvec.pages[i];
-
- if (page_seek_hole_data(inode, page, &lastoff, whence))
- goto check_range;
- lastoff = page_offset(page) + PAGE_SIZE;
- }
- pagevec_release(&pvec);
- } while (index < end);
-
- /* When no page at lastoff and we are not done, we found a hole. */
- if (whence != SEEK_HOLE)
- goto not_found;
-
-check_range:
- if (lastoff < offset + length)
- goto out;
-not_found:
- lastoff = -ENOENT;
-out:
- pagevec_release(&pvec);
- return lastoff;
-}
-
-
-static loff_t
-iomap_seek_hole_actor(struct inode *inode, loff_t offset, loff_t length,
- void *data, struct iomap *iomap)
-{
- switch (iomap->type) {
- case IOMAP_UNWRITTEN:
- offset = page_cache_seek_hole_data(inode, offset, length,
- SEEK_HOLE);
- if (offset < 0)
- return length;
- /* fall through */
- case IOMAP_HOLE:
- *(loff_t *)data = offset;
- return 0;
- default:
- return length;
- }
-}
-
-loff_t
-iomap_seek_hole(struct inode *inode, loff_t offset, const struct iomap_ops *ops)
-{
- loff_t size = i_size_read(inode);
- loff_t length = size - offset;
- loff_t ret;
-
- /* Nothing to be found before or beyond the end of the file. */
- if (offset < 0 || offset >= size)
- return -ENXIO;
-
- while (length > 0) {
- ret = iomap_apply(inode, offset, length, IOMAP_REPORT, ops,
- &offset, iomap_seek_hole_actor);
- if (ret < 0)
- return ret;
- if (ret == 0)
- break;
-
- offset += ret;
- length -= ret;
- }
-
- return offset;
-}
-EXPORT_SYMBOL_GPL(iomap_seek_hole);
-
-static loff_t
-iomap_seek_data_actor(struct inode *inode, loff_t offset, loff_t length,
- void *data, struct iomap *iomap)
-{
- switch (iomap->type) {
- case IOMAP_HOLE:
- return length;
- case IOMAP_UNWRITTEN:
- offset = page_cache_seek_hole_data(inode, offset, length,
- SEEK_DATA);
- if (offset < 0)
- return length;
- /*FALLTHRU*/
- default:
- *(loff_t *)data = offset;
- return 0;
- }
-}
-
-loff_t
-iomap_seek_data(struct inode *inode, loff_t offset, const struct iomap_ops *ops)
-{
- loff_t size = i_size_read(inode);
- loff_t length = size - offset;
- loff_t ret;
-
- /* Nothing to be found before or beyond the end of the file. */
- if (offset < 0 || offset >= size)
- return -ENXIO;
-
- while (length > 0) {
- ret = iomap_apply(inode, offset, length, IOMAP_REPORT, ops,
- &offset, iomap_seek_data_actor);
- if (ret < 0)
- return ret;
- if (ret == 0)
- break;
-
- offset += ret;
- length -= ret;
- }
-
- if (length <= 0)
- return -ENXIO;
- return offset;
-}
-EXPORT_SYMBOL_GPL(iomap_seek_data);
-
-/*
- * Private flags for iomap_dio, must not overlap with the public ones in
- * iomap.h:
- */
-#define IOMAP_DIO_WRITE_FUA (1 << 28)
-#define IOMAP_DIO_NEED_SYNC (1 << 29)
-#define IOMAP_DIO_WRITE (1 << 30)
-#define IOMAP_DIO_DIRTY (1 << 31)
-
-struct iomap_dio {
- struct kiocb *iocb;
- iomap_dio_end_io_t *end_io;
- loff_t i_size;
- loff_t size;
- atomic_t ref;
- unsigned flags;
- int error;
- bool wait_for_completion;
-
- union {
- /* used during submission and for synchronous completion: */
- struct {
- struct iov_iter *iter;
- struct task_struct *waiter;
- struct request_queue *last_queue;
- blk_qc_t cookie;
- } submit;
-
- /* used for aio completion: */
- struct {
- struct work_struct work;
- } aio;
- };
-};
-
-int iomap_dio_iopoll(struct kiocb *kiocb, bool spin)
-{
- struct request_queue *q = READ_ONCE(kiocb->private);
-
- if (!q)
- return 0;
- return blk_poll(q, READ_ONCE(kiocb->ki_cookie), spin);
-}
-EXPORT_SYMBOL_GPL(iomap_dio_iopoll);
-
-static void iomap_dio_submit_bio(struct iomap_dio *dio, struct iomap *iomap,
- struct bio *bio)
-{
- atomic_inc(&dio->ref);
-
- if (dio->iocb->ki_flags & IOCB_HIPRI)
- bio_set_polled(bio, dio->iocb);
-
- dio->submit.last_queue = bdev_get_queue(iomap->bdev);
- dio->submit.cookie = submit_bio(bio);
-}
-
-static ssize_t iomap_dio_complete(struct iomap_dio *dio)
-{
- struct kiocb *iocb = dio->iocb;
- struct inode *inode = file_inode(iocb->ki_filp);
- loff_t offset = iocb->ki_pos;
- ssize_t ret;
-
- if (dio->end_io) {
- ret = dio->end_io(iocb,
- dio->error ? dio->error : dio->size,
- dio->flags);
- } else {
- ret = dio->error;
- }
-
- if (likely(!ret)) {
- ret = dio->size;
- /* check for short read */
- if (offset + ret > dio->i_size &&
- !(dio->flags & IOMAP_DIO_WRITE))
- ret = dio->i_size - offset;
- iocb->ki_pos += ret;
- }
-
- /*
- * Try again to invalidate clean pages which might have been cached by
- * non-direct readahead, or faulted in by get_user_pages() if the source
- * of the write was an mmap'ed region of the file we're writing. Either
- * one is a pretty crazy thing to do, so we don't support it 100%. If
- * this invalidation fails, tough, the write still worked...
- *
- * And this page cache invalidation has to be after dio->end_io(), as
- * some filesystems convert unwritten extents to real allocations in
- * end_io() when necessary, otherwise a racing buffer read would cache
- * zeros from unwritten extents.
- */
- if (!dio->error &&
- (dio->flags & IOMAP_DIO_WRITE) && inode->i_mapping->nrpages) {
- int err;
- err = invalidate_inode_pages2_range(inode->i_mapping,
- offset >> PAGE_SHIFT,
- (offset + dio->size - 1) >> PAGE_SHIFT);
- if (err)
- dio_warn_stale_pagecache(iocb->ki_filp);
- }
-
- /*
- * If this is a DSYNC write, make sure we push it to stable storage now
- * that we've written data.
- */
- if (ret > 0 && (dio->flags & IOMAP_DIO_NEED_SYNC))
- ret = generic_write_sync(iocb, ret);
-
- inode_dio_end(file_inode(iocb->ki_filp));
- kfree(dio);
-
- return ret;
-}
-
-static void iomap_dio_complete_work(struct work_struct *work)
-{
- struct iomap_dio *dio = container_of(work, struct iomap_dio, aio.work);
- struct kiocb *iocb = dio->iocb;
-
- iocb->ki_complete(iocb, iomap_dio_complete(dio), 0);
-}
-
-/*
- * Set an error in the dio if none is set yet. We have to use cmpxchg
- * as the submission context and the completion context(s) can race to
- * update the error.
- */
-static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret)
-{
- cmpxchg(&dio->error, 0, ret);
-}
-
-static void iomap_dio_bio_end_io(struct bio *bio)
-{
- struct iomap_dio *dio = bio->bi_private;
- bool should_dirty = (dio->flags & IOMAP_DIO_DIRTY);
-
- if (bio->bi_status)
- iomap_dio_set_error(dio, blk_status_to_errno(bio->bi_status));
-
- if (atomic_dec_and_test(&dio->ref)) {
- if (dio->wait_for_completion) {
- struct task_struct *waiter = dio->submit.waiter;
- WRITE_ONCE(dio->submit.waiter, NULL);
- blk_wake_io_task(waiter);
- } else if (dio->flags & IOMAP_DIO_WRITE) {
- struct inode *inode = file_inode(dio->iocb->ki_filp);
-
- INIT_WORK(&dio->aio.work, iomap_dio_complete_work);
- queue_work(inode->i_sb->s_dio_done_wq, &dio->aio.work);
- } else {
- iomap_dio_complete_work(&dio->aio.work);
- }
- }
-
- if (should_dirty) {
- bio_check_pages_dirty(bio);
- } else {
- bio_release_pages(bio, false);
- bio_put(bio);
- }
-}
-
-static void
-iomap_dio_zero(struct iomap_dio *dio, struct iomap *iomap, loff_t pos,
- unsigned len)
-{
- struct page *page = ZERO_PAGE(0);
- int flags = REQ_SYNC | REQ_IDLE;
- struct bio *bio;
-
- bio = bio_alloc(GFP_KERNEL, 1);
- bio_set_dev(bio, iomap->bdev);
- bio->bi_iter.bi_sector = iomap_sector(iomap, pos);
- bio->bi_private = dio;
- bio->bi_end_io = iomap_dio_bio_end_io;
-
- get_page(page);
- __bio_add_page(bio, page, len, 0);
- bio_set_op_attrs(bio, REQ_OP_WRITE, flags);
- iomap_dio_submit_bio(dio, iomap, bio);
-}
-
-static loff_t
-iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length,
- struct iomap_dio *dio, struct iomap *iomap)
-{
- unsigned int blkbits = blksize_bits(bdev_logical_block_size(iomap->bdev));
- unsigned int fs_block_size = i_blocksize(inode), pad;
- unsigned int align = iov_iter_alignment(dio->submit.iter);
- struct iov_iter iter;
- struct bio *bio;
- bool need_zeroout = false;
- bool use_fua = false;
- int nr_pages, ret = 0;
- size_t copied = 0;
-
- if ((pos | length | align) & ((1 << blkbits) - 1))
- return -EINVAL;
-
- if (iomap->type == IOMAP_UNWRITTEN) {
- dio->flags |= IOMAP_DIO_UNWRITTEN;
- need_zeroout = true;
- }
-
- if (iomap->flags & IOMAP_F_SHARED)
- dio->flags |= IOMAP_DIO_COW;
-
- if (iomap->flags & IOMAP_F_NEW) {
- need_zeroout = true;
- } else if (iomap->type == IOMAP_MAPPED) {
- /*
- * Use a FUA write if we need datasync semantics, this is a pure
- * data IO that doesn't require any metadata updates (including
- * after IO completion such as unwritten extent conversion) and
- * the underlying device supports FUA. This allows us to avoid
- * cache flushes on IO completion.
- */
- if (!(iomap->flags & (IOMAP_F_SHARED|IOMAP_F_DIRTY)) &&
- (dio->flags & IOMAP_DIO_WRITE_FUA) &&
- blk_queue_fua(bdev_get_queue(iomap->bdev)))
- use_fua = true;
- }
-
- /*
- * Operate on a partial iter trimmed to the extent we were called for.
- * We'll update the iter in the dio once we're done with this extent.
- */
- iter = *dio->submit.iter;
- iov_iter_truncate(&iter, length);
-
- nr_pages = iov_iter_npages(&iter, BIO_MAX_PAGES);
- if (nr_pages <= 0)
- return nr_pages;
-
- if (need_zeroout) {
- /* zero out from the start of the block to the write offset */
- pad = pos & (fs_block_size - 1);
- if (pad)
- iomap_dio_zero(dio, iomap, pos - pad, pad);
- }
-
- do {
- size_t n;
- if (dio->error) {
- iov_iter_revert(dio->submit.iter, copied);
- return 0;
- }
-
- bio = bio_alloc(GFP_KERNEL, nr_pages);
- bio_set_dev(bio, iomap->bdev);
- bio->bi_iter.bi_sector = iomap_sector(iomap, pos);
- bio->bi_write_hint = dio->iocb->ki_hint;
- bio->bi_ioprio = dio->iocb->ki_ioprio;
- bio->bi_private = dio;
- bio->bi_end_io = iomap_dio_bio_end_io;
-
- ret = bio_iov_iter_get_pages(bio, &iter);
- if (unlikely(ret)) {
- /*
- * We have to stop part way through an IO. We must fall
- * through to the sub-block tail zeroing here, otherwise
- * this short IO may expose stale data in the tail of
- * the block we haven't written data to.
- */
- bio_put(bio);
- goto zero_tail;
- }
-
- n = bio->bi_iter.bi_size;
- if (dio->flags & IOMAP_DIO_WRITE) {
- bio->bi_opf = REQ_OP_WRITE | REQ_SYNC | REQ_IDLE;
- if (use_fua)
- bio->bi_opf |= REQ_FUA;
- else
- dio->flags &= ~IOMAP_DIO_WRITE_FUA;
- task_io_account_write(n);
- } else {
- bio->bi_opf = REQ_OP_READ;
- if (dio->flags & IOMAP_DIO_DIRTY)
- bio_set_pages_dirty(bio);
- }
-
- iov_iter_advance(dio->submit.iter, n);
-
- dio->size += n;
- pos += n;
- copied += n;
-
- nr_pages = iov_iter_npages(&iter, BIO_MAX_PAGES);
- iomap_dio_submit_bio(dio, iomap, bio);
- } while (nr_pages);
-
- /*
- * We need to zeroout the tail of a sub-block write if the extent type
- * requires zeroing or the write extends beyond EOF. If we don't zero
- * the block tail in the latter case, we can expose stale data via mmap
- * reads of the EOF block.
- */
-zero_tail:
- if (need_zeroout ||
- ((dio->flags & IOMAP_DIO_WRITE) && pos >= i_size_read(inode))) {
- /* zero out from the end of the write to the end of the block */
- pad = pos & (fs_block_size - 1);
- if (pad)
- iomap_dio_zero(dio, iomap, pos, fs_block_size - pad);
- }
- return copied ? copied : ret;
-}
-
-static loff_t
-iomap_dio_hole_actor(loff_t length, struct iomap_dio *dio)
-{
- length = iov_iter_zero(length, dio->submit.iter);
- dio->size += length;
- return length;
-}
-
-static loff_t
-iomap_dio_inline_actor(struct inode *inode, loff_t pos, loff_t length,
- struct iomap_dio *dio, struct iomap *iomap)
-{
- struct iov_iter *iter = dio->submit.iter;
- size_t copied;
-
- BUG_ON(pos + length > PAGE_SIZE - offset_in_page(iomap->inline_data));
-
- if (dio->flags & IOMAP_DIO_WRITE) {
- loff_t size = inode->i_size;
-
- if (pos > size)
- memset(iomap->inline_data + size, 0, pos - size);
- copied = copy_from_iter(iomap->inline_data + pos, length, iter);
- if (copied) {
- if (pos + copied > size)
- i_size_write(inode, pos + copied);
- mark_inode_dirty(inode);
- }
- } else {
- copied = copy_to_iter(iomap->inline_data + pos, length, iter);
- }
- dio->size += copied;
- return copied;
-}
-
-static loff_t
-iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
-{
- struct iomap_dio *dio = data;
-
- switch (iomap->type) {
- case IOMAP_HOLE:
- if (WARN_ON_ONCE(dio->flags & IOMAP_DIO_WRITE))
- return -EIO;
- return iomap_dio_hole_actor(length, dio);
- case IOMAP_UNWRITTEN:
- if (!(dio->flags & IOMAP_DIO_WRITE))
- return iomap_dio_hole_actor(length, dio);
- return iomap_dio_bio_actor(inode, pos, length, dio, iomap);
- case IOMAP_MAPPED:
- return iomap_dio_bio_actor(inode, pos, length, dio, iomap);
- case IOMAP_INLINE:
- return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
- default:
- WARN_ON_ONCE(1);
- return -EIO;
- }
-}
-
-/*
- * iomap_dio_rw() always completes O_[D]SYNC writes regardless of whether the IO
- * is being issued as AIO or not. This allows us to optimise pure data writes
- * to use REQ_FUA rather than requiring generic_write_sync() to issue a
- * REQ_FLUSH post write. This is slightly tricky because a single request here
- * can be mapped into multiple disjoint IOs and only a subset of the IOs issued
- * may be pure data writes. In that case, we still need to do a full data sync
- * completion.
- */
-ssize_t
-iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
- const struct iomap_ops *ops, iomap_dio_end_io_t end_io)
-{
- struct address_space *mapping = iocb->ki_filp->f_mapping;
- struct inode *inode = file_inode(iocb->ki_filp);
- size_t count = iov_iter_count(iter);
- loff_t pos = iocb->ki_pos, start = pos;
- loff_t end = iocb->ki_pos + count - 1, ret = 0;
- unsigned int flags = IOMAP_DIRECT;
- bool wait_for_completion = is_sync_kiocb(iocb);
- struct blk_plug plug;
- struct iomap_dio *dio;
-
- lockdep_assert_held(&inode->i_rwsem);
-
- if (!count)
- return 0;
-
- dio = kmalloc(sizeof(*dio), GFP_KERNEL);
- if (!dio)
- return -ENOMEM;
-
- dio->iocb = iocb;
- atomic_set(&dio->ref, 1);
- dio->size = 0;
- dio->i_size = i_size_read(inode);
- dio->end_io = end_io;
- dio->error = 0;
- dio->flags = 0;
-
- dio->submit.iter = iter;
- dio->submit.waiter = current;
- dio->submit.cookie = BLK_QC_T_NONE;
- dio->submit.last_queue = NULL;
-
- if (iov_iter_rw(iter) == READ) {
- if (pos >= dio->i_size)
- goto out_free_dio;
-
- if (iter_is_iovec(iter) && iov_iter_rw(iter) == READ)
- dio->flags |= IOMAP_DIO_DIRTY;
- } else {
- flags |= IOMAP_WRITE;
- dio->flags |= IOMAP_DIO_WRITE;
-
- /* for data sync or sync, we need sync completion processing */
- if (iocb->ki_flags & IOCB_DSYNC)
- dio->flags |= IOMAP_DIO_NEED_SYNC;
-
- /*
- * For datasync only writes, we optimistically try using FUA for
- * this IO. Any non-FUA write that occurs will clear this flag,
- * hence we know before completion whether a cache flush is
- * necessary.
- */
- if ((iocb->ki_flags & (IOCB_DSYNC | IOCB_SYNC)) == IOCB_DSYNC)
- dio->flags |= IOMAP_DIO_WRITE_FUA;
- }
-
- if (iocb->ki_flags & IOCB_NOWAIT) {
- if (filemap_range_has_page(mapping, start, end)) {
- ret = -EAGAIN;
- goto out_free_dio;
- }
- flags |= IOMAP_NOWAIT;
- }
-
- ret = filemap_write_and_wait_range(mapping, start, end);
- if (ret)
- goto out_free_dio;
-
- /*
- * Try to invalidate cache pages for the range we're direct
- * writing. If this invalidation fails, tough, the write will
- * still work, but racing two incompatible write paths is a
- * pretty crazy thing to do, so we don't support it 100%.
- */
- ret = invalidate_inode_pages2_range(mapping,
- start >> PAGE_SHIFT, end >> PAGE_SHIFT);
- if (ret)
- dio_warn_stale_pagecache(iocb->ki_filp);
- ret = 0;
-
- if (iov_iter_rw(iter) == WRITE && !wait_for_completion &&
- !inode->i_sb->s_dio_done_wq) {
- ret = sb_init_dio_done_wq(inode->i_sb);
- if (ret < 0)
- goto out_free_dio;
- }
-
- inode_dio_begin(inode);
-
- blk_start_plug(&plug);
- do {
- ret = iomap_apply(inode, pos, count, flags, ops, dio,
- iomap_dio_actor);
- if (ret <= 0) {
- /* magic error code to fall back to buffered I/O */
- if (ret == -ENOTBLK) {
- wait_for_completion = true;
- ret = 0;
- }
- break;
- }
- pos += ret;
-
- if (iov_iter_rw(iter) == READ && pos >= dio->i_size)
- break;
- } while ((count = iov_iter_count(iter)) > 0);
- blk_finish_plug(&plug);
-
- if (ret < 0)
- iomap_dio_set_error(dio, ret);
-
- /*
- * If all the writes we issued were FUA, we don't need to flush the
- * cache on IO completion. Clear the sync flag for this case.
- */
- if (dio->flags & IOMAP_DIO_WRITE_FUA)
- dio->flags &= ~IOMAP_DIO_NEED_SYNC;
-
- WRITE_ONCE(iocb->ki_cookie, dio->submit.cookie);
- WRITE_ONCE(iocb->private, dio->submit.last_queue);
-
- /*
- * We are about to drop our additional submission reference, which
- * might be the last reference to the dio. There are three three
- * different ways we can progress here:
- *
- * (a) If this is the last reference we will always complete and free
- * the dio ourselves.
- * (b) If this is not the last reference, and we serve an asynchronous
- * iocb, we must never touch the dio after the decrement, the
- * I/O completion handler will complete and free it.
- * (c) If this is not the last reference, but we serve a synchronous
- * iocb, the I/O completion handler will wake us up on the drop
- * of the final reference, and we will complete and free it here
- * after we got woken by the I/O completion handler.
- */
- dio->wait_for_completion = wait_for_completion;
- if (!atomic_dec_and_test(&dio->ref)) {
- if (!wait_for_completion)
- return -EIOCBQUEUED;
-
- for (;;) {
- set_current_state(TASK_UNINTERRUPTIBLE);
- if (!READ_ONCE(dio->submit.waiter))
- break;
-
- if (!(iocb->ki_flags & IOCB_HIPRI) ||
- !dio->submit.last_queue ||
- !blk_poll(dio->submit.last_queue,
- dio->submit.cookie, true))
- io_schedule();
- }
- __set_current_state(TASK_RUNNING);
- }
-
- return iomap_dio_complete(dio);
-
-out_free_dio:
- kfree(dio);
- return ret;
-}
-EXPORT_SYMBOL_GPL(iomap_dio_rw);
-
-/* Swapfile activation */
-
-#ifdef CONFIG_SWAP
-struct iomap_swapfile_info {
- struct iomap iomap; /* accumulated iomap */
- struct swap_info_struct *sis;
- uint64_t lowest_ppage; /* lowest physical addr seen (pages) */
- uint64_t highest_ppage; /* highest physical addr seen (pages) */
- unsigned long nr_pages; /* number of pages collected */
- int nr_extents; /* extent count */
-};
-
-/*
- * Collect physical extents for this swap file. Physical extents reported to
- * the swap code must be trimmed to align to a page boundary. The logical
- * offset within the file is irrelevant since the swapfile code maps logical
- * page numbers of the swap device to the physical page-aligned extents.
- */
-static int iomap_swapfile_add_extent(struct iomap_swapfile_info *isi)
-{
- struct iomap *iomap = &isi->iomap;
- unsigned long nr_pages;
- uint64_t first_ppage;
- uint64_t first_ppage_reported;
- uint64_t next_ppage;
- int error;
-
- /*
- * Round the start up and the end down so that the physical
- * extent aligns to a page boundary.
- */
- first_ppage = ALIGN(iomap->addr, PAGE_SIZE) >> PAGE_SHIFT;
- next_ppage = ALIGN_DOWN(iomap->addr + iomap->length, PAGE_SIZE) >>
- PAGE_SHIFT;
-
- /* Skip too-short physical extents. */
- if (first_ppage >= next_ppage)
- return 0;
- nr_pages = next_ppage - first_ppage;
-
- /*
- * Calculate how much swap space we're adding; the first page contains
- * the swap header and doesn't count. The mm still wants that first
- * page fed to add_swap_extent, however.
- */
- first_ppage_reported = first_ppage;
- if (iomap->offset == 0)
- first_ppage_reported++;
- if (isi->lowest_ppage > first_ppage_reported)
- isi->lowest_ppage = first_ppage_reported;
- if (isi->highest_ppage < (next_ppage - 1))
- isi->highest_ppage = next_ppage - 1;
-
- /* Add extent, set up for the next call. */
- error = add_swap_extent(isi->sis, isi->nr_pages, nr_pages, first_ppage);
- if (error < 0)
- return error;
- isi->nr_extents += error;
- isi->nr_pages += nr_pages;
- return 0;
-}
-
-/*
- * Accumulate iomaps for this swap file. We have to accumulate iomaps because
- * swap only cares about contiguous page-aligned physical extents and makes no
- * distinction between written and unwritten extents.
- */
-static loff_t iomap_swapfile_activate_actor(struct inode *inode, loff_t pos,
- loff_t count, void *data, struct iomap *iomap)
-{
- struct iomap_swapfile_info *isi = data;
- int error;
-
- switch (iomap->type) {
- case IOMAP_MAPPED:
- case IOMAP_UNWRITTEN:
- /* Only real or unwritten extents. */
- break;
- case IOMAP_INLINE:
- /* No inline data. */
- pr_err("swapon: file is inline\n");
- return -EINVAL;
- default:
- pr_err("swapon: file has unallocated extents\n");
- return -EINVAL;
- }
-
- /* No uncommitted metadata or shared blocks. */
- if (iomap->flags & IOMAP_F_DIRTY) {
- pr_err("swapon: file is not committed\n");
- return -EINVAL;
- }
- if (iomap->flags & IOMAP_F_SHARED) {
- pr_err("swapon: file has shared extents\n");
- return -EINVAL;
- }
-
- /* Only one bdev per swap file. */
- if (iomap->bdev != isi->sis->bdev) {
- pr_err("swapon: file is on multiple devices\n");
- return -EINVAL;
- }
-
- if (isi->iomap.length == 0) {
- /* No accumulated extent, so just store it. */
- memcpy(&isi->iomap, iomap, sizeof(isi->iomap));
- } else if (isi->iomap.addr + isi->iomap.length == iomap->addr) {
- /* Append this to the accumulated extent. */
- isi->iomap.length += iomap->length;
- } else {
- /* Otherwise, add the retained iomap and store this one. */
- error = iomap_swapfile_add_extent(isi);
- if (error)
- return error;
- memcpy(&isi->iomap, iomap, sizeof(isi->iomap));
- }
- return count;
-}
-
-/*
- * Iterate a swap file's iomaps to construct physical extents that can be
- * passed to the swapfile subsystem.
- */
-int iomap_swapfile_activate(struct swap_info_struct *sis,
- struct file *swap_file, sector_t *pagespan,
- const struct iomap_ops *ops)
-{
- struct iomap_swapfile_info isi = {
- .sis = sis,
- .lowest_ppage = (sector_t)-1ULL,
- };
- struct address_space *mapping = swap_file->f_mapping;
- struct inode *inode = mapping->host;
- loff_t pos = 0;
- loff_t len = ALIGN_DOWN(i_size_read(inode), PAGE_SIZE);
- loff_t ret;
-
- /*
- * Persist all file mapping metadata so that we won't have any
- * IOMAP_F_DIRTY iomaps.
- */
- ret = vfs_fsync(swap_file, 1);
- if (ret)
- return ret;
-
- while (len > 0) {
- ret = iomap_apply(inode, pos, len, IOMAP_REPORT,
- ops, &isi, iomap_swapfile_activate_actor);
- if (ret <= 0)
- return ret;
-
- pos += ret;
- len -= ret;
- }
-
- if (isi.iomap.length) {
- ret = iomap_swapfile_add_extent(&isi);
- if (ret)
- return ret;
- }
-
- *pagespan = 1 + isi.highest_ppage - isi.lowest_ppage;
- sis->max = isi.nr_pages;
- sis->pages = isi.nr_pages - 1;
- sis->highest_bit = isi.nr_pages - 1;
- return isi.nr_extents;
-}
-EXPORT_SYMBOL_GPL(iomap_swapfile_activate);
-#endif /* CONFIG_SWAP */
-
-static loff_t
-iomap_bmap_actor(struct inode *inode, loff_t pos, loff_t length,
- void *data, struct iomap *iomap)
-{
- sector_t *bno = data, addr;
-
- if (iomap->type == IOMAP_MAPPED) {
- addr = (pos - iomap->offset + iomap->addr) >> inode->i_blkbits;
- if (addr > INT_MAX)
- WARN(1, "would truncate bmap result\n");
- else
- *bno = addr;
- }
- return 0;
-}
-
-/* legacy ->bmap interface. 0 is the error return (!) */
-sector_t
-iomap_bmap(struct address_space *mapping, sector_t bno,
- const struct iomap_ops *ops)
-{
- struct inode *inode = mapping->host;
- loff_t pos = bno << inode->i_blkbits;
- unsigned blocksize = i_blocksize(inode);
-
- if (filemap_write_and_wait(mapping))
- return 0;
-
- bno = 0;
- iomap_apply(inode, pos, blocksize, 0, ops, &bno, iomap_bmap_actor);
- return bno;
-}
-EXPORT_SYMBOL_GPL(iomap_bmap);
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0-or-newer
+#
+# Copyright (c) 2019 Oracle.
+# All Rights Reserved.
+#
+obj-$(CONFIG_FS_IOMAP) += iomap.o
+
+iomap-y += \
+ apply.o \
+ buffered-io.o \
+ direct-io.o \
+ fiemap.o \
+ seek.o
+
+iomap-$(CONFIG_SWAP) += swapfile.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2010 Red Hat, Inc.
+ * Copyright (c) 2016-2018 Christoph Hellwig.
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+
+/*
+ * Execute a iomap write on a segment of the mapping that spans a
+ * contiguous range of pages that have identical block mapping state.
+ *
+ * This avoids the need to map pages individually, do individual allocations
+ * for each page and most importantly avoid the need for filesystem specific
+ * locking per page. Instead, all the operations are amortised over the entire
+ * range of pages. It is assumed that the filesystems will lock whatever
+ * resources they require in the iomap_begin call, and release them in the
+ * iomap_end call.
+ */
+loff_t
+iomap_apply(struct inode *inode, loff_t pos, loff_t length, unsigned flags,
+ const struct iomap_ops *ops, void *data, iomap_actor_t actor)
+{
+ struct iomap iomap = { 0 };
+ loff_t written = 0, ret;
+
+ /*
+ * Need to map a range from start position for length bytes. This can
+ * span multiple pages - it is only guaranteed to return a range of a
+ * single type of pages (e.g. all into a hole, all mapped or all
+ * unwritten). Failure at this point has nothing to undo.
+ *
+ * If allocation is required for this range, reserve the space now so
+ * that the allocation is guaranteed to succeed later on. Once we copy
+ * the data into the page cache pages, then we cannot fail otherwise we
+ * expose transient stale data. If the reserve fails, we can safely
+ * back out at this point as there is nothing to undo.
+ */
+ ret = ops->iomap_begin(inode, pos, length, flags, &iomap);
+ if (ret)
+ return ret;
+ if (WARN_ON(iomap.offset > pos))
+ return -EIO;
+ if (WARN_ON(iomap.length == 0))
+ return -EIO;
+
+ /*
+ * Cut down the length to the one actually provided by the filesystem,
+ * as it might not be able to give us the whole size that we requested.
+ */
+ if (iomap.offset + iomap.length < pos + length)
+ length = iomap.offset + iomap.length - pos;
+
+ /*
+ * Now that we have guaranteed that the space allocation will succeed.
+ * we can do the copy-in page by page without having to worry about
+ * failures exposing transient data.
+ */
+ written = actor(inode, pos, length, data, &iomap);
+
+ /*
+ * Now the data has been copied, commit the range we've copied. This
+ * should not fail unless the filesystem has had a fatal error.
+ */
+ if (ops->iomap_end) {
+ ret = ops->iomap_end(inode, pos, length,
+ written > 0 ? written : 0,
+ flags, &iomap);
+ }
+
+ return written ? written : ret;
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2010 Red Hat, Inc.
+ * Copyright (c) 2016-2018 Christoph Hellwig.
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+#include <linux/pagemap.h>
+#include <linux/uio.h>
+#include <linux/buffer_head.h>
+#include <linux/dax.h>
+#include <linux/writeback.h>
+#include <linux/swap.h>
+#include <linux/bio.h>
+#include <linux/sched/signal.h>
+#include <linux/migrate.h>
+
+#include "../internal.h"
+
+static struct iomap_page *
+iomap_page_create(struct inode *inode, struct page *page)
+{
+ struct iomap_page *iop = to_iomap_page(page);
+
+ if (iop || i_blocksize(inode) == PAGE_SIZE)
+ return iop;
+
+ iop = kmalloc(sizeof(*iop), GFP_NOFS | __GFP_NOFAIL);
+ atomic_set(&iop->read_count, 0);
+ atomic_set(&iop->write_count, 0);
+ bitmap_zero(iop->uptodate, PAGE_SIZE / SECTOR_SIZE);
+
+ /*
+ * migrate_page_move_mapping() assumes that pages with private data have
+ * their count elevated by 1.
+ */
+ get_page(page);
+ set_page_private(page, (unsigned long)iop);
+ SetPagePrivate(page);
+ return iop;
+}
+
+static void
+iomap_page_release(struct page *page)
+{
+ struct iomap_page *iop = to_iomap_page(page);
+
+ if (!iop)
+ return;
+ WARN_ON_ONCE(atomic_read(&iop->read_count));
+ WARN_ON_ONCE(atomic_read(&iop->write_count));
+ ClearPagePrivate(page);
+ set_page_private(page, 0);
+ put_page(page);
+ kfree(iop);
+}
+
+/*
+ * Calculate the range inside the page that we actually need to read.
+ */
+static void
+iomap_adjust_read_range(struct inode *inode, struct iomap_page *iop,
+ loff_t *pos, loff_t length, unsigned *offp, unsigned *lenp)
+{
+ loff_t orig_pos = *pos;
+ loff_t isize = i_size_read(inode);
+ unsigned block_bits = inode->i_blkbits;
+ unsigned block_size = (1 << block_bits);
+ unsigned poff = offset_in_page(*pos);
+ unsigned plen = min_t(loff_t, PAGE_SIZE - poff, length);
+ unsigned first = poff >> block_bits;
+ unsigned last = (poff + plen - 1) >> block_bits;
+
+ /*
+ * If the block size is smaller than the page size we need to check the
+ * per-block uptodate status and adjust the offset and length if needed
+ * to avoid reading in already uptodate ranges.
+ */
+ if (iop) {
+ unsigned int i;
+
+ /* move forward for each leading block marked uptodate */
+ for (i = first; i <= last; i++) {
+ if (!test_bit(i, iop->uptodate))
+ break;
+ *pos += block_size;
+ poff += block_size;
+ plen -= block_size;
+ first++;
+ }
+
+ /* truncate len if we find any trailing uptodate block(s) */
+ for ( ; i <= last; i++) {
+ if (test_bit(i, iop->uptodate)) {
+ plen -= (last - i + 1) * block_size;
+ last = i - 1;
+ break;
+ }
+ }
+ }
+
+ /*
+ * If the extent spans the block that contains the i_size we need to
+ * handle both halves separately so that we properly zero data in the
+ * page cache for blocks that are entirely outside of i_size.
+ */
+ if (orig_pos <= isize && orig_pos + length > isize) {
+ unsigned end = offset_in_page(isize - 1) >> block_bits;
+
+ if (first <= end && last > end)
+ plen -= (last - end) * block_size;
+ }
+
+ *offp = poff;
+ *lenp = plen;
+}
+
+static void
+iomap_set_range_uptodate(struct page *page, unsigned off, unsigned len)
+{
+ struct iomap_page *iop = to_iomap_page(page);
+ struct inode *inode = page->mapping->host;
+ unsigned first = off >> inode->i_blkbits;
+ unsigned last = (off + len - 1) >> inode->i_blkbits;
+ unsigned int i;
+ bool uptodate = true;
+
+ if (iop) {
+ for (i = 0; i < PAGE_SIZE / i_blocksize(inode); i++) {
+ if (i >= first && i <= last)
+ set_bit(i, iop->uptodate);
+ else if (!test_bit(i, iop->uptodate))
+ uptodate = false;
+ }
+ }
+
+ if (uptodate && !PageError(page))
+ SetPageUptodate(page);
+}
+
+static void
+iomap_read_finish(struct iomap_page *iop, struct page *page)
+{
+ if (!iop || atomic_dec_and_test(&iop->read_count))
+ unlock_page(page);
+}
+
+static void
+iomap_read_page_end_io(struct bio_vec *bvec, int error)
+{
+ struct page *page = bvec->bv_page;
+ struct iomap_page *iop = to_iomap_page(page);
+
+ if (unlikely(error)) {
+ ClearPageUptodate(page);
+ SetPageError(page);
+ } else {
+ iomap_set_range_uptodate(page, bvec->bv_offset, bvec->bv_len);
+ }
+
+ iomap_read_finish(iop, page);
+}
+
+static void
+iomap_read_end_io(struct bio *bio)
+{
+ int error = blk_status_to_errno(bio->bi_status);
+ struct bio_vec *bvec;
+ struct bvec_iter_all iter_all;
+
+ bio_for_each_segment_all(bvec, bio, iter_all)
+ iomap_read_page_end_io(bvec, error);
+ bio_put(bio);
+}
+
+struct iomap_readpage_ctx {
+ struct page *cur_page;
+ bool cur_page_in_bio;
+ bool is_readahead;
+ struct bio *bio;
+ struct list_head *pages;
+};
+
+static void
+iomap_read_inline_data(struct inode *inode, struct page *page,
+ struct iomap *iomap)
+{
+ size_t size = i_size_read(inode);
+ void *addr;
+
+ if (PageUptodate(page))
+ return;
+
+ BUG_ON(page->index);
+ BUG_ON(size > PAGE_SIZE - offset_in_page(iomap->inline_data));
+
+ addr = kmap_atomic(page);
+ memcpy(addr, iomap->inline_data, size);
+ memset(addr + size, 0, PAGE_SIZE - size);
+ kunmap_atomic(addr);
+ SetPageUptodate(page);
+}
+
+static loff_t
+iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
+ struct iomap *iomap)
+{
+ struct iomap_readpage_ctx *ctx = data;
+ struct page *page = ctx->cur_page;
+ struct iomap_page *iop = iomap_page_create(inode, page);
+ bool same_page = false, is_contig = false;
+ loff_t orig_pos = pos;
+ unsigned poff, plen;
+ sector_t sector;
+
+ if (iomap->type == IOMAP_INLINE) {
+ WARN_ON_ONCE(pos);
+ iomap_read_inline_data(inode, page, iomap);
+ return PAGE_SIZE;
+ }
+
+ /* zero post-eof blocks as the page may be mapped */
+ iomap_adjust_read_range(inode, iop, &pos, length, &poff, &plen);
+ if (plen == 0)
+ goto done;
+
+ if (iomap->type != IOMAP_MAPPED || pos >= i_size_read(inode)) {
+ zero_user(page, poff, plen);
+ iomap_set_range_uptodate(page, poff, plen);
+ goto done;
+ }
+
+ ctx->cur_page_in_bio = true;
+
+ /*
+ * Try to merge into a previous segment if we can.
+ */
+ sector = iomap_sector(iomap, pos);
+ if (ctx->bio && bio_end_sector(ctx->bio) == sector)
+ is_contig = true;
+
+ if (is_contig &&
+ __bio_try_merge_page(ctx->bio, page, plen, poff, &same_page)) {
+ if (!same_page && iop)
+ atomic_inc(&iop->read_count);
+ goto done;
+ }
+
+ /*
+ * If we start a new segment we need to increase the read count, and we
+ * need to do so before submitting any previous full bio to make sure
+ * that we don't prematurely unlock the page.
+ */
+ if (iop)
+ atomic_inc(&iop->read_count);
+
+ if (!ctx->bio || !is_contig || bio_full(ctx->bio, plen)) {
+ gfp_t gfp = mapping_gfp_constraint(page->mapping, GFP_KERNEL);
+ int nr_vecs = (length + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ if (ctx->bio)
+ submit_bio(ctx->bio);
+
+ if (ctx->is_readahead) /* same as readahead_gfp_mask */
+ gfp |= __GFP_NORETRY | __GFP_NOWARN;
+ ctx->bio = bio_alloc(gfp, min(BIO_MAX_PAGES, nr_vecs));
+ ctx->bio->bi_opf = REQ_OP_READ;
+ if (ctx->is_readahead)
+ ctx->bio->bi_opf |= REQ_RAHEAD;
+ ctx->bio->bi_iter.bi_sector = sector;
+ bio_set_dev(ctx->bio, iomap->bdev);
+ ctx->bio->bi_end_io = iomap_read_end_io;
+ }
+
+ bio_add_page(ctx->bio, page, plen, poff);
+done:
+ /*
+ * Move the caller beyond our range so that it keeps making progress.
+ * For that we have to include any leading non-uptodate ranges, but
+ * we can skip trailing ones as they will be handled in the next
+ * iteration.
+ */
+ return pos - orig_pos + plen;
+}
+
+int
+iomap_readpage(struct page *page, const struct iomap_ops *ops)
+{
+ struct iomap_readpage_ctx ctx = { .cur_page = page };
+ struct inode *inode = page->mapping->host;
+ unsigned poff;
+ loff_t ret;
+
+ for (poff = 0; poff < PAGE_SIZE; poff += ret) {
+ ret = iomap_apply(inode, page_offset(page) + poff,
+ PAGE_SIZE - poff, 0, ops, &ctx,
+ iomap_readpage_actor);
+ if (ret <= 0) {
+ WARN_ON_ONCE(ret == 0);
+ SetPageError(page);
+ break;
+ }
+ }
+
+ if (ctx.bio) {
+ submit_bio(ctx.bio);
+ WARN_ON_ONCE(!ctx.cur_page_in_bio);
+ } else {
+ WARN_ON_ONCE(ctx.cur_page_in_bio);
+ unlock_page(page);
+ }
+
+ /*
+ * Just like mpage_readpages and block_read_full_page we always
+ * return 0 and just mark the page as PageError on errors. This
+ * should be cleaned up all through the stack eventually.
+ */
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iomap_readpage);
+
+static struct page *
+iomap_next_page(struct inode *inode, struct list_head *pages, loff_t pos,
+ loff_t length, loff_t *done)
+{
+ while (!list_empty(pages)) {
+ struct page *page = lru_to_page(pages);
+
+ if (page_offset(page) >= (u64)pos + length)
+ break;
+
+ list_del(&page->lru);
+ if (!add_to_page_cache_lru(page, inode->i_mapping, page->index,
+ GFP_NOFS))
+ return page;
+
+ /*
+ * If we already have a page in the page cache at index we are
+ * done. Upper layers don't care if it is uptodate after the
+ * readpages call itself as every page gets checked again once
+ * actually needed.
+ */
+ *done += PAGE_SIZE;
+ put_page(page);
+ }
+
+ return NULL;
+}
+
+static loff_t
+iomap_readpages_actor(struct inode *inode, loff_t pos, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ struct iomap_readpage_ctx *ctx = data;
+ loff_t done, ret;
+
+ for (done = 0; done < length; done += ret) {
+ if (ctx->cur_page && offset_in_page(pos + done) == 0) {
+ if (!ctx->cur_page_in_bio)
+ unlock_page(ctx->cur_page);
+ put_page(ctx->cur_page);
+ ctx->cur_page = NULL;
+ }
+ if (!ctx->cur_page) {
+ ctx->cur_page = iomap_next_page(inode, ctx->pages,
+ pos, length, &done);
+ if (!ctx->cur_page)
+ break;
+ ctx->cur_page_in_bio = false;
+ }
+ ret = iomap_readpage_actor(inode, pos + done, length - done,
+ ctx, iomap);
+ }
+
+ return done;
+}
+
+int
+iomap_readpages(struct address_space *mapping, struct list_head *pages,
+ unsigned nr_pages, const struct iomap_ops *ops)
+{
+ struct iomap_readpage_ctx ctx = {
+ .pages = pages,
+ .is_readahead = true,
+ };
+ loff_t pos = page_offset(list_entry(pages->prev, struct page, lru));
+ loff_t last = page_offset(list_entry(pages->next, struct page, lru));
+ loff_t length = last - pos + PAGE_SIZE, ret = 0;
+
+ while (length > 0) {
+ ret = iomap_apply(mapping->host, pos, length, 0, ops,
+ &ctx, iomap_readpages_actor);
+ if (ret <= 0) {
+ WARN_ON_ONCE(ret == 0);
+ goto done;
+ }
+ pos += ret;
+ length -= ret;
+ }
+ ret = 0;
+done:
+ if (ctx.bio)
+ submit_bio(ctx.bio);
+ if (ctx.cur_page) {
+ if (!ctx.cur_page_in_bio)
+ unlock_page(ctx.cur_page);
+ put_page(ctx.cur_page);
+ }
+
+ /*
+ * Check that we didn't lose a page due to the arcance calling
+ * conventions..
+ */
+ WARN_ON_ONCE(!ret && !list_empty(ctx.pages));
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iomap_readpages);
+
+/*
+ * iomap_is_partially_uptodate checks whether blocks within a page are
+ * uptodate or not.
+ *
+ * Returns true if all blocks which correspond to a file portion
+ * we want to read within the page are uptodate.
+ */
+int
+iomap_is_partially_uptodate(struct page *page, unsigned long from,
+ unsigned long count)
+{
+ struct iomap_page *iop = to_iomap_page(page);
+ struct inode *inode = page->mapping->host;
+ unsigned len, first, last;
+ unsigned i;
+
+ /* Limit range to one page */
+ len = min_t(unsigned, PAGE_SIZE - from, count);
+
+ /* First and last blocks in range within page */
+ first = from >> inode->i_blkbits;
+ last = (from + len - 1) >> inode->i_blkbits;
+
+ if (iop) {
+ for (i = first; i <= last; i++)
+ if (!test_bit(i, iop->uptodate))
+ return 0;
+ return 1;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate);
+
+int
+iomap_releasepage(struct page *page, gfp_t gfp_mask)
+{
+ /*
+ * mm accommodates an old ext3 case where clean pages might not have had
+ * the dirty bit cleared. Thus, it can send actual dirty pages to
+ * ->releasepage() via shrink_active_list(), skip those here.
+ */
+ if (PageDirty(page) || PageWriteback(page))
+ return 0;
+ iomap_page_release(page);
+ return 1;
+}
+EXPORT_SYMBOL_GPL(iomap_releasepage);
+
+void
+iomap_invalidatepage(struct page *page, unsigned int offset, unsigned int len)
+{
+ /*
+ * If we are invalidating the entire page, clear the dirty state from it
+ * and release it to avoid unnecessary buildup of the LRU.
+ */
+ if (offset == 0 && len == PAGE_SIZE) {
+ WARN_ON_ONCE(PageWriteback(page));
+ cancel_dirty_page(page);
+ iomap_page_release(page);
+ }
+}
+EXPORT_SYMBOL_GPL(iomap_invalidatepage);
+
+#ifdef CONFIG_MIGRATION
+int
+iomap_migrate_page(struct address_space *mapping, struct page *newpage,
+ struct page *page, enum migrate_mode mode)
+{
+ int ret;
+
+ ret = migrate_page_move_mapping(mapping, newpage, page, 0);
+ if (ret != MIGRATEPAGE_SUCCESS)
+ return ret;
+
+ if (page_has_private(page)) {
+ ClearPagePrivate(page);
+ get_page(newpage);
+ set_page_private(newpage, page_private(page));
+ set_page_private(page, 0);
+ put_page(page);
+ SetPagePrivate(newpage);
+ }
+
+ if (mode != MIGRATE_SYNC_NO_COPY)
+ migrate_page_copy(newpage, page);
+ else
+ migrate_page_states(newpage, page);
+ return MIGRATEPAGE_SUCCESS;
+}
+EXPORT_SYMBOL_GPL(iomap_migrate_page);
+#endif /* CONFIG_MIGRATION */
+
+static void
+iomap_write_failed(struct inode *inode, loff_t pos, unsigned len)
+{
+ loff_t i_size = i_size_read(inode);
+
+ /*
+ * Only truncate newly allocated pages beyoned EOF, even if the
+ * write started inside the existing inode size.
+ */
+ if (pos + len > i_size)
+ truncate_pagecache_range(inode, max(pos, i_size), pos + len);
+}
+
+static int
+iomap_read_page_sync(struct inode *inode, loff_t block_start, struct page *page,
+ unsigned poff, unsigned plen, unsigned from, unsigned to,
+ struct iomap *iomap)
+{
+ struct bio_vec bvec;
+ struct bio bio;
+
+ if (iomap->type != IOMAP_MAPPED || block_start >= i_size_read(inode)) {
+ zero_user_segments(page, poff, from, to, poff + plen);
+ iomap_set_range_uptodate(page, poff, plen);
+ return 0;
+ }
+
+ bio_init(&bio, &bvec, 1);
+ bio.bi_opf = REQ_OP_READ;
+ bio.bi_iter.bi_sector = iomap_sector(iomap, block_start);
+ bio_set_dev(&bio, iomap->bdev);
+ __bio_add_page(&bio, page, plen, poff);
+ return submit_bio_wait(&bio);
+}
+
+static int
+__iomap_write_begin(struct inode *inode, loff_t pos, unsigned len,
+ struct page *page, struct iomap *iomap)
+{
+ struct iomap_page *iop = iomap_page_create(inode, page);
+ loff_t block_size = i_blocksize(inode);
+ loff_t block_start = pos & ~(block_size - 1);
+ loff_t block_end = (pos + len + block_size - 1) & ~(block_size - 1);
+ unsigned from = offset_in_page(pos), to = from + len, poff, plen;
+ int status = 0;
+
+ if (PageUptodate(page))
+ return 0;
+
+ do {
+ iomap_adjust_read_range(inode, iop, &block_start,
+ block_end - block_start, &poff, &plen);
+ if (plen == 0)
+ break;
+
+ if ((from > poff && from < poff + plen) ||
+ (to > poff && to < poff + plen)) {
+ status = iomap_read_page_sync(inode, block_start, page,
+ poff, plen, from, to, iomap);
+ if (status)
+ break;
+ }
+
+ } while ((block_start += plen) < block_end);
+
+ return status;
+}
+
+static int
+iomap_write_begin(struct inode *inode, loff_t pos, unsigned len, unsigned flags,
+ struct page **pagep, struct iomap *iomap)
+{
+ const struct iomap_page_ops *page_ops = iomap->page_ops;
+ pgoff_t index = pos >> PAGE_SHIFT;
+ struct page *page;
+ int status = 0;
+
+ BUG_ON(pos + len > iomap->offset + iomap->length);
+
+ if (fatal_signal_pending(current))
+ return -EINTR;
+
+ if (page_ops && page_ops->page_prepare) {
+ status = page_ops->page_prepare(inode, pos, len, iomap);
+ if (status)
+ return status;
+ }
+
+ page = grab_cache_page_write_begin(inode->i_mapping, index, flags);
+ if (!page) {
+ status = -ENOMEM;
+ goto out_no_page;
+ }
+
+ if (iomap->type == IOMAP_INLINE)
+ iomap_read_inline_data(inode, page, iomap);
+ else if (iomap->flags & IOMAP_F_BUFFER_HEAD)
+ status = __block_write_begin_int(page, pos, len, NULL, iomap);
+ else
+ status = __iomap_write_begin(inode, pos, len, page, iomap);
+
+ if (unlikely(status))
+ goto out_unlock;
+
+ *pagep = page;
+ return 0;
+
+out_unlock:
+ unlock_page(page);
+ put_page(page);
+ iomap_write_failed(inode, pos, len);
+
+out_no_page:
+ if (page_ops && page_ops->page_done)
+ page_ops->page_done(inode, pos, 0, NULL, iomap);
+ return status;
+}
+
+int
+iomap_set_page_dirty(struct page *page)
+{
+ struct address_space *mapping = page_mapping(page);
+ int newly_dirty;
+
+ if (unlikely(!mapping))
+ return !TestSetPageDirty(page);
+
+ /*
+ * Lock out page->mem_cgroup migration to keep PageDirty
+ * synchronized with per-memcg dirty page counters.
+ */
+ lock_page_memcg(page);
+ newly_dirty = !TestSetPageDirty(page);
+ if (newly_dirty)
+ __set_page_dirty(page, mapping, 0);
+ unlock_page_memcg(page);
+
+ if (newly_dirty)
+ __mark_inode_dirty(mapping->host, I_DIRTY_PAGES);
+ return newly_dirty;
+}
+EXPORT_SYMBOL_GPL(iomap_set_page_dirty);
+
+static int
+__iomap_write_end(struct inode *inode, loff_t pos, unsigned len,
+ unsigned copied, struct page *page, struct iomap *iomap)
+{
+ flush_dcache_page(page);
+
+ /*
+ * The blocks that were entirely written will now be uptodate, so we
+ * don't have to worry about a readpage reading them and overwriting a
+ * partial write. However if we have encountered a short write and only
+ * partially written into a block, it will not be marked uptodate, so a
+ * readpage might come in and destroy our partial write.
+ *
+ * Do the simplest thing, and just treat any short write to a non
+ * uptodate page as a zero-length write, and force the caller to redo
+ * the whole thing.
+ */
+ if (unlikely(copied < len && !PageUptodate(page)))
+ return 0;
+ iomap_set_range_uptodate(page, offset_in_page(pos), len);
+ iomap_set_page_dirty(page);
+ return copied;
+}
+
+static int
+iomap_write_end_inline(struct inode *inode, struct page *page,
+ struct iomap *iomap, loff_t pos, unsigned copied)
+{
+ void *addr;
+
+ WARN_ON_ONCE(!PageUptodate(page));
+ BUG_ON(pos + copied > PAGE_SIZE - offset_in_page(iomap->inline_data));
+
+ addr = kmap_atomic(page);
+ memcpy(iomap->inline_data + pos, addr + pos, copied);
+ kunmap_atomic(addr);
+
+ mark_inode_dirty(inode);
+ return copied;
+}
+
+static int
+iomap_write_end(struct inode *inode, loff_t pos, unsigned len,
+ unsigned copied, struct page *page, struct iomap *iomap)
+{
+ const struct iomap_page_ops *page_ops = iomap->page_ops;
+ loff_t old_size = inode->i_size;
+ int ret;
+
+ if (iomap->type == IOMAP_INLINE) {
+ ret = iomap_write_end_inline(inode, page, iomap, pos, copied);
+ } else if (iomap->flags & IOMAP_F_BUFFER_HEAD) {
+ ret = block_write_end(NULL, inode->i_mapping, pos, len, copied,
+ page, NULL);
+ } else {
+ ret = __iomap_write_end(inode, pos, len, copied, page, iomap);
+ }
+
+ /*
+ * Update the in-memory inode size after copying the data into the page
+ * cache. It's up to the file system to write the updated size to disk,
+ * preferably after I/O completion so that no stale data is exposed.
+ */
+ if (pos + ret > old_size) {
+ i_size_write(inode, pos + ret);
+ iomap->flags |= IOMAP_F_SIZE_CHANGED;
+ }
+ unlock_page(page);
+
+ if (old_size < pos)
+ pagecache_isize_extended(inode, old_size, pos);
+ if (page_ops && page_ops->page_done)
+ page_ops->page_done(inode, pos, ret, page, iomap);
+ put_page(page);
+
+ if (ret < len)
+ iomap_write_failed(inode, pos, len);
+ return ret;
+}
+
+static loff_t
+iomap_write_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
+ struct iomap *iomap)
+{
+ struct iov_iter *i = data;
+ long status = 0;
+ ssize_t written = 0;
+ unsigned int flags = AOP_FLAG_NOFS;
+
+ do {
+ struct page *page;
+ unsigned long offset; /* Offset into pagecache page */
+ unsigned long bytes; /* Bytes to write to page */
+ size_t copied; /* Bytes copied from user */
+
+ offset = offset_in_page(pos);
+ bytes = min_t(unsigned long, PAGE_SIZE - offset,
+ iov_iter_count(i));
+again:
+ if (bytes > length)
+ bytes = length;
+
+ /*
+ * Bring in the user page that we will copy from _first_.
+ * Otherwise there's a nasty deadlock on copying from the
+ * same page as we're writing to, without it being marked
+ * up-to-date.
+ *
+ * Not only is this an optimisation, but it is also required
+ * to check that the address is actually valid, when atomic
+ * usercopies are used, below.
+ */
+ if (unlikely(iov_iter_fault_in_readable(i, bytes))) {
+ status = -EFAULT;
+ break;
+ }
+
+ status = iomap_write_begin(inode, pos, bytes, flags, &page,
+ iomap);
+ if (unlikely(status))
+ break;
+
+ if (mapping_writably_mapped(inode->i_mapping))
+ flush_dcache_page(page);
+
+ copied = iov_iter_copy_from_user_atomic(page, i, offset, bytes);
+
+ flush_dcache_page(page);
+
+ status = iomap_write_end(inode, pos, bytes, copied, page,
+ iomap);
+ if (unlikely(status < 0))
+ break;
+ copied = status;
+
+ cond_resched();
+
+ iov_iter_advance(i, copied);
+ if (unlikely(copied == 0)) {
+ /*
+ * If we were unable to copy any data at all, we must
+ * fall back to a single segment length write.
+ *
+ * If we didn't fallback here, we could livelock
+ * because not all segments in the iov can be copied at
+ * once without a pagefault.
+ */
+ bytes = min_t(unsigned long, PAGE_SIZE - offset,
+ iov_iter_single_seg_count(i));
+ goto again;
+ }
+ pos += copied;
+ written += copied;
+ length -= copied;
+
+ balance_dirty_pages_ratelimited(inode->i_mapping);
+ } while (iov_iter_count(i) && length);
+
+ return written ? written : status;
+}
+
+ssize_t
+iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *iter,
+ const struct iomap_ops *ops)
+{
+ struct inode *inode = iocb->ki_filp->f_mapping->host;
+ loff_t pos = iocb->ki_pos, ret = 0, written = 0;
+
+ while (iov_iter_count(iter)) {
+ ret = iomap_apply(inode, pos, iov_iter_count(iter),
+ IOMAP_WRITE, ops, iter, iomap_write_actor);
+ if (ret <= 0)
+ break;
+ pos += ret;
+ written += ret;
+ }
+
+ return written ? written : ret;
+}
+EXPORT_SYMBOL_GPL(iomap_file_buffered_write);
+
+static struct page *
+__iomap_read_page(struct inode *inode, loff_t offset)
+{
+ struct address_space *mapping = inode->i_mapping;
+ struct page *page;
+
+ page = read_mapping_page(mapping, offset >> PAGE_SHIFT, NULL);
+ if (IS_ERR(page))
+ return page;
+ if (!PageUptodate(page)) {
+ put_page(page);
+ return ERR_PTR(-EIO);
+ }
+ return page;
+}
+
+static loff_t
+iomap_dirty_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
+ struct iomap *iomap)
+{
+ long status = 0;
+ ssize_t written = 0;
+
+ do {
+ struct page *page, *rpage;
+ unsigned long offset; /* Offset into pagecache page */
+ unsigned long bytes; /* Bytes to write to page */
+
+ offset = offset_in_page(pos);
+ bytes = min_t(loff_t, PAGE_SIZE - offset, length);
+
+ rpage = __iomap_read_page(inode, pos);
+ if (IS_ERR(rpage))
+ return PTR_ERR(rpage);
+
+ status = iomap_write_begin(inode, pos, bytes,
+ AOP_FLAG_NOFS, &page, iomap);
+ put_page(rpage);
+ if (unlikely(status))
+ return status;
+
+ WARN_ON_ONCE(!PageUptodate(page));
+
+ status = iomap_write_end(inode, pos, bytes, bytes, page, iomap);
+ if (unlikely(status <= 0)) {
+ if (WARN_ON_ONCE(status == 0))
+ return -EIO;
+ return status;
+ }
+
+ cond_resched();
+
+ pos += status;
+ written += status;
+ length -= status;
+
+ balance_dirty_pages_ratelimited(inode->i_mapping);
+ } while (length);
+
+ return written;
+}
+
+int
+iomap_file_dirty(struct inode *inode, loff_t pos, loff_t len,
+ const struct iomap_ops *ops)
+{
+ loff_t ret;
+
+ while (len) {
+ ret = iomap_apply(inode, pos, len, IOMAP_WRITE, ops, NULL,
+ iomap_dirty_actor);
+ if (ret <= 0)
+ return ret;
+ pos += ret;
+ len -= ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iomap_file_dirty);
+
+static int iomap_zero(struct inode *inode, loff_t pos, unsigned offset,
+ unsigned bytes, struct iomap *iomap)
+{
+ struct page *page;
+ int status;
+
+ status = iomap_write_begin(inode, pos, bytes, AOP_FLAG_NOFS, &page,
+ iomap);
+ if (status)
+ return status;
+
+ zero_user(page, offset, bytes);
+ mark_page_accessed(page);
+
+ return iomap_write_end(inode, pos, bytes, bytes, page, iomap);
+}
+
+static int iomap_dax_zero(loff_t pos, unsigned offset, unsigned bytes,
+ struct iomap *iomap)
+{
+ return __dax_zero_page_range(iomap->bdev, iomap->dax_dev,
+ iomap_sector(iomap, pos & PAGE_MASK), offset, bytes);
+}
+
+static loff_t
+iomap_zero_range_actor(struct inode *inode, loff_t pos, loff_t count,
+ void *data, struct iomap *iomap)
+{
+ bool *did_zero = data;
+ loff_t written = 0;
+ int status;
+
+ /* already zeroed? we're done. */
+ if (iomap->type == IOMAP_HOLE || iomap->type == IOMAP_UNWRITTEN)
+ return count;
+
+ do {
+ unsigned offset, bytes;
+
+ offset = offset_in_page(pos);
+ bytes = min_t(loff_t, PAGE_SIZE - offset, count);
+
+ if (IS_DAX(inode))
+ status = iomap_dax_zero(pos, offset, bytes, iomap);
+ else
+ status = iomap_zero(inode, pos, offset, bytes, iomap);
+ if (status < 0)
+ return status;
+
+ pos += bytes;
+ count -= bytes;
+ written += bytes;
+ if (did_zero)
+ *did_zero = true;
+ } while (count > 0);
+
+ return written;
+}
+
+int
+iomap_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero,
+ const struct iomap_ops *ops)
+{
+ loff_t ret;
+
+ while (len > 0) {
+ ret = iomap_apply(inode, pos, len, IOMAP_ZERO,
+ ops, did_zero, iomap_zero_range_actor);
+ if (ret <= 0)
+ return ret;
+
+ pos += ret;
+ len -= ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iomap_zero_range);
+
+int
+iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,
+ const struct iomap_ops *ops)
+{
+ unsigned int blocksize = i_blocksize(inode);
+ unsigned int off = pos & (blocksize - 1);
+
+ /* Block boundary? Nothing to do */
+ if (!off)
+ return 0;
+ return iomap_zero_range(inode, pos, blocksize - off, did_zero, ops);
+}
+EXPORT_SYMBOL_GPL(iomap_truncate_page);
+
+static loff_t
+iomap_page_mkwrite_actor(struct inode *inode, loff_t pos, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ struct page *page = data;
+ int ret;
+
+ if (iomap->flags & IOMAP_F_BUFFER_HEAD) {
+ ret = __block_write_begin_int(page, pos, length, NULL, iomap);
+ if (ret)
+ return ret;
+ block_commit_write(page, 0, length);
+ } else {
+ WARN_ON_ONCE(!PageUptodate(page));
+ iomap_page_create(inode, page);
+ set_page_dirty(page);
+ }
+
+ return length;
+}
+
+vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops)
+{
+ struct page *page = vmf->page;
+ struct inode *inode = file_inode(vmf->vma->vm_file);
+ unsigned long length;
+ loff_t offset, size;
+ ssize_t ret;
+
+ lock_page(page);
+ size = i_size_read(inode);
+ if ((page->mapping != inode->i_mapping) ||
+ (page_offset(page) > size)) {
+ /* We overload EFAULT to mean page got truncated */
+ ret = -EFAULT;
+ goto out_unlock;
+ }
+
+ /* page is wholly or partially inside EOF */
+ if (((page->index + 1) << PAGE_SHIFT) > size)
+ length = offset_in_page(size);
+ else
+ length = PAGE_SIZE;
+
+ offset = page_offset(page);
+ while (length > 0) {
+ ret = iomap_apply(inode, offset, length,
+ IOMAP_WRITE | IOMAP_FAULT, ops, page,
+ iomap_page_mkwrite_actor);
+ if (unlikely(ret <= 0))
+ goto out_unlock;
+ offset += ret;
+ length -= ret;
+ }
+
+ wait_for_stable_page(page);
+ return VM_FAULT_LOCKED;
+out_unlock:
+ unlock_page(page);
+ return block_page_mkwrite_return(ret);
+}
+EXPORT_SYMBOL_GPL(iomap_page_mkwrite);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2010 Red Hat, Inc.
+ * Copyright (c) 2016-2018 Christoph Hellwig.
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+#include <linux/backing-dev.h>
+#include <linux/uio.h>
+#include <linux/task_io_accounting_ops.h>
+
+#include "../internal.h"
+
+/*
+ * Private flags for iomap_dio, must not overlap with the public ones in
+ * iomap.h:
+ */
+#define IOMAP_DIO_WRITE_FUA (1 << 28)
+#define IOMAP_DIO_NEED_SYNC (1 << 29)
+#define IOMAP_DIO_WRITE (1 << 30)
+#define IOMAP_DIO_DIRTY (1 << 31)
+
+struct iomap_dio {
+ struct kiocb *iocb;
+ iomap_dio_end_io_t *end_io;
+ loff_t i_size;
+ loff_t size;
+ atomic_t ref;
+ unsigned flags;
+ int error;
+ bool wait_for_completion;
+
+ union {
+ /* used during submission and for synchronous completion: */
+ struct {
+ struct iov_iter *iter;
+ struct task_struct *waiter;
+ struct request_queue *last_queue;
+ blk_qc_t cookie;
+ } submit;
+
+ /* used for aio completion: */
+ struct {
+ struct work_struct work;
+ } aio;
+ };
+};
+
+int iomap_dio_iopoll(struct kiocb *kiocb, bool spin)
+{
+ struct request_queue *q = READ_ONCE(kiocb->private);
+
+ if (!q)
+ return 0;
+ return blk_poll(q, READ_ONCE(kiocb->ki_cookie), spin);
+}
+EXPORT_SYMBOL_GPL(iomap_dio_iopoll);
+
+static void iomap_dio_submit_bio(struct iomap_dio *dio, struct iomap *iomap,
+ struct bio *bio)
+{
+ atomic_inc(&dio->ref);
+
+ if (dio->iocb->ki_flags & IOCB_HIPRI)
+ bio_set_polled(bio, dio->iocb);
+
+ dio->submit.last_queue = bdev_get_queue(iomap->bdev);
+ dio->submit.cookie = submit_bio(bio);
+}
+
+static ssize_t iomap_dio_complete(struct iomap_dio *dio)
+{
+ struct kiocb *iocb = dio->iocb;
+ struct inode *inode = file_inode(iocb->ki_filp);
+ loff_t offset = iocb->ki_pos;
+ ssize_t ret;
+
+ if (dio->end_io) {
+ ret = dio->end_io(iocb,
+ dio->error ? dio->error : dio->size,
+ dio->flags);
+ } else {
+ ret = dio->error;
+ }
+
+ if (likely(!ret)) {
+ ret = dio->size;
+ /* check for short read */
+ if (offset + ret > dio->i_size &&
+ !(dio->flags & IOMAP_DIO_WRITE))
+ ret = dio->i_size - offset;
+ iocb->ki_pos += ret;
+ }
+
+ /*
+ * Try again to invalidate clean pages which might have been cached by
+ * non-direct readahead, or faulted in by get_user_pages() if the source
+ * of the write was an mmap'ed region of the file we're writing. Either
+ * one is a pretty crazy thing to do, so we don't support it 100%. If
+ * this invalidation fails, tough, the write still worked...
+ *
+ * And this page cache invalidation has to be after dio->end_io(), as
+ * some filesystems convert unwritten extents to real allocations in
+ * end_io() when necessary, otherwise a racing buffer read would cache
+ * zeros from unwritten extents.
+ */
+ if (!dio->error &&
+ (dio->flags & IOMAP_DIO_WRITE) && inode->i_mapping->nrpages) {
+ int err;
+ err = invalidate_inode_pages2_range(inode->i_mapping,
+ offset >> PAGE_SHIFT,
+ (offset + dio->size - 1) >> PAGE_SHIFT);
+ if (err)
+ dio_warn_stale_pagecache(iocb->ki_filp);
+ }
+
+ /*
+ * If this is a DSYNC write, make sure we push it to stable storage now
+ * that we've written data.
+ */
+ if (ret > 0 && (dio->flags & IOMAP_DIO_NEED_SYNC))
+ ret = generic_write_sync(iocb, ret);
+
+ inode_dio_end(file_inode(iocb->ki_filp));
+ kfree(dio);
+
+ return ret;
+}
+
+static void iomap_dio_complete_work(struct work_struct *work)
+{
+ struct iomap_dio *dio = container_of(work, struct iomap_dio, aio.work);
+ struct kiocb *iocb = dio->iocb;
+
+ iocb->ki_complete(iocb, iomap_dio_complete(dio), 0);
+}
+
+/*
+ * Set an error in the dio if none is set yet. We have to use cmpxchg
+ * as the submission context and the completion context(s) can race to
+ * update the error.
+ */
+static inline void iomap_dio_set_error(struct iomap_dio *dio, int ret)
+{
+ cmpxchg(&dio->error, 0, ret);
+}
+
+static void iomap_dio_bio_end_io(struct bio *bio)
+{
+ struct iomap_dio *dio = bio->bi_private;
+ bool should_dirty = (dio->flags & IOMAP_DIO_DIRTY);
+
+ if (bio->bi_status)
+ iomap_dio_set_error(dio, blk_status_to_errno(bio->bi_status));
+
+ if (atomic_dec_and_test(&dio->ref)) {
+ if (dio->wait_for_completion) {
+ struct task_struct *waiter = dio->submit.waiter;
+ WRITE_ONCE(dio->submit.waiter, NULL);
+ blk_wake_io_task(waiter);
+ } else if (dio->flags & IOMAP_DIO_WRITE) {
+ struct inode *inode = file_inode(dio->iocb->ki_filp);
+
+ INIT_WORK(&dio->aio.work, iomap_dio_complete_work);
+ queue_work(inode->i_sb->s_dio_done_wq, &dio->aio.work);
+ } else {
+ iomap_dio_complete_work(&dio->aio.work);
+ }
+ }
+
+ if (should_dirty) {
+ bio_check_pages_dirty(bio);
+ } else {
+ bio_release_pages(bio, false);
+ bio_put(bio);
+ }
+}
+
+static void
+iomap_dio_zero(struct iomap_dio *dio, struct iomap *iomap, loff_t pos,
+ unsigned len)
+{
+ struct page *page = ZERO_PAGE(0);
+ int flags = REQ_SYNC | REQ_IDLE;
+ struct bio *bio;
+
+ bio = bio_alloc(GFP_KERNEL, 1);
+ bio_set_dev(bio, iomap->bdev);
+ bio->bi_iter.bi_sector = iomap_sector(iomap, pos);
+ bio->bi_private = dio;
+ bio->bi_end_io = iomap_dio_bio_end_io;
+
+ get_page(page);
+ __bio_add_page(bio, page, len, 0);
+ bio_set_op_attrs(bio, REQ_OP_WRITE, flags);
+ iomap_dio_submit_bio(dio, iomap, bio);
+}
+
+static loff_t
+iomap_dio_bio_actor(struct inode *inode, loff_t pos, loff_t length,
+ struct iomap_dio *dio, struct iomap *iomap)
+{
+ unsigned int blkbits = blksize_bits(bdev_logical_block_size(iomap->bdev));
+ unsigned int fs_block_size = i_blocksize(inode), pad;
+ unsigned int align = iov_iter_alignment(dio->submit.iter);
+ struct iov_iter iter;
+ struct bio *bio;
+ bool need_zeroout = false;
+ bool use_fua = false;
+ int nr_pages, ret = 0;
+ size_t copied = 0;
+
+ if ((pos | length | align) & ((1 << blkbits) - 1))
+ return -EINVAL;
+
+ if (iomap->type == IOMAP_UNWRITTEN) {
+ dio->flags |= IOMAP_DIO_UNWRITTEN;
+ need_zeroout = true;
+ }
+
+ if (iomap->flags & IOMAP_F_SHARED)
+ dio->flags |= IOMAP_DIO_COW;
+
+ if (iomap->flags & IOMAP_F_NEW) {
+ need_zeroout = true;
+ } else if (iomap->type == IOMAP_MAPPED) {
+ /*
+ * Use a FUA write if we need datasync semantics, this is a pure
+ * data IO that doesn't require any metadata updates (including
+ * after IO completion such as unwritten extent conversion) and
+ * the underlying device supports FUA. This allows us to avoid
+ * cache flushes on IO completion.
+ */
+ if (!(iomap->flags & (IOMAP_F_SHARED|IOMAP_F_DIRTY)) &&
+ (dio->flags & IOMAP_DIO_WRITE_FUA) &&
+ blk_queue_fua(bdev_get_queue(iomap->bdev)))
+ use_fua = true;
+ }
+
+ /*
+ * Operate on a partial iter trimmed to the extent we were called for.
+ * We'll update the iter in the dio once we're done with this extent.
+ */
+ iter = *dio->submit.iter;
+ iov_iter_truncate(&iter, length);
+
+ nr_pages = iov_iter_npages(&iter, BIO_MAX_PAGES);
+ if (nr_pages <= 0)
+ return nr_pages;
+
+ if (need_zeroout) {
+ /* zero out from the start of the block to the write offset */
+ pad = pos & (fs_block_size - 1);
+ if (pad)
+ iomap_dio_zero(dio, iomap, pos - pad, pad);
+ }
+
+ do {
+ size_t n;
+ if (dio->error) {
+ iov_iter_revert(dio->submit.iter, copied);
+ return 0;
+ }
+
+ bio = bio_alloc(GFP_KERNEL, nr_pages);
+ bio_set_dev(bio, iomap->bdev);
+ bio->bi_iter.bi_sector = iomap_sector(iomap, pos);
+ bio->bi_write_hint = dio->iocb->ki_hint;
+ bio->bi_ioprio = dio->iocb->ki_ioprio;
+ bio->bi_private = dio;
+ bio->bi_end_io = iomap_dio_bio_end_io;
+
+ ret = bio_iov_iter_get_pages(bio, &iter);
+ if (unlikely(ret)) {
+ /*
+ * We have to stop part way through an IO. We must fall
+ * through to the sub-block tail zeroing here, otherwise
+ * this short IO may expose stale data in the tail of
+ * the block we haven't written data to.
+ */
+ bio_put(bio);
+ goto zero_tail;
+ }
+
+ n = bio->bi_iter.bi_size;
+ if (dio->flags & IOMAP_DIO_WRITE) {
+ bio->bi_opf = REQ_OP_WRITE | REQ_SYNC | REQ_IDLE;
+ if (use_fua)
+ bio->bi_opf |= REQ_FUA;
+ else
+ dio->flags &= ~IOMAP_DIO_WRITE_FUA;
+ task_io_account_write(n);
+ } else {
+ bio->bi_opf = REQ_OP_READ;
+ if (dio->flags & IOMAP_DIO_DIRTY)
+ bio_set_pages_dirty(bio);
+ }
+
+ iov_iter_advance(dio->submit.iter, n);
+
+ dio->size += n;
+ pos += n;
+ copied += n;
+
+ nr_pages = iov_iter_npages(&iter, BIO_MAX_PAGES);
+ iomap_dio_submit_bio(dio, iomap, bio);
+ } while (nr_pages);
+
+ /*
+ * We need to zeroout the tail of a sub-block write if the extent type
+ * requires zeroing or the write extends beyond EOF. If we don't zero
+ * the block tail in the latter case, we can expose stale data via mmap
+ * reads of the EOF block.
+ */
+zero_tail:
+ if (need_zeroout ||
+ ((dio->flags & IOMAP_DIO_WRITE) && pos >= i_size_read(inode))) {
+ /* zero out from the end of the write to the end of the block */
+ pad = pos & (fs_block_size - 1);
+ if (pad)
+ iomap_dio_zero(dio, iomap, pos, fs_block_size - pad);
+ }
+ return copied ? copied : ret;
+}
+
+static loff_t
+iomap_dio_hole_actor(loff_t length, struct iomap_dio *dio)
+{
+ length = iov_iter_zero(length, dio->submit.iter);
+ dio->size += length;
+ return length;
+}
+
+static loff_t
+iomap_dio_inline_actor(struct inode *inode, loff_t pos, loff_t length,
+ struct iomap_dio *dio, struct iomap *iomap)
+{
+ struct iov_iter *iter = dio->submit.iter;
+ size_t copied;
+
+ BUG_ON(pos + length > PAGE_SIZE - offset_in_page(iomap->inline_data));
+
+ if (dio->flags & IOMAP_DIO_WRITE) {
+ loff_t size = inode->i_size;
+
+ if (pos > size)
+ memset(iomap->inline_data + size, 0, pos - size);
+ copied = copy_from_iter(iomap->inline_data + pos, length, iter);
+ if (copied) {
+ if (pos + copied > size)
+ i_size_write(inode, pos + copied);
+ mark_inode_dirty(inode);
+ }
+ } else {
+ copied = copy_to_iter(iomap->inline_data + pos, length, iter);
+ }
+ dio->size += copied;
+ return copied;
+}
+
+static loff_t
+iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ struct iomap_dio *dio = data;
+
+ switch (iomap->type) {
+ case IOMAP_HOLE:
+ if (WARN_ON_ONCE(dio->flags & IOMAP_DIO_WRITE))
+ return -EIO;
+ return iomap_dio_hole_actor(length, dio);
+ case IOMAP_UNWRITTEN:
+ if (!(dio->flags & IOMAP_DIO_WRITE))
+ return iomap_dio_hole_actor(length, dio);
+ return iomap_dio_bio_actor(inode, pos, length, dio, iomap);
+ case IOMAP_MAPPED:
+ return iomap_dio_bio_actor(inode, pos, length, dio, iomap);
+ case IOMAP_INLINE:
+ return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
+ default:
+ WARN_ON_ONCE(1);
+ return -EIO;
+ }
+}
+
+/*
+ * iomap_dio_rw() always completes O_[D]SYNC writes regardless of whether the IO
+ * is being issued as AIO or not. This allows us to optimise pure data writes
+ * to use REQ_FUA rather than requiring generic_write_sync() to issue a
+ * REQ_FLUSH post write. This is slightly tricky because a single request here
+ * can be mapped into multiple disjoint IOs and only a subset of the IOs issued
+ * may be pure data writes. In that case, we still need to do a full data sync
+ * completion.
+ */
+ssize_t
+iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
+ const struct iomap_ops *ops, iomap_dio_end_io_t end_io)
+{
+ struct address_space *mapping = iocb->ki_filp->f_mapping;
+ struct inode *inode = file_inode(iocb->ki_filp);
+ size_t count = iov_iter_count(iter);
+ loff_t pos = iocb->ki_pos, start = pos;
+ loff_t end = iocb->ki_pos + count - 1, ret = 0;
+ unsigned int flags = IOMAP_DIRECT;
+ bool wait_for_completion = is_sync_kiocb(iocb);
+ struct blk_plug plug;
+ struct iomap_dio *dio;
+
+ lockdep_assert_held(&inode->i_rwsem);
+
+ if (!count)
+ return 0;
+
+ dio = kmalloc(sizeof(*dio), GFP_KERNEL);
+ if (!dio)
+ return -ENOMEM;
+
+ dio->iocb = iocb;
+ atomic_set(&dio->ref, 1);
+ dio->size = 0;
+ dio->i_size = i_size_read(inode);
+ dio->end_io = end_io;
+ dio->error = 0;
+ dio->flags = 0;
+
+ dio->submit.iter = iter;
+ dio->submit.waiter = current;
+ dio->submit.cookie = BLK_QC_T_NONE;
+ dio->submit.last_queue = NULL;
+
+ if (iov_iter_rw(iter) == READ) {
+ if (pos >= dio->i_size)
+ goto out_free_dio;
+
+ if (iter_is_iovec(iter) && iov_iter_rw(iter) == READ)
+ dio->flags |= IOMAP_DIO_DIRTY;
+ } else {
+ flags |= IOMAP_WRITE;
+ dio->flags |= IOMAP_DIO_WRITE;
+
+ /* for data sync or sync, we need sync completion processing */
+ if (iocb->ki_flags & IOCB_DSYNC)
+ dio->flags |= IOMAP_DIO_NEED_SYNC;
+
+ /*
+ * For datasync only writes, we optimistically try using FUA for
+ * this IO. Any non-FUA write that occurs will clear this flag,
+ * hence we know before completion whether a cache flush is
+ * necessary.
+ */
+ if ((iocb->ki_flags & (IOCB_DSYNC | IOCB_SYNC)) == IOCB_DSYNC)
+ dio->flags |= IOMAP_DIO_WRITE_FUA;
+ }
+
+ if (iocb->ki_flags & IOCB_NOWAIT) {
+ if (filemap_range_has_page(mapping, start, end)) {
+ ret = -EAGAIN;
+ goto out_free_dio;
+ }
+ flags |= IOMAP_NOWAIT;
+ }
+
+ ret = filemap_write_and_wait_range(mapping, start, end);
+ if (ret)
+ goto out_free_dio;
+
+ /*
+ * Try to invalidate cache pages for the range we're direct
+ * writing. If this invalidation fails, tough, the write will
+ * still work, but racing two incompatible write paths is a
+ * pretty crazy thing to do, so we don't support it 100%.
+ */
+ ret = invalidate_inode_pages2_range(mapping,
+ start >> PAGE_SHIFT, end >> PAGE_SHIFT);
+ if (ret)
+ dio_warn_stale_pagecache(iocb->ki_filp);
+ ret = 0;
+
+ if (iov_iter_rw(iter) == WRITE && !wait_for_completion &&
+ !inode->i_sb->s_dio_done_wq) {
+ ret = sb_init_dio_done_wq(inode->i_sb);
+ if (ret < 0)
+ goto out_free_dio;
+ }
+
+ inode_dio_begin(inode);
+
+ blk_start_plug(&plug);
+ do {
+ ret = iomap_apply(inode, pos, count, flags, ops, dio,
+ iomap_dio_actor);
+ if (ret <= 0) {
+ /* magic error code to fall back to buffered I/O */
+ if (ret == -ENOTBLK) {
+ wait_for_completion = true;
+ ret = 0;
+ }
+ break;
+ }
+ pos += ret;
+
+ if (iov_iter_rw(iter) == READ && pos >= dio->i_size)
+ break;
+ } while ((count = iov_iter_count(iter)) > 0);
+ blk_finish_plug(&plug);
+
+ if (ret < 0)
+ iomap_dio_set_error(dio, ret);
+
+ /*
+ * If all the writes we issued were FUA, we don't need to flush the
+ * cache on IO completion. Clear the sync flag for this case.
+ */
+ if (dio->flags & IOMAP_DIO_WRITE_FUA)
+ dio->flags &= ~IOMAP_DIO_NEED_SYNC;
+
+ WRITE_ONCE(iocb->ki_cookie, dio->submit.cookie);
+ WRITE_ONCE(iocb->private, dio->submit.last_queue);
+
+ /*
+ * We are about to drop our additional submission reference, which
+ * might be the last reference to the dio. There are three three
+ * different ways we can progress here:
+ *
+ * (a) If this is the last reference we will always complete and free
+ * the dio ourselves.
+ * (b) If this is not the last reference, and we serve an asynchronous
+ * iocb, we must never touch the dio after the decrement, the
+ * I/O completion handler will complete and free it.
+ * (c) If this is not the last reference, but we serve a synchronous
+ * iocb, the I/O completion handler will wake us up on the drop
+ * of the final reference, and we will complete and free it here
+ * after we got woken by the I/O completion handler.
+ */
+ dio->wait_for_completion = wait_for_completion;
+ if (!atomic_dec_and_test(&dio->ref)) {
+ if (!wait_for_completion)
+ return -EIOCBQUEUED;
+
+ for (;;) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ if (!READ_ONCE(dio->submit.waiter))
+ break;
+
+ if (!(iocb->ki_flags & IOCB_HIPRI) ||
+ !dio->submit.last_queue ||
+ !blk_poll(dio->submit.last_queue,
+ dio->submit.cookie, true))
+ io_schedule();
+ }
+ __set_current_state(TASK_RUNNING);
+ }
+
+ return iomap_dio_complete(dio);
+
+out_free_dio:
+ kfree(dio);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iomap_dio_rw);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2016-2018 Christoph Hellwig.
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+
+struct fiemap_ctx {
+ struct fiemap_extent_info *fi;
+ struct iomap prev;
+};
+
+static int iomap_to_fiemap(struct fiemap_extent_info *fi,
+ struct iomap *iomap, u32 flags)
+{
+ switch (iomap->type) {
+ case IOMAP_HOLE:
+ /* skip holes */
+ return 0;
+ case IOMAP_DELALLOC:
+ flags |= FIEMAP_EXTENT_DELALLOC | FIEMAP_EXTENT_UNKNOWN;
+ break;
+ case IOMAP_MAPPED:
+ break;
+ case IOMAP_UNWRITTEN:
+ flags |= FIEMAP_EXTENT_UNWRITTEN;
+ break;
+ case IOMAP_INLINE:
+ flags |= FIEMAP_EXTENT_DATA_INLINE;
+ break;
+ }
+
+ if (iomap->flags & IOMAP_F_MERGED)
+ flags |= FIEMAP_EXTENT_MERGED;
+ if (iomap->flags & IOMAP_F_SHARED)
+ flags |= FIEMAP_EXTENT_SHARED;
+
+ return fiemap_fill_next_extent(fi, iomap->offset,
+ iomap->addr != IOMAP_NULL_ADDR ? iomap->addr : 0,
+ iomap->length, flags);
+}
+
+static loff_t
+iomap_fiemap_actor(struct inode *inode, loff_t pos, loff_t length, void *data,
+ struct iomap *iomap)
+{
+ struct fiemap_ctx *ctx = data;
+ loff_t ret = length;
+
+ if (iomap->type == IOMAP_HOLE)
+ return length;
+
+ ret = iomap_to_fiemap(ctx->fi, &ctx->prev, 0);
+ ctx->prev = *iomap;
+ switch (ret) {
+ case 0: /* success */
+ return length;
+ case 1: /* extent array full */
+ return 0;
+ default:
+ return ret;
+ }
+}
+
+int iomap_fiemap(struct inode *inode, struct fiemap_extent_info *fi,
+ loff_t start, loff_t len, const struct iomap_ops *ops)
+{
+ struct fiemap_ctx ctx;
+ loff_t ret;
+
+ memset(&ctx, 0, sizeof(ctx));
+ ctx.fi = fi;
+ ctx.prev.type = IOMAP_HOLE;
+
+ ret = fiemap_check_flags(fi, FIEMAP_FLAG_SYNC);
+ if (ret)
+ return ret;
+
+ if (fi->fi_flags & FIEMAP_FLAG_SYNC) {
+ ret = filemap_write_and_wait(inode->i_mapping);
+ if (ret)
+ return ret;
+ }
+
+ while (len > 0) {
+ ret = iomap_apply(inode, start, len, IOMAP_REPORT, ops, &ctx,
+ iomap_fiemap_actor);
+ /* inode with no (attribute) mapping will give ENOENT */
+ if (ret == -ENOENT)
+ break;
+ if (ret < 0)
+ return ret;
+ if (ret == 0)
+ break;
+
+ start += ret;
+ len -= ret;
+ }
+
+ if (ctx.prev.type != IOMAP_HOLE) {
+ ret = iomap_to_fiemap(fi, &ctx.prev, FIEMAP_EXTENT_LAST);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(iomap_fiemap);
+
+static loff_t
+iomap_bmap_actor(struct inode *inode, loff_t pos, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ sector_t *bno = data, addr;
+
+ if (iomap->type == IOMAP_MAPPED) {
+ addr = (pos - iomap->offset + iomap->addr) >> inode->i_blkbits;
+ if (addr > INT_MAX)
+ WARN(1, "would truncate bmap result\n");
+ else
+ *bno = addr;
+ }
+ return 0;
+}
+
+/* legacy ->bmap interface. 0 is the error return (!) */
+sector_t
+iomap_bmap(struct address_space *mapping, sector_t bno,
+ const struct iomap_ops *ops)
+{
+ struct inode *inode = mapping->host;
+ loff_t pos = bno << inode->i_blkbits;
+ unsigned blocksize = i_blocksize(inode);
+
+ if (filemap_write_and_wait(mapping))
+ return 0;
+
+ bno = 0;
+ iomap_apply(inode, pos, blocksize, 0, ops, &bno, iomap_bmap_actor);
+ return bno;
+}
+EXPORT_SYMBOL_GPL(iomap_bmap);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2017 Red Hat, Inc.
+ * Copyright (c) 2018 Christoph Hellwig.
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+#include <linux/pagemap.h>
+#include <linux/pagevec.h>
+
+/*
+ * Seek for SEEK_DATA / SEEK_HOLE within @page, starting at @lastoff.
+ * Returns true if found and updates @lastoff to the offset in file.
+ */
+static bool
+page_seek_hole_data(struct inode *inode, struct page *page, loff_t *lastoff,
+ int whence)
+{
+ const struct address_space_operations *ops = inode->i_mapping->a_ops;
+ unsigned int bsize = i_blocksize(inode), off;
+ bool seek_data = whence == SEEK_DATA;
+ loff_t poff = page_offset(page);
+
+ if (WARN_ON_ONCE(*lastoff >= poff + PAGE_SIZE))
+ return false;
+
+ if (*lastoff < poff) {
+ /*
+ * Last offset smaller than the start of the page means we found
+ * a hole:
+ */
+ if (whence == SEEK_HOLE)
+ return true;
+ *lastoff = poff;
+ }
+
+ /*
+ * Just check the page unless we can and should check block ranges:
+ */
+ if (bsize == PAGE_SIZE || !ops->is_partially_uptodate)
+ return PageUptodate(page) == seek_data;
+
+ lock_page(page);
+ if (unlikely(page->mapping != inode->i_mapping))
+ goto out_unlock_not_found;
+
+ for (off = 0; off < PAGE_SIZE; off += bsize) {
+ if (offset_in_page(*lastoff) >= off + bsize)
+ continue;
+ if (ops->is_partially_uptodate(page, off, bsize) == seek_data) {
+ unlock_page(page);
+ return true;
+ }
+ *lastoff = poff + off + bsize;
+ }
+
+out_unlock_not_found:
+ unlock_page(page);
+ return false;
+}
+
+/*
+ * Seek for SEEK_DATA / SEEK_HOLE in the page cache.
+ *
+ * Within unwritten extents, the page cache determines which parts are holes
+ * and which are data: uptodate buffer heads count as data; everything else
+ * counts as a hole.
+ *
+ * Returns the resulting offset on successs, and -ENOENT otherwise.
+ */
+static loff_t
+page_cache_seek_hole_data(struct inode *inode, loff_t offset, loff_t length,
+ int whence)
+{
+ pgoff_t index = offset >> PAGE_SHIFT;
+ pgoff_t end = DIV_ROUND_UP(offset + length, PAGE_SIZE);
+ loff_t lastoff = offset;
+ struct pagevec pvec;
+
+ if (length <= 0)
+ return -ENOENT;
+
+ pagevec_init(&pvec);
+
+ do {
+ unsigned nr_pages, i;
+
+ nr_pages = pagevec_lookup_range(&pvec, inode->i_mapping, &index,
+ end - 1);
+ if (nr_pages == 0)
+ break;
+
+ for (i = 0; i < nr_pages; i++) {
+ struct page *page = pvec.pages[i];
+
+ if (page_seek_hole_data(inode, page, &lastoff, whence))
+ goto check_range;
+ lastoff = page_offset(page) + PAGE_SIZE;
+ }
+ pagevec_release(&pvec);
+ } while (index < end);
+
+ /* When no page at lastoff and we are not done, we found a hole. */
+ if (whence != SEEK_HOLE)
+ goto not_found;
+
+check_range:
+ if (lastoff < offset + length)
+ goto out;
+not_found:
+ lastoff = -ENOENT;
+out:
+ pagevec_release(&pvec);
+ return lastoff;
+}
+
+
+static loff_t
+iomap_seek_hole_actor(struct inode *inode, loff_t offset, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ switch (iomap->type) {
+ case IOMAP_UNWRITTEN:
+ offset = page_cache_seek_hole_data(inode, offset, length,
+ SEEK_HOLE);
+ if (offset < 0)
+ return length;
+ /* fall through */
+ case IOMAP_HOLE:
+ *(loff_t *)data = offset;
+ return 0;
+ default:
+ return length;
+ }
+}
+
+loff_t
+iomap_seek_hole(struct inode *inode, loff_t offset, const struct iomap_ops *ops)
+{
+ loff_t size = i_size_read(inode);
+ loff_t length = size - offset;
+ loff_t ret;
+
+ /* Nothing to be found before or beyond the end of the file. */
+ if (offset < 0 || offset >= size)
+ return -ENXIO;
+
+ while (length > 0) {
+ ret = iomap_apply(inode, offset, length, IOMAP_REPORT, ops,
+ &offset, iomap_seek_hole_actor);
+ if (ret < 0)
+ return ret;
+ if (ret == 0)
+ break;
+
+ offset += ret;
+ length -= ret;
+ }
+
+ return offset;
+}
+EXPORT_SYMBOL_GPL(iomap_seek_hole);
+
+static loff_t
+iomap_seek_data_actor(struct inode *inode, loff_t offset, loff_t length,
+ void *data, struct iomap *iomap)
+{
+ switch (iomap->type) {
+ case IOMAP_HOLE:
+ return length;
+ case IOMAP_UNWRITTEN:
+ offset = page_cache_seek_hole_data(inode, offset, length,
+ SEEK_DATA);
+ if (offset < 0)
+ return length;
+ /*FALLTHRU*/
+ default:
+ *(loff_t *)data = offset;
+ return 0;
+ }
+}
+
+loff_t
+iomap_seek_data(struct inode *inode, loff_t offset, const struct iomap_ops *ops)
+{
+ loff_t size = i_size_read(inode);
+ loff_t length = size - offset;
+ loff_t ret;
+
+ /* Nothing to be found before or beyond the end of the file. */
+ if (offset < 0 || offset >= size)
+ return -ENXIO;
+
+ while (length > 0) {
+ ret = iomap_apply(inode, offset, length, IOMAP_REPORT, ops,
+ &offset, iomap_seek_data_actor);
+ if (ret < 0)
+ return ret;
+ if (ret == 0)
+ break;
+
+ offset += ret;
+ length -= ret;
+ }
+
+ if (length <= 0)
+ return -ENXIO;
+ return offset;
+}
+EXPORT_SYMBOL_GPL(iomap_seek_data);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Oracle. All Rights Reserved.
+ * Author: Darrick J. Wong <darrick.wong@oracle.com>
+ */
+#include <linux/module.h>
+#include <linux/compiler.h>
+#include <linux/fs.h>
+#include <linux/iomap.h>
+#include <linux/swap.h>
+
+/* Swapfile activation */
+
+struct iomap_swapfile_info {
+ struct iomap iomap; /* accumulated iomap */
+ struct swap_info_struct *sis;
+ uint64_t lowest_ppage; /* lowest physical addr seen (pages) */
+ uint64_t highest_ppage; /* highest physical addr seen (pages) */
+ unsigned long nr_pages; /* number of pages collected */
+ int nr_extents; /* extent count */
+};
+
+/*
+ * Collect physical extents for this swap file. Physical extents reported to
+ * the swap code must be trimmed to align to a page boundary. The logical
+ * offset within the file is irrelevant since the swapfile code maps logical
+ * page numbers of the swap device to the physical page-aligned extents.
+ */
+static int iomap_swapfile_add_extent(struct iomap_swapfile_info *isi)
+{
+ struct iomap *iomap = &isi->iomap;
+ unsigned long nr_pages;
+ uint64_t first_ppage;
+ uint64_t first_ppage_reported;
+ uint64_t next_ppage;
+ int error;
+
+ /*
+ * Round the start up and the end down so that the physical
+ * extent aligns to a page boundary.
+ */
+ first_ppage = ALIGN(iomap->addr, PAGE_SIZE) >> PAGE_SHIFT;
+ next_ppage = ALIGN_DOWN(iomap->addr + iomap->length, PAGE_SIZE) >>
+ PAGE_SHIFT;
+
+ /* Skip too-short physical extents. */
+ if (first_ppage >= next_ppage)
+ return 0;
+ nr_pages = next_ppage - first_ppage;
+
+ /*
+ * Calculate how much swap space we're adding; the first page contains
+ * the swap header and doesn't count. The mm still wants that first
+ * page fed to add_swap_extent, however.
+ */
+ first_ppage_reported = first_ppage;
+ if (iomap->offset == 0)
+ first_ppage_reported++;
+ if (isi->lowest_ppage > first_ppage_reported)
+ isi->lowest_ppage = first_ppage_reported;
+ if (isi->highest_ppage < (next_ppage - 1))
+ isi->highest_ppage = next_ppage - 1;
+
+ /* Add extent, set up for the next call. */
+ error = add_swap_extent(isi->sis, isi->nr_pages, nr_pages, first_ppage);
+ if (error < 0)
+ return error;
+ isi->nr_extents += error;
+ isi->nr_pages += nr_pages;
+ return 0;
+}
+
+/*
+ * Accumulate iomaps for this swap file. We have to accumulate iomaps because
+ * swap only cares about contiguous page-aligned physical extents and makes no
+ * distinction between written and unwritten extents.
+ */
+static loff_t iomap_swapfile_activate_actor(struct inode *inode, loff_t pos,
+ loff_t count, void *data, struct iomap *iomap)
+{
+ struct iomap_swapfile_info *isi = data;
+ int error;
+
+ switch (iomap->type) {
+ case IOMAP_MAPPED:
+ case IOMAP_UNWRITTEN:
+ /* Only real or unwritten extents. */
+ break;
+ case IOMAP_INLINE:
+ /* No inline data. */
+ pr_err("swapon: file is inline\n");
+ return -EINVAL;
+ default:
+ pr_err("swapon: file has unallocated extents\n");
+ return -EINVAL;
+ }
+
+ /* No uncommitted metadata or shared blocks. */
+ if (iomap->flags & IOMAP_F_DIRTY) {
+ pr_err("swapon: file is not committed\n");
+ return -EINVAL;
+ }
+ if (iomap->flags & IOMAP_F_SHARED) {
+ pr_err("swapon: file has shared extents\n");
+ return -EINVAL;
+ }
+
+ /* Only one bdev per swap file. */
+ if (iomap->bdev != isi->sis->bdev) {
+ pr_err("swapon: file is on multiple devices\n");
+ return -EINVAL;
+ }
+
+ if (isi->iomap.length == 0) {
+ /* No accumulated extent, so just store it. */
+ memcpy(&isi->iomap, iomap, sizeof(isi->iomap));
+ } else if (isi->iomap.addr + isi->iomap.length == iomap->addr) {
+ /* Append this to the accumulated extent. */
+ isi->iomap.length += iomap->length;
+ } else {
+ /* Otherwise, add the retained iomap and store this one. */
+ error = iomap_swapfile_add_extent(isi);
+ if (error)
+ return error;
+ memcpy(&isi->iomap, iomap, sizeof(isi->iomap));
+ }
+ return count;
+}
+
+/*
+ * Iterate a swap file's iomaps to construct physical extents that can be
+ * passed to the swapfile subsystem.
+ */
+int iomap_swapfile_activate(struct swap_info_struct *sis,
+ struct file *swap_file, sector_t *pagespan,
+ const struct iomap_ops *ops)
+{
+ struct iomap_swapfile_info isi = {
+ .sis = sis,
+ .lowest_ppage = (sector_t)-1ULL,
+ };
+ struct address_space *mapping = swap_file->f_mapping;
+ struct inode *inode = mapping->host;
+ loff_t pos = 0;
+ loff_t len = ALIGN_DOWN(i_size_read(inode), PAGE_SIZE);
+ loff_t ret;
+
+ /*
+ * Persist all file mapping metadata so that we won't have any
+ * IOMAP_F_DIRTY iomaps.
+ */
+ ret = vfs_fsync(swap_file, 1);
+ if (ret)
+ return ret;
+
+ while (len > 0) {
+ ret = iomap_apply(inode, pos, len, IOMAP_REPORT,
+ ops, &isi, iomap_swapfile_activate_actor);
+ if (ret <= 0)
+ return ret;
+
+ pos += ret;
+ len -= ret;
+ }
+
+ if (isi.iomap.length) {
+ ret = iomap_swapfile_add_extent(&isi);
+ if (ret)
+ return ret;
+ }
+
+ *pagespan = 1 + isi.highest_ppage - isi.lowest_ppage;
+ sis->max = isi.nr_pages;
+ sis->pages = isi.nr_pages - 1;
+ sis->highest_bit = isi.nr_pages - 1;
+ return isi.nr_extents;
+}
+EXPORT_SYMBOL_GPL(iomap_swapfile_activate);
#include <linux/exportfs.h>
#include <linux/writeback.h>
#include <linux/buffer_head.h> /* sync_mapping_buffers */
+#include <linux/fs_context.h>
+#include <linux/pseudo_fs.h>
#include <linux/uaccess.h>
.statfs = simple_statfs,
};
-/*
- * Common helper for pseudo-filesystems (sockfs, pipefs, bdev - stuff that
- * will never be mountable)
- */
-struct dentry *mount_pseudo_xattr(struct file_system_type *fs_type, char *name,
- const struct super_operations *ops, const struct xattr_handler **xattr,
- const struct dentry_operations *dops, unsigned long magic)
+static int pseudo_fs_fill_super(struct super_block *s, struct fs_context *fc)
{
- struct super_block *s;
- struct dentry *dentry;
+ struct pseudo_fs_context *ctx = fc->fs_private;
struct inode *root;
- struct qstr d_name = QSTR_INIT(name, strlen(name));
-
- s = sget_userns(fs_type, NULL, set_anon_super, SB_KERNMOUNT|SB_NOUSER,
- &init_user_ns, NULL);
- if (IS_ERR(s))
- return ERR_CAST(s);
s->s_maxbytes = MAX_LFS_FILESIZE;
s->s_blocksize = PAGE_SIZE;
s->s_blocksize_bits = PAGE_SHIFT;
- s->s_magic = magic;
- s->s_op = ops ? ops : &simple_super_operations;
- s->s_xattr = xattr;
+ s->s_magic = ctx->magic;
+ s->s_op = ctx->ops ?: &simple_super_operations;
+ s->s_xattr = ctx->xattr;
s->s_time_gran = 1;
root = new_inode(s);
if (!root)
- goto Enomem;
+ return -ENOMEM;
+
/*
* since this is the first inode, make it number 1. New inodes created
* after this must take care not to collide with it (by passing
root->i_ino = 1;
root->i_mode = S_IFDIR | S_IRUSR | S_IWUSR;
root->i_atime = root->i_mtime = root->i_ctime = current_time(root);
- dentry = __d_alloc(s, &d_name);
- if (!dentry) {
- iput(root);
- goto Enomem;
+ s->s_root = d_make_root(root);
+ if (!s->s_root)
+ return -ENOMEM;
+ s->s_d_op = ctx->dops;
+ return 0;
+}
+
+static int pseudo_fs_get_tree(struct fs_context *fc)
+{
+ return get_tree_nodev(fc, pseudo_fs_fill_super);
+}
+
+static void pseudo_fs_free(struct fs_context *fc)
+{
+ kfree(fc->fs_private);
+}
+
+static const struct fs_context_operations pseudo_fs_context_ops = {
+ .free = pseudo_fs_free,
+ .get_tree = pseudo_fs_get_tree,
+};
+
+/*
+ * Common helper for pseudo-filesystems (sockfs, pipefs, bdev - stuff that
+ * will never be mountable)
+ */
+struct pseudo_fs_context *init_pseudo(struct fs_context *fc,
+ unsigned long magic)
+{
+ struct pseudo_fs_context *ctx;
+
+ ctx = kzalloc(sizeof(struct pseudo_fs_context), GFP_KERNEL);
+ if (likely(ctx)) {
+ ctx->magic = magic;
+ fc->fs_private = ctx;
+ fc->ops = &pseudo_fs_context_ops;
+ fc->sb_flags |= SB_NOUSER;
+ fc->global = true;
}
- d_instantiate(dentry, root);
- s->s_root = dentry;
- s->s_d_op = dops;
- s->s_flags |= SB_ACTIVE;
- return dget(s->s_root);
-
-Enomem:
- deactivate_locked_super(s);
- return ERR_PTR(-ENOMEM);
+ return ctx;
}
-EXPORT_SYMBOL(mount_pseudo_xattr);
+EXPORT_SYMBOL(init_pseudo);
int simple_open(struct inode *inode, struct file *file)
{
#include <linux/sched/task.h>
#include <uapi/linux/mount.h>
#include <linux/fs_context.h>
+#include <linux/shmem_fs.h>
#include "pnode.h"
#include "internal.h"
err = vfs_parse_fs_string(fc, "source", name, strlen(name));
if (!err)
err = parse_monolithic_mount_data(fc, data);
+ if (!err && !mount_capable(fc))
+ err = -EPERM;
if (!err)
err = vfs_get_tree(fc);
if (!err)
}
EXPORT_SYMBOL(mount_subtree);
-int ksys_mount(char __user *dev_name, char __user *dir_name, char __user *type,
- unsigned long flags, void __user *data)
+int ksys_mount(const char __user *dev_name, const char __user *dir_name,
+ const char __user *type, unsigned long flags, void __user *data)
{
int ret;
char *kernel_type;
struct mount *m;
struct mnt_namespace *ns;
struct path root;
- struct file_system_type *type;
- type = get_fs_type("rootfs");
- if (!type)
- panic("Can't find rootfs type");
- mnt = vfs_kern_mount(type, 0, "rootfs", NULL);
- put_filesystem(type);
+ mnt = vfs_kern_mount(&rootfs_fs_type, 0, "rootfs", NULL);
if (IS_ERR(mnt))
panic("Can't create rootfs");
fs_kobj = kobject_create_and_add("fs", NULL);
if (!fs_kobj)
printk(KERN_WARNING "%s: kobj create error\n", __func__);
+ shmem_init();
init_rootfs();
init_mount_tree();
}
CFLAGS_nfstrace.o += -I$(src)
nfs-y := client.o dir.o file.o getroot.o inode.o super.o \
io.o direct.o pagelist.o read.o symlink.o unlink.o \
- write.o namespace.o mount_clnt.o nfstrace.o export.o
+ write.o namespace.o mount_clnt.o nfstrace.o \
+ export.o sysfs.o
nfs-$(CONFIG_ROOT_NFS) += nfsroot.o
nfs-$(CONFIG_SYSCTL) += sysctl.o
nfs-$(CONFIG_NFS_FSCACHE) += fscache.o fscache-index.o
validate_seqid(const struct nfs4_slot_table *tbl, const struct nfs4_slot *slot,
const struct cb_sequenceargs * args)
{
+ __be32 ret;
+
+ ret = cpu_to_be32(NFS4ERR_BADSLOT);
if (args->csa_slotid > tbl->server_highest_slotid)
- return htonl(NFS4ERR_BADSLOT);
+ goto out_err;
/* Replay */
if (args->csa_sequenceid == slot->seq_nr) {
+ ret = cpu_to_be32(NFS4ERR_DELAY);
if (nfs4_test_locked_slot(tbl, slot->slot_nr))
- return htonl(NFS4ERR_DELAY);
+ goto out_err;
+
/* Signal process_op to set this error on next op */
+ ret = cpu_to_be32(NFS4ERR_RETRY_UNCACHED_REP);
if (args->csa_cachethis == 0)
- return htonl(NFS4ERR_RETRY_UNCACHED_REP);
+ goto out_err;
/* Liar! We never allowed you to set csa_cachethis != 0 */
- return htonl(NFS4ERR_SEQ_FALSE_RETRY);
+ ret = cpu_to_be32(NFS4ERR_SEQ_FALSE_RETRY);
+ goto out_err;
}
/* Note: wraparound relies on seq_nr being of type u32 */
- if (likely(args->csa_sequenceid == slot->seq_nr + 1))
- return htonl(NFS4_OK);
-
/* Misordered request */
- return htonl(NFS4ERR_SEQ_MISORDERED);
+ ret = cpu_to_be32(NFS4ERR_SEQ_MISORDERED);
+ if (args->csa_sequenceid != slot->seq_nr + 1)
+ goto out_err;
+
+ return cpu_to_be32(NFS4_OK);
+
+out_err:
+ trace_nfs4_cb_seqid_err(args, ret);
+ return ret;
}
/*
#include "pnfs.h"
#include "nfs.h"
#include "netns.h"
+#include "sysfs.h"
#define NFSDBG_FACILITY NFSDBG_CLIENT
clp->cl_rpcclient = ERR_PTR(-EINVAL);
clp->cl_proto = cl_init->proto;
+ clp->cl_nconnect = cl_init->nconnect;
clp->cl_net = get_net(cl_init->net);
clp->cl_principal = "*";
EXPORT_SYMBOL_GPL(nfs_alloc_client);
#if IS_ENABLED(CONFIG_NFS_V4)
-void nfs_cleanup_cb_ident_idr(struct net *net)
+static void nfs_cleanup_cb_ident_idr(struct net *net)
{
struct nfs_net *nn = net_generic(net, nfs_net_id);
}
#else
-void nfs_cleanup_cb_ident_idr(struct net *net)
+static void nfs_cleanup_cb_ident_idr(struct net *net)
{
}
clp = nfs_match_client(cl_init);
if (clp) {
spin_unlock(&nn->nfs_client_lock);
- if (IS_ERR(clp))
- return clp;
if (new)
new->rpc_ops->free_client(new);
+ if (IS_ERR(clp))
+ return clp;
return nfs_found_client(cl_init, clp);
}
if (new) {
struct rpc_create_args args = {
.net = clp->cl_net,
.protocol = clp->cl_proto,
+ .nconnect = clp->cl_nconnect,
.address = (struct sockaddr *)&clp->cl_addr,
.addrsize = clp->cl_addrlen,
.timeout = cl_init->timeparms,
.net = data->net,
.timeparms = &timeparms,
.cred = server->cred,
+ .nconnect = data->nfs_server.nconnect,
};
struct nfs_client *clp;
int error;
#endif
spin_lock_init(&nn->nfs_client_lock);
nn->boot_time = ktime_get_real();
+
+ nfs_netns_sysfs_setup(nn, net);
+}
+
+void nfs_clients_exit(struct net *net)
+{
+ struct nfs_net *nn = net_generic(net, nfs_net_id);
+
+ nfs_netns_sysfs_destroy(nn);
+ nfs_cleanup_cb_ident_idr(net);
+ WARN_ON_ONCE(!list_empty(&nn->nfs_client_list));
+ WARN_ON_ONCE(!list_empty(&nn->nfs_volume_list));
}
#ifdef CONFIG_PROC_FS
ctx->dup_cookie = 0;
ctx->cred = get_cred(cred);
spin_lock(&dir->i_lock);
+ if (list_empty(&nfsi->open_files) &&
+ (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER))
+ nfsi->cache_validity |= NFS_INO_INVALID_DATA |
+ NFS_INO_REVAL_FORCED;
list_add(&ctx->list, &nfsi->open_files);
spin_unlock(&dir->i_lock);
return ctx;
struct nfs_cache_array_entry array[0];
};
-struct readdirvec {
- unsigned long nr;
- unsigned long index;
- struct page *pages[NFS_MAX_READDIR_RAPAGES];
-};
-
typedef int (*decode_dirent_t)(struct xdr_stream *, struct nfs_entry *, bool);
typedef struct {
struct file *file;
struct page *page;
struct dir_context *ctx;
unsigned long page_index;
- struct readdirvec pvec;
u64 *dir_cookie;
u64 last_cookie;
loff_t current_index;
struct nfs_cache_array *array;
unsigned int count = 0;
int status;
- int max_rapages = NFS_MAX_READDIR_RAPAGES;
-
- desc->pvec.index = desc->page_index;
- desc->pvec.nr = 0;
scratch = alloc_page(GFP_KERNEL);
if (scratch == NULL)
if (desc->plus)
nfs_prime_dcache(file_dentry(desc->file), entry);
- status = nfs_readdir_add_to_array(entry, desc->pvec.pages[desc->pvec.nr]);
- if (status == -ENOSPC) {
- desc->pvec.nr++;
- if (desc->pvec.nr == max_rapages)
- break;
- status = nfs_readdir_add_to_array(entry, desc->pvec.pages[desc->pvec.nr]);
- }
+ status = nfs_readdir_add_to_array(entry, page);
if (status != 0)
break;
} while (!entry->eof);
- /*
- * page and desc->pvec.pages[0] are valid, don't need to check
- * whether or not to be NULL.
- */
- copy_highpage(page, desc->pvec.pages[0]);
-
out_nopages:
if (count == 0 || (status == -EBADCOOKIE && entry->eof != 0)) {
- array = kmap_atomic(desc->pvec.pages[desc->pvec.nr]);
+ array = kmap(page);
array->eof_index = array->size;
status = 0;
- kunmap_atomic(array);
+ kunmap(page);
}
put_page(scratch);
-
- /*
- * desc->pvec.nr > 0 means at least one page was completely filled,
- * we should return -ENOSPC. Otherwise function
- * nfs_readdir_xdr_to_array will enter infinite loop.
- */
- if (desc->pvec.nr > 0)
- return -ENOSPC;
return status;
}
return -ENOMEM;
}
-/*
- * nfs_readdir_rapages_init initialize rapages by nfs_cache_array structure.
- */
-static
-void nfs_readdir_rapages_init(nfs_readdir_descriptor_t *desc)
-{
- struct nfs_cache_array *array;
- int max_rapages = NFS_MAX_READDIR_RAPAGES;
- int index;
-
- for (index = 0; index < max_rapages; index++) {
- array = kmap_atomic(desc->pvec.pages[index]);
- memset(array, 0, sizeof(struct nfs_cache_array));
- array->eof_index = -1;
- kunmap_atomic(array);
- }
-}
-
static
int nfs_readdir_xdr_to_array(nfs_readdir_descriptor_t *desc, struct page *page, struct inode *inode)
{
int status = -ENOMEM;
unsigned int array_size = ARRAY_SIZE(pages);
- /*
- * This means we hit readdir rdpages miss, the preallocated rdpages
- * are useless, the preallocate rdpages should be reinitialized.
- */
- nfs_readdir_rapages_init(desc);
-
entry.prev_cookie = 0;
entry.cookie = desc->last_cookie;
entry.eof = 0;
struct inode *inode = file_inode(desc->file);
int ret;
- /*
- * If desc->page_index in range desc->pvec.index and
- * desc->pvec.index + desc->pvec.nr, we get readdir cache hit.
- */
- if (desc->page_index >= desc->pvec.index &&
- desc->page_index < (desc->pvec.index + desc->pvec.nr)) {
- /*
- * page and desc->pvec.pages[x] are valid, don't need to check
- * whether or not to be NULL.
- */
- copy_highpage(page, desc->pvec.pages[desc->page_index - desc->pvec.index]);
- ret = 0;
- } else {
- ret = nfs_readdir_xdr_to_array(desc, page, inode);
- if (ret < 0)
- goto error;
- }
-
+ ret = nfs_readdir_xdr_to_array(desc, page, inode);
+ if (ret < 0)
+ goto error;
SetPageUptodate(page);
if (invalidate_inode_pages2_range(inode->i_mapping, page->index + 1, -1) < 0) {
*desc = &my_desc;
struct nfs_open_dir_context *dir_ctx = file->private_data;
int res = 0;
- int max_rapages = NFS_MAX_READDIR_RAPAGES;
dfprintk(FILE, "NFS: readdir(%pD2) starting at cookie %llu\n",
file, (long long)ctx->pos);
desc->decode = NFS_PROTO(inode)->decode_dirent;
desc->plus = nfs_use_readdirplus(inode, ctx);
- res = nfs_readdir_alloc_pages(desc->pvec.pages, max_rapages);
- if (res < 0)
- return -ENOMEM;
-
- nfs_readdir_rapages_init(desc);
-
if (ctx->pos == 0 || nfs_attribute_cache_expired(inode))
res = nfs_revalidate_mapping(inode, file->f_mapping);
if (res < 0)
break;
} while (!desc->eof);
out:
- nfs_readdir_free_pages(desc->pvec.pages, max_rapages);
if (res > 0)
res = 0;
dfprintk(FILE, "NFS: readdir(%pD2) returns %d\n", file, res);
if (pgio->pg_error < 0)
return;
out_mds:
+ trace_pnfs_mds_fallback_pg_init_read(pgio->pg_inode,
+ 0, NFS4_MAX_UINT64, IOMODE_READ,
+ NFS_I(pgio->pg_inode)->layout,
+ pgio->pg_lseg);
pnfs_put_lseg(pgio->pg_lseg);
pgio->pg_lseg = NULL;
nfs_pageio_reset_read_mds(pgio);
return;
out_mds:
+ trace_pnfs_mds_fallback_pg_init_write(pgio->pg_inode,
+ 0, NFS4_MAX_UINT64, IOMODE_RW,
+ NFS_I(pgio->pg_inode)->layout,
+ pgio->pg_lseg);
pnfs_put_lseg(pgio->pg_lseg);
pgio->pg_lseg = NULL;
nfs_pageio_reset_write_mds(pgio);
if (pgio->pg_lseg)
return FF_LAYOUT_MIRROR_COUNT(pgio->pg_lseg);
+ trace_pnfs_mds_fallback_pg_get_mirror_count(pgio->pg_inode,
+ 0, NFS4_MAX_UINT64, IOMODE_RW,
+ NFS_I(pgio->pg_inode)->layout,
+ pgio->pg_lseg);
/* no lseg means that pnfs is not in use, so no mirroring here */
nfs_pageio_reset_write_mds(pgio);
out:
hdr->args.count,
(unsigned long long)hdr->args.offset);
+ trace_pnfs_mds_fallback_write_done(hdr->inode,
+ hdr->args.offset, hdr->args.count,
+ IOMODE_RW, NFS_I(hdr->inode)->layout,
+ hdr->lseg);
task->tk_status = pnfs_write_done_resend_to_mds(hdr);
}
}
hdr->args.count,
(unsigned long long)hdr->args.offset);
+ trace_pnfs_mds_fallback_read_done(hdr->inode,
+ hdr->args.offset, hdr->args.count,
+ IOMODE_READ, NFS_I(hdr->inode)->layout,
+ hdr->lseg);
task->tk_status = pnfs_read_done_resend_to_mds(hdr);
}
}
out_failed:
if (ff_layout_avoid_mds_available_ds(lseg))
return PNFS_TRY_AGAIN;
+ trace_pnfs_mds_fallback_read_pagelist(hdr->inode,
+ hdr->args.offset, hdr->args.count,
+ IOMODE_READ, NFS_I(hdr->inode)->layout, lseg);
return PNFS_NOT_ATTEMPTED;
}
out_failed:
if (ff_layout_avoid_mds_available_ds(lseg))
return PNFS_TRY_AGAIN;
+ trace_pnfs_mds_fallback_write_pagelist(hdr->inode,
+ hdr->args.offset, hdr->args.count,
+ IOMODE_RW, NFS_I(hdr->inode)->layout, lseg);
return PNFS_NOT_ATTEMPTED;
}
if (status == 0)
return 0;
- if (mirror->mirror_ds == NULL)
+ if (IS_ERR_OR_NULL(mirror->mirror_ds))
return -EINVAL;
dserr = kmalloc(sizeof(*dserr), gfp_flags);
#include "pnfs.h"
#include "nfs.h"
#include "netns.h"
+#include "sysfs.h"
#include "nfstrace.h"
}
if (inode->i_mapping->nrpages == 0)
- flags &= ~NFS_INO_INVALID_DATA;
+ flags &= ~(NFS_INO_INVALID_DATA|NFS_INO_DATA_INVAL_DEFER);
nfsi->cache_validity |= flags;
if (flags & NFS_INO_INVALID_DATA)
nfs_fscache_invalidate(inode);
i_size_write(inode, offset);
/* Optimisation */
if (offset == 0)
- NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_DATA;
+ NFS_I(inode)->cache_validity &= ~(NFS_INO_INVALID_DATA |
+ NFS_INO_DATA_INVAL_DEFER);
NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_SIZE;
spin_unlock(&inode->i_lock);
struct nfs_inode *nfsi = NFS_I(inode);
spin_lock(&inode->i_lock);
+ if (list_empty(&nfsi->open_files) &&
+ (nfsi->cache_validity & NFS_INO_DATA_INVAL_DEFER))
+ nfsi->cache_validity |= NFS_INO_INVALID_DATA |
+ NFS_INO_REVAL_FORCED;
list_add_tail_rcu(&ctx->list, &nfsi->open_files);
spin_unlock(&inode->i_lock);
}
nfs_fscache_open_file(inode, filp);
return 0;
}
+EXPORT_SYMBOL_GPL(nfs_open);
/*
* This function is called whenever some part of NFS notices that
set_bit(NFS_INO_INVALIDATING, bitlock);
smp_wmb();
- nfsi->cache_validity &= ~NFS_INO_INVALID_DATA;
+ nfsi->cache_validity &= ~(NFS_INO_INVALID_DATA|
+ NFS_INO_DATA_INVAL_DEFER);
spin_unlock(&inode->i_lock);
trace_nfs_invalidate_mapping_enter(inode);
ret = nfs_invalidate_mapping(inode, mapping);
dprintk("NFS: change_attr change on server for file %s/%ld\n",
inode->i_sb->s_id,
inode->i_ino);
- }
+ } else if (!have_delegation)
+ nfsi->cache_validity |= NFS_INO_DATA_INVAL_DEFER;
inode_set_iversion_raw(inode, fattr->change_attr);
attr_changed = true;
}
static void nfs_net_exit(struct net *net)
{
- struct nfs_net *nn = net_generic(net, nfs_net_id);
-
nfs_fs_proc_net_exit(net);
- nfs_cleanup_cb_ident_idr(net);
- WARN_ON_ONCE(!list_empty(&nn->nfs_client_list));
- WARN_ON_ONCE(!list_empty(&nn->nfs_volume_list));
+ nfs_clients_exit(net);
}
static struct pernet_operations nfs_net_ops = {
{
int err;
+ err = nfs_sysfs_init();
+ if (err < 0)
+ goto out10;
+
err = register_pernet_subsys(&nfs_net_ops);
if (err < 0)
goto out9;
out8:
unregister_pernet_subsys(&nfs_net_ops);
out9:
+ nfs_sysfs_exit();
+out10:
return err;
}
unregister_nfs_fs();
nfs_fs_proc_exit();
nfsiod_stop();
+ nfs_sysfs_exit();
}
/* Not quite true; I just maintain it */
* Maximum number of pages that readdir can use for creating
* a vmapped array of pages.
*/
-#define NFS_MAX_READDIR_PAGES 64
-#define NFS_MAX_READDIR_RAPAGES 8
+#define NFS_MAX_READDIR_PAGES 8
struct nfs_client_initdata {
unsigned long init_flags;
struct nfs_subversion *nfs_mod;
int proto;
u32 minorversion;
+ unsigned int nconnect;
struct net *net;
const struct rpc_timeout *timeparms;
const struct cred *cred;
char *export_path;
int port;
unsigned short protocol;
+ unsigned short nconnect;
} nfs_server;
void *lsm_opts;
/* client.c */
extern const struct rpc_program nfs_program;
extern void nfs_clients_init(struct net *net);
+extern void nfs_clients_exit(struct net *net);
extern struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *);
int nfs_create_rpc_client(struct nfs_client *, const struct nfs_client_initdata *, rpc_authflavor_t);
struct nfs_client *nfs_get_client(const struct nfs_client_initdata *);
struct nfs_server *nfs_alloc_server(void);
void nfs_server_copy_userdata(struct nfs_server *, struct nfs_server *);
-extern void nfs_cleanup_cb_ident_idr(struct net *);
extern void nfs_put_client(struct nfs_client *);
extern void nfs_free_client(struct nfs_client *);
extern struct nfs_client *nfs4_find_client_ident(struct net *, int);
uint32_t major, minor;
};
+struct nfs_netns_client;
+
struct nfs_net {
struct cache_detail *nfs_dns_resolve;
struct rpc_pipe *bl_device_pipe;
unsigned short nfs_callback_tcpport6;
int cb_users[NFS4_MAX_MINOR_VERSION + 1];
#endif
+ struct nfs_netns_client *nfs_client;
spinlock_t nfs_client_lock;
ktime_t boot_time;
#ifdef CONFIG_PROC_FS
return 0;
out_status:
*status = be32_to_cpup(p);
- trace_nfs_xdr_status((int)*status);
+ trace_nfs_xdr_status(xdr, (int)*status);
return 0;
}
return ERR_PTR(-EINVAL);
cl_init.hostname = buf;
+ if (mds_clp->cl_nconnect > 1 && ds_proto == XPRT_TRANSPORT_TCP)
+ cl_init.nconnect = mds_clp->cl_nconnect;
+
if (mds_srv->flags & NFS_MOUNT_NORESVPORT)
set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags);
return 0;
out_status:
*status = be32_to_cpup(p);
- trace_nfs_xdr_status((int)*status);
+ trace_nfs_xdr_status(xdr, (int)*status);
return 0;
}
const struct nfs_lock_context *l_ctx,
fmode_t fmode);
+extern int nfs4_proc_get_lease_time(struct nfs_client *clp,
+ struct nfs_fsinfo *fsinfo);
#if defined(CONFIG_NFS_V4_1)
extern int nfs41_sequence_done(struct rpc_task *, struct nfs4_sequence_res *);
extern int nfs4_proc_create_session(struct nfs_client *, const struct cred *);
extern int nfs4_proc_destroy_session(struct nfs4_session *, const struct cred *);
-extern int nfs4_proc_get_lease_time(struct nfs_client *clp,
- struct nfs_fsinfo *fsinfo);
extern int nfs4_proc_layoutcommit(struct nfs4_layoutcommit_data *data,
bool sync);
extern int nfs4_detect_session_trunking(struct nfs_client *clp,
const size_t addrlen,
const char *ip_addr,
int proto, const struct rpc_timeout *timeparms,
- u32 minorversion, struct net *net)
+ u32 minorversion, unsigned int nconnect,
+ struct net *net)
{
struct nfs_client_initdata cl_init = {
.hostname = hostname,
};
struct nfs_client *clp;
+ if (minorversion > 0 && proto == XPRT_TRANSPORT_TCP)
+ cl_init.nconnect = nconnect;
if (server->flags & NFS_MOUNT_NORESVPORT)
set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags);
if (server->options & NFS_OPTION_MIGRATION)
return ERR_PTR(-EINVAL);
cl_init.hostname = buf;
+ if (mds_clp->cl_nconnect > 1 && ds_proto == XPRT_TRANSPORT_TCP)
+ cl_init.nconnect = mds_clp->cl_nconnect;
+
if (mds_srv->flags & NFS_MOUNT_NORESVPORT)
__set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags);
data->nfs_server.protocol,
&timeparms,
data->minorversion,
+ data->nfs_server.nconnect,
data->net);
if (error < 0)
return error;
XPRT_TRANSPORT_RDMA,
parent_server->client->cl_timeout,
parent_client->cl_mvops->minor_version,
+ parent_client->cl_nconnect,
parent_client->cl_net);
if (!error)
goto init_server;
XPRT_TRANSPORT_TCP,
parent_server->client->cl_timeout,
parent_client->cl_mvops->minor_version,
+ parent_client->cl_nconnect,
parent_client->cl_net);
if (error < 0)
goto error;
set_bit(NFS_MIG_TSM_POSSIBLE, &server->mig_status);
error = nfs4_set_client(server, hostname, sap, salen, buf,
clp->cl_proto, clnt->cl_timeout,
- clp->cl_minorversion, net);
+ clp->cl_minorversion,
+ clp->cl_nconnect, net);
clear_bit(NFS_MIG_TSM_POSSIBLE, &server->mig_status);
if (error != 0) {
nfs_server_insert_lists(server);
return err;
if ((openflags & O_ACCMODE) == 3)
- openflags--;
+ return nfs_open(inode, filp);
/* We can't create new files here */
openflags &= ~(O_CREAT|O_EXCL);
bool same_inode = false;
int ret;
- if (remap_flags & ~(REMAP_FILE_DEDUP | REMAP_FILE_ADVISORY))
+ /* NFS does not support deduplication. */
+ if (remap_flags & REMAP_FILE_DEDUP)
+ return -EOPNOTSUPP;
+
+ if (remap_flags & ~REMAP_FILE_ADVISORY)
return -EINVAL;
/* check alignment w.r.t. clone_blksize */
return nfs4_delay_killable(timeout);
}
+static const nfs4_stateid *
+nfs4_recoverable_stateid(const nfs4_stateid *stateid)
+{
+ if (!stateid)
+ return NULL;
+ switch (stateid->type) {
+ case NFS4_OPEN_STATEID_TYPE:
+ case NFS4_LOCK_STATEID_TYPE:
+ case NFS4_DELEGATION_STATEID_TYPE:
+ return stateid;
+ default:
+ break;
+ }
+ return NULL;
+}
+
/* This is the error handling routine for processes that are allowed
* to sleep.
*/
{
struct nfs_client *clp = server->nfs_client;
struct nfs4_state *state = exception->state;
- const nfs4_stateid *stateid = exception->stateid;
+ const nfs4_stateid *stateid;
struct inode *inode = exception->inode;
int ret = errorcode;
exception->recovering = 0;
exception->retry = 0;
+ stateid = nfs4_recoverable_stateid(exception->stateid);
if (stateid == NULL && state != NULL)
- stateid = &state->stateid;
+ stateid = nfs4_recoverable_stateid(&state->stateid);
switch(errorcode) {
case 0:
return true;
}
+static fmode_t _nfs4_ctx_to_accessmode(const struct nfs_open_context *ctx)
+{
+ return ctx->mode & (FMODE_READ|FMODE_WRITE|FMODE_EXEC);
+}
+
+static fmode_t _nfs4_ctx_to_openmode(const struct nfs_open_context *ctx)
+{
+ fmode_t ret = ctx->mode & (FMODE_READ|FMODE_WRITE);
+
+ return (ctx->mode & FMODE_EXEC) ? FMODE_READ | ret : ret;
+}
+
static u32
nfs4_map_atomic_open_share(struct nfs_server *server,
fmode_t fmode, int openflags)
}
static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
- fmode_t fmode,
- int flags,
- struct nfs_open_context *ctx)
+ int flags, struct nfs_open_context *ctx)
{
struct nfs4_state_owner *sp = opendata->owner;
struct nfs_server *server = sp->so_server;
struct dentry *dentry;
struct nfs4_state *state;
+ fmode_t acc_mode = _nfs4_ctx_to_accessmode(ctx);
unsigned int seq;
int ret;
/* Parse layoutget results before we check for access */
pnfs_parse_lgopen(state->inode, opendata->lgp, ctx);
- ret = nfs4_opendata_access(sp->so_cred, opendata, state, fmode, flags);
+ ret = nfs4_opendata_access(sp->so_cred, opendata, state,
+ acc_mode, flags);
if (ret != 0)
goto out;
struct dentry *dentry = ctx->dentry;
const struct cred *cred = ctx->cred;
struct nfs4_threshold **ctx_th = &ctx->mdsthreshold;
- fmode_t fmode = ctx->mode & (FMODE_READ|FMODE_WRITE|FMODE_EXEC);
+ fmode_t fmode = _nfs4_ctx_to_openmode(ctx);
enum open_claim_type4 claim = NFS4_OPEN_CLAIM_NULL;
struct iattr *sattr = c->sattr;
struct nfs4_label *label = c->label;
if (d_really_is_positive(dentry))
opendata->state = nfs4_get_open_state(d_inode(dentry), sp);
- status = _nfs4_open_and_get_state(opendata, fmode, flags, ctx);
+ status = _nfs4_open_and_get_state(opendata, flags, ctx);
if (status != 0)
goto err_free_label;
state = ctx->state;
if (ctx->state == NULL)
return;
if (is_sync)
- nfs4_close_sync(ctx->state, ctx->mode);
+ nfs4_close_sync(ctx->state, _nfs4_ctx_to_openmode(ctx));
else
- nfs4_close_state(ctx->state, ctx->mode);
+ nfs4_close_state(ctx->state, _nfs4_ctx_to_openmode(ctx));
}
#define FATTR4_WORD1_NFS40_MASK (2*FATTR4_WORD1_MOUNTED_ON_FILEID - 1UL)
.rpc_message = &msg,
.callback_ops = &nfs4_setclientid_ops,
.callback_data = &setclientid,
- .flags = RPC_TASK_TIMEOUT,
+ .flags = RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN,
};
int status;
dprintk("NFS call setclientid_confirm auth=%s, (client ID %llx)\n",
clp->cl_rpcclient->cl_auth->au_ops->au_name,
clp->cl_clientid);
- status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+ status = rpc_call_sync(clp->cl_rpcclient, &msg,
+ RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN);
trace_nfs4_setclientid_confirm(clp, status);
dprintk("NFS reply setclientid_confirm: %d\n", status);
return status;
NFS_SP4_MACH_CRED_SECINFO, &clnt, &msg);
status = nfs4_call_sync(clnt, NFS_SERVER(dir), &msg, &args.seq_args,
- &res.seq_res, 0);
+ &res.seq_res, RPC_TASK_NO_ROUND_ROBIN);
dprintk("NFS reply secinfo: %d\n", status);
put_cred(cred);
.rpc_client = clp->cl_rpcclient,
.callback_ops = &nfs4_exchange_id_call_ops,
.rpc_message = &msg,
- .flags = RPC_TASK_TIMEOUT,
+ .flags = RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN,
};
struct nfs41_exchange_id_data *calldata;
int status;
};
int status;
- status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+ status = rpc_call_sync(clp->cl_rpcclient, &msg,
+ RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN);
trace_nfs4_destroy_clientid(clp, status);
if (status)
dprintk("NFS: Got error %d from the server %s on "
return ret;
}
+#endif /* CONFIG_NFS_V4_1 */
+
struct nfs4_get_lease_time_data {
struct nfs4_get_lease_time_args *args;
struct nfs4_get_lease_time_res *res;
(struct nfs4_get_lease_time_data *)calldata;
dprintk("--> %s\n", __func__);
- if (!nfs41_sequence_done(task, &data->res->lr_seq_res))
+ if (!nfs4_sequence_done(task, &data->res->lr_seq_res))
return;
switch (task->tk_status) {
case -NFS4ERR_DELAY:
return status;
}
+#ifdef CONFIG_NFS_V4_1
+
/*
* Initialize the values to be used by the client in CREATE_SESSION
* If nfs4_init_session set the fore channel request and response sizes,
{
unsigned int max_rqst_sz, max_resp_sz;
unsigned int max_bc_payload = rpc_max_bc_payload(clnt);
+ unsigned int max_bc_slots = rpc_num_bc_slots(clnt);
max_rqst_sz = NFS_MAX_FILE_IO_SIZE + nfs41_maxwrite_overhead;
max_resp_sz = NFS_MAX_FILE_IO_SIZE + nfs41_maxread_overhead;
args->bc_attrs.max_resp_sz_cached = 0;
args->bc_attrs.max_ops = NFS4_MAX_BACK_CHANNEL_OPS;
args->bc_attrs.max_reqs = max_t(unsigned short, max_session_cb_slots, 1);
+ if (args->bc_attrs.max_reqs > max_bc_slots)
+ args->bc_attrs.max_reqs = max_bc_slots;
dprintk("%s: Back Channel : max_rqst_sz=%u max_resp_sz=%u "
"max_resp_sz_cached=%u max_ops=%u max_reqs=%u\n",
nfs4_init_channel_attrs(&args, clp->cl_rpcclient);
args.flags = (SESSION4_PERSIST | SESSION4_BACK_CHAN);
- status = rpc_call_sync(session->clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+ status = rpc_call_sync(session->clp->cl_rpcclient, &msg,
+ RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN);
trace_nfs4_create_session(clp, status);
switch (status) {
if (!test_and_clear_bit(NFS4_SESSION_ESTABLISHED, &session->session_state))
return 0;
- status = rpc_call_sync(session->clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+ status = rpc_call_sync(session->clp->cl_rpcclient, &msg,
+ RPC_TASK_TIMEOUT | RPC_TASK_NO_ROUND_ROBIN);
trace_nfs4_destroy_session(session->clp, status);
if (status)
.rpc_client = clp->cl_rpcclient,
.rpc_message = &msg,
.callback_ops = &nfs4_reclaim_complete_call_ops,
- .flags = RPC_TASK_ASYNC,
+ .flags = RPC_TASK_ASYNC | RPC_TASK_NO_ROUND_ROBIN,
};
int status = -ENOMEM;
dprintk("--> %s\n", __func__);
status = nfs4_call_sync(clnt, server, &msg, &args.seq_args,
- &res.seq_res, 0);
+ &res.seq_res, RPC_TASK_NO_ROUND_ROBIN);
dprintk("<-- %s status=%d\n", __func__, status);
put_cred(cred);
static DEFINE_MUTEX(nfs_clid_init_mutex);
+static int nfs4_setup_state_renewal(struct nfs_client *clp)
+{
+ int status;
+ struct nfs_fsinfo fsinfo;
+ unsigned long now;
+
+ if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
+ nfs4_schedule_state_renewal(clp);
+ return 0;
+ }
+
+ now = jiffies;
+ status = nfs4_proc_get_lease_time(clp, &fsinfo);
+ if (status == 0) {
+ nfs4_set_lease_period(clp, fsinfo.lease_time * HZ, now);
+ nfs4_schedule_state_renewal(clp);
+ }
+
+ return status;
+}
+
int nfs4_init_clientid(struct nfs_client *clp, const struct cred *cred)
{
struct nfs4_setclientid_res clid = {
if (status != 0)
goto out;
clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
- nfs4_schedule_state_renewal(clp);
+ nfs4_setup_state_renewal(clp);
out:
return status;
}
#if defined(CONFIG_NFS_V4_1)
-static int nfs41_setup_state_renewal(struct nfs_client *clp)
-{
- int status;
- struct nfs_fsinfo fsinfo;
- unsigned long now;
-
- if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
- nfs4_schedule_state_renewal(clp);
- return 0;
- }
-
- now = jiffies;
- status = nfs4_proc_get_lease_time(clp, &fsinfo);
- if (status == 0) {
- nfs4_set_lease_period(clp, fsinfo.lease_time * HZ, now);
- nfs4_schedule_state_renewal(clp);
- }
-
- return status;
-}
-
static void nfs41_finish_session_reset(struct nfs_client *clp)
{
clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
clear_bit(NFS4CLNT_SESSION_RESET, &clp->cl_state);
/* create_session negotiated new slot table */
clear_bit(NFS4CLNT_BIND_CONN_TO_SESSION, &clp->cl_state);
- nfs41_setup_state_renewal(clp);
+ nfs4_setup_state_renewal(clp);
}
int nfs41_init_clientid(struct nfs_client *clp, const struct cred *cred)
* choose to use.
*/
goto out;
- nfs4_copy_open_stateid(dst, state);
- ret = 0;
+ ret = nfs4_copy_open_stateid(dst, state) ? 0 : -EAGAIN;
out:
if (nfs_server_capable(state->inode, NFS_CAP_STATEID_NFSV41))
dst->seqid = 0;
EXPORT_TRACEPOINT_SYMBOL_GPL(nfs4_pnfs_read);
EXPORT_TRACEPOINT_SYMBOL_GPL(nfs4_pnfs_write);
EXPORT_TRACEPOINT_SYMBOL_GPL(nfs4_pnfs_commit_ds);
+
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_pg_init_read);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_pg_init_write);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_pg_get_mirror_count);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_read_done);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_write_done);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_read_pagelist);
+EXPORT_TRACEPOINT_SYMBOL_GPL(pnfs_mds_fallback_write_pagelist);
#endif
TRACE_DEFINE_ENUM(NFS4ERR_XDEV);
#define show_nfsv4_errors(error) \
- __print_symbolic(-(error), \
+ __print_symbolic(error, \
{ NFS4_OK, "OK" }, \
/* Mapped by nfs4_stat_to_errno() */ \
{ EPERM, "EPERM" }, \
TP_STRUCT__entry(
__string(dstaddr, clp->cl_hostname)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
),
TP_printk(
- "error=%d (%s) dstaddr=%s",
- __entry->error,
+ "error=%ld (%s) dstaddr=%s",
+ -__entry->error,
show_nfsv4_errors(__entry->error),
__get_str(dstaddr)
)
__field(unsigned int, highest_slotid)
__field(unsigned int, target_highest_slotid)
__field(unsigned int, status_flags)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
__entry->error = res->sr_status;
),
TP_printk(
- "error=%d (%s) session=0x%08x slot_nr=%u seq_nr=%u "
+ "error=%ld (%s) session=0x%08x slot_nr=%u seq_nr=%u "
"highest_slotid=%u target_highest_slotid=%u "
"status_flags=%u (%s)",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
__entry->session,
__entry->slot_nr,
__field(unsigned int, seq_nr)
__field(unsigned int, highest_slotid)
__field(unsigned int, cachethis)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
__entry->seq_nr = args->csa_sequenceid;
__entry->highest_slotid = args->csa_highestslotid;
__entry->cachethis = args->csa_cachethis;
- __entry->error = -be32_to_cpu(status);
+ __entry->error = be32_to_cpu(status);
),
TP_printk(
- "error=%d (%s) session=0x%08x slot_nr=%u seq_nr=%u "
+ "error=%ld (%s) session=0x%08x slot_nr=%u seq_nr=%u "
"highest_slotid=%u",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
__entry->session,
__entry->slot_nr,
__entry->highest_slotid
)
);
+
+TRACE_EVENT(nfs4_cb_seqid_err,
+ TP_PROTO(
+ const struct cb_sequenceargs *args,
+ __be32 status
+ ),
+ TP_ARGS(args, status),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, session)
+ __field(unsigned int, slot_nr)
+ __field(unsigned int, seq_nr)
+ __field(unsigned int, highest_slotid)
+ __field(unsigned int, cachethis)
+ __field(unsigned long, error)
+ ),
+
+ TP_fast_assign(
+ __entry->session = nfs_session_id_hash(&args->csa_sessionid);
+ __entry->slot_nr = args->csa_slotid;
+ __entry->seq_nr = args->csa_sequenceid;
+ __entry->highest_slotid = args->csa_highestslotid;
+ __entry->cachethis = args->csa_cachethis;
+ __entry->error = be32_to_cpu(status);
+ ),
+
+ TP_printk(
+ "error=%ld (%s) session=0x%08x slot_nr=%u seq_nr=%u "
+ "highest_slotid=%u",
+ -__entry->error,
+ show_nfsv4_errors(__entry->error),
+ __entry->session,
+ __entry->slot_nr,
+ __entry->seq_nr,
+ __entry->highest_slotid
+ )
+);
+
#endif /* CONFIG_NFS_V4_1 */
TRACE_EVENT(nfs4_setup_sequence,
TRACE_EVENT(nfs4_xdr_status,
TP_PROTO(
+ const struct xdr_stream *xdr,
u32 op,
int error
),
- TP_ARGS(op, error),
+ TP_ARGS(xdr, op, error),
TP_STRUCT__entry(
+ __field(unsigned int, task_id)
+ __field(unsigned int, client_id)
+ __field(u32, xid)
__field(u32, op)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
+ const struct rpc_rqst *rqstp = xdr->rqst;
+ const struct rpc_task *task = rqstp->rq_task;
+
+ __entry->task_id = task->tk_pid;
+ __entry->client_id = task->tk_client->cl_clid;
+ __entry->xid = be32_to_cpu(rqstp->rq_xid);
__entry->op = op;
- __entry->error = -error;
+ __entry->error = error;
),
TP_printk(
- "operation %d: nfs status %d (%s)",
- __entry->op,
- __entry->error, show_nfsv4_errors(__entry->error)
+ "task:%u@%d xid=0x%08x error=%ld (%s) operation=%u",
+ __entry->task_id, __entry->client_id, __entry->xid,
+ -__entry->error, show_nfsv4_errors(__entry->error),
+ __entry->op
)
);
TP_ARGS(ctx, flags, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(unsigned int, flags)
__field(unsigned int, fmode)
__field(dev_t, dev)
const struct nfs4_state *state = ctx->state;
const struct inode *inode = NULL;
- __entry->error = error;
+ __entry->error = -error;
__entry->flags = flags;
__entry->fmode = (__force unsigned int)ctx->mode;
__entry->dev = ctx->dentry->d_sb->s_dev;
),
TP_printk(
- "error=%d (%s) flags=%d (%s) fmode=%s "
+ "error=%ld (%s) flags=%d (%s) fmode=%s "
"fileid=%02x:%02x:%llu fhandle=0x%08x "
"name=%02x:%02x:%llu/%s stateid=%d:0x%08x "
"openstateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
__entry->flags,
show_open_flags(__entry->flags),
__field(u32, fhandle)
__field(u64, fileid)
__field(unsigned int, fmode)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
),
),
TP_printk(
- "error=%d (%s) fmode=%s fileid=%02x:%02x:%llu "
+ "error=%ld (%s) fmode=%s fileid=%02x:%02x:%llu "
"fhandle=0x%08x openstateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
__entry->fmode ? show_fmode_flags(__entry->fmode) :
"closed",
TP_ARGS(request, state, cmd, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(int, cmd)
__field(char, type)
__field(loff_t, start)
),
TP_printk(
- "error=%d (%s) cmd=%s:%s range=%lld:%lld "
+ "error=%ld (%s) cmd=%s:%s range=%lld:%lld "
"fileid=%02x:%02x:%llu fhandle=0x%08x "
"stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
show_lock_cmd(__entry->cmd),
show_lock_type(__entry->type),
TP_ARGS(request, state, lockstateid, cmd, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(int, cmd)
__field(char, type)
__field(loff_t, start)
),
TP_printk(
- "error=%d (%s) cmd=%s:%s range=%lld:%lld "
+ "error=%ld (%s) cmd=%s:%s range=%lld:%lld "
"fileid=%02x:%02x:%llu fhandle=0x%08x "
"stateid=%d:0x%08x lockstateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
show_lock_cmd(__entry->cmd),
show_lock_type(__entry->type),
TP_STRUCT__entry(
__field(dev_t, dev)
__field(u32, fhandle)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
),
),
TP_printk(
- "error=%d (%s) dev=%02x:%02x fhandle=0x%08x "
+ "error=%ld (%s) dev=%02x:%02x fhandle=0x%08x "
"stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
__entry->fhandle,
TP_ARGS(state, lsp, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u32, fhandle)
__field(u64, fileid)
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
TP_STRUCT__entry(
__field(dev_t, dev)
- __field(int, error)
+ __field(unsigned long, error)
__field(u64, dir)
__string(name, name->name)
),
TP_fast_assign(
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
- __entry->error = error;
+ __entry->error = -error;
__assign_str(name, name->name);
),
TP_printk(
- "error=%d (%s) name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) name=%02x:%02x:%llu/%s",
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->dir,
TP_STRUCT__entry(
__field(dev_t, dev)
__field(u64, ino)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
),
TP_printk(
- "error=%d (%s) inode=%02x:%02x:%llu",
- __entry->error,
+ "error=%ld (%s) inode=%02x:%02x:%llu",
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->ino
TP_STRUCT__entry(
__field(dev_t, dev)
- __field(int, error)
+ __field(unsigned long, error)
__field(u64, olddir)
__string(oldname, oldname->name)
__field(u64, newdir)
),
TP_printk(
- "error=%d (%s) oldname=%02x:%02x:%llu/%s "
+ "error=%ld (%s) oldname=%02x:%02x:%llu/%s "
"newname=%02x:%02x:%llu/%s",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->olddir,
__field(dev_t, dev)
__field(u32, fhandle)
__field(u64, fileid)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
__entry->dev = inode->i_sb->s_dev;
__entry->fileid = NFS_FILEID(inode);
__entry->fhandle = nfs_fhandle_hash(NFS_FH(inode));
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x",
- __entry->error,
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x",
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__field(dev_t, dev)
__field(u32, fhandle)
__field(u64, fileid)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
),
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__field(u32, fhandle)
__field(u64, fileid)
__field(unsigned int, valid)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"valid=%s",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
TP_ARGS(clp, fhandle, inode, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u32, fhandle)
__field(u64, fileid)
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"dstaddr=%s",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
TP_ARGS(clp, fhandle, inode, stateid, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u32, fhandle)
__field(u64, fileid)
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"stateid=%d:0x%08x dstaddr=%s",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
TP_ARGS(name, len, id, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(u32, id)
__dynamic_array(char, name, len > 0 ? len + 1 : 1)
),
),
TP_printk(
- "error=%d id=%u name=%s",
- __entry->error,
+ "error=%ld (%s) id=%u name=%s",
+ -__entry->error, show_nfsv4_errors(__entry->error),
__entry->id,
__get_str(name)
)
__field(u64, fileid)
__field(loff_t, offset)
__field(size_t, count)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
),
__entry->fhandle = nfs_fhandle_hash(NFS_FH(inode));
__entry->offset = hdr->args.offset;
__entry->count = hdr->args.count;
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__entry->stateid_seq =
be32_to_cpu(state->stateid.seqid);
__entry->stateid_hash =
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"offset=%lld count=%zu stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__field(u64, fileid)
__field(loff_t, offset)
__field(size_t, count)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
),
__entry->fhandle = nfs_fhandle_hash(NFS_FH(inode));
__entry->offset = hdr->args.offset;
__entry->count = hdr->args.count;
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__entry->stateid_seq =
be32_to_cpu(state->stateid.seqid);
__entry->stateid_hash =
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"offset=%lld count=%zu stateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__field(u64, fileid)
__field(loff_t, offset)
__field(size_t, count)
- __field(int, error)
+ __field(unsigned long, error)
),
TP_fast_assign(
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"offset=%lld count=%zu",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__field(u32, iomode)
__field(u64, offset)
__field(u64, count)
- __field(int, error)
+ __field(unsigned long, error)
__field(int, stateid_seq)
__field(u32, stateid_hash)
__field(int, layoutstateid_seq)
),
TP_printk(
- "error=%d (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"iomode=%s offset=%llu count=%llu stateid=%d:0x%08x "
"layoutstateid=%d:0x%08x",
- __entry->error,
+ -__entry->error,
show_nfsv4_errors(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
TRACE_DEFINE_ENUM(PNFS_UPDATE_LAYOUT_INVALID_OPEN);
TRACE_DEFINE_ENUM(PNFS_UPDATE_LAYOUT_RETRY);
TRACE_DEFINE_ENUM(PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET);
+TRACE_DEFINE_ENUM(PNFS_UPDATE_LAYOUT_EXIT);
#define show_pnfs_update_layout_reason(reason) \
__print_symbolic(reason, \
{ PNFS_UPDATE_LAYOUT_BLOCKED, "layouts blocked" }, \
{ PNFS_UPDATE_LAYOUT_INVALID_OPEN, "invalid open" }, \
{ PNFS_UPDATE_LAYOUT_RETRY, "retrying" }, \
- { PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET, "sent layoutget" })
+ { PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET, "sent layoutget" }, \
+ { PNFS_UPDATE_LAYOUT_EXIT, "exit" })
TRACE_EVENT(pnfs_update_layout,
TP_PROTO(struct inode *inode,
)
);
+DECLARE_EVENT_CLASS(pnfs_layout_event,
+ TP_PROTO(struct inode *inode,
+ loff_t pos,
+ u64 count,
+ enum pnfs_iomode iomode,
+ struct pnfs_layout_hdr *lo,
+ struct pnfs_layout_segment *lseg
+ ),
+ TP_ARGS(inode, pos, count, iomode, lo, lseg),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(u64, fileid)
+ __field(u32, fhandle)
+ __field(loff_t, pos)
+ __field(u64, count)
+ __field(enum pnfs_iomode, iomode)
+ __field(int, layoutstateid_seq)
+ __field(u32, layoutstateid_hash)
+ __field(long, lseg)
+ ),
+ TP_fast_assign(
+ __entry->dev = inode->i_sb->s_dev;
+ __entry->fileid = NFS_FILEID(inode);
+ __entry->fhandle = nfs_fhandle_hash(NFS_FH(inode));
+ __entry->pos = pos;
+ __entry->count = count;
+ __entry->iomode = iomode;
+ if (lo != NULL) {
+ __entry->layoutstateid_seq =
+ be32_to_cpu(lo->plh_stateid.seqid);
+ __entry->layoutstateid_hash =
+ nfs_stateid_hash(&lo->plh_stateid);
+ } else {
+ __entry->layoutstateid_seq = 0;
+ __entry->layoutstateid_hash = 0;
+ }
+ __entry->lseg = (long)lseg;
+ ),
+ TP_printk(
+ "fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "iomode=%s pos=%llu count=%llu "
+ "layoutstateid=%d:0x%08x lseg=0x%lx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ (unsigned long long)__entry->fileid,
+ __entry->fhandle,
+ show_pnfs_iomode(__entry->iomode),
+ (unsigned long long)__entry->pos,
+ (unsigned long long)__entry->count,
+ __entry->layoutstateid_seq, __entry->layoutstateid_hash,
+ __entry->lseg
+ )
+);
+
+#define DEFINE_PNFS_LAYOUT_EVENT(name) \
+ DEFINE_EVENT(pnfs_layout_event, name, \
+ TP_PROTO(struct inode *inode, \
+ loff_t pos, \
+ u64 count, \
+ enum pnfs_iomode iomode, \
+ struct pnfs_layout_hdr *lo, \
+ struct pnfs_layout_segment *lseg \
+ ), \
+ TP_ARGS(inode, pos, count, iomode, lo, lseg))
+
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_pg_init_read);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_pg_init_write);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_pg_get_mirror_count);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_read_done);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_write_done);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_read_pagelist);
+DEFINE_PNFS_LAYOUT_EVENT(pnfs_mds_fallback_write_pagelist);
+
#endif /* CONFIG_NFS_V4_1 */
#endif /* _TRACE_NFS4_H */
#define NFS4_dec_sequence_sz \
(compound_decode_hdr_maxsz + \
decode_sequence_maxsz)
+#endif
#define NFS4_enc_get_lease_time_sz (compound_encode_hdr_maxsz + \
encode_sequence_maxsz + \
encode_putrootfh_maxsz + \
decode_sequence_maxsz + \
decode_putrootfh_maxsz + \
decode_fsinfo_maxsz)
+#if defined(CONFIG_NFS_V4_1)
#define NFS4_enc_reclaim_complete_sz (compound_encode_hdr_maxsz + \
encode_sequence_maxsz + \
encode_reclaim_complete_maxsz)
encode_nops(&hdr);
}
+#endif
+
/*
* a GET_LEASE_TIME request
*/
encode_nops(&hdr);
}
+#ifdef CONFIG_NFS_V4_1
+
/*
* a RECLAIM_COMPLETE request
*/
return true;
out_status:
nfserr = be32_to_cpup(p);
- trace_nfs4_xdr_status(opnum, nfserr);
+ trace_nfs4_xdr_status(xdr, opnum, nfserr);
*nfs_retval = nfs4_stat_to_errno(nfserr);
return true;
out_bad_operation:
*res = be32_to_cpup(p);
bitmap[0] &= ~FATTR4_WORD0_LEASE_TIME;
}
- dprintk("%s: file size=%u\n", __func__, (unsigned int)*res);
+ dprintk("%s: lease time=%u\n", __func__, (unsigned int)*res);
return 0;
}
return status;
}
+#endif
+
/*
* Decode GET_LEASE_TIME response
*/
return status;
}
+#ifdef CONFIG_NFS_V4_1
+
/*
* Decode RECLAIM_COMPLETE response
*/
PROC41(CREATE_SESSION, enc_create_session, dec_create_session),
PROC41(DESTROY_SESSION, enc_destroy_session, dec_destroy_session),
PROC41(SEQUENCE, enc_sequence, dec_sequence),
- PROC41(GET_LEASE_TIME, enc_get_lease_time, dec_get_lease_time),
+ PROC(GET_LEASE_TIME, enc_get_lease_time, dec_get_lease_time),
PROC41(RECLAIM_COMPLETE,enc_reclaim_complete, dec_reclaim_complete),
PROC41(GETDEVICEINFO, enc_getdeviceinfo, dec_getdeviceinfo),
PROC41(LAYOUTGET, enc_layoutget, dec_layoutget),
#include <linux/tracepoint.h>
#include <linux/iversion.h>
+TRACE_DEFINE_ENUM(DT_UNKNOWN);
+TRACE_DEFINE_ENUM(DT_FIFO);
+TRACE_DEFINE_ENUM(DT_CHR);
+TRACE_DEFINE_ENUM(DT_DIR);
+TRACE_DEFINE_ENUM(DT_BLK);
+TRACE_DEFINE_ENUM(DT_REG);
+TRACE_DEFINE_ENUM(DT_LNK);
+TRACE_DEFINE_ENUM(DT_SOCK);
+TRACE_DEFINE_ENUM(DT_WHT);
+
#define nfs_show_file_type(ftype) \
__print_symbolic(ftype, \
{ DT_UNKNOWN, "UNKNOWN" }, \
{ DT_SOCK, "SOCK" }, \
{ DT_WHT, "WHT" })
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_DATA);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_ATIME);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_ACCESS);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_ACL);
+TRACE_DEFINE_ENUM(NFS_INO_REVAL_PAGECACHE);
+TRACE_DEFINE_ENUM(NFS_INO_REVAL_FORCED);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_LABEL);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_CHANGE);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_CTIME);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_MTIME);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_SIZE);
+TRACE_DEFINE_ENUM(NFS_INO_INVALID_OTHER);
+
#define nfs_show_cache_validity(v) \
__print_flags(v, "|", \
- { NFS_INO_INVALID_ATTR, "INVALID_ATTR" }, \
{ NFS_INO_INVALID_DATA, "INVALID_DATA" }, \
{ NFS_INO_INVALID_ATIME, "INVALID_ATIME" }, \
{ NFS_INO_INVALID_ACCESS, "INVALID_ACCESS" }, \
{ NFS_INO_INVALID_ACL, "INVALID_ACL" }, \
{ NFS_INO_REVAL_PAGECACHE, "REVAL_PAGECACHE" }, \
{ NFS_INO_REVAL_FORCED, "REVAL_FORCED" }, \
- { NFS_INO_INVALID_LABEL, "INVALID_LABEL" })
+ { NFS_INO_INVALID_LABEL, "INVALID_LABEL" }, \
+ { NFS_INO_INVALID_CHANGE, "INVALID_CHANGE" }, \
+ { NFS_INO_INVALID_CTIME, "INVALID_CTIME" }, \
+ { NFS_INO_INVALID_MTIME, "INVALID_MTIME" }, \
+ { NFS_INO_INVALID_SIZE, "INVALID_SIZE" }, \
+ { NFS_INO_INVALID_OTHER, "INVALID_OTHER" })
+
+TRACE_DEFINE_ENUM(NFS_INO_ADVISE_RDPLUS);
+TRACE_DEFINE_ENUM(NFS_INO_STALE);
+TRACE_DEFINE_ENUM(NFS_INO_ACL_LRU_SET);
+TRACE_DEFINE_ENUM(NFS_INO_INVALIDATING);
+TRACE_DEFINE_ENUM(NFS_INO_FSCACHE);
+TRACE_DEFINE_ENUM(NFS_INO_FSCACHE_LOCK);
+TRACE_DEFINE_ENUM(NFS_INO_LAYOUTCOMMIT);
+TRACE_DEFINE_ENUM(NFS_INO_LAYOUTCOMMITTING);
+TRACE_DEFINE_ENUM(NFS_INO_LAYOUTSTATS);
+TRACE_DEFINE_ENUM(NFS_INO_ODIRECT);
#define nfs_show_nfsi_flags(v) \
__print_flags(v, "|", \
- { 1 << NFS_INO_ADVISE_RDPLUS, "ADVISE_RDPLUS" }, \
- { 1 << NFS_INO_STALE, "STALE" }, \
- { 1 << NFS_INO_INVALIDATING, "INVALIDATING" }, \
- { 1 << NFS_INO_FSCACHE, "FSCACHE" }, \
- { 1 << NFS_INO_LAYOUTCOMMIT, "NEED_LAYOUTCOMMIT" }, \
- { 1 << NFS_INO_LAYOUTCOMMITTING, "LAYOUTCOMMIT" })
+ { BIT(NFS_INO_ADVISE_RDPLUS), "ADVISE_RDPLUS" }, \
+ { BIT(NFS_INO_STALE), "STALE" }, \
+ { BIT(NFS_INO_ACL_LRU_SET), "ACL_LRU_SET" }, \
+ { BIT(NFS_INO_INVALIDATING), "INVALIDATING" }, \
+ { BIT(NFS_INO_FSCACHE), "FSCACHE" }, \
+ { BIT(NFS_INO_FSCACHE_LOCK), "FSCACHE_LOCK" }, \
+ { BIT(NFS_INO_LAYOUTCOMMIT), "NEED_LAYOUTCOMMIT" }, \
+ { BIT(NFS_INO_LAYOUTCOMMITTING), "LAYOUTCOMMIT" }, \
+ { BIT(NFS_INO_LAYOUTSTATS), "LAYOUTSTATS" }, \
+ { BIT(NFS_INO_ODIRECT), "ODIRECT" })
DECLARE_EVENT_CLASS(nfs_inode_event,
TP_PROTO(
TP_ARGS(inode, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u32, fhandle)
__field(unsigned char, type)
TP_fast_assign(
const struct nfs_inode *nfsi = NFS_I(inode);
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__entry->dev = inode->i_sb->s_dev;
__entry->fileid = nfsi->fileid;
__entry->fhandle = nfs_fhandle_hash(&nfsi->fh);
),
TP_printk(
- "error=%d fileid=%02x:%02x:%llu fhandle=0x%08x "
+ "error=%ld (%s) fileid=%02x:%02x:%llu fhandle=0x%08x "
"type=%u (%s) version=%llu size=%lld "
- "cache_validity=%lu (%s) nfs_flags=%ld (%s)",
- __entry->error,
+ "cache_validity=0x%lx (%s) nfs_flags=0x%lx (%s)",
+ -__entry->error, nfs_show_status(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->fileid,
__entry->fhandle,
DEFINE_NFS_INODE_EVENT(nfs_access_enter);
DEFINE_NFS_INODE_EVENT_DONE(nfs_access_exit);
+TRACE_DEFINE_ENUM(LOOKUP_FOLLOW);
+TRACE_DEFINE_ENUM(LOOKUP_DIRECTORY);
+TRACE_DEFINE_ENUM(LOOKUP_AUTOMOUNT);
+TRACE_DEFINE_ENUM(LOOKUP_PARENT);
+TRACE_DEFINE_ENUM(LOOKUP_REVAL);
+TRACE_DEFINE_ENUM(LOOKUP_RCU);
+TRACE_DEFINE_ENUM(LOOKUP_NO_REVAL);
+TRACE_DEFINE_ENUM(LOOKUP_NO_EVAL);
+TRACE_DEFINE_ENUM(LOOKUP_OPEN);
+TRACE_DEFINE_ENUM(LOOKUP_CREATE);
+TRACE_DEFINE_ENUM(LOOKUP_EXCL);
+TRACE_DEFINE_ENUM(LOOKUP_RENAME_TARGET);
+TRACE_DEFINE_ENUM(LOOKUP_JUMPED);
+TRACE_DEFINE_ENUM(LOOKUP_ROOT);
+TRACE_DEFINE_ENUM(LOOKUP_EMPTY);
+TRACE_DEFINE_ENUM(LOOKUP_DOWN);
+
#define show_lookup_flags(flags) \
- __print_flags((unsigned long)flags, "|", \
- { LOOKUP_AUTOMOUNT, "AUTOMOUNT" }, \
+ __print_flags(flags, "|", \
+ { LOOKUP_FOLLOW, "FOLLOW" }, \
{ LOOKUP_DIRECTORY, "DIRECTORY" }, \
+ { LOOKUP_AUTOMOUNT, "AUTOMOUNT" }, \
+ { LOOKUP_PARENT, "PARENT" }, \
+ { LOOKUP_REVAL, "REVAL" }, \
+ { LOOKUP_RCU, "RCU" }, \
+ { LOOKUP_NO_REVAL, "NO_REVAL" }, \
+ { LOOKUP_NO_EVAL, "NO_EVAL" }, \
{ LOOKUP_OPEN, "OPEN" }, \
{ LOOKUP_CREATE, "CREATE" }, \
- { LOOKUP_EXCL, "EXCL" })
+ { LOOKUP_EXCL, "EXCL" }, \
+ { LOOKUP_RENAME_TARGET, "RENAME_TARGET" }, \
+ { LOOKUP_JUMPED, "JUMPED" }, \
+ { LOOKUP_ROOT, "ROOT" }, \
+ { LOOKUP_EMPTY, "EMPTY" }, \
+ { LOOKUP_DOWN, "DOWN" })
DECLARE_EVENT_CLASS(nfs_lookup_event,
TP_PROTO(
TP_ARGS(dir, dentry, flags),
TP_STRUCT__entry(
- __field(unsigned int, flags)
+ __field(unsigned long, flags)
__field(dev_t, dev)
__field(u64, dir)
__string(name, dentry->d_name.name)
),
TP_printk(
- "flags=%u (%s) name=%02x:%02x:%llu/%s",
+ "flags=0x%lx (%s) name=%02x:%02x:%llu/%s",
__entry->flags,
show_lookup_flags(__entry->flags),
MAJOR(__entry->dev), MINOR(__entry->dev),
TP_ARGS(dir, dentry, flags, error),
TP_STRUCT__entry(
- __field(int, error)
- __field(unsigned int, flags)
+ __field(unsigned long, error)
+ __field(unsigned long, flags)
__field(dev_t, dev)
__field(u64, dir)
__string(name, dentry->d_name.name)
TP_fast_assign(
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__entry->flags = flags;
__assign_str(name, dentry->d_name.name);
),
TP_printk(
- "error=%d flags=%u (%s) name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) flags=0x%lx (%s) name=%02x:%02x:%llu/%s",
+ -__entry->error, nfs_show_status(__entry->error),
__entry->flags,
show_lookup_flags(__entry->flags),
MAJOR(__entry->dev), MINOR(__entry->dev),
DEFINE_NFS_LOOKUP_EVENT(nfs_lookup_revalidate_enter);
DEFINE_NFS_LOOKUP_EVENT_DONE(nfs_lookup_revalidate_exit);
+TRACE_DEFINE_ENUM(O_WRONLY);
+TRACE_DEFINE_ENUM(O_RDWR);
+TRACE_DEFINE_ENUM(O_CREAT);
+TRACE_DEFINE_ENUM(O_EXCL);
+TRACE_DEFINE_ENUM(O_NOCTTY);
+TRACE_DEFINE_ENUM(O_TRUNC);
+TRACE_DEFINE_ENUM(O_APPEND);
+TRACE_DEFINE_ENUM(O_NONBLOCK);
+TRACE_DEFINE_ENUM(O_DSYNC);
+TRACE_DEFINE_ENUM(O_DIRECT);
+TRACE_DEFINE_ENUM(O_LARGEFILE);
+TRACE_DEFINE_ENUM(O_DIRECTORY);
+TRACE_DEFINE_ENUM(O_NOFOLLOW);
+TRACE_DEFINE_ENUM(O_NOATIME);
+TRACE_DEFINE_ENUM(O_CLOEXEC);
+
#define show_open_flags(flags) \
- __print_flags((unsigned long)flags, "|", \
+ __print_flags(flags, "|", \
+ { O_WRONLY, "O_WRONLY" }, \
+ { O_RDWR, "O_RDWR" }, \
{ O_CREAT, "O_CREAT" }, \
{ O_EXCL, "O_EXCL" }, \
+ { O_NOCTTY, "O_NOCTTY" }, \
{ O_TRUNC, "O_TRUNC" }, \
{ O_APPEND, "O_APPEND" }, \
+ { O_NONBLOCK, "O_NONBLOCK" }, \
{ O_DSYNC, "O_DSYNC" }, \
{ O_DIRECT, "O_DIRECT" }, \
- { O_DIRECTORY, "O_DIRECTORY" })
+ { O_LARGEFILE, "O_LARGEFILE" }, \
+ { O_DIRECTORY, "O_DIRECTORY" }, \
+ { O_NOFOLLOW, "O_NOFOLLOW" }, \
+ { O_NOATIME, "O_NOATIME" }, \
+ { O_CLOEXEC, "O_CLOEXEC" })
+
+TRACE_DEFINE_ENUM(FMODE_READ);
+TRACE_DEFINE_ENUM(FMODE_WRITE);
+TRACE_DEFINE_ENUM(FMODE_EXEC);
#define show_fmode_flags(mode) \
__print_flags(mode, "|", \
TP_ARGS(dir, ctx, flags),
TP_STRUCT__entry(
- __field(unsigned int, flags)
+ __field(unsigned long, flags)
__field(unsigned int, fmode)
__field(dev_t, dev)
__field(u64, dir)
),
TP_printk(
- "flags=%u (%s) fmode=%s name=%02x:%02x:%llu/%s",
+ "flags=0x%lx (%s) fmode=%s name=%02x:%02x:%llu/%s",
__entry->flags,
show_open_flags(__entry->flags),
show_fmode_flags(__entry->fmode),
TP_ARGS(dir, ctx, flags, error),
TP_STRUCT__entry(
- __field(int, error)
- __field(unsigned int, flags)
+ __field(unsigned long, error)
+ __field(unsigned long, flags)
__field(unsigned int, fmode)
__field(dev_t, dev)
__field(u64, dir)
),
TP_fast_assign(
- __entry->error = error;
+ __entry->error = -error;
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
__entry->flags = flags;
),
TP_printk(
- "error=%d flags=%u (%s) fmode=%s "
+ "error=%ld (%s) flags=0x%lx (%s) fmode=%s "
"name=%02x:%02x:%llu/%s",
- __entry->error,
+ -__entry->error, nfs_show_status(__entry->error),
__entry->flags,
show_open_flags(__entry->flags),
show_fmode_flags(__entry->fmode),
TP_ARGS(dir, dentry, flags),
TP_STRUCT__entry(
- __field(unsigned int, flags)
+ __field(unsigned long, flags)
__field(dev_t, dev)
__field(u64, dir)
__string(name, dentry->d_name.name)
),
TP_printk(
- "flags=%u (%s) name=%02x:%02x:%llu/%s",
+ "flags=0x%lx (%s) name=%02x:%02x:%llu/%s",
__entry->flags,
show_open_flags(__entry->flags),
MAJOR(__entry->dev), MINOR(__entry->dev),
TP_ARGS(dir, dentry, flags, error),
TP_STRUCT__entry(
- __field(int, error)
- __field(unsigned int, flags)
+ __field(unsigned long, error)
+ __field(unsigned long, flags)
__field(dev_t, dev)
__field(u64, dir)
__string(name, dentry->d_name.name)
),
TP_fast_assign(
- __entry->error = error;
+ __entry->error = -error;
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
__entry->flags = flags;
),
TP_printk(
- "error=%d flags=%u (%s) name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) flags=0x%lx (%s) name=%02x:%02x:%llu/%s",
+ -__entry->error, nfs_show_status(__entry->error),
__entry->flags,
show_open_flags(__entry->flags),
MAJOR(__entry->dev), MINOR(__entry->dev),
TP_ARGS(dir, dentry, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u64, dir)
__string(name, dentry->d_name.name)
TP_fast_assign(
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__assign_str(name, dentry->d_name.name);
),
TP_printk(
- "error=%d name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) name=%02x:%02x:%llu/%s",
+ -__entry->error, nfs_show_status(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->dir,
__get_str(name)
TP_ARGS(inode, dir, dentry, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned long, error)
__field(dev_t, dev)
__field(u64, fileid)
__field(u64, dir)
__entry->dev = inode->i_sb->s_dev;
__entry->fileid = NFS_FILEID(inode);
__entry->dir = NFS_FILEID(dir);
- __entry->error = error;
+ __entry->error = error < 0 ? -error : 0;
__assign_str(name, dentry->d_name.name);
),
TP_printk(
- "error=%d fileid=%02x:%02x:%llu name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) fileid=%02x:%02x:%llu name=%02x:%02x:%llu/%s",
+ -__entry->error, nfs_show_status(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
__entry->fileid,
MAJOR(__entry->dev), MINOR(__entry->dev),
TP_STRUCT__entry(
__field(dev_t, dev)
- __field(int, error)
+ __field(unsigned long, error)
__field(u64, old_dir)
__string(old_name, old_dentry->d_name.name)
__field(u64, new_dir)
TP_fast_assign(
__entry->dev = old_dir->i_sb->s_dev;
+ __entry->error = -error;
__entry->old_dir = NFS_FILEID(old_dir);
__entry->new_dir = NFS_FILEID(new_dir);
- __entry->error = error;
__assign_str(old_name, old_dentry->d_name.name);
__assign_str(new_name, new_dentry->d_name.name);
),
TP_printk(
- "error=%d old_name=%02x:%02x:%llu/%s "
+ "error=%ld (%s) old_name=%02x:%02x:%llu/%s "
"new_name=%02x:%02x:%llu/%s",
- __entry->error,
+ -__entry->error, nfs_show_status(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->old_dir,
__get_str(old_name),
TP_STRUCT__entry(
__field(dev_t, dev)
- __field(int, error)
+ __field(unsigned long, error)
__field(u64, dir)
__dynamic_array(char, name, data->args.name.len + 1)
),
size_t len = data->args.name.len;
__entry->dev = dir->i_sb->s_dev;
__entry->dir = NFS_FILEID(dir);
- __entry->error = error;
+ __entry->error = -error;
memcpy(__get_str(name),
data->args.name.name, len);
__get_str(name)[len] = 0;
),
TP_printk(
- "error=%d name=%02x:%02x:%llu/%s",
- __entry->error,
+ "error=%ld (%s) name=%02x:%02x:%llu/%s",
+ -__entry->error, nfs_show_status(__entry->error),
MAJOR(__entry->dev), MINOR(__entry->dev),
(unsigned long long)__entry->dir,
__get_str(name)
TRACE_DEFINE_ENUM(NFSERR_NOENT);
TRACE_DEFINE_ENUM(NFSERR_IO);
TRACE_DEFINE_ENUM(NFSERR_NXIO);
+TRACE_DEFINE_ENUM(ECHILD);
+TRACE_DEFINE_ENUM(NFSERR_EAGAIN);
TRACE_DEFINE_ENUM(NFSERR_ACCES);
TRACE_DEFINE_ENUM(NFSERR_EXIST);
TRACE_DEFINE_ENUM(NFSERR_XDEV);
TRACE_DEFINE_ENUM(NFSERR_NOSPC);
TRACE_DEFINE_ENUM(NFSERR_ROFS);
TRACE_DEFINE_ENUM(NFSERR_MLINK);
+TRACE_DEFINE_ENUM(NFSERR_OPNOTSUPP);
TRACE_DEFINE_ENUM(NFSERR_NAMETOOLONG);
TRACE_DEFINE_ENUM(NFSERR_NOTEMPTY);
TRACE_DEFINE_ENUM(NFSERR_DQUOT);
{ NFSERR_NOENT, "NOENT" }, \
{ NFSERR_IO, "IO" }, \
{ NFSERR_NXIO, "NXIO" }, \
+ { ECHILD, "CHILD" }, \
+ { NFSERR_EAGAIN, "AGAIN" }, \
{ NFSERR_ACCES, "ACCES" }, \
{ NFSERR_EXIST, "EXIST" }, \
{ NFSERR_XDEV, "XDEV" }, \
{ NFSERR_NOSPC, "NOSPC" }, \
{ NFSERR_ROFS, "ROFS" }, \
{ NFSERR_MLINK, "MLINK" }, \
+ { NFSERR_OPNOTSUPP, "OPNOTSUPP" }, \
{ NFSERR_NAMETOOLONG, "NAMETOOLONG" }, \
{ NFSERR_NOTEMPTY, "NOTEMPTY" }, \
{ NFSERR_DQUOT, "DQUOT" }, \
TRACE_EVENT(nfs_xdr_status,
TP_PROTO(
+ const struct xdr_stream *xdr,
int error
),
- TP_ARGS(error),
+ TP_ARGS(xdr, error),
TP_STRUCT__entry(
- __field(int, error)
+ __field(unsigned int, task_id)
+ __field(unsigned int, client_id)
+ __field(u32, xid)
+ __field(unsigned long, error)
),
TP_fast_assign(
+ const struct rpc_rqst *rqstp = xdr->rqst;
+ const struct rpc_task *task = rqstp->rq_task;
+
+ __entry->task_id = task->tk_pid;
+ __entry->client_id = task->tk_client->cl_clid;
+ __entry->xid = be32_to_cpu(rqstp->rq_xid);
__entry->error = error;
),
TP_printk(
- "error=%d (%s)",
- __entry->error, nfs_show_status(__entry->error)
+ "task:%u@%d xid=0x%08x error=%ld (%s)",
+ __entry->task_id, __entry->client_id, __entry->xid,
+ -__entry->error, nfs_show_status(__entry->error)
)
);
static inline struct nfs_page *
nfs_page_alloc(void)
{
- struct nfs_page *p = kmem_cache_zalloc(nfs_page_cachep, GFP_NOIO);
+ struct nfs_page *p = kmem_cache_zalloc(nfs_page_cachep, GFP_KERNEL);
if (p)
INIT_LIST_HEAD(&p->wb_list);
return p;
if (pagecount <= ARRAY_SIZE(pg_array->page_array))
pg_array->pagevec = pg_array->page_array;
else {
- if (hdr->rw_mode == FMODE_WRITE)
- gfp_flags = GFP_NOIO;
pg_array->pagevec = kcalloc(pagecount, sizeof(struct page *), gfp_flags);
if (!pg_array->pagevec) {
pg_array->npages = 0;
desc->pg_mirrors_dynamic = NULL;
if (mirror_count == 1)
return desc->pg_mirrors_static;
- ret = kmalloc_array(mirror_count, sizeof(*ret), GFP_NOFS);
+ ret = kmalloc_array(mirror_count, sizeof(*ret), GFP_KERNEL);
if (ret != NULL) {
for (i = 0; i < mirror_count; i++)
nfs_pageio_mirror_init(&ret[i], desc->pg_bsize);
spin_unlock(&ino->i_lock);
lseg = ERR_PTR(wait_var_event_killable(&lo->plh_outstanding,
!atomic_read(&lo->plh_outstanding)));
- if (IS_ERR(lseg) || !list_empty(&lo->plh_segs))
+ if (IS_ERR(lseg))
goto out_put_layout_hdr;
pnfs_put_layout_hdr(lo);
goto lookup_again;
* stateid.
*/
if (test_bit(NFS_LAYOUT_INVALID_STID, &lo->plh_flags)) {
+ int status;
/*
* The first layoutget for the file. Need to serialize per
}
first = true;
- if (nfs4_select_rw_stateid(ctx->state,
+ status = nfs4_select_rw_stateid(ctx->state,
iomode == IOMODE_RW ? FMODE_WRITE : FMODE_READ,
- NULL, &stateid, NULL) != 0) {
+ NULL, &stateid, NULL);
+ if (status != 0) {
trace_pnfs_update_layout(ino, pos, count,
iomode, lo, lseg,
PNFS_UPDATE_LAYOUT_INVALID_OPEN);
- goto out_unlock;
+ if (status != -EAGAIN)
+ goto out_unlock;
+ spin_unlock(&ino->i_lock);
+ nfs4_schedule_stateid_recovery(server, ctx->state);
+ pnfs_clear_first_layoutget(lo);
+ pnfs_put_layout_hdr(lo);
+ goto lookup_again;
}
} else {
nfs4_stateid_copy(&stateid, &lo->plh_stateid);
out_put_layout_hdr:
if (first)
pnfs_clear_first_layoutget(lo);
+ trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg,
+ PNFS_UPDATE_LAYOUT_EXIT);
pnfs_put_layout_hdr(lo);
out:
dprintk("%s: inode %s/%llu pNFS layout segment %s for "
wb_size,
IOMODE_RW,
false,
- GFP_NOFS);
+ GFP_KERNEL);
if (IS_ERR(pgio->pg_lseg)) {
pgio->pg_error = PTR_ERR(pgio->pg_lseg);
pgio->pg_lseg = NULL;
#define NFS_DEFAULT_VERSION 2
#endif
+#define NFS_MAX_CONNECTIONS 16
+
enum {
/* Mount options that take no arguments */
Opt_soft, Opt_softerr, Opt_hard,
Opt_nfsvers,
Opt_sec, Opt_proto, Opt_mountproto, Opt_mounthost,
Opt_addr, Opt_mountaddr, Opt_clientaddr,
+ Opt_nconnect,
Opt_lookupcache,
Opt_fscache_uniq,
Opt_local_lock,
{ Opt_mounthost, "mounthost=%s" },
{ Opt_mountaddr, "mountaddr=%s" },
+ { Opt_nconnect, "nconnect=%s" },
+
{ Opt_lookupcache, "lookupcache=%s" },
{ Opt_fscache_uniq, "fsc=%s" },
{ Opt_local_lock, "local_lock=%s" },
}
default:
if (showdefaults)
- seq_printf(m, ",mountaddr=unspecified");
+ seq_puts(m, ",mountaddr=unspecified");
}
if (nfss->mountd_version || showdefaults)
seq_printf(m, ",proto=%s",
rpc_peeraddr2str(nfss->client, RPC_DISPLAY_NETID));
rcu_read_unlock();
+ if (clp->cl_nconnect > 0)
+ seq_printf(m, ",nconnect=%u", clp->cl_nconnect);
if (version == 4) {
if (nfss->port != NFS_PORT)
seq_printf(m, ",port=%u", nfss->port);
nfs_show_nfsv4_options(m, nfss, showdefaults);
if (nfss->options & NFS_OPTION_FSCACHE)
- seq_printf(m, ",fsc");
+ seq_puts(m, ",fsc");
if (nfss->options & NFS_OPTION_MIGRATION)
- seq_printf(m, ",migration");
+ seq_puts(m, ",migration");
if (nfss->flags & NFS_MOUNT_LOOKUP_CACHE_NONEG) {
if (nfss->flags & NFS_MOUNT_LOOKUP_CACHE_NONE)
- seq_printf(m, ",lookupcache=none");
+ seq_puts(m, ",lookupcache=none");
else
- seq_printf(m, ",lookupcache=pos");
+ seq_puts(m, ",lookupcache=pos");
}
local_flock = nfss->flags & NFS_MOUNT_LOCAL_FLOCK;
local_fcntl = nfss->flags & NFS_MOUNT_LOCAL_FCNTL;
if (!local_flock && !local_fcntl)
- seq_printf(m, ",local_lock=none");
+ seq_puts(m, ",local_lock=none");
else if (local_flock && local_fcntl)
- seq_printf(m, ",local_lock=all");
+ seq_puts(m, ",local_lock=all");
else if (local_flock)
- seq_printf(m, ",local_lock=flock");
+ seq_puts(m, ",local_lock=flock");
else
- seq_printf(m, ",local_lock=posix");
+ seq_puts(m, ",local_lock=posix");
}
/*
EXPORT_SYMBOL_GPL(nfs_show_options);
#if IS_ENABLED(CONFIG_NFS_V4)
+static void show_lease(struct seq_file *m, struct nfs_server *server)
+{
+ struct nfs_client *clp = server->nfs_client;
+ unsigned long expire;
+
+ seq_printf(m, ",lease_time=%ld", clp->cl_lease_time / HZ);
+ expire = clp->cl_last_renewal + clp->cl_lease_time;
+ seq_printf(m, ",lease_expired=%ld",
+ time_after(expire, jiffies) ? 0 : (jiffies - expire) / HZ);
+}
#ifdef CONFIG_NFS_V4_1
static void show_sessions(struct seq_file *m, struct nfs_server *server)
{
if (nfs4_has_session(server->nfs_client))
- seq_printf(m, ",sessions");
+ seq_puts(m, ",sessions");
}
#else
static void show_sessions(struct seq_file *m, struct nfs_server *server) {}
/*
* Display all mount option settings
*/
- seq_printf(m, "\n\topts:\t");
+ seq_puts(m, "\n\topts:\t");
seq_puts(m, sb_rdonly(root->d_sb) ? "ro" : "rw");
seq_puts(m, root->d_sb->s_flags & SB_SYNCHRONOUS ? ",sync" : "");
seq_puts(m, root->d_sb->s_flags & SB_NOATIME ? ",noatime" : "");
show_implementation_id(m, nfss);
- seq_printf(m, "\n\tcaps:\t");
+ seq_puts(m, "\n\tcaps:\t");
seq_printf(m, "caps=0x%x", nfss->caps);
seq_printf(m, ",wtmult=%u", nfss->wtmult);
seq_printf(m, ",dtsize=%u", nfss->dtsize);
#if IS_ENABLED(CONFIG_NFS_V4)
if (nfss->nfs_client->rpc_ops->version == 4) {
- seq_printf(m, "\n\tnfsv4:\t");
+ seq_puts(m, "\n\tnfsv4:\t");
seq_printf(m, "bm0=0x%x", nfss->attr_bitmask[0]);
seq_printf(m, ",bm1=0x%x", nfss->attr_bitmask[1]);
seq_printf(m, ",bm2=0x%x", nfss->attr_bitmask[2]);
seq_printf(m, ",acl=0x%x", nfss->acl_bitmask);
show_sessions(m, nfss);
show_pnfs(m, nfss);
+ show_lease(m, nfss);
}
#endif
preempt_enable();
}
- seq_printf(m, "\n\tevents:\t");
+ seq_puts(m, "\n\tevents:\t");
for (i = 0; i < __NFSIOS_COUNTSMAX; i++)
seq_printf(m, "%lu ", totals.events[i]);
- seq_printf(m, "\n\tbytes:\t");
+ seq_puts(m, "\n\tbytes:\t");
for (i = 0; i < __NFSIOS_BYTESMAX; i++)
seq_printf(m, "%Lu ", totals.bytes[i]);
#ifdef CONFIG_NFS_FSCACHE
if (nfss->options & NFS_OPTION_FSCACHE) {
- seq_printf(m, "\n\tfsc:\t");
+ seq_puts(m, "\n\tfsc:\t");
for (i = 0; i < __NFSIOS_FSCACHEMAX; i++)
seq_printf(m, "%Lu ", totals.fscache[i]);
}
#endif
- seq_printf(m, "\n");
+ seq_putc(m, '\n');
rpc_clnt_show_stats(m, nfss->client);
if (mnt->mount_server.addrlen == 0)
goto out_invalid_address;
break;
+ case Opt_nconnect:
+ if (nfs_get_option_ul_bound(args, &option, 1, NFS_MAX_CONNECTIONS))
+ goto out_invalid_value;
+ mnt->nfs_server.nconnect = option;
+ break;
case Opt_lookupcache:
string = match_strdup(args);
if (string == NULL)
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 Hammerspace Inc
+ */
+
+#include <linux/module.h>
+#include <linux/kobject.h>
+#include <linux/sysfs.h>
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/netdevice.h>
+#include <linux/string.h>
+#include <linux/nfs_fs.h>
+#include <linux/rcupdate.h>
+
+#include "nfs4_fs.h"
+#include "netns.h"
+#include "sysfs.h"
+
+struct kobject *nfs_client_kobj;
+static struct kset *nfs_client_kset;
+
+static void nfs_netns_object_release(struct kobject *kobj)
+{
+ kfree(kobj);
+}
+
+static const struct kobj_ns_type_operations *nfs_netns_object_child_ns_type(
+ struct kobject *kobj)
+{
+ return &net_ns_type_operations;
+}
+
+static struct kobj_type nfs_netns_object_type = {
+ .release = nfs_netns_object_release,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .child_ns_type = nfs_netns_object_child_ns_type,
+};
+
+static struct kobject *nfs_netns_object_alloc(const char *name,
+ struct kset *kset, struct kobject *parent)
+{
+ struct kobject *kobj;
+
+ kobj = kzalloc(sizeof(*kobj), GFP_KERNEL);
+ if (kobj) {
+ kobj->kset = kset;
+ if (kobject_init_and_add(kobj, &nfs_netns_object_type,
+ parent, "%s", name) == 0)
+ return kobj;
+ kobject_put(kobj);
+ }
+ return NULL;
+}
+
+int nfs_sysfs_init(void)
+{
+ nfs_client_kset = kset_create_and_add("nfs", NULL, fs_kobj);
+ if (!nfs_client_kset)
+ return -ENOMEM;
+ nfs_client_kobj = nfs_netns_object_alloc("net", nfs_client_kset, NULL);
+ if (!nfs_client_kobj) {
+ kset_unregister(nfs_client_kset);
+ nfs_client_kset = NULL;
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+void nfs_sysfs_exit(void)
+{
+ kobject_put(nfs_client_kobj);
+ kset_unregister(nfs_client_kset);
+}
+
+static ssize_t nfs_netns_identifier_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *buf)
+{
+ struct nfs_netns_client *c = container_of(kobj,
+ struct nfs_netns_client,
+ kobject);
+ return scnprintf(buf, PAGE_SIZE, "%s\n", c->identifier);
+}
+
+/* Strip trailing '\n' */
+static size_t nfs_string_strip(const char *c, size_t len)
+{
+ while (len > 0 && c[len-1] == '\n')
+ --len;
+ return len;
+}
+
+static ssize_t nfs_netns_identifier_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct nfs_netns_client *c = container_of(kobj,
+ struct nfs_netns_client,
+ kobject);
+ const char *old;
+ char *p;
+ size_t len;
+
+ len = nfs_string_strip(buf, min_t(size_t, count, CONTAINER_ID_MAXLEN));
+ if (!len)
+ return 0;
+ p = kmemdup_nul(buf, len, GFP_KERNEL);
+ if (!p)
+ return -ENOMEM;
+ old = xchg(&c->identifier, p);
+ if (old) {
+ synchronize_rcu();
+ kfree(old);
+ }
+ return count;
+}
+
+static void nfs_netns_client_release(struct kobject *kobj)
+{
+ struct nfs_netns_client *c = container_of(kobj,
+ struct nfs_netns_client,
+ kobject);
+
+ if (c->identifier)
+ kfree(c->identifier);
+ kfree(c);
+}
+
+static const void *nfs_netns_client_namespace(struct kobject *kobj)
+{
+ return container_of(kobj, struct nfs_netns_client, kobject)->net;
+}
+
+static struct kobj_attribute nfs_netns_client_id = __ATTR(identifier,
+ 0644, nfs_netns_identifier_show, nfs_netns_identifier_store);
+
+static struct attribute *nfs_netns_client_attrs[] = {
+ &nfs_netns_client_id.attr,
+ NULL,
+};
+
+static struct kobj_type nfs_netns_client_type = {
+ .release = nfs_netns_client_release,
+ .default_attrs = nfs_netns_client_attrs,
+ .sysfs_ops = &kobj_sysfs_ops,
+ .namespace = nfs_netns_client_namespace,
+};
+
+static struct nfs_netns_client *nfs_netns_client_alloc(struct kobject *parent,
+ struct net *net)
+{
+ struct nfs_netns_client *p;
+
+ p = kzalloc(sizeof(*p), GFP_KERNEL);
+ if (p) {
+ p->net = net;
+ p->kobject.kset = nfs_client_kset;
+ if (kobject_init_and_add(&p->kobject, &nfs_netns_client_type,
+ parent, "nfs_client") == 0)
+ return p;
+ kobject_put(&p->kobject);
+ }
+ return NULL;
+}
+
+void nfs_netns_sysfs_setup(struct nfs_net *netns, struct net *net)
+{
+ struct nfs_netns_client *clp;
+
+ clp = nfs_netns_client_alloc(nfs_client_kobj, net);
+ if (clp) {
+ netns->nfs_client = clp;
+ kobject_uevent(&clp->kobject, KOBJ_ADD);
+ }
+}
+
+void nfs_netns_sysfs_destroy(struct nfs_net *netns)
+{
+ struct nfs_netns_client *clp = netns->nfs_client;
+
+ if (clp) {
+ kobject_uevent(&clp->kobject, KOBJ_REMOVE);
+ kobject_del(&clp->kobject);
+ kobject_put(&clp->kobject);
+ netns->nfs_client = NULL;
+ }
+}
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2019 Hammerspace Inc
+ */
+
+#ifndef __NFS_SYSFS_H
+#define __NFS_SYSFS_H
+
+#define CONTAINER_ID_MAXLEN (64)
+
+struct nfs_netns_client {
+ struct kobject kobject;
+ struct net *net;
+ const char *identifier;
+};
+
+extern struct kobject *nfs_client_kobj;
+
+extern int nfs_sysfs_init(void);
+extern void nfs_sysfs_exit(void);
+
+void nfs_netns_sysfs_setup(struct nfs_net *netns, struct net *net);
+void nfs_netns_sysfs_destroy(struct nfs_net *netns);
+
+#endif
static struct nfs_pgio_header *nfs_writehdr_alloc(void)
{
- struct nfs_pgio_header *p = mempool_alloc(nfs_wdata_mempool, GFP_NOIO);
+ struct nfs_pgio_header *p = mempool_alloc(nfs_wdata_mempool, GFP_KERNEL);
memset(p, 0, sizeof(*p));
p->rw_mode = FMODE_WRITE;
struct inode *inode = mapping->host;
struct nfs_pageio_descriptor pgio;
struct nfs_io_completion *ioc;
- unsigned int pflags = memalloc_nofs_save();
int err;
nfs_inc_stats(inode, NFSIOS_VFSWRITEPAGES);
- ioc = nfs_io_completion_alloc(GFP_NOFS);
+ ioc = nfs_io_completion_alloc(GFP_KERNEL);
if (ioc)
nfs_io_completion_init(ioc, nfs_io_completion_commit, inode);
nfs_pageio_complete(&pgio);
nfs_io_completion_put(ioc);
- memalloc_nofs_restore(pflags);
-
if (err < 0)
goto out_err;
err = pgio.pg_error;
#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/ctype.h>
+#include <linux/fs_context.h>
#include <linux/sunrpc/svcsock.h>
#include <linux/lockd/lockd.h>
inode_unlock(dir);
}
-static int nfsd_fill_super(struct super_block * sb, void * data, int silent)
+static int nfsd_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct nfsd_net *nn = net_generic(current->nsproxy->net_ns,
nfsd_net_id);
#endif
/* last one */ {""}
};
- get_net(sb->s_fs_info);
+
ret = simple_fill_super(sb, 0x6e667364, nfsd_files);
if (ret)
return ret;
return PTR_ERR(dentry);
nn->nfsd_client_dir = dentry;
return 0;
+}
+static int nfsd_fs_get_tree(struct fs_context *fc)
+{
+ fc->s_fs_info = get_net(fc->net_ns);
+ return vfs_get_super(fc, vfs_get_keyed_super, nfsd_fill_super);
}
-static struct dentry *nfsd_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static void nfsd_fs_free_fc(struct fs_context *fc)
{
- struct net *net = current->nsproxy->net_ns;
- return mount_ns(fs_type, flags, data, net, net->user_ns, nfsd_fill_super);
+ if (fc->s_fs_info)
+ put_net(fc->s_fs_info);
+}
+
+static const struct fs_context_operations nfsd_fs_context_ops = {
+ .free = nfsd_fs_free_fc,
+ .get_tree = nfsd_fs_get_tree,
+};
+
+static int nfsd_init_fs_context(struct fs_context *fc)
+{
+ put_user_ns(fc->user_ns);
+ fc->user_ns = get_user_ns(fc->net_ns->user_ns);
+ fc->ops = &nfsd_fs_context_ops;
+ return 0;
}
static void nfsd_umount(struct super_block *sb)
static struct file_system_type nfsd_fs_type = {
.owner = THIS_MODULE,
.name = "nfsd",
- .mount = nfsd_mount,
+ .init_fs_context = nfsd_init_fs_context,
.kill_sb = nfsd_umount,
};
MODULE_ALIAS_FS("nfsd");
#include <linux/sysctl.h>
-static int zero;
-
struct ctl_table inotify_table[] = {
{
.procname = "max_user_instances",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "max_user_watches",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "max_queued_events",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero
+ .extra1 = SYSCTL_ZERO
},
{ }
};
// SPDX-License-Identifier: GPL-2.0
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/file.h>
#include <linux/fs.h>
#include <linux/proc_ns.h>
.evict_inode = nsfs_evict,
.show_path = nsfs_show_path,
};
-static struct dentry *nsfs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+
+static int nsfs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "nsfs:", &nsfs_ops,
- &ns_dentry_operations, NSFS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, NSFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->ops = &nsfs_ops;
+ ctx->dops = &ns_dentry_operations;
+ return 0;
}
+
static struct file_system_type nsfs = {
.name = "nsfs",
- .mount = nsfs_mount,
+ .init_fs_context = nsfs_init_fs_context,
.kill_sb = kill_anon_super,
};
#include <linux/types.h>
#include <linux/string.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/init.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
.remount_fs = openprom_remount,
};
-static int openprom_fill_super(struct super_block *s, void *data, int silent)
+static int openprom_fill_super(struct super_block *s, struct fs_context *fc)
{
struct inode *root_inode;
struct op_inode_info *oi;
return ret;
}
-static struct dentry *openprom_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int openpromfs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, openprom_fill_super);
+ return get_tree_single(fc, openprom_fill_super);
+}
+
+static const struct fs_context_operations openpromfs_context_ops = {
+ .get_tree = openpromfs_get_tree,
+};
+
+static int openpromfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &openpromfs_context_ops;
+ return 0;
}
static struct file_system_type openprom_fs_type = {
.owner = THIS_MODULE,
.name = "openpromfs",
- .mount = openprom_mount,
+ .init_fs_context = openpromfs_init_fs_context,
.kill_sb = kill_anon_super,
};
MODULE_ALIAS_FS("openpromfs");
struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode);
struct orangefs_khandle *handle = &orangefs_inode->refn.khandle;
struct orangefs_kernel_op_s *new_op = NULL;
- int buffer_index = -1;
+ int buffer_index;
ssize_t ret;
size_t copy_amount;
*/
if (ret == -EAGAIN && op_state_purged(new_op)) {
orangefs_bufmap_put(buffer_index);
- buffer_index = -1;
if (type == ORANGEFS_IO_WRITE)
iov_iter_revert(iter, total_size);
gossip_debug(GOSSIP_FILE_DEBUG,
"%s(%pU): PUT buffer_index %d\n",
__func__, handle, buffer_index);
}
- buffer_index = -1;
}
op_release(new_op);
return ret;
#include <linux/fs.h>
#include <linux/log2.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/magic.h>
#include <linux/pipe_fs_i.h>
#include <linux/uio.h>
* any operations on the root directory. However, we need a non-trivial
* d_name - pipe: will go nicely and kill the special-casing in procfs.
*/
-static struct dentry *pipefs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+
+static int pipefs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo(fs_type, "pipe:", &pipefs_ops,
- &pipefs_dentry_operations, PIPEFS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, PIPEFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->ops = &pipefs_ops;
+ ctx->dops = &pipefs_dentry_operations;
+ return 0;
}
static struct file_system_type pipe_fs_type = {
.name = "pipefs",
- .mount = pipefs_mount,
+ .init_fs_context = pipefs_init_fs_context,
.kill_sb = kill_anon_super,
};
snapshot.
If you say Y here, the collected device dumps will be added
- as ELF notes to /proc/vmcore.
+ as ELF notes to /proc/vmcore. You can still disable device
+ dump using the kernel command line option 'novmcoredd'.
config PROC_SYSCTL
bool "Sysctl support (/proc/sys)" if EXPERT
interface is through /proc/sys. If you say Y here a tree of
modifiable sysctl entries will be generated beneath the
/proc/sys directory. They are explained in the files
- in <file:Documentation/sysctl/>. Note that enabling this
+ in <file:Documentation/admin-guide/sysctl/>. Note that enabling this
option will enlarge the kernel by at least 8 KB.
As it is generally a good thing, you should say Y here unless
return result;
}
+/*
+ * If the user used setproctitle(), we just get the string from
+ * user space at arg_start, and limit it to a maximum of one page.
+ */
+static ssize_t get_mm_proctitle(struct mm_struct *mm, char __user *buf,
+ size_t count, unsigned long pos,
+ unsigned long arg_start)
+{
+ char *page;
+ int ret, got;
+
+ if (pos >= PAGE_SIZE)
+ return 0;
+
+ page = (char *)__get_free_page(GFP_KERNEL);
+ if (!page)
+ return -ENOMEM;
+
+ ret = 0;
+ got = access_remote_vm(mm, arg_start, page, PAGE_SIZE, FOLL_ANON);
+ if (got > 0) {
+ int len = strnlen(page, got);
+
+ /* Include the NUL character if it was found */
+ if (len < got)
+ len++;
+
+ if (len > pos) {
+ len -= pos;
+ if (len > count)
+ len = count;
+ len -= copy_to_user(buf, page+pos, len);
+ if (!len)
+ len = -EFAULT;
+ ret = len;
+ }
+ }
+ free_page((unsigned long)page);
+ return ret;
+}
+
static ssize_t get_mm_cmdline(struct mm_struct *mm, char __user *buf,
size_t count, loff_t *ppos)
{
unsigned long arg_start, arg_end, env_start, env_end;
unsigned long pos, len;
- char *page;
+ char *page, c;
/* Check if process spawned far enough to have cmdline. */
if (!mm->env_end)
return 0;
/*
- * We have traditionally allowed the user to re-write
- * the argument strings and overflow the end result
- * into the environment section. But only do that if
- * the environment area is contiguous to the arguments.
+ * We allow setproctitle() to overwrite the argument
+ * strings, and overflow past the original end. But
+ * only when it overflows into the environment area.
*/
- if (env_start != arg_end || env_start >= env_end)
+ if (env_start != arg_end || env_end < env_start)
env_start = env_end = arg_end;
-
- /* .. and limit it to a maximum of one page of slop */
- if (env_end >= arg_end + PAGE_SIZE)
- env_end = arg_end + PAGE_SIZE - 1;
+ len = env_end - arg_start;
/* We're not going to care if "*ppos" has high bits set */
- pos = arg_start + *ppos;
-
- /* .. but we do check the result is in the proper range */
- if (pos < arg_start || pos >= env_end)
+ pos = *ppos;
+ if (pos >= len)
return 0;
+ if (count > len - pos)
+ count = len - pos;
+ if (!count)
+ return 0;
+
+ /*
+ * Magical special case: if the argv[] end byte is not
+ * zero, the user has overwritten it with setproctitle(3).
+ *
+ * Possible future enhancement: do this only once when
+ * pos is 0, and set a flag in the 'struct file'.
+ */
+ if (access_remote_vm(mm, arg_end-1, &c, 1, FOLL_ANON) == 1 && c)
+ return get_mm_proctitle(mm, buf, count, pos, arg_start);
- /* .. and we never go past env_end */
- if (env_end - pos < count)
- count = env_end - pos;
+ /*
+ * For the non-setproctitle() case we limit things strictly
+ * to the [arg_start, arg_end[ range.
+ */
+ pos += arg_start;
+ if (pos < arg_start || pos >= arg_end)
+ return 0;
+ if (count > arg_end - pos)
+ count = arg_end - pos;
page = (char *)__get_free_page(GFP_KERNEL);
if (!page)
while (count) {
int got;
size_t size = min_t(size_t, PAGE_SIZE, count);
- long offset;
- /*
- * Are we already starting past the official end?
- * We always include the last byte that is *supposed*
- * to be NUL
- */
- offset = (pos >= arg_end) ? pos - arg_end + 1 : 0;
-
- got = access_remote_vm(mm, pos - offset, page, size + offset, FOLL_ANON);
- if (got <= offset)
+ got = access_remote_vm(mm, pos, page, size, FOLL_ANON);
+ if (got <= 0)
break;
- got -= offset;
-
- /* Don't walk past a NUL character once you hit arg_end */
- if (pos + got >= arg_end) {
- int n = 0;
-
- /*
- * If we started before 'arg_end' but ended up
- * at or after it, we start the NUL character
- * check at arg_end-1 (where we expect the normal
- * EOF to be).
- *
- * NOTE! This is smaller than 'got', because
- * pos + got >= arg_end
- */
- if (pos < arg_end)
- n = arg_end - pos - 1;
-
- /* Cut off at first NUL after 'n' */
- got = n + strnlen(page+n, offset+got-n);
- if (got < offset)
- break;
- got -= offset;
-
- /* Include the NUL if it existed */
- if (got < size)
- got++;
- }
-
- got -= copy_to_user(buf, page+offset, got);
+ got -= copy_to_user(buf, page, got);
if (unlikely(!got)) {
if (!len)
len = -EFAULT;
struct proc_dir_entry *pde = PDE(file_inode(file));
loff_t rv = -EINVAL;
if (use_pde(pde)) {
- loff_t (*llseek)(struct file *, loff_t, int);
+ typeof_member(struct file_operations, llseek) llseek;
+
llseek = pde->proc_fops->llseek;
if (!llseek)
llseek = default_llseek;
static ssize_t proc_reg_read(struct file *file, char __user *buf, size_t count, loff_t *ppos)
{
- ssize_t (*read)(struct file *, char __user *, size_t, loff_t *);
struct proc_dir_entry *pde = PDE(file_inode(file));
ssize_t rv = -EIO;
if (use_pde(pde)) {
+ typeof_member(struct file_operations, read) read;
+
read = pde->proc_fops->read;
if (read)
rv = read(file, buf, count, ppos);
static ssize_t proc_reg_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos)
{
- ssize_t (*write)(struct file *, const char __user *, size_t, loff_t *);
struct proc_dir_entry *pde = PDE(file_inode(file));
ssize_t rv = -EIO;
if (use_pde(pde)) {
+ typeof_member(struct file_operations, write) write;
+
write = pde->proc_fops->write;
if (write)
rv = write(file, buf, count, ppos);
{
struct proc_dir_entry *pde = PDE(file_inode(file));
__poll_t rv = DEFAULT_POLLMASK;
- __poll_t (*poll)(struct file *, struct poll_table_struct *);
if (use_pde(pde)) {
+ typeof_member(struct file_operations, poll) poll;
+
poll = pde->proc_fops->poll;
if (poll)
rv = poll(file, pts);
{
struct proc_dir_entry *pde = PDE(file_inode(file));
long rv = -ENOTTY;
- long (*ioctl)(struct file *, unsigned int, unsigned long);
if (use_pde(pde)) {
+ typeof_member(struct file_operations, unlocked_ioctl) ioctl;
+
ioctl = pde->proc_fops->unlocked_ioctl;
if (ioctl)
rv = ioctl(file, cmd, arg);
{
struct proc_dir_entry *pde = PDE(file_inode(file));
long rv = -ENOTTY;
- long (*compat_ioctl)(struct file *, unsigned int, unsigned long);
if (use_pde(pde)) {
+ typeof_member(struct file_operations, compat_ioctl) compat_ioctl;
+
compat_ioctl = pde->proc_fops->compat_ioctl;
if (compat_ioctl)
rv = compat_ioctl(file, cmd, arg);
{
struct proc_dir_entry *pde = PDE(file_inode(file));
int rv = -EIO;
- int (*mmap)(struct file *, struct vm_area_struct *);
if (use_pde(pde)) {
+ typeof_member(struct file_operations, mmap) mmap;
+
mmap = pde->proc_fops->mmap;
if (mmap)
rv = mmap(file, vma);
unsigned long rv = -EIO;
if (use_pde(pde)) {
- typeof(proc_reg_get_unmapped_area) *get_area;
+ typeof_member(struct file_operations, get_unmapped_area) get_area;
get_area = pde->proc_fops->get_unmapped_area;
#ifdef CONFIG_MMU
{
struct proc_dir_entry *pde = PDE(inode);
int rv = 0;
- int (*open)(struct inode *, struct file *);
- int (*release)(struct inode *, struct file *);
+ typeof_member(struct file_operations, open) open;
+ typeof_member(struct file_operations, release) release;
struct pde_opener *pdeo;
/*
static const struct file_operations proc_sys_dir_file_operations;
static const struct inode_operations proc_sys_dir_operations;
+/* shared constants to be used in various sysctls */
+const int sysctl_vals[] = { 0, 1, INT_MAX };
+EXPORT_SYMBOL(sysctl_vals);
+
/* Support for permanently empty directories */
struct ctl_table sysctl_mount_point[] = {
if (root->set_ownership)
root->set_ownership(head, table, &inode->i_uid, &inode->i_gid);
+ else {
+ inode->i_uid = GLOBAL_ROOT_UID;
+ inode->i_gid = GLOBAL_ROOT_GID;
+ }
return inode;
}
{
struct proc_fs_context *ctx = fc->fs_private;
- put_user_ns(fc->user_ns);
- fc->user_ns = get_user_ns(ctx->pid_ns->user_ns);
fc->s_fs_info = ctx->pid_ns;
return vfs_get_super(fc, vfs_get_keyed_super, proc_fill_super);
}
{
struct proc_fs_context *ctx = fc->fs_private;
- if (ctx->pid_ns)
- put_pid_ns(ctx->pid_ns);
+ put_pid_ns(ctx->pid_ns);
kfree(ctx);
}
return -ENOMEM;
ctx->pid_ns = get_pid_ns(task_active_pid_ns(current));
+ put_user_ns(fc->user_ns);
+ fc->user_ns = get_user_ns(ctx->pid_ns->user_ns);
fc->fs_private = ctx;
fc->ops = &proc_fs_context_ops;
return 0;
__show_smap(m, &mss, false);
- seq_printf(m, "THPeligible: %d\n", transparent_hugepage_enabled(vma));
+ seq_printf(m, "THPeligible: %d\n",
+ transparent_hugepage_enabled(vma));
if (arch_pkeys_enabled())
seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma));
#include <linux/init.h>
#include <linux/crash_dump.h>
#include <linux/list.h>
+#include <linux/moduleparam.h>
#include <linux/mutex.h>
#include <linux/vmalloc.h>
#include <linux/pagemap.h>
/* Device Dump list and mutex to synchronize access to list */
static LIST_HEAD(vmcoredd_list);
static DEFINE_MUTEX(vmcoredd_mutex);
+
+static bool vmcoredd_disabled;
+core_param(novmcoredd, vmcoredd_disabled, bool, 0);
#endif /* CONFIG_PROC_VMCORE_DEVICE_DUMP */
/* Device Dump Size */
size_t data_size;
int ret;
+ if (vmcoredd_disabled) {
+ pr_err_once("Device dump is disabled\n");
+ return -EINVAL;
+ }
+
if (!data || !strlen(data->dump_name) ||
!data->vmcoredd_callback || !data->size)
return -EINVAL;
.fs_flags = FS_USERNS_MOUNT,
};
-int __init init_ramfs_fs(void)
+static int __init init_ramfs_fs(void)
{
- static unsigned long once;
-
- if (test_and_set_bit(0, &once))
- return 0;
return register_filesystem(&ramfs_fs_type);
}
fs_initcall(init_ramfs_fs);
struct super_block *sb);
static void release_journal_dev(struct super_block *super,
struct reiserfs_journal *journal);
-static int dirty_one_transaction(struct super_block *s,
+static void dirty_one_transaction(struct super_block *s,
struct reiserfs_journal_list *jl);
static void flush_async_commits(struct work_struct *work);
static void queue_log_writer(struct super_block *s);
}
/* used by flush_commit_list */
-static int dirty_one_transaction(struct super_block *s,
+static void dirty_one_transaction(struct super_block *s,
struct reiserfs_journal_list *jl)
{
struct reiserfs_journal_cnode *cn;
struct reiserfs_journal_list *pjl;
- int ret = 0;
jl->j_state |= LIST_DIRTY;
cn = jl->j_realblock;
}
cn = cn->next;
}
- return ret;
}
static int kupdate_transactions(struct super_block *s,
PT_OLD_TIMESPEC = 3,
};
-static int poll_select_copy_remaining(struct timespec64 *end_time,
- void __user *p,
- enum poll_time_type pt_type, int ret)
+static int poll_select_finish(struct timespec64 *end_time,
+ void __user *p,
+ enum poll_time_type pt_type, int ret)
{
struct timespec64 rts;
+ restore_saved_sigmask_unless(ret == -ERESTARTNOHAND);
+
if (!p)
return ret;
}
ret = core_sys_select(n, inp, outp, exp, to);
- ret = poll_select_copy_remaining(&end_time, tvp, PT_TIMEVAL, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tvp, PT_TIMEVAL, ret);
}
SYSCALL_DEFINE5(select, int, n, fd_set __user *, inp, fd_set __user *, outp,
const sigset_t __user *sigmask, size_t sigsetsize,
enum poll_time_type type)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = core_sys_select(n, inp, outp, exp, to);
- restore_user_sigmask(sigmask, &sigsaved, ret == -ERESTARTNOHAND);
- ret = poll_select_copy_remaining(&end_time, tsp, type, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, type, ret);
}
/*
if (!count) {
count = wait->error;
if (signal_pending(current))
- count = -EINTR;
+ count = -ERESTARTNOHAND;
}
if (count || timed_out)
break;
struct timespec64 *end_time)
{
struct poll_wqueues table;
- int err = -EFAULT, fdcount, len, size;
+ int err = -EFAULT, fdcount, len;
/* Allocate small arguments on the stack to save memory and be
faster - use long to make sure the buffer is aligned properly
on 64 bit archs to avoid unaligned access */
break;
len = min(todo, POLLFD_PER_PAGE);
- size = sizeof(struct poll_list) + sizeof(struct pollfd) * len;
- walk = walk->next = kmalloc(size, GFP_KERNEL);
+ walk = walk->next = kmalloc(struct_size(walk, entries, len),
+ GFP_KERNEL);
if (!walk) {
err = -ENOMEM;
goto out_fds;
ret = do_sys_poll(ufds, nfds, to);
- if (ret == -EINTR) {
+ if (ret == -ERESTARTNOHAND) {
restart_block->fn = do_restart_poll;
ret = -ERESTART_RESTARTBLOCK;
}
ret = do_sys_poll(ufds, nfds, to);
- if (ret == -EINTR) {
+ if (ret == -ERESTARTNOHAND) {
struct restart_block *restart_block;
restart_block = ¤t->restart_block;
struct __kernel_timespec __user *, tsp, const sigset_t __user *, sigmask,
size_t, sigsetsize)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = do_sys_poll(ufds, nfds, to);
-
- restore_user_sigmask(sigmask, &sigsaved, ret == -EINTR);
- /* We can restart this syscall, usually */
- if (ret == -EINTR)
- ret = -ERESTARTNOHAND;
-
- ret = poll_select_copy_remaining(&end_time, tsp, PT_TIMESPEC, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, PT_TIMESPEC, ret);
}
#if defined(CONFIG_COMPAT_32BIT_TIME) && !defined(CONFIG_64BIT)
struct old_timespec32 __user *, tsp, const sigset_t __user *, sigmask,
size_t, sigsetsize)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = do_sys_poll(ufds, nfds, to);
-
- restore_user_sigmask(sigmask, &sigsaved, ret == -EINTR);
- /* We can restart this syscall, usually */
- if (ret == -EINTR)
- ret = -ERESTARTNOHAND;
-
- ret = poll_select_copy_remaining(&end_time, tsp, PT_OLD_TIMESPEC, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, PT_OLD_TIMESPEC, ret);
}
#endif
}
ret = compat_core_sys_select(n, inp, outp, exp, to);
- ret = poll_select_copy_remaining(&end_time, tvp, PT_OLD_TIMEVAL, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tvp, PT_OLD_TIMEVAL, ret);
}
COMPAT_SYSCALL_DEFINE5(select, int, n, compat_ulong_t __user *, inp,
void __user *tsp, compat_sigset_t __user *sigmask,
compat_size_t sigsetsize, enum poll_time_type type)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_compat_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_compat_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = compat_core_sys_select(n, inp, outp, exp, to);
- restore_user_sigmask(sigmask, &sigsaved, ret == -ERESTARTNOHAND);
- ret = poll_select_copy_remaining(&end_time, tsp, type, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, type, ret);
}
COMPAT_SYSCALL_DEFINE6(pselect6_time64, int, n, compat_ulong_t __user *, inp,
unsigned int, nfds, struct old_timespec32 __user *, tsp,
const compat_sigset_t __user *, sigmask, compat_size_t, sigsetsize)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_compat_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_compat_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = do_sys_poll(ufds, nfds, to);
-
- restore_user_sigmask(sigmask, &sigsaved, ret == -EINTR);
- /* We can restart this syscall, usually */
- if (ret == -EINTR)
- ret = -ERESTARTNOHAND;
-
- ret = poll_select_copy_remaining(&end_time, tsp, PT_OLD_TIMESPEC, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, PT_OLD_TIMESPEC, ret);
}
#endif
unsigned int, nfds, struct __kernel_timespec __user *, tsp,
const compat_sigset_t __user *, sigmask, compat_size_t, sigsetsize)
{
- sigset_t ksigmask, sigsaved;
struct timespec64 ts, end_time, *to = NULL;
int ret;
return -EINVAL;
}
- ret = set_compat_user_sigmask(sigmask, &ksigmask, &sigsaved, sigsetsize);
+ ret = set_compat_user_sigmask(sigmask, sigsetsize);
if (ret)
return ret;
ret = do_sys_poll(ufds, nfds, to);
-
- restore_user_sigmask(sigmask, &sigsaved, ret == -EINTR);
- /* We can restart this syscall, usually */
- if (ret == -EINTR)
- ret = -ERESTARTNOHAND;
-
- ret = poll_select_copy_remaining(&end_time, tsp, PT_TIMESPEC, ret);
-
- return ret;
+ return poll_select_finish(&end_time, tsp, PT_TIMESPEC, ret);
}
#endif
EXPORT_SYMBOL(generic_shutdown_super);
+bool mount_capable(struct fs_context *fc)
+{
+ struct user_namespace *user_ns = fc->global ? &init_user_ns
+ : fc->user_ns;
+
+ if (!(fc->fs_type->fs_flags & FS_USERNS_MOUNT))
+ return capable(CAP_SYS_ADMIN);
+ else
+ return ns_capable(user_ns, CAP_SYS_ADMIN);
+}
+
/**
* sget_fc - Find or create a superblock
* @fc: Filesystem context.
struct user_namespace *user_ns = fc->global ? &init_user_ns : fc->user_ns;
int err;
- if (!(fc->sb_flags & SB_KERNMOUNT) &&
- fc->purpose != FS_CONTEXT_FOR_SUBMOUNT) {
- /* Don't allow mounting unless the caller has CAP_SYS_ADMIN
- * over the namespace.
- */
- if (!(fc->fs_type->fs_flags & FS_USERNS_MOUNT)) {
- if (!capable(CAP_SYS_ADMIN))
- return ERR_PTR(-EPERM);
- } else {
- if (!ns_capable(fc->user_ns, CAP_SYS_ADMIN))
- return ERR_PTR(-EPERM);
- }
- }
-
retry:
spin_lock(&sb_lock);
if (test) {
}
fc->s_fs_info = NULL;
s->s_type = fc->fs_type;
+ s->s_iflags |= fc->s_iflags;
strlcpy(s->s_id, s->s_type->name, sizeof(s->s_id));
list_add_tail(&s->s_list, &super_blocks);
hlist_add_head(&s->s_instances, &s->s_type->fs_supers);
EXPORT_SYMBOL(sget_fc);
/**
- * sget_userns - find or create a superblock
- * @type: filesystem type superblock should belong to
- * @test: comparison callback
- * @set: setup callback
- * @flags: mount flags
- * @user_ns: User namespace for the super_block
- * @data: argument to each of them
+ * sget - find or create a superblock
+ * @type: filesystem type superblock should belong to
+ * @test: comparison callback
+ * @set: setup callback
+ * @flags: mount flags
+ * @data: argument to each of them
*/
-struct super_block *sget_userns(struct file_system_type *type,
+struct super_block *sget(struct file_system_type *type,
int (*test)(struct super_block *,void *),
int (*set)(struct super_block *,void *),
- int flags, struct user_namespace *user_ns,
+ int flags,
void *data)
{
+ struct user_namespace *user_ns = current_user_ns();
struct super_block *s = NULL;
struct super_block *old;
int err;
- if (!(flags & (SB_KERNMOUNT|SB_SUBMOUNT)) &&
- !(type->fs_flags & FS_USERNS_MOUNT) &&
- !capable(CAP_SYS_ADMIN))
- return ERR_PTR(-EPERM);
+ /* We don't yet pass the user namespace of the parent
+ * mount through to here so always use &init_user_ns
+ * until that changes.
+ */
+ if (flags & SB_SUBMOUNT)
+ user_ns = &init_user_ns;
+
retry:
spin_lock(&sb_lock);
if (test) {
register_shrinker_prepared(&s->s_shrink);
return s;
}
-
-EXPORT_SYMBOL(sget_userns);
-
-/**
- * sget - find or create a superblock
- * @type: filesystem type superblock should belong to
- * @test: comparison callback
- * @set: setup callback
- * @flags: mount flags
- * @data: argument to each of them
- */
-struct super_block *sget(struct file_system_type *type,
- int (*test)(struct super_block *,void *),
- int (*set)(struct super_block *,void *),
- int flags,
- void *data)
-{
- struct user_namespace *user_ns = current_user_ns();
-
- /* We don't yet pass the user namespace of the parent
- * mount through to here so always use &init_user_ns
- * until that changes.
- */
- if (flags & SB_SUBMOUNT)
- user_ns = &init_user_ns;
-
- /* Ensure the requestor has permissions over the target filesystem */
- if (!(flags & (SB_KERNMOUNT|SB_SUBMOUNT)) && !ns_capable(user_ns, CAP_SYS_ADMIN))
- return ERR_PTR(-EPERM);
-
- return sget_userns(type, test, set, flags, user_ns, data);
-}
-
EXPORT_SYMBOL(sget);
void drop_super(struct super_block *sb)
}
EXPORT_SYMBOL(kill_litter_super);
-static int ns_test_super(struct super_block *sb, void *data)
-{
- return sb->s_fs_info == data;
-}
-
-static int ns_set_super(struct super_block *sb, void *data)
-{
- sb->s_fs_info = data;
- return set_anon_super(sb, NULL);
-}
-
-struct dentry *mount_ns(struct file_system_type *fs_type,
- int flags, void *data, void *ns, struct user_namespace *user_ns,
- int (*fill_super)(struct super_block *, void *, int))
-{
- struct super_block *sb;
-
- /* Don't allow mounting unless the caller has CAP_SYS_ADMIN
- * over the namespace.
- */
- if (!(flags & SB_KERNMOUNT) && !ns_capable(user_ns, CAP_SYS_ADMIN))
- return ERR_PTR(-EPERM);
-
- sb = sget_userns(fs_type, ns_test_super, ns_set_super, flags,
- user_ns, ns);
- if (IS_ERR(sb))
- return ERR_CAST(sb);
-
- if (!sb->s_root) {
- int err;
- err = fill_super(sb, data, flags & SB_SILENT ? 1 : 0);
- if (err) {
- deactivate_locked_super(sb);
- return ERR_PTR(err);
- }
-
- sb->s_flags |= SB_ACTIVE;
- }
-
- return dget(sb->s_root);
-}
-
-EXPORT_SYMBOL(mount_ns);
-
int set_anon_super_fc(struct super_block *sb, struct fs_context *fc)
{
return set_anon_super(sb, NULL);
}
EXPORT_SYMBOL(vfs_get_super);
+int get_tree_nodev(struct fs_context *fc,
+ int (*fill_super)(struct super_block *sb,
+ struct fs_context *fc))
+{
+ return vfs_get_super(fc, vfs_get_independent_super, fill_super);
+}
+EXPORT_SYMBOL(get_tree_nodev);
+
+int get_tree_single(struct fs_context *fc,
+ int (*fill_super)(struct super_block *sb,
+ struct fs_context *fc))
+{
+ return vfs_get_super(fc, vfs_get_single_super, fill_super);
+}
+EXPORT_SYMBOL(get_tree_single);
+
#ifdef CONFIG_BLOCK
static int set_bdev_super(struct super_block *s, void *data)
{
fc->fs_private = kfc;
fc->ops = &sysfs_fs_context_ops;
if (netns) {
- if (fc->user_ns)
- put_user_ns(fc->user_ns);
+ put_user_ns(fc->user_ns);
fc->user_ns = get_user_ns(netns->user_ns);
}
fc->global = true;
{
int rc;
- rc = migrate_page_move_mapping(mapping, newpage, page, mode, 0);
+ rc = migrate_page_move_mapping(mapping, newpage, page, 0);
if (rc != MIGRATEPAGE_SUCCESS)
return rc;
struct super_block *sb = dentry->d_sb;
struct ufs_sb_private_info *uspi= UFS_SB(sb)->s_uspi;
unsigned flags = UFS_SB(sb)->s_flags;
- struct ufs_super_block_third *usb3;
u64 id = huge_encode_dev(sb->s_bdev->bd_dev);
mutex_lock(&UFS_SB(sb)->s_lock);
- usb3 = ubh_get_usb_third(uspi);
if ((flags & UFS_TYPE_MASK) == UFS_TYPE_UFS2)
buf->f_type = UFS2_MAGIC;
xfs_refcount_btree.o \
xfs_sb.o \
xfs_symlink_remote.o \
+ xfs_trans_inode.o \
xfs_trans_resv.o \
xfs_types.o \
)
xfs_rmap_item.o \
xfs_log_recover.o \
xfs_trans_ail.o \
- xfs_trans_buf.o \
- xfs_trans_inode.o
+ xfs_trans_buf.o
# optional features
xfs-$(CONFIG_XFS_QUOTA) += xfs_dquot.o \
inode->i_mtime = tv;
if (flags & XFS_ICHGTIME_CHG)
inode->i_ctime = tv;
+ if (flags & XFS_ICHGTIME_CREATE) {
+ ip->i_d.di_crtime.t_sec = (int32_t)tv.tv_sec;
+ ip->i_d.di_crtime.t_nsec = (int32_t)tv.tv_nsec;
+ }
}
/*
struct file *filp,
struct vm_area_struct *vma)
{
+ struct dax_device *dax_dev;
+
+ dax_dev = xfs_find_daxdev_for_inode(file_inode(filp));
/*
- * We don't support synchronous mappings for non-DAX files. At least
- * until someone comes with a sensible use case.
+ * We don't support synchronous mappings for non-DAX files and
+ * for DAX files if underneath dax_device is not synchronous.
*/
- if (!IS_DAX(file_inode(filp)) && (vma->vm_flags & VM_SYNC))
+ if (!daxdev_mapping_supported(vma, dax_dev))
return -EOPNOTSUPP;
file_accessed(filp);
warn_slowpath_fmt_taint(__FILE__, __LINE__, taint, arg)
#else
extern __printf(1, 2) void __warn_printk(const char *fmt, ...);
-#define __WARN() __WARN_TAINT(TAINT_WARN)
-#define __WARN_printf(arg...) do { __warn_printk(arg); __WARN(); } while (0)
+#define __WARN() do { \
+ printk(KERN_WARNING CUT_HERE); __WARN_TAINT(TAINT_WARN); \
+} while (0)
+#define __WARN_printf(arg...) __WARN_printf_taint(TAINT_WARN, arg)
#define __WARN_printf_taint(taint, arg...) \
do { __warn_printk(arg); __WARN_TAINT(taint); } while (0)
#endif
/* Keep includes the same across arches. */
#include <linux/mm.h>
+#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
+
/*
* The cache doesn't need to be flushed when TLB entries change when
* the cache is mapped to physical memory, not virtual memory
*/
-#define flush_cache_all() do { } while (0)
-#define flush_cache_mm(mm) do { } while (0)
-#define flush_cache_dup_mm(mm) do { } while (0)
-#define flush_cache_range(vma, start, end) do { } while (0)
-#define flush_cache_page(vma, vmaddr, pfn) do { } while (0)
-#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 0
-#define flush_dcache_page(page) do { } while (0)
-#define flush_dcache_mmap_lock(mapping) do { } while (0)
-#define flush_dcache_mmap_unlock(mapping) do { } while (0)
-#define flush_icache_range(start, end) do { } while (0)
-#define flush_icache_page(vma,pg) do { } while (0)
-#define flush_icache_user_range(vma,pg,adr,len) do { } while (0)
-#define flush_cache_vmap(start, end) do { } while (0)
-#define flush_cache_vunmap(start, end) do { } while (0)
+static inline void flush_cache_all(void)
+{
+}
+
+static inline void flush_cache_mm(struct mm_struct *mm)
+{
+}
+
+static inline void flush_cache_dup_mm(struct mm_struct *mm)
+{
+}
+
+static inline void flush_cache_range(struct vm_area_struct *vma,
+ unsigned long start,
+ unsigned long end)
+{
+}
+
+static inline void flush_cache_page(struct vm_area_struct *vma,
+ unsigned long vmaddr,
+ unsigned long pfn)
+{
+}
+
+static inline void flush_dcache_page(struct page *page)
+{
+}
+
+static inline void flush_dcache_mmap_lock(struct address_space *mapping)
+{
+}
+
+static inline void flush_dcache_mmap_unlock(struct address_space *mapping)
+{
+}
+
+static inline void flush_icache_range(unsigned long start, unsigned long end)
+{
+}
+
+static inline void flush_icache_page(struct vm_area_struct *vma,
+ struct page *page)
+{
+}
+
+static inline void flush_icache_user_range(struct vm_area_struct *vma,
+ struct page *page,
+ unsigned long addr, int len)
+{
+}
+
+static inline void flush_cache_vmap(unsigned long start, unsigned long end)
+{
+}
+
+static inline void flush_cache_vunmap(unsigned long start, unsigned long end)
+{
+}
#define copy_to_user_page(vma, page, vaddr, dst, src, len) \
do { \
#define ACPI_PROBE_TABLE(name)
#endif
+#ifdef CONFIG_THERMAL
+#define THERMAL_TABLE(name) \
+ . = ALIGN(8); \
+ __##name##_thermal_table = .; \
+ KEEP(*(__##name##_thermal_table)) \
+ __##name##_thermal_table_end = .;
+#else
+#define THERMAL_TABLE(name)
+#endif
+
#define KERNEL_DTB() \
STRUCT_ALIGN(); \
__dtb_start = .; \
IRQCHIP_OF_MATCH_TABLE() \
ACPI_PROBE_TABLE(irqchip) \
ACPI_PROBE_TABLE(timer) \
+ THERMAL_TABLE(governor) \
EARLYCON_TABLE() \
LSM_TABLE()
#define CLK_MIPI_HSI 349 /* Exynos4210 only */
#define CLK_PIXELASYNCM0 351
#define CLK_PIXELASYNCM1 352
+#define CLK_ASYNC_G3D 353 /* Exynos4x12 only */
#define CLK_PWM_ISP_SCLK 379 /* Exynos4x12 only */
#define CLK_SPI0_ISP_SCLK 380 /* Exynos4x12 only */
#define CLK_SPI1_ISP_SCLK 381 /* Exynos4x12 only */
#define CLK_MAU_EPLL 159
#define CLK_SCLK_HSIC_12M 160
#define CLK_SCLK_MPHY_IXTAL24 161
+#define CLK_SCLK_BPLL 162
/* gate clocks */
#define CLK_UART0 257
#define CLK_ACLK432_CAM 518
#define CLK_ACLK_FL1550_CAM 519
#define CLK_ACLK550_CAM 520
+#define CLK_CLKM_PHY0 521
+#define CLK_CLKM_PHY1 522
+#define CLK_ACLK_PPMU_DREX0_0 523
+#define CLK_ACLK_PPMU_DREX0_1 524
+#define CLK_ACLK_PPMU_DREX1_0 525
+#define CLK_ACLK_PPMU_DREX1_1 526
+#define CLK_PCLK_PPMU_DREX0_0 527
+#define CLK_PCLK_PPMU_DREX0_1 528
+#define CLK_PCLK_PPMU_DREX1_0 529
+#define CLK_PCLK_PPMU_DREX1_1 530
/* mux clocks */
#define CLK_MOUT_HDMI 640
#define CLK_MOUT_EPLL 657
#define CLK_MOUT_MAU_EPLL 658
#define CLK_MOUT_USER_MAU_EPLL 659
+#define CLK_MOUT_SCLK_SPLL 660
+#define CLK_MOUT_MX_MSPLL_CCORE_PHY 661
/* divider clocks */
#define CLK_DOUT_PIXEL 768
#define CLK_DOUT_CCLK_DREX0 794
#define CLK_DOUT_CLK2X_PHY0 795
#define CLK_DOUT_PCLK_CORE_MEM 796
+#define CLK_FF_DOUT_SPLL2 797
+#define CLK_DOUT_PCLK_DREX0 798
+#define CLK_DOUT_PCLK_DREX1 799
/* must be greater than maximal clock id */
-#define CLK_NR_CLKS 797
+#define CLK_NR_CLKS 800
#endif /* _DT_BINDINGS_CLOCK_EXYNOS_5420_H */
#define CLKID_VDEC_1 204
#define CLKID_VDEC_HEVC 207
#define CLKID_VDEC_HEVCF 210
+#define CLKID_TS 212
#endif /* __G12A_CLKC_H */
#define IMX8MM_CLK_NAND_USDHC_BUS_RAWNAND_CLK 222
-#define IMX8MM_CLK_END 223
+#define IMX8MM_CLK_GPIO1_ROOT 223
+#define IMX8MM_CLK_GPIO2_ROOT 224
+#define IMX8MM_CLK_GPIO3_ROOT 225
+#define IMX8MM_CLK_GPIO4_ROOT 226
+#define IMX8MM_CLK_GPIO5_ROOT 227
+
+#define IMX8MM_CLK_SNVS_ROOT 228
+#define IMX8MM_CLK_GIC 229
+
+#define IMX8MM_CLK_END 230
#endif
#define IMX8MQ_CLK_GPIO4_ROOT 262
#define IMX8MQ_CLK_GPIO5_ROOT 263
-#define IMX8MQ_CLK_END 264
+#define IMX8MQ_CLK_SNVS_ROOT 264
+#define IMX8MQ_CLK_GIC 265
+
+#define IMX8MQ_CLK_END 266
#endif /* __DT_BINDINGS_CLOCK_IMX8MQ_H */
#define CLKID_VDEC_HCODEC 199
#define CLKID_VDEC_2 202
#define CLKID_VDEC_HEVC 206
+#define CLKID_CTS_AMCLK 209
+#define CLKID_CTS_MCLK_I958 212
+#define CLKID_CTS_I958 213
#endif /* __MESON8B_CLKC_H */
#define CLK_TOP_MSDC2_INFRA 176
#define CLK_TOP_NR_CLK 177
+/* AUDSYS */
+
+#define CLK_AUD_AFE 0
+#define CLK_AUD_I2S 1
+#define CLK_AUD_22M 2
+#define CLK_AUD_24M 3
+#define CLK_AUD_INTDIR 4
+#define CLK_AUD_APLL2_TUNER 5
+#define CLK_AUD_APLL_TUNER 6
+#define CLK_AUD_HDMI 7
+#define CLK_AUD_SPDF 8
+#define CLK_AUD_ADC 9
+#define CLK_AUD_DAC 10
+#define CLK_AUD_DAC_PREDIS 11
+#define CLK_AUD_TML 12
+#define CLK_AUD_NR_CLK 13
+
#endif /* _DT_BINDINGS_CLK_MT8516_H */
#define GCC_PCIEPHY_0_PHY_BCR 12
#define GCC_EMAC_BCR 13
#define GCC_CDSP_RESTART 14
+#define GCC_PCIE_0_AXI_MASTER_STICKY_ARES 15
+#define GCC_PCIE_0_AHB_ARES 16
+#define GCC_PCIE_0_AXI_SLAVE_ARES 17
+#define GCC_PCIE_0_AXI_MASTER_ARES 18
+#define GCC_PCIE_0_CORE_STICKY_ARES 19
+#define GCC_PCIE_0_SLEEP_ARES 20
+#define GCC_PCIE_0_PIPE_ARES 21
#endif
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2019, Jeffrey Hugo
+ */
+
+#ifndef _DT_BINDINGS_CLK_MSM_GPUCC_8998_H
+#define _DT_BINDINGS_CLK_MSM_GPUCC_8998_H
+
+#define GPUPLL0 0
+#define GPUPLL0_OUT_EVEN 1
+#define RBCPR_CLK_SRC 2
+#define GFX3D_CLK_SRC 3
+#define RBBMTIMER_CLK_SRC 4
+#define GFX3D_ISENSE_CLK_SRC 5
+#define RBCPR_CLK 6
+#define GFX3D_CLK 7
+#define RBBMTIMER_CLK 8
+#define GFX3D_ISENSE_CLK 9
+#define GPUCC_CXO_CLK 10
+
+#define GPU_CX_BCR 0
+#define RBCPR_BCR 1
+#define GPU_GX_BCR 2
+#define GPU_ISENSE_BCR 3
+
+#define GPU_CX_GDSC 1
+#define GPU_GX_GDSC 2
+
+#endif
#define SCLK_WIFI 141
#define SCLK_OTGPHY0 142
#define SCLK_OTGPHY1 143
+#define SCLK_HDMI_PHY 144
/* dclk gates */
#define DCLK_VOP 190
#define PCLK_DCF 233
#define PCLK_SARADC 234
#define PCLK_ACODECPHY 235
+#define PCLK_WDT 236
/* hclk gates */
#define HCLK_PERI 308
#define STRATIX10_USB_CLK 59
#define STRATIX10_SPI_M_CLK 60
#define STRATIX10_NAND_CLK 61
-#define STRATIX10_NUM_CLKS 62
+#define STRATIX10_NAND_X_CLK 62
+#define STRATIX10_NAND_ECC_CLK 63
+#define STRATIX10_NUM_CLKS 64
#endif /* __STRATIX10_CLOCK_H */
#ifdef CONFIG_X86_IO_APIC
extern int acpi_get_override_irq(u32 gsi, int *trigger, int *polarity);
#else
-#define acpi_get_override_irq(gsi, trigger, polarity) (-1)
+static inline int acpi_get_override_irq(u32 gsi, int *trigger, int *polarity)
+{
+ return -1;
+}
#endif
/*
* This function undoes the effect of one call to acpi_register_gsi().
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef __LINUX_BITS_H
#define __LINUX_BITS_H
+
+#include <linux/const.h>
#include <asm/bitsperlong.h>
-#define BIT(nr) (1UL << (nr))
-#define BIT_ULL(nr) (1ULL << (nr))
-#define BIT_MASK(nr) (1UL << ((nr) % BITS_PER_LONG))
+#define BIT(nr) (UL(1) << (nr))
+#define BIT_ULL(nr) (ULL(1) << (nr))
+#define BIT_MASK(nr) (UL(1) << ((nr) % BITS_PER_LONG))
#define BIT_WORD(nr) ((nr) / BITS_PER_LONG)
-#define BIT_ULL_MASK(nr) (1ULL << ((nr) % BITS_PER_LONG_LONG))
+#define BIT_ULL_MASK(nr) (ULL(1) << ((nr) % BITS_PER_LONG_LONG))
#define BIT_ULL_WORD(nr) ((nr) / BITS_PER_LONG_LONG)
#define BITS_PER_BYTE 8
* GENMASK_ULL(39, 21) gives us the 64bit vector 0x000000ffffe00000.
*/
#define GENMASK(h, l) \
- (((~0UL) - (1UL << (l)) + 1) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+ (((~UL(0)) - (UL(1) << (l)) + 1) & \
+ (~UL(0) >> (BITS_PER_LONG - 1 - (h))))
#define GENMASK_ULL(h, l) \
- (((~0ULL) - (1ULL << (l)) + 1) & \
- (~0ULL >> (BITS_PER_LONG_LONG - 1 - (h))))
+ (((~ULL(0)) - (ULL(1) << (l)) + 1) & \
+ (~ULL(0) >> (BITS_PER_LONG_LONG - 1 - (h))))
#endif /* __LINUX_BITS_H */
CEPH_FEATURE_MON_STATEFUL_SUB | \
CEPH_FEATURE_CRUSH_TUNABLES5 | \
CEPH_FEATURE_NEW_OSDOPREPLY_ENCODING | \
+ CEPH_FEATURE_MSG_ADDR2 | \
CEPH_FEATURE_CEPHX_V2)
#define CEPH_FEATURES_REQUIRED_DEFAULT 0
/* flags field in client cap messages (version >= 10) */
#define CEPH_CLIENT_CAPS_SYNC (1<<0)
#define CEPH_CLIENT_CAPS_NO_CAPSNAP (1<<1)
-#define CEPH_CLIENT_CAPS_PENDING_CAPSNAP (1<<2);
+#define CEPH_CLIENT_CAPS_PENDING_CAPSNAP (1<<2)
/*
* caps message, used for capability callbacks, acks, requests, etc.
char *lock_name, u8 *type, char **tag,
struct ceph_locker **lockers, u32 *num_lockers);
+int ceph_cls_assert_locked(struct ceph_osd_request *req, int which,
+ char *lock_name, u8 type, char *cookie, char *tag);
+
#endif
/*
* sockaddr_storage <-> ceph_sockaddr
*/
-static inline void ceph_encode_addr(struct ceph_entity_addr *a)
+#define CEPH_ENTITY_ADDR_TYPE_NONE 0
+#define CEPH_ENTITY_ADDR_TYPE_LEGACY __cpu_to_le32(1)
+
+static inline void ceph_encode_banner_addr(struct ceph_entity_addr *a)
{
__be16 ss_family = htons(a->in_addr.ss_family);
a->in_addr.ss_family = *(__u16 *)&ss_family;
+
+ /* Banner addresses require TYPE_NONE */
+ a->type = CEPH_ENTITY_ADDR_TYPE_NONE;
}
-static inline void ceph_decode_addr(struct ceph_entity_addr *a)
+static inline void ceph_decode_banner_addr(struct ceph_entity_addr *a)
{
__be16 ss_family = *(__be16 *)&a->in_addr.ss_family;
a->in_addr.ss_family = ntohs(ss_family);
WARN_ON(a->in_addr.ss_family == 512);
+ a->type = CEPH_ENTITY_ADDR_TYPE_LEGACY;
}
+extern int ceph_decode_entity_addr(void **p, void *end,
+ struct ceph_entity_addr *addr);
/*
* encoders
*/
#define CEPH_MSG_MAX_MIDDLE_LEN (16*1024*1024)
/*
- * Handle the largest possible rbd object in one message.
+ * The largest possible rbd data object is 32M.
+ * The largest possible rbd object map object is 64M.
+ *
* There is no limit on the size of cephfs objects, but it has to obey
* rsize and wsize mount options anyway.
*/
-#define CEPH_MSG_MAX_DATA_LEN (32*1024*1024)
+#define CEPH_MSG_MAX_DATA_LEN (64*1024*1024)
#define CEPH_AUTH_NAME_DEFAULT "guest"
/* pagevec.c */
extern void ceph_release_page_vector(struct page **pages, int num_pages);
-
-extern struct page **ceph_get_direct_page_vector(const void __user *data,
- int num_pages,
- bool write_page);
extern void ceph_put_page_vector(struct page **pages, int num_pages,
bool dirty);
extern struct page **ceph_alloc_page_vector(int num_pages, gfp_t flags);
#endif
};
-extern struct ceph_monmap *ceph_monmap_decode(void *p, void *end);
extern int ceph_monmap_contains(struct ceph_monmap *m,
struct ceph_entity_addr *addr);
bool r_mempool;
struct completion r_completion; /* private to osd_client.c */
ceph_osdc_callback_t r_callback;
- struct list_head r_unsafe_item;
struct inode *r_inode; /* for use by callbacks */
+ struct list_head r_private_item; /* ditto */
void *r_priv; /* ditto */
/* set by submitter */
void ceph_osdc_update_epoch_barrier(struct ceph_osd_client *osdc, u32 eb);
void ceph_osdc_abort_requests(struct ceph_osd_client *osdc, int err);
+#define osd_req_op_data(oreq, whch, typ, fld) \
+({ \
+ struct ceph_osd_request *__oreq = (oreq); \
+ unsigned int __whch = (whch); \
+ BUG_ON(__whch >= __oreq->r_num_ops); \
+ &__oreq->r_ops[__whch].typ.fld; \
+})
+
extern void osd_req_op_init(struct ceph_osd_request *osd_req,
unsigned int which, u16 opcode, u32 flags);
const char *class, const char *method,
unsigned int flags,
struct page *req_page, size_t req_len,
- struct page *resp_page, size_t *resp_len);
+ struct page **resp_pages, size_t *resp_len);
extern int ceph_osdc_readpages(struct ceph_osd_client *osdc,
struct ceph_vino vino,
struct ceph_file_extent **file_extents,
u32 *num_file_extents);
+u64 ceph_get_num_objects(struct ceph_file_layout *l, u64 size);
+
#endif
/*
* Control Group subsystem type.
- * See Documentation/cgroup-v1/cgroups.rst for details
+ * See Documentation/admin-guide/cgroup-v1/cgroups.rst for details
*/
struct cgroup_subsys {
struct cgroup_subsys_state *(*css_alloc)(struct cgroup_subsys_state *parent_css);
#include <linux/of.h>
#include <linux/of_clk.h>
-#ifdef CONFIG_COMMON_CLK
-
/*
* flags used across common struct clk. these flags should only affect the
* top-level framework. custom flags for dealing with hardware specifics
/* helper functions */
const char *__clk_get_name(const struct clk *clk);
const char *clk_hw_get_name(const struct clk_hw *hw);
+#ifdef CONFIG_COMMON_CLK
struct clk_hw *__clk_get_hw(struct clk *clk);
+#else
+static inline struct clk_hw *__clk_get_hw(struct clk *clk)
+{
+ return (struct clk_hw *)clk;
+}
+#endif
unsigned int clk_hw_get_num_parents(const struct clk_hw *hw);
struct clk_hw *clk_hw_get_parent(const struct clk_hw *hw);
struct clk_hw *clk_hw_get_parent_by_index(const struct clk_hw *hw,
*/
unsigned long clk_hw_round_rate(struct clk_hw *hw, unsigned long rate);
-struct of_device_id;
-
struct clk_onecell_data {
struct clk **clks;
unsigned int clk_num;
struct clk_hw *hws[];
};
-extern struct of_device_id __clk_of_table;
-
#define CLK_OF_DECLARE(name, compat, fn) OF_DECLARE_1(clk, name, compat, fn)
/*
.ops = _ops, \
})
+#define CLK_HW_INIT_HW(_name, _parent, _ops, _flags) \
+ (&(struct clk_init_data) { \
+ .flags = _flags, \
+ .name = _name, \
+ .parent_hws = (const struct clk_hw*[]) { _parent }, \
+ .num_parents = 1, \
+ .ops = _ops, \
+ })
+
+/*
+ * This macro is intended for drivers to be able to share the otherwise
+ * individual struct clk_hw[] compound literals created by the compiler
+ * when using CLK_HW_INIT_HW. It does NOT support multiple parents.
+ */
+#define CLK_HW_INIT_HWS(_name, _parent, _ops, _flags) \
+ (&(struct clk_init_data) { \
+ .flags = _flags, \
+ .name = _name, \
+ .parent_hws = _parent, \
+ .num_parents = 1, \
+ .ops = _ops, \
+ })
+
+#define CLK_HW_INIT_FW_NAME(_name, _parent, _ops, _flags) \
+ (&(struct clk_init_data) { \
+ .flags = _flags, \
+ .name = _name, \
+ .parent_data = (const struct clk_parent_data[]) { \
+ { .fw_name = _parent }, \
+ }, \
+ .num_parents = 1, \
+ .ops = _ops, \
+ })
+
#define CLK_HW_INIT_PARENTS(_name, _parents, _ops, _flags) \
(&(struct clk_init_data) { \
.flags = _flags, \
.ops = _ops, \
})
+#define CLK_HW_INIT_PARENTS_HW(_name, _parents, _ops, _flags) \
+ (&(struct clk_init_data) { \
+ .flags = _flags, \
+ .name = _name, \
+ .parent_hws = _parents, \
+ .num_parents = ARRAY_SIZE(_parents), \
+ .ops = _ops, \
+ })
+
+#define CLK_HW_INIT_PARENTS_DATA(_name, _parents, _ops, _flags) \
+ (&(struct clk_init_data) { \
+ .flags = _flags, \
+ .name = _name, \
+ .parent_data = _parents, \
+ .num_parents = ARRAY_SIZE(_parents), \
+ .ops = _ops, \
+ })
+
#define CLK_HW_INIT_NO_PARENT(_name, _ops, _flags) \
(&(struct clk_init_data) { \
.flags = _flags, \
_flags), \
}
+#define CLK_FIXED_FACTOR_HW(_struct, _name, _parent, \
+ _div, _mult, _flags) \
+ struct clk_fixed_factor _struct = { \
+ .div = _div, \
+ .mult = _mult, \
+ .hw.init = CLK_HW_INIT_HW(_name, \
+ _parent, \
+ &clk_fixed_factor_ops, \
+ _flags), \
+ }
+
+/*
+ * This macro allows the driver to reuse the _parent array for multiple
+ * fixed factor clk declarations.
+ */
+#define CLK_FIXED_FACTOR_HWS(_struct, _name, _parent, \
+ _div, _mult, _flags) \
+ struct clk_fixed_factor _struct = { \
+ .div = _div, \
+ .mult = _mult, \
+ .hw.init = CLK_HW_INIT_HWS(_name, \
+ _parent, \
+ &clk_fixed_factor_ops, \
+ _flags), \
+ }
+
+#define CLK_FIXED_FACTOR_FW_NAME(_struct, _name, _parent, \
+ _div, _mult, _flags) \
+ struct clk_fixed_factor _struct = { \
+ .div = _div, \
+ .mult = _mult, \
+ .hw.init = CLK_HW_INIT_FW_NAME(_name, \
+ _parent, \
+ &clk_fixed_factor_ops, \
+ _flags), \
+ }
+
#ifdef CONFIG_OF
int of_clk_add_provider(struct device_node *np,
struct clk *(*clk_src_get)(struct of_phandle_args *args,
void clk_gate_restore_context(struct clk_hw *hw);
-#endif /* CONFIG_COMMON_CLK */
#endif /* CLK_PROVIDER_H */
*/
int __must_check clk_bulk_get_all(struct device *dev,
struct clk_bulk_data **clks);
+
+/**
+ * clk_bulk_get_optional - lookup and obtain a number of references to clock producer
+ * @dev: device for clock "consumer"
+ * @num_clks: the number of clk_bulk_data
+ * @clks: the clk_bulk_data table of consumer
+ *
+ * Behaves the same as clk_bulk_get() except where there is no clock producer.
+ * In this case, instead of returning -ENOENT, the function returns 0 and
+ * NULL for a clk for which a clock producer could not be determined.
+ */
+int __must_check clk_bulk_get_optional(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks);
/**
* devm_clk_bulk_get - managed get multiple clk consumers
* @dev: device for clock "consumer"
int __must_check devm_clk_bulk_get(struct device *dev, int num_clks,
struct clk_bulk_data *clks);
/**
+ * devm_clk_bulk_get_optional - managed get multiple optional consumer clocks
+ * @dev: device for clock "consumer"
+ * @clks: pointer to the clk_bulk_data table of consumer
+ *
+ * Behaves the same as devm_clk_bulk_get() except where there is no clock
+ * producer. In this case, instead of returning -ENOENT, the function returns
+ * NULL for given clk. It is assumed all clocks in clk_bulk_data are optional.
+ *
+ * Returns 0 if all clocks specified in clk_bulk_data table are obtained
+ * successfully or for any clk there was no clk provider available, otherwise
+ * returns valid IS_ERR() condition containing errno.
+ * The implementation uses @dev and @clk_bulk_data.id to determine the
+ * clock consumer, and thereby the clock producer.
+ * The clock returned is stored in each @clk_bulk_data.clk field.
+ *
+ * Drivers must assume that the clock source is not enabled.
+ *
+ * clk_bulk_get should not be called from within interrupt context.
+ */
+int __must_check devm_clk_bulk_get_optional(struct device *dev, int num_clks,
+ struct clk_bulk_data *clks);
+/**
* devm_clk_bulk_get_all - managed get multiple clk consumers
* @dev: device for clock "consumer"
* @clks: pointer to the clk_bulk_data table of consumer
return 0;
}
+static inline int __must_check clk_bulk_get_optional(struct device *dev,
+ int num_clks, struct clk_bulk_data *clks)
+{
+ return 0;
+}
+
static inline int __must_check clk_bulk_get_all(struct device *dev,
struct clk_bulk_data **clks)
{
return 0;
}
+static inline int __must_check devm_clk_bulk_get_optional(struct device *dev,
+ int num_clks, struct clk_bulk_data *clks)
+{
+ return 0;
+}
+
static inline int __must_check devm_clk_bulk_get_all(struct device *dev,
struct clk_bulk_data **clks)
{
#ifndef _CODA_HEADER_
#define _CODA_HEADER_
-#if defined(__linux__)
typedef unsigned long long u_quad_t;
-#endif
+
#include <uapi/linux/coda.h>
#endif
compat_sigset_word sig[_COMPAT_NSIG_WORDS];
} compat_sigset_t;
-int set_compat_user_sigmask(const compat_sigset_t __user *usigmask,
- sigset_t *set, sigset_t *oldset,
+int set_compat_user_sigmask(const compat_sigset_t __user *umask,
size_t sigsetsize);
struct compat_sigaction {
struct cn_queue_dev *cbdev;
};
+/**
+ * cn_add_callback() - Registers new callback with connector core.
+ *
+ * @id: unique connector's user identifier.
+ * It must be registered in connector.h for legal
+ * in-kernel users.
+ * @name: connector's callback symbolic name.
+ * @callback: connector's callback.
+ * parameters are %cn_msg and the sender's credentials
+ */
int cn_add_callback(struct cb_id *id, const char *name,
void (*callback)(struct cn_msg *, struct netlink_skb_parms *));
-void cn_del_callback(struct cb_id *);
+/**
+ * cn_del_callback() - Unregisters new callback with connector core.
+ *
+ * @id: unique connector's user identifier.
+ */
+void cn_del_callback(struct cb_id *id);
+
+
+/**
+ * cn_netlink_send_mult - Sends message to the specified groups.
+ *
+ * @msg: message header(with attached data).
+ * @len: Number of @msg to be sent.
+ * @portid: destination port.
+ * If non-zero the message will be sent to the given port,
+ * which should be set to the original sender.
+ * @group: destination group.
+ * If @portid and @group is zero, then appropriate group will
+ * be searched through all registered connector users, and
+ * message will be delivered to the group which was created
+ * for user with the same ID as in @msg.
+ * If @group is not zero, then message will be delivered
+ * to the specified group.
+ * @gfp_mask: GFP mask.
+ *
+ * It can be safely called from softirq context, but may silently
+ * fail under strong memory pressure.
+ *
+ * If there are no listeners for given group %-ESRCH can be returned.
+ */
int cn_netlink_send_mult(struct cn_msg *msg, u16 len, u32 portid, u32 group, gfp_t gfp_mask);
+
+/**
+ * cn_netlink_send_mult - Sends message to the specified groups.
+ *
+ * @msg: message header(with attached data).
+ * @portid: destination port.
+ * If non-zero the message will be sent to the given port,
+ * which should be set to the original sender.
+ * @group: destination group.
+ * If @portid and @group is zero, then appropriate group will
+ * be searched through all registered connector users, and
+ * message will be delivered to the group which was created
+ * for user with the same ID as in @msg.
+ * If @group is not zero, then message will be delivered
+ * to the specified group.
+ * @gfp_mask: GFP mask.
+ *
+ * It can be safely called from softirq context, but may silently
+ * fail under strong memory pressure.
+ *
+ * If there are no listeners for given group %-ESRCH can be returned.
+ */
int cn_netlink_send(struct cn_msg *msg, u32 portid, u32 group, gfp_t gfp_mask);
int cn_queue_add_callback(struct cn_queue_dev *dev, const char *name,
unsigned int transition_latency;
};
-struct cpufreq_user_policy {
- unsigned int min; /* in kHz */
- unsigned int max; /* in kHz */
-};
-
struct cpufreq_policy {
/* CPUs sharing clock, require sw coordination */
cpumask_var_t cpus; /* Online CPUs only */
struct work_struct update; /* if update_policy() needs to be
* called, but you're in IRQ context */
- struct cpufreq_user_policy user_policy;
+ struct dev_pm_qos_request *min_freq_req;
+ struct dev_pm_qos_request *max_freq_req;
struct cpufreq_frequency_table *freq_table;
enum cpufreq_table_sorting freq_table_sorted;
/* Pointer to the cooling device if used for thermal mitigation */
struct thermal_cooling_device *cdev;
+
+ struct notifier_block nb_min;
+ struct notifier_block nb_max;
};
struct cpufreq_freqs {
int cpufreq_get_policy(struct cpufreq_policy *policy, unsigned int cpu);
int cpufreq_set_policy(struct cpufreq_policy *policy,
struct cpufreq_policy *new_policy);
+void refresh_frequency_limits(struct cpufreq_policy *policy);
void cpufreq_update_policy(unsigned int cpu);
void cpufreq_update_limits(unsigned int cpu);
bool have_governor_per_policy(void);
int cpufreq_table_validate_and_sort(struct cpufreq_policy *policy);
unsigned int cpufreq_generic_get(unsigned int cpu);
-int cpufreq_generic_init(struct cpufreq_policy *policy,
+void cpufreq_generic_init(struct cpufreq_policy *policy,
struct cpufreq_frequency_table *table,
unsigned int transition_latency);
#endif /* _LINUX_CPUFREQ_H */
#include <linux/radix-tree.h>
#include <asm/pgtable.h>
+/* Flag for synchronous flush */
+#define DAXDEV_F_SYNC (1UL << 0)
+
typedef unsigned long dax_entry_t;
struct iomap_ops;
#if IS_ENABLED(CONFIG_DAX)
struct dax_device *dax_get_by_host(const char *host);
struct dax_device *alloc_dax(void *private, const char *host,
- const struct dax_operations *ops);
+ const struct dax_operations *ops, unsigned long flags);
void put_dax(struct dax_device *dax_dev);
void kill_dax(struct dax_device *dax_dev);
void dax_write_cache(struct dax_device *dax_dev, bool wc);
bool dax_write_cache_enabled(struct dax_device *dax_dev);
+bool __dax_synchronous(struct dax_device *dax_dev);
+static inline bool dax_synchronous(struct dax_device *dax_dev)
+{
+ return __dax_synchronous(dax_dev);
+}
+void __set_dax_synchronous(struct dax_device *dax_dev);
+static inline void set_dax_synchronous(struct dax_device *dax_dev)
+{
+ __set_dax_synchronous(dax_dev);
+}
+/*
+ * Check if given mapping is supported by the file / underlying device.
+ */
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+ struct dax_device *dax_dev)
+{
+ if (!(vma->vm_flags & VM_SYNC))
+ return true;
+ if (!IS_DAX(file_inode(vma->vm_file)))
+ return false;
+ return dax_synchronous(dax_dev);
+}
#else
static inline struct dax_device *dax_get_by_host(const char *host)
{
return NULL;
}
static inline struct dax_device *alloc_dax(void *private, const char *host,
- const struct dax_operations *ops)
+ const struct dax_operations *ops, unsigned long flags)
{
/*
* Callers should check IS_ENABLED(CONFIG_DAX) to know if this
{
return false;
}
+static inline bool dax_synchronous(struct dax_device *dax_dev)
+{
+ return true;
+}
+static inline void set_dax_synchronous(struct dax_device *dax_dev)
+{
+}
+static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
+ struct dax_device *dax_dev)
+{
+ return !(vma->vm_flags & VM_SYNC);
+}
#endif
struct writeback_control;
*/
extern __printf(4, 5)
char *dynamic_dname(struct dentry *, char *, int, const char *, ...);
-extern char *simple_dname(struct dentry *, char *, int);
extern char *__d_path(const struct path *, const struct path *, char *, int);
extern char *d_absolute_path(const struct path *, char *, int);
*---------------------------------------------------------------*/
#define DM_NAME "device-mapper"
-#define DM_RATELIMIT(pr_func, fmt, ...) \
-do { \
- static DEFINE_RATELIMIT_STATE(rs, DEFAULT_RATELIMIT_INTERVAL, \
- DEFAULT_RATELIMIT_BURST); \
- \
- if (__ratelimit(&rs)) \
- pr_func(DM_FMT(fmt), ##__VA_ARGS__); \
-} while (0)
-
#define DM_FMT(fmt) DM_NAME ": " DM_MSG_PREFIX ": " fmt "\n"
#define DMCRIT(fmt, ...) pr_crit(DM_FMT(fmt), ##__VA_ARGS__)
#define DMERR(fmt, ...) pr_err(DM_FMT(fmt), ##__VA_ARGS__)
-#define DMERR_LIMIT(fmt, ...) DM_RATELIMIT(pr_err, fmt, ##__VA_ARGS__)
+#define DMERR_LIMIT(fmt, ...) pr_err_ratelimited(DM_FMT(fmt), ##__VA_ARGS__)
#define DMWARN(fmt, ...) pr_warn(DM_FMT(fmt), ##__VA_ARGS__)
-#define DMWARN_LIMIT(fmt, ...) DM_RATELIMIT(pr_warn, fmt, ##__VA_ARGS__)
+#define DMWARN_LIMIT(fmt, ...) pr_warn_ratelimited(DM_FMT(fmt), ##__VA_ARGS__)
#define DMINFO(fmt, ...) pr_info(DM_FMT(fmt), ##__VA_ARGS__)
-#define DMINFO_LIMIT(fmt, ...) DM_RATELIMIT(pr_info, fmt, ##__VA_ARGS__)
+#define DMINFO_LIMIT(fmt, ...) pr_info_ratelimited(DM_FMT(fmt), ##__VA_ARGS__)
#ifdef CONFIG_DM_DEBUG
#define DMDEBUG(fmt, ...) printk(KERN_DEBUG DM_FMT(fmt), ##__VA_ARGS__)
-#define DMDEBUG_LIMIT(fmt, ...) DM_RATELIMIT(pr_debug, fmt, ##__VA_ARGS__)
+#define DMDEBUG_LIMIT(fmt, ...) pr_debug_ratelimited(DM_FMT(fmt), ##__VA_ARGS__)
#else
#define DMDEBUG(fmt, ...) no_printk(fmt, ##__VA_ARGS__)
#define DMDEBUG_LIMIT(fmt, ...) no_printk(fmt, ##__VA_ARGS__)
* Copyright (c) 2004-2009 Greg Kroah-Hartman <gregkh@suse.de>
* Copyright (c) 2008-2009 Novell Inc.
*
- * See Documentation/driver-model/ for more information.
+ * See Documentation/driver-api/driver-model/ for more information.
*/
#ifndef _DEVICE_H_
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2018-2019 Synopsys, Inc. and/or its affiliates.
+ * Synopsys DesignWare eDMA core driver
+ *
+ * Author: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
+ */
+
+#ifndef _DW_EDMA_H
+#define _DW_EDMA_H
+
+#include <linux/device.h>
+#include <linux/dmaengine.h>
+
+struct dw_edma;
+
+/**
+ * struct dw_edma_chip - representation of DesignWare eDMA controller hardware
+ * @dev: struct device of the eDMA controller
+ * @id: instance ID
+ * @irq: irq line
+ * @dw: struct dw_edma that is filed by dw_edma_probe()
+ */
+struct dw_edma_chip {
+ struct device *dev;
+ int id;
+ int irq;
+ struct dw_edma *dw;
+};
+
+/* Export to the platform drivers */
+#if IS_ENABLED(CONFIG_DW_EDMA)
+int dw_edma_probe(struct dw_edma_chip *chip);
+int dw_edma_remove(struct dw_edma_chip *chip);
+#else
+static inline int dw_edma_probe(struct dw_edma_chip *chip)
+{
+ return -ENODEV;
+}
+
+static inline int dw_edma_remove(struct dw_edma_chip *chip)
+{
+ return 0;
+}
+#endif /* CONFIG_DW_EDMA */
+
+#endif /* _DW_EDMA_H */
enum dma_status dma_wait_for_async_tx(struct dma_async_tx_descriptor *tx);
void dma_issue_pending_all(void);
struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
- dma_filter_fn fn, void *fn_param);
+ dma_filter_fn fn, void *fn_param,
+ struct device_node *np);
struct dma_chan *dma_request_slave_channel(struct device *dev, const char *name);
struct dma_chan *dma_request_chan(struct device *dev, const char *name);
{
}
static inline struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
- dma_filter_fn fn, void *fn_param)
+ dma_filter_fn fn,
+ void *fn_param,
+ struct device_node *np)
{
return NULL;
}
void dma_run_dependencies(struct dma_async_tx_descriptor *tx);
struct dma_chan *dma_get_slave_channel(struct dma_chan *chan);
struct dma_chan *dma_get_any_slave_channel(struct dma_device *device);
-#define dma_request_channel(mask, x, y) __dma_request_channel(&(mask), x, y)
+#define dma_request_channel(mask, x, y) \
+ __dma_request_channel(&(mask), x, y, NULL)
#define dma_request_slave_channel_compat(mask, x, y, dev, name) \
__dma_request_slave_channel_compat(&(mask), x, y, dev, name)
if (!fn || !fn_param)
return NULL;
- return __dma_request_channel(mask, fn, fn_param);
+ return __dma_request_channel(mask, fn, fn_param, NULL);
}
#endif /* DMAENGINE_H */
return size <= size_default && (size & (size - 1)) == 0;
}
-#define bpf_ctx_wide_store_ok(off, size, type, field) \
+#define bpf_ctx_wide_access_ok(off, size, type, field) \
(size == sizeof(__u64) && \
off >= offsetof(type, field) && \
off + sizeof(__u64) <= offsetofend(type, field) && \
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Analog Devices AXI common registers & definitions
+ *
+ * Copyright 2019 Analog Devices Inc.
+ *
+ * https://wiki.analog.com/resources/fpga/docs/axi_ip
+ * https://wiki.analog.com/resources/fpga/docs/hdl/regmap
+ */
+
+#ifndef ADI_AXI_COMMON_H_
+#define ADI_AXI_COMMON_H_
+
+#define ADI_AXI_REG_VERSION 0x0000
+
+#define ADI_AXI_PCORE_VER(major, minor, patch) \
+ (((major) << 16) | ((minor) << 8) | (patch))
+
+#endif /* ADI_AXI_COMMON_H_ */
#define MODULE_ALIAS_FS(NAME) MODULE_ALIAS("fs-" NAME)
-extern struct dentry *mount_ns(struct file_system_type *fs_type,
- int flags, void *data, void *ns, struct user_namespace *user_ns,
- int (*fill_super)(struct super_block *, void *, int));
#ifdef CONFIG_BLOCK
extern struct dentry *mount_bdev(struct file_system_type *fs_type,
int flags, const char *dev_name, void *data,
struct super_block *sget_fc(struct fs_context *fc,
int (*test)(struct super_block *, struct fs_context *),
int (*set)(struct super_block *, struct fs_context *));
-struct super_block *sget_userns(struct file_system_type *type,
- int (*test)(struct super_block *,void *),
- int (*set)(struct super_block *,void *),
- int flags, struct user_namespace *user_ns,
- void *data);
struct super_block *sget(struct file_system_type *type,
int (*test)(struct super_block *,void *),
int (*set)(struct super_block *,void *),
int flags, void *data);
-extern struct dentry *mount_pseudo_xattr(struct file_system_type *, char *,
- const struct super_operations *ops,
- const struct xattr_handler **xattr,
- const struct dentry_operations *dops,
- unsigned long);
-
-static inline struct dentry *
-mount_pseudo(struct file_system_type *fs_type, char *name,
- const struct super_operations *ops,
- const struct dentry_operations *dops, unsigned long magic)
-{
- return mount_pseudo_xattr(fs_type, name, ops, NULL, dops, magic);
-}
/* Alas, no aliases. Too much hassle with bringing module.h everywhere */
#define fops_get(fops) \
void *s_fs_info; /* Proposed s_fs_info */
unsigned int sb_flags; /* Proposed superblock flags (SB_*) */
unsigned int sb_flags_mask; /* Superblock flags that were changed */
+ unsigned int s_iflags; /* OR'd with sb->s_iflags */
unsigned int lsm_flags; /* Information flags from the fs to the LSM */
enum fs_context_purpose purpose:8;
enum fs_context_phase phase:8; /* The phase the context is in */
enum vfs_get_super_keying keying,
int (*fill_super)(struct super_block *sb,
struct fs_context *fc));
+extern int get_tree_nodev(struct fs_context *fc,
+ int (*fill_super)(struct super_block *sb,
+ struct fs_context *fc));
+extern int get_tree_single(struct fs_context *fc,
+ int (*fill_super)(struct super_block *sb,
+ struct fs_context *fc));
extern const struct file_operations fscontext_fops;
iter = ftrace_rec_iter_next(iter))
-int ftrace_update_record(struct dyn_ftrace *rec, int enable);
-int ftrace_test_record(struct dyn_ftrace *rec, int enable);
+int ftrace_update_record(struct dyn_ftrace *rec, bool enable);
+int ftrace_test_record(struct dyn_ftrace *rec, bool enable);
void ftrace_run_stop_machine(int command);
unsigned long ftrace_location(unsigned long ip);
unsigned long ftrace_location_range(unsigned long start, unsigned long end);
bool transparent_hugepage_enabled(struct vm_area_struct *vma);
+#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1)
+
+static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
+ unsigned long haddr)
+{
+ /* Don't have to check pgoff for anonymous vma */
+ if (!vma_is_anonymous(vma)) {
+ if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) !=
+ (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK))
+ return false;
+ }
+
+ if (haddr < vma->vm_start || haddr + HPAGE_PMD_SIZE > vma->vm_end)
+ return false;
+ return true;
+}
+
#define transparent_hugepage_use_zero_page() \
(transparent_hugepage_flags & \
(1<<TRANSPARENT_HUGEPAGE_USE_ZERO_PAGE_FLAG))
return false;
}
+static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
+ unsigned long haddr)
+{
+ return false;
+}
+
static inline void prep_transhuge_page(struct page *page) {}
#define transparent_hugepage_flags 0UL
/*
Hardware Random Number Generator
- Please read Documentation/hw_random.txt for details on use.
+ Please read Documentation/admin-guide/hw_random.rst for details on use.
----------------------------------------------------------
This software may be used and distributed according to the terms
#include <linux/sched.h>
/* hwspinlock mode argument */
-#define HWLOCK_IRQSTATE 0x01 /* Disable interrupts, save state */
-#define HWLOCK_IRQ 0x02 /* Disable interrupts, don't save state */
-#define HWLOCK_RAW 0x03
+#define HWLOCK_IRQSTATE 0x01 /* Disable interrupts, save state */
+#define HWLOCK_IRQ 0x02 /* Disable interrupts, don't save state */
+#define HWLOCK_RAW 0x03
+#define HWLOCK_IN_ATOMIC 0x04 /* Called while in atomic context */
struct device;
struct device_node;
}
/**
+ * hwspin_trylock_in_atomic() - attempt to lock a specific hwspinlock
+ * @hwlock: an hwspinlock which we want to trylock
+ *
+ * This function attempts to lock an hwspinlock, and will immediately fail
+ * if the hwspinlock is already taken.
+ *
+ * This function shall be called only from an atomic context.
+ *
+ * Returns 0 if we successfully locked the hwspinlock, -EBUSY if
+ * the hwspinlock was already taken, and -EINVAL if @hwlock is invalid.
+ */
+static inline int hwspin_trylock_in_atomic(struct hwspinlock *hwlock)
+{
+ return __hwspin_trylock(hwlock, HWLOCK_IN_ATOMIC, NULL);
+}
+
+/**
* hwspin_trylock() - attempt to lock a specific hwspinlock
* @hwlock: an hwspinlock which we want to trylock
*
}
/**
+ * hwspin_lock_timeout_in_atomic() - lock an hwspinlock with timeout limit
+ * @hwlock: the hwspinlock to be locked
+ * @to: timeout value in msecs
+ *
+ * This function locks the underlying @hwlock. If the @hwlock
+ * is already taken, the function will busy loop waiting for it to
+ * be released, but give up when @timeout msecs have elapsed.
+ *
+ * This function shall be called only from an atomic context and the timeout
+ * value shall not exceed a few msecs.
+ *
+ * Returns 0 when the @hwlock was successfully taken, and an appropriate
+ * error code otherwise (most notably an -ETIMEDOUT if the @hwlock is still
+ * busy after @timeout msecs). The function will never sleep.
+ */
+static inline
+int hwspin_lock_timeout_in_atomic(struct hwspinlock *hwlock, unsigned int to)
+{
+ return __hwspin_lock_timeout(hwlock, to, HWLOCK_IN_ATOMIC, NULL);
+}
+
+/**
* hwspin_lock_timeout() - lock an hwspinlock with timeout limit
* @hwlock: the hwspinlock to be locked
* @to: timeout value in msecs
}
/**
+ * hwspin_unlock_in_atomic() - unlock hwspinlock
+ * @hwlock: a previously-acquired hwspinlock which we want to unlock
+ *
+ * This function will unlock a specific hwspinlock.
+ *
+ * @hwlock must be already locked (e.g. by hwspin_trylock()) before calling
+ * this function: it is a bug to call unlock on a @hwlock that is already
+ * unlocked.
+ */
+static inline void hwspin_unlock_in_atomic(struct hwspinlock *hwlock)
+{
+ __hwspin_unlock(hwlock, HWLOCK_IN_ATOMIC, NULL);
+}
+
+/**
* hwspin_unlock() - unlock hwspinlock
* @hwlock: a previously-acquired hwspinlock which we want to unlock
*
/* Used for contructor calls. */
typedef void (*ctor_fn_t)(void);
+struct file_system_type;
+
/* Defined in init/main.c */
extern int do_one_initcall(initcall_t fn);
extern char __initdata boot_command_line[];
/* used by init/main.c */
void setup_arch(char **);
void prepare_namespace(void);
-int __init init_rootfs(void);
+void __init init_rootfs(void);
+extern struct file_system_type rootfs_fs_type;
#if defined(CONFIG_STRICT_KERNEL_RWX) || defined(CONFIG_STRICT_MODULE_RWX)
extern bool rodata_enabled;
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Data types and headers for RAPL support
+ *
+ * Copyright (C) 2019 Intel Corporation.
+ *
+ * Author: Zhang Rui <rui.zhang@intel.com>
+ */
+
+#ifndef __INTEL_RAPL_H__
+#define __INTEL_RAPL_H__
+
+#include <linux/types.h>
+#include <linux/powercap.h>
+#include <linux/cpuhotplug.h>
+
+enum rapl_domain_type {
+ RAPL_DOMAIN_PACKAGE, /* entire package/socket */
+ RAPL_DOMAIN_PP0, /* core power plane */
+ RAPL_DOMAIN_PP1, /* graphics uncore */
+ RAPL_DOMAIN_DRAM, /* DRAM control_type */
+ RAPL_DOMAIN_PLATFORM, /* PSys control_type */
+ RAPL_DOMAIN_MAX,
+};
+
+enum rapl_domain_reg_id {
+ RAPL_DOMAIN_REG_LIMIT,
+ RAPL_DOMAIN_REG_STATUS,
+ RAPL_DOMAIN_REG_PERF,
+ RAPL_DOMAIN_REG_POLICY,
+ RAPL_DOMAIN_REG_INFO,
+ RAPL_DOMAIN_REG_MAX,
+};
+
+struct rapl_package;
+
+enum rapl_primitives {
+ ENERGY_COUNTER,
+ POWER_LIMIT1,
+ POWER_LIMIT2,
+ FW_LOCK,
+
+ PL1_ENABLE, /* power limit 1, aka long term */
+ PL1_CLAMP, /* allow frequency to go below OS request */
+ PL2_ENABLE, /* power limit 2, aka short term, instantaneous */
+ PL2_CLAMP,
+
+ TIME_WINDOW1, /* long term */
+ TIME_WINDOW2, /* short term */
+ THERMAL_SPEC_POWER,
+ MAX_POWER,
+
+ MIN_POWER,
+ MAX_TIME_WINDOW,
+ THROTTLED_TIME,
+ PRIORITY_LEVEL,
+
+ /* below are not raw primitive data */
+ AVERAGE_POWER,
+ NR_RAPL_PRIMITIVES,
+};
+
+struct rapl_domain_data {
+ u64 primitives[NR_RAPL_PRIMITIVES];
+ unsigned long timestamp;
+};
+
+#define NR_POWER_LIMITS (2)
+struct rapl_power_limit {
+ struct powercap_zone_constraint *constraint;
+ int prim_id; /* primitive ID used to enable */
+ struct rapl_domain *domain;
+ const char *name;
+ u64 last_power_limit;
+};
+
+struct rapl_package;
+
+struct rapl_domain {
+ const char *name;
+ enum rapl_domain_type id;
+ u64 regs[RAPL_DOMAIN_REG_MAX];
+ struct powercap_zone power_zone;
+ struct rapl_domain_data rdd;
+ struct rapl_power_limit rpl[NR_POWER_LIMITS];
+ u64 attr_map; /* track capabilities */
+ unsigned int state;
+ unsigned int domain_energy_unit;
+ struct rapl_package *rp;
+};
+
+struct reg_action {
+ u64 reg;
+ u64 mask;
+ u64 value;
+ int err;
+};
+
+/**
+ * struct rapl_if_priv: private data for different RAPL interfaces
+ * @control_type: Each RAPL interface must have its own powercap
+ * control type.
+ * @platform_rapl_domain: Optional. Some RAPL interface may have platform
+ * level RAPL control.
+ * @pcap_rapl_online: CPU hotplug state for each RAPL interface.
+ * @reg_unit: Register for getting energy/power/time unit.
+ * @regs: Register sets for different RAPL Domains.
+ * @limits: Number of power limits supported by each domain.
+ * @read_raw: Callback for reading RAPL interface specific
+ * registers.
+ * @write_raw: Callback for writing RAPL interface specific
+ * registers.
+ */
+struct rapl_if_priv {
+ struct powercap_control_type *control_type;
+ struct rapl_domain *platform_rapl_domain;
+ enum cpuhp_state pcap_rapl_online;
+ u64 reg_unit;
+ u64 regs[RAPL_DOMAIN_MAX][RAPL_DOMAIN_REG_MAX];
+ int limits[RAPL_DOMAIN_MAX];
+ int (*read_raw)(int cpu, struct reg_action *ra);
+ int (*write_raw)(int cpu, struct reg_action *ra);
+};
+
+/* maximum rapl package domain name: package-%d-die-%d */
+#define PACKAGE_DOMAIN_NAME_LENGTH 30
+
+struct rapl_package {
+ unsigned int id; /* logical die id, equals physical 1-die systems */
+ unsigned int nr_domains;
+ unsigned long domain_map; /* bit map of active domains */
+ unsigned int power_unit;
+ unsigned int energy_unit;
+ unsigned int time_unit;
+ struct rapl_domain *domains; /* array of domains, sized at runtime */
+ struct powercap_zone *power_zone; /* keep track of parent zone */
+ unsigned long power_limit_irq; /* keep track of package power limit
+ * notify interrupt enable status.
+ */
+ struct list_head plist;
+ int lead_cpu; /* one active cpu per package for access */
+ /* Track active cpus */
+ struct cpumask cpumask;
+ char name[PACKAGE_DOMAIN_NAME_LENGTH];
+ struct rapl_if_priv *priv;
+};
+
+struct rapl_package *rapl_find_package_domain(int cpu, struct rapl_if_priv *priv);
+struct rapl_package *rapl_add_package(int cpu, struct rapl_if_priv *priv);
+void rapl_remove_package(struct rapl_package *rp);
+
+int rapl_add_platform_domain(struct rapl_if_priv *priv);
+void rapl_remove_platform_domain(struct rapl_if_priv *priv);
+
+#endif /* __INTEL_RAPL_H__ */
#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
void __init ioremap_huge_init(void);
+int arch_ioremap_p4d_supported(void);
int arch_ioremap_pud_supported(void);
int arch_ioremap_pmd_supported(void);
#else
#include <linux/mm.h>
#include <linux/types.h>
#include <linux/mm_types.h>
+#include <linux/blkdev.h>
struct address_space;
struct fiemap_extent_info;
const struct iomap_page_ops *page_ops;
};
+static inline sector_t
+iomap_sector(struct iomap *iomap, loff_t pos)
+{
+ return (iomap->addr + pos - iomap->offset) >> SECTOR_SHIFT;
+}
+
/*
* When a filesystem sets page_ops in an iomap mapping it returns, page_prepare
* and page_done will be called for each page written to. This only applies to
};
/*
+ * Main iomap iterator function.
+ */
+typedef loff_t (*iomap_actor_t)(struct inode *inode, loff_t pos, loff_t len,
+ void *data, struct iomap *iomap);
+
+loff_t iomap_apply(struct inode *inode, loff_t pos, loff_t length,
+ unsigned flags, const struct iomap_ops *ops, void *data,
+ iomap_actor_t actor);
+
+/*
* Structure allocate for each page when block size < PAGE_SIZE to track
* sub-page uptodate status and I/O completions.
*/
}
/**
+ * inode_set_max_iversion_raw - update i_version new value is larger
+ * @inode: inode to set
+ * @val: new i_version to set
+ *
+ * Some self-managed filesystems (e.g Ceph) will only update the i_version
+ * value if the new value is larger than the one we already have.
+ */
+static inline void
+inode_set_max_iversion_raw(struct inode *inode, u64 val)
+{
+ u64 cur, old;
+
+ cur = inode_peek_iversion_raw(inode);
+ for (;;) {
+ if (cur > val)
+ break;
+ old = atomic64_cmpxchg(&inode->i_version, cur, val);
+ if (likely(old == cur))
+ break;
+ cur = old;
+ }
+}
+
+/**
* inode_set_iversion - set i_version to a particular value
* @inode: inode to set
* @val: new i_version value to set
*/
#define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
+#define typeof_member(T, m) typeof(((T*)0)->m)
+
#define DIV_ROUND_UP __KERNEL_DIV_ROUND_UP
#define DIV_ROUND_DOWN_ULL(ll, d) \
}
#endif
+/* Returns true if kprobes handled the fault */
+static nokprobe_inline bool kprobe_page_fault(struct pt_regs *regs,
+ unsigned int trap)
+{
+ if (!kprobes_built_in())
+ return false;
+ if (user_mode(regs))
+ return false;
+ /*
+ * To be potentially processing a kprobe fault and to be allowed
+ * to call kprobe_running(), we have to be non-preemptible.
+ */
+ if (preemptible())
+ return false;
+ if (!kprobe_running())
+ return false;
+ return kprobe_fault_handler(regs, trap);
+}
+
#endif /* _LINUX_KPROBES_H */
#include <linux/types.h>
#include <linux/uuid.h>
#include <linux/spinlock.h>
+#include <linux/bio.h>
struct badrange_entry {
u64 start;
*/
ND_REGION_PERSIST_MEMCTRL = 2,
+ /* Platform provides asynchronous flush mechanism */
+ ND_REGION_ASYNC = 3,
+
/* mark newly adjusted resources as requiring a label update */
DPA_RESOURCE_ADJUSTED = 1 << 0,
};
int position;
};
+struct nd_region;
struct nd_region_desc {
struct resource *res;
struct nd_mapping_desc *mapping;
int target_node;
unsigned long flags;
struct device_node *of_node;
+ int (*flush)(struct nd_region *nd_region, struct bio *bio);
};
struct device;
unsigned int nd_region_acquire_lane(struct nd_region *nd_region);
void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane);
u64 nd_fletcher64(void *addr, size_t len, bool le);
-void nvdimm_flush(struct nd_region *nd_region);
+int nvdimm_flush(struct nd_region *nd_region, struct bio *bio);
+int generic_nvdimm_flush(struct nd_region *nd_region);
int nvdimm_has_flush(struct nd_region *nd_region);
int nvdimm_has_cache(struct nd_region *nd_region);
int nvdimm_in_overwrite(struct nvdimm *nvdimm);
+bool is_nvdimm_sync(struct nd_region *nd_region);
static inline int nvdimm_ctl(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
unsigned int buf_len, int *cmd_rc)
* Copyright (C) 2006,2007 Red Hat, Inc., Ingo Molnar <mingo@redhat.com>
* Copyright (C) 2007 Red Hat, Inc., Peter Zijlstra
*
- * see Documentation/locking/lockdep-design.txt for more details.
+ * see Documentation/locking/lockdep-design.rst for more details.
*/
#ifndef __LINUX_LOCKDEP_H
#define __LINUX_LOCKDEP_H
* @compressedSize: is the precise full size of the compressed block
* @maxDecompressedSize: is the size of 'dest' buffer
*
- * Decompresses data fom 'source' into 'dest'.
+ * Decompresses data from 'source' into 'dest'.
* If the source stream is detected malformed, the function will
* stop decoding and return a negative result.
* This function is protected against buffer overflow exploits,
const char *dictionary, int dictSize);
/**
- * LZ4_decompress_fast_continue() - Decompress blocks in streaming mode
+ * LZ4_decompress_safe_continue() - Decompress blocks in streaming mode
* @LZ4_streamDecode: the 'LZ4_streamDecode_t' structure
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* @compressedSize: is the precise full size of the compressed block
* @maxDecompressedSize: is the size of 'dest' buffer
*
- * These decoding function allows decompression of multiple blocks
+ * This decoding function allows decompression of multiple blocks
* in "streaming" mode.
* Previously decoded blocks *must* remain available at the memory position
* where they were decoded (up to 64 KB)
* which must be already allocated with 'originalSize' bytes
* @originalSize: is the original and therefore uncompressed size
*
- * These decoding function allows decompression of multiple blocks
+ * This decoding function allows decompression of multiple blocks
* in "streaming" mode.
* Previously decoded blocks *must* remain available at the memory position
* where they were decoded (up to 64 KB)
* @dictStart: pointer to the start of the dictionary in memory
* @dictSize: size of dictionary
*
- * These decoding function works the same as
+ * This decoding function works the same as
* a combination of LZ4_setStreamDecode() followed by
* LZ4_decompress_safe_continue()
- * It is stand-alone, and don'tn eed a LZ4_streamDecode_t structure.
+ * It is stand-alone, and doesn't need an LZ4_streamDecode_t structure.
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* @dictStart: pointer to the start of the dictionary in memory
* @dictSize: size of dictionary
*
- * These decoding function works the same as
+ * This decoding function works the same as
* a combination of LZ4_setStreamDecode() followed by
- * LZ4_decompress_safe_continue()
- * It is stand-alone, and don'tn eed a LZ4_streamDecode_t structure.
+ * LZ4_decompress_fast_continue()
+ * It is stand-alone, and doesn't need an LZ4_streamDecode_t structure.
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
extern void unregister_memory_notifier(struct notifier_block *nb);
extern int register_memory_isolate_notifier(struct notifier_block *nb);
extern void unregister_memory_isolate_notifier(struct notifier_block *nb);
-int hotplug_memory_register(int nid, struct mem_section *section);
-#ifdef CONFIG_MEMORY_HOTREMOVE
-extern void unregister_memory_section(struct mem_section *);
-#endif
+int create_memory_block_devices(unsigned long start, unsigned long size);
+void remove_memory_block_devices(unsigned long start, unsigned long size);
extern int memory_dev_init(void);
extern int memory_notify(unsigned long val, void *v);
extern int memory_isolate_notify(unsigned long val, void *v);
-extern struct memory_block *find_memory_block_hinted(struct mem_section *,
- struct memory_block *);
extern struct memory_block *find_memory_block(struct mem_section *);
+typedef int (*walk_memory_blocks_func_t)(struct memory_block *, void *);
+extern int walk_memory_blocks(unsigned long start, unsigned long size,
+ void *arg, walk_memory_blocks_func_t func);
#define CONFIG_MEM_BLOCK_SIZE (PAGES_PER_SECTION<<PAGE_SHIFT)
#endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
return movable_node_enabled;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
extern void arch_remove_memory(int nid, u64 start, u64 size,
struct vmem_altmap *altmap);
extern void __remove_pages(struct zone *zone, unsigned long start_pfn,
unsigned long nr_pages, struct vmem_altmap *altmap);
-#endif /* CONFIG_MEMORY_HOTREMOVE */
-
-/*
- * Do we want sysfs memblock files created. This will allow userspace to online
- * and offline memory explicitly. Lack of this bit means that the caller has to
- * call move_pfn_range_to_zone to finish the initialization.
- */
-
-#define MHP_MEMBLOCK_API (1<<0)
/* reasonably generic interface to expand the physical pages */
extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,
extern bool is_mem_section_removable(unsigned long pfn, unsigned long nr_pages);
extern void try_offline_node(int nid);
extern int offline_pages(unsigned long start_pfn, unsigned long nr_pages);
-extern void remove_memory(int nid, u64 start, u64 size);
+extern int remove_memory(int nid, u64 start, u64 size);
extern void __remove_memory(int nid, u64 start, u64 size);
#else
return -EINVAL;
}
-static inline void remove_memory(int nid, u64 start, u64 size) {}
+static inline int remove_memory(int nid, u64 start, u64 size)
+{
+ return -EBUSY;
+}
+
static inline void __remove_memory(int nid, u64 start, u64 size) {}
#endif /* CONFIG_MEMORY_HOTREMOVE */
extern void __ref free_area_init_core_hotplug(int nid);
-extern int walk_memory_range(unsigned long start_pfn, unsigned long end_pfn,
- void *arg, int (*func)(struct memory_block *, void *));
extern int __add_memory(int nid, u64 start, u64 size);
extern int add_memory(int nid, u64 start, u64 size);
extern int add_memory_resource(int nid, struct resource *resource);
extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn,
unsigned long nr_pages, struct vmem_altmap *altmap);
extern bool is_memblock_offlined(struct memory_block *mem);
-extern int sparse_add_one_section(int nid, unsigned long start_pfn,
- struct vmem_altmap *altmap);
-extern void sparse_remove_one_section(struct zone *zone, struct mem_section *ms,
+extern int sparse_add_section(int nid, unsigned long pfn,
+ unsigned long nr_pages, struct vmem_altmap *altmap);
+extern void sparse_remove_section(struct mem_section *ms,
+ unsigned long pfn, unsigned long nr_pages,
unsigned long map_offset, struct vmem_altmap *altmap);
extern struct page *sparse_decode_mem_map(unsigned long coded_mem_map,
unsigned long pnum);
extern int migrate_huge_page_move_mapping(struct address_space *mapping,
struct page *newpage, struct page *page);
extern int migrate_page_move_mapping(struct address_space *mapping,
- struct page *newpage, struct page *page, enum migrate_mode mode,
- int extra_count);
+ struct page *newpage, struct page *page, int extra_count);
#else
static inline void putback_movable_pages(struct list_head *l) {}
vma->vm_ops = NULL;
}
+static inline bool vma_is_anonymous(struct vm_area_struct *vma)
+{
+ return !vma->vm_ops;
+}
+
+#ifdef CONFIG_SHMEM
+/*
+ * The vma_is_shmem is not inline because it is used only by slow
+ * paths in userfault.
+ */
+bool vma_is_shmem(struct vm_area_struct *vma);
+#else
+static inline bool vma_is_shmem(struct vm_area_struct *vma) { return false; }
+#endif
+
+int vma_is_stack_for_current(struct vm_area_struct *vma);
+
/* flush_tlb_range() takes a vma, not a mm, and can care about flags */
#define TLB_FLUSH_VMA(mm,flags) { .vm_mm = (mm), .vm_flags = (flags) }
struct mmu_gather;
struct inode;
-#if !defined(__HAVE_ARCH_PTE_DEVMAP) || !defined(CONFIG_TRANSPARENT_HUGEPAGE)
+#if !defined(CONFIG_ARCH_HAS_PTE_DEVMAP) || !defined(CONFIG_TRANSPARENT_HUGEPAGE)
static inline int pmd_devmap(pmd_t pmd)
{
return 0;
return false;
}
-static inline bool is_device_private_page(const struct page *page)
-{
- return is_zone_device_page(page) &&
- page->pgmap->type == MEMORY_DEVICE_PRIVATE;
-}
-
-#ifdef CONFIG_PCI_P2PDMA
-static inline bool is_pci_p2pdma_page(const struct page *page)
-{
- return is_zone_device_page(page) &&
- page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA;
-}
-#else /* CONFIG_PCI_P2PDMA */
-static inline bool is_pci_p2pdma_page(const struct page *page)
-{
- return false;
-}
-#endif /* CONFIG_PCI_P2PDMA */
-
#else /* CONFIG_DEV_PAGEMAP_OPS */
static inline bool put_devmap_managed_page(struct page *page)
{
return false;
}
+#endif /* CONFIG_DEV_PAGEMAP_OPS */
static inline bool is_device_private_page(const struct page *page)
{
- return false;
+ return IS_ENABLED(CONFIG_DEV_PAGEMAP_OPS) &&
+ IS_ENABLED(CONFIG_DEVICE_PRIVATE) &&
+ is_zone_device_page(page) &&
+ page->pgmap->type == MEMORY_DEVICE_PRIVATE;
}
static inline bool is_pci_p2pdma_page(const struct page *page)
{
- return false;
+ return IS_ENABLED(CONFIG_DEV_PAGEMAP_OPS) &&
+ IS_ENABLED(CONFIG_PCI_P2PDMA) &&
+ is_zone_device_page(page) &&
+ page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA;
}
-#endif /* CONFIG_DEV_PAGEMAP_OPS */
/* 127: arbitrary random number, small enough to assemble well */
#define page_ref_zero_or_close_to_overflow(page) \
int get_user_pages_fast(unsigned long start, int nr_pages,
unsigned int gup_flags, struct page **pages);
+int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc);
+int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc,
+ struct task_struct *task, bool bypass_rlim);
+
/* Container for pinned pfns / pages */
struct frame_vector {
unsigned int nr_allocated; /* Number of frames we have space for */
int get_cmdline(struct task_struct *task, char *buffer, int buflen);
-static inline bool vma_is_anonymous(struct vm_area_struct *vma)
-{
- return !vma->vm_ops;
-}
-
-#ifdef CONFIG_SHMEM
-/*
- * The vma_is_shmem is not inline because it is used only by slow
- * paths in userfault.
- */
-bool vma_is_shmem(struct vm_area_struct *vma);
-#else
-static inline bool vma_is_shmem(struct vm_area_struct *vma) { return false; }
-#endif
-
-int vma_is_stack_for_current(struct vm_area_struct *vma);
-
extern unsigned long move_page_tables(struct vm_area_struct *vma,
unsigned long old_addr, struct vm_area_struct *new_vma,
unsigned long new_addr, unsigned long len,
}
#endif
-#ifndef __HAVE_ARCH_PTE_DEVMAP
+#ifndef CONFIG_ARCH_HAS_PTE_DEVMAP
static inline int pte_devmap(pte_t pte)
{
return 0;
#endif
const char * arch_vma_name(struct vm_area_struct *vma);
+#ifdef CONFIG_MMU
void print_vma_addr(char *prefix, unsigned long rip);
+#else
+static inline void print_vma_addr(char *prefix, unsigned long rip)
+{
+}
+#endif
void *sparse_buffer_alloc(unsigned long size);
-struct page *sparse_mem_map_populate(unsigned long pnum, int nid,
- struct vmem_altmap *altmap);
+struct page * __populate_section_memmap(unsigned long pfn,
+ unsigned long nr_pages, int nid, struct vmem_altmap *altmap);
pgd_t *vmemmap_pgd_populate(unsigned long addr, int node);
p4d_t *vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node);
pud_t *vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node);
*/
#define zone_idx(zone) ((zone) - (zone)->zone_pgdat->node_zones)
-#ifdef CONFIG_ZONE_DEVICE
-static inline bool is_dev_zone(const struct zone *zone)
-{
- return zone_idx(zone) == ZONE_DEVICE;
-}
-#else
-static inline bool is_dev_zone(const struct zone *zone)
-{
- return false;
-}
-#endif
-
/*
* Returns true if a zone has pages managed by the buddy allocator.
* All the reclaim decisions have to use this function rather than
#define SECTION_ALIGN_UP(pfn) (((pfn) + PAGES_PER_SECTION - 1) & PAGE_SECTION_MASK)
#define SECTION_ALIGN_DOWN(pfn) ((pfn) & PAGE_SECTION_MASK)
+#define SUBSECTION_SHIFT 21
+
+#define PFN_SUBSECTION_SHIFT (SUBSECTION_SHIFT - PAGE_SHIFT)
+#define PAGES_PER_SUBSECTION (1UL << PFN_SUBSECTION_SHIFT)
+#define PAGE_SUBSECTION_MASK (~(PAGES_PER_SUBSECTION-1))
+
+#if SUBSECTION_SHIFT > SECTION_SIZE_BITS
+#error Subsection size exceeds section size
+#else
+#define SUBSECTIONS_PER_SECTION (1UL << (SECTION_SIZE_BITS - SUBSECTION_SHIFT))
+#endif
+
+#define SUBSECTION_ALIGN_UP(pfn) ALIGN((pfn), PAGES_PER_SUBSECTION)
+#define SUBSECTION_ALIGN_DOWN(pfn) ((pfn) & PAGE_SUBSECTION_MASK)
+
+struct mem_section_usage {
+ DECLARE_BITMAP(subsection_map, SUBSECTIONS_PER_SECTION);
+ /* See declaration of similar field in struct zone */
+ unsigned long pageblock_flags[0];
+};
+
+void subsection_map_init(unsigned long pfn, unsigned long nr_pages);
+
struct page;
struct page_ext;
struct mem_section {
*/
unsigned long section_mem_map;
- /* See declaration of similar field in struct zone */
- unsigned long *pageblock_flags;
+ struct mem_section_usage *usage;
#ifdef CONFIG_PAGE_EXTENSION
/*
* If SPARSEMEM, pgdat doesn't have page_ext pointer. We use
extern struct mem_section mem_section[NR_SECTION_ROOTS][SECTIONS_PER_ROOT];
#endif
+static inline unsigned long *section_to_usemap(struct mem_section *ms)
+{
+ return ms->usage->pageblock_flags;
+}
+
static inline struct mem_section *__nr_to_section(unsigned long nr)
{
#ifdef CONFIG_SPARSEMEM_EXTREME
return NULL;
return &mem_section[SECTION_NR_TO_ROOT(nr)][nr & SECTION_ROOT_MASK];
}
-extern int __section_nr(struct mem_section* ms);
-extern unsigned long usemap_size(void);
+extern unsigned long __section_nr(struct mem_section *ms);
+extern size_t mem_section_usage_size(void);
/*
* We use the lower bits of the mem_map pointer to store
#define SECTION_MARKED_PRESENT (1UL<<0)
#define SECTION_HAS_MEM_MAP (1UL<<1)
#define SECTION_IS_ONLINE (1UL<<2)
-#define SECTION_MAP_LAST_BIT (1UL<<3)
+#define SECTION_IS_EARLY (1UL<<3)
+#define SECTION_MAP_LAST_BIT (1UL<<4)
#define SECTION_MAP_MASK (~(SECTION_MAP_LAST_BIT-1))
#define SECTION_NID_SHIFT 3
return (section && (section->section_mem_map & SECTION_HAS_MEM_MAP));
}
+static inline int early_section(struct mem_section *section)
+{
+ return (section && (section->section_mem_map & SECTION_IS_EARLY));
+}
+
static inline int valid_section_nr(unsigned long nr)
{
return valid_section(__nr_to_section(nr));
return __nr_to_section(pfn_to_section_nr(pfn));
}
-extern int __highest_present_section_nr;
+extern unsigned long __highest_present_section_nr;
+
+static inline int subsection_map_index(unsigned long pfn)
+{
+ return (pfn & ~(PAGE_SECTION_MASK)) / PAGES_PER_SUBSECTION;
+}
+
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+static inline int pfn_section_valid(struct mem_section *ms, unsigned long pfn)
+{
+ int idx = subsection_map_index(pfn);
+
+ return test_bit(idx, ms->usage->subsection_map);
+}
+#else
+static inline int pfn_section_valid(struct mem_section *ms, unsigned long pfn)
+{
+ return 1;
+}
+#endif
#ifndef CONFIG_HAVE_ARCH_PFN_VALID
static inline int pfn_valid(unsigned long pfn)
{
+ struct mem_section *ms;
+
if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS)
return 0;
- return valid_section(__nr_to_section(pfn_to_section_nr(pfn)));
+ ms = __nr_to_section(pfn_to_section_nr(pfn));
+ if (!valid_section(ms))
+ return 0;
+ /*
+ * Traditionally early sections always returned pfn_valid() for
+ * the entire section-sized span.
+ */
+ return early_section(ms) || pfn_section_valid(ms, pfn);
}
#endif
#define sparse_init() do {} while (0)
#define sparse_index_init(_sec, _nid) do {} while (0)
#define pfn_present pfn_valid
+#define subsection_map_init(_pfn, _nr_pages) do {} while (0)
#endif /* CONFIG_SPARSEMEM */
/*
/* Free memory returned from module_alloc. */
void module_memfree(void *module_region);
+/* Determines if the section name is an exit section (that is only used during
+ * module unloading)
+ */
+bool module_exit_section(const char *name);
+
/*
* Apply the given relocation to the (simplified) ELF. Return -error
* or 0.
/*
* See kernel/locking/mutex.c for detailed documentation of these APIs.
- * Also see Documentation/locking/mutex-design.txt.
+ * Also see Documentation/locking/mutex-design.rst.
*/
#ifdef CONFIG_DEBUG_LOCK_ALLOC
extern void mutex_lock_nested(struct mutex *lock, unsigned int subclass);
PNFS_UPDATE_LAYOUT_BLOCKED,
PNFS_UPDATE_LAYOUT_INVALID_OPEN,
PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET,
+ PNFS_UPDATE_LAYOUT_EXIT,
};
#define NFS4_OP_MAP_NUM_LONGS \
#define NFS_INO_INVALID_MTIME BIT(10) /* cached mtime is invalid */
#define NFS_INO_INVALID_SIZE BIT(11) /* cached size is invalid */
#define NFS_INO_INVALID_OTHER BIT(12) /* other attrs are invalid */
+#define NFS_INO_DATA_INVAL_DEFER \
+ BIT(13) /* Deferred cache invalidation */
#define NFS_INO_INVALID_ATTR (NFS_INO_INVALID_CHANGE \
| NFS_INO_INVALID_CTIME \
struct nfs_subversion * cl_nfs_mod; /* pointer to nfs version module */
u32 cl_minorversion;/* NFSv4 minorversion */
+ unsigned int cl_nconnect; /* Number of connections */
const char * cl_principal; /* used for machine cred */
#if IS_ENABLED(CONFIG_NFS_V4)
extern void unregister_one_node(int nid);
extern int register_cpu_under_node(unsigned int cpu, unsigned int nid);
extern int unregister_cpu_under_node(unsigned int cpu, unsigned int nid);
-extern int register_mem_sect_under_node(struct memory_block *mem_blk,
- void *arg);
-extern int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,
- unsigned long phys_index);
+extern void unregister_memory_block_under_nodes(struct memory_block *mem_blk);
extern int register_memory_node_under_compute_node(unsigned int mem_nid,
unsigned int cpu_nid,
{
return 0;
}
-static inline int register_mem_sect_under_node(struct memory_block *mem_blk,
- void *arg)
+static inline void unregister_memory_block_under_nodes(struct memory_block *mem_blk)
{
- return 0;
-}
-static inline int unregister_mem_sect_under_nodes(struct memory_block *mem_blk,
- unsigned long phys_index)
-{
- return 0;
}
static inline void register_hugetlbfs_with_node(node_registration_func_t reg,
#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3 0xabcd
#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3_AXI 0xabce
#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB31 0xabcf
+#define PCI_DEVICE_ID_SYNOPSYS_EDDA 0xedda
#define PCI_VENDOR_ID_USR 0x16ec
#endif
#endif
-#ifdef __HAVE_ARCH_PTE_DEVMAP
+#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP
static inline bool pfn_t_devmap(pfn_t pfn)
{
const u64 flags = PFN_DEV|PFN_MAP;
defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)
pud_t pud_mkdevmap(pud_t pud);
#endif
-#endif /* __HAVE_ARCH_PTE_DEVMAP */
+#endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */
#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL
static inline bool pfn_t_special(pfn_t pfn)
#include <linux/rculist.h>
#include <linux/wait.h>
+#include <linux/refcount.h>
enum pid_type
{
struct pid
{
- atomic_t count;
+ refcount_t count;
unsigned int level;
/* lists of tasks that use this pid */
struct hlist_head tasks[PIDTYPE_MAX];
static inline struct pid *get_pid(struct pid *pid)
{
if (pid)
- atomic_inc(&pid->count);
+ refcount_inc(&pid->count);
return pid;
}
int dma_request2; /* secondary DMA request line */
enum sdma_peripheral_type peripheral_type;
int priority;
- struct device_node *of_node;
};
static inline int imx_dma_is_ipu(struct dma_chan *chan)
#define ASUS_WMI_DEVID_KBD_BACKLIGHT 0x00050021
#define ASUS_WMI_DEVID_LIGHT_SENSOR 0x00050022 /* ?? */
#define ASUS_WMI_DEVID_LIGHTBAR 0x00050025
-#define ASUS_WMI_DEVID_FAN_MODE 0x00110018
+#define ASUS_WMI_DEVID_FAN_BOOST_MODE 0x00110018
/* Misc */
#define ASUS_WMI_DEVID_CAMERA 0x00060013
*
* Copyright (c) 2001-2003 Patrick Mochel <mochel@osdl.org>
*
- * See Documentation/driver-model/ for more information.
+ * See Documentation/driver-api/driver-model/ for more information.
*/
#ifndef _PLATFORM_DEVICE_H_
#define PM_QOS_RESUME_LATENCY_NO_CONSTRAINT PM_QOS_LATENCY_ANY
#define PM_QOS_RESUME_LATENCY_NO_CONSTRAINT_NS PM_QOS_LATENCY_ANY_NS
#define PM_QOS_LATENCY_TOLERANCE_DEFAULT_VALUE 0
+#define PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE 0
+#define PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE (-1)
#define PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT (-1)
#define PM_QOS_FLAG_NO_POWER_OFF (1 << 0)
enum dev_pm_qos_req_type {
DEV_PM_QOS_RESUME_LATENCY = 1,
DEV_PM_QOS_LATENCY_TOLERANCE,
+ DEV_PM_QOS_MIN_FREQUENCY,
+ DEV_PM_QOS_MAX_FREQUENCY,
DEV_PM_QOS_FLAGS,
};
struct dev_pm_qos {
struct pm_qos_constraints resume_latency;
struct pm_qos_constraints latency_tolerance;
+ struct pm_qos_constraints min_frequency;
+ struct pm_qos_constraints max_frequency;
struct pm_qos_flags flags;
struct dev_pm_qos_request *resume_latency_req;
struct dev_pm_qos_request *latency_tolerance_req;
struct dev_pm_qos_request *flags_req;
+ struct dev_pm_qos_request *min_frequency_req;
+ struct dev_pm_qos_request *max_frequency_req;
};
/* Action requested to pm_qos_update_target */
#ifdef CONFIG_PM
enum pm_qos_flags_status __dev_pm_qos_flags(struct device *dev, s32 mask);
enum pm_qos_flags_status dev_pm_qos_flags(struct device *dev, s32 mask);
-s32 __dev_pm_qos_read_value(struct device *dev);
-s32 dev_pm_qos_read_value(struct device *dev);
+s32 __dev_pm_qos_resume_latency(struct device *dev);
+s32 dev_pm_qos_read_value(struct device *dev, enum dev_pm_qos_req_type type);
int dev_pm_qos_add_request(struct device *dev, struct dev_pm_qos_request *req,
enum dev_pm_qos_req_type type, s32 value);
int dev_pm_qos_update_request(struct dev_pm_qos_request *req, s32 new_value);
int dev_pm_qos_remove_request(struct dev_pm_qos_request *req);
int dev_pm_qos_add_notifier(struct device *dev,
- struct notifier_block *notifier);
+ struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type);
int dev_pm_qos_remove_notifier(struct device *dev,
- struct notifier_block *notifier);
+ struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type);
void dev_pm_qos_constraints_init(struct device *dev);
void dev_pm_qos_constraints_destroy(struct device *dev);
int dev_pm_qos_add_ancestor_request(struct device *dev,
return dev->power.qos->flags_req->data.flr.flags;
}
-static inline s32 dev_pm_qos_raw_read_value(struct device *dev)
+static inline s32 dev_pm_qos_raw_resume_latency(struct device *dev)
{
return IS_ERR_OR_NULL(dev->power.qos) ?
PM_QOS_RESUME_LATENCY_NO_CONSTRAINT :
static inline enum pm_qos_flags_status dev_pm_qos_flags(struct device *dev,
s32 mask)
{ return PM_QOS_FLAGS_UNDEFINED; }
-static inline s32 __dev_pm_qos_read_value(struct device *dev)
- { return PM_QOS_RESUME_LATENCY_NO_CONSTRAINT; }
-static inline s32 dev_pm_qos_read_value(struct device *dev)
+static inline s32 __dev_pm_qos_resume_latency(struct device *dev)
{ return PM_QOS_RESUME_LATENCY_NO_CONSTRAINT; }
+static inline s32 dev_pm_qos_read_value(struct device *dev,
+ enum dev_pm_qos_req_type type)
+{
+ switch (type) {
+ case DEV_PM_QOS_RESUME_LATENCY:
+ return PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
+ case DEV_PM_QOS_MIN_FREQUENCY:
+ return PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE;
+ case DEV_PM_QOS_MAX_FREQUENCY:
+ return PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE;
+ default:
+ WARN_ON(1);
+ return 0;
+ }
+}
+
static inline int dev_pm_qos_add_request(struct device *dev,
struct dev_pm_qos_request *req,
enum dev_pm_qos_req_type type,
static inline int dev_pm_qos_remove_request(struct dev_pm_qos_request *req)
{ return 0; }
static inline int dev_pm_qos_add_notifier(struct device *dev,
- struct notifier_block *notifier)
+ struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type)
{ return 0; }
static inline int dev_pm_qos_remove_notifier(struct device *dev,
- struct notifier_block *notifier)
+ struct notifier_block *notifier,
+ enum dev_pm_qos_req_type type)
{ return 0; }
static inline void dev_pm_qos_constraints_init(struct device *dev)
{
return PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
}
static inline s32 dev_pm_qos_requested_flags(struct device *dev) { return 0; }
-static inline s32 dev_pm_qos_raw_read_value(struct device *dev)
+static inline s32 dev_pm_qos_raw_resume_latency(struct device *dev)
{
return PM_QOS_RESUME_LATENCY_NO_CONSTRAINT;
}
* non-initialized list entries.
*/
#define LIST_POISON1 ((void *) 0x100 + POISON_POINTER_DELTA)
-#define LIST_POISON2 ((void *) 0x200 + POISON_POINTER_DELTA)
+#define LIST_POISON2 ((void *) 0x122 + POISON_POINTER_DELTA)
/********** include/linux/timer.h **********/
/*
--- /dev/null
+#ifndef __LINUX_PSEUDO_FS__
+#define __LINUX_PSEUDO_FS__
+
+#include <linux/fs_context.h>
+
+struct pseudo_fs_context {
+ const struct super_operations *ops;
+ const struct xattr_handler **xattr;
+ const struct dentry_operations *dops;
+ unsigned long magic;
+};
+
+struct pseudo_fs_context *init_pseudo(struct fs_context *fc,
+ unsigned long magic);
+
+#endif
extern const struct file_operations ramfs_file_operations;
extern const struct vm_operations_struct generic_file_vm_ops;
-extern int __init init_ramfs_fs(void);
int ramfs_fill_super(struct super_block *sb, void *data, int silent);
struct rb_node *rb_node;
};
-/*
- * Leftmost-cached rbtrees.
- *
- * We do not cache the rightmost node based on footprint
- * size vs number of potential users that could benefit
- * from O(1) rb_last(). Just not worth it, users that want
- * this feature can always implement the logic explicitly.
- * Furthermore, users that want to cache both pointers may
- * find it a bit asymmetric, but that's ok.
- */
-struct rb_root_cached {
- struct rb_root rb_root;
- struct rb_node *rb_leftmost;
-};
-
#define rb_parent(r) ((struct rb_node *)((r)->__rb_parent_color & ~3))
#define RB_ROOT (struct rb_root) { NULL, }
-#define RB_ROOT_CACHED (struct rb_root_cached) { {NULL, }, NULL }
#define rb_entry(ptr, type, member) container_of(ptr, type, member)
#define RB_EMPTY_ROOT(root) (READ_ONCE((root)->rb_node) == NULL)
extern struct rb_node *rb_first(const struct rb_root *);
extern struct rb_node *rb_last(const struct rb_root *);
-extern void rb_insert_color_cached(struct rb_node *,
- struct rb_root_cached *, bool);
-extern void rb_erase_cached(struct rb_node *node, struct rb_root_cached *);
-/* Same as rb_first(), but O(1) */
-#define rb_first_cached(root) (root)->rb_leftmost
-
/* Postorder iteration - always visit the parent after its children */
extern struct rb_node *rb_first_postorder(const struct rb_root *);
extern struct rb_node *rb_next_postorder(const struct rb_node *);
struct rb_root *root);
extern void rb_replace_node_rcu(struct rb_node *victim, struct rb_node *new,
struct rb_root *root);
-extern void rb_replace_node_cached(struct rb_node *victim, struct rb_node *new,
- struct rb_root_cached *root);
static inline void rb_link_node(struct rb_node *node, struct rb_node *parent,
struct rb_node **rb_link)
typeof(*pos), field); 1; }); \
pos = n)
+/*
+ * Leftmost-cached rbtrees.
+ *
+ * We do not cache the rightmost node based on footprint
+ * size vs number of potential users that could benefit
+ * from O(1) rb_last(). Just not worth it, users that want
+ * this feature can always implement the logic explicitly.
+ * Furthermore, users that want to cache both pointers may
+ * find it a bit asymmetric, but that's ok.
+ */
+struct rb_root_cached {
+ struct rb_root rb_root;
+ struct rb_node *rb_leftmost;
+};
+
+#define RB_ROOT_CACHED (struct rb_root_cached) { {NULL, }, NULL }
+
+/* Same as rb_first(), but O(1) */
+#define rb_first_cached(root) (root)->rb_leftmost
+
+static inline void rb_insert_color_cached(struct rb_node *node,
+ struct rb_root_cached *root,
+ bool leftmost)
+{
+ if (leftmost)
+ root->rb_leftmost = node;
+ rb_insert_color(node, &root->rb_root);
+}
+
+static inline void rb_erase_cached(struct rb_node *node,
+ struct rb_root_cached *root)
+{
+ if (root->rb_leftmost == node)
+ root->rb_leftmost = rb_next(node);
+ rb_erase(node, &root->rb_root);
+}
+
+static inline void rb_replace_node_cached(struct rb_node *victim,
+ struct rb_node *new,
+ struct rb_root_cached *root)
+{
+ if (root->rb_leftmost == victim)
+ root->rb_leftmost = new;
+ rb_replace_node(victim, new, &root->rb_root);
+}
+
#endif /* _LINUX_RBTREE_H */
void (*rotate)(struct rb_node *old, struct rb_node *new);
};
-extern void __rb_insert_augmented(struct rb_node *node,
- struct rb_root *root,
- bool newleft, struct rb_node **leftmost,
+extern void __rb_insert_augmented(struct rb_node *node, struct rb_root *root,
void (*augment_rotate)(struct rb_node *old, struct rb_node *new));
+
/*
* Fixup the rbtree and update the augmented information when rebalancing.
*
rb_insert_augmented(struct rb_node *node, struct rb_root *root,
const struct rb_augment_callbacks *augment)
{
- __rb_insert_augmented(node, root, false, NULL, augment->rotate);
+ __rb_insert_augmented(node, root, augment->rotate);
}
static inline void
struct rb_root_cached *root, bool newleft,
const struct rb_augment_callbacks *augment)
{
- __rb_insert_augmented(node, &root->rb_root,
- newleft, &root->rb_leftmost, augment->rotate);
+ if (newleft)
+ root->rb_leftmost = node;
+ rb_insert_augmented(node, &root->rb_root, augment);
}
#define RB_DECLARE_CALLBACKS(rbstatic, rbname, rbstruct, rbfield, \
static __always_inline struct rb_node *
__rb_erase_augmented(struct rb_node *node, struct rb_root *root,
- struct rb_node **leftmost,
const struct rb_augment_callbacks *augment)
{
struct rb_node *child = node->rb_right;
struct rb_node *parent, *rebalance;
unsigned long pc;
- if (leftmost && node == *leftmost)
- *leftmost = rb_next(node);
-
if (!tmp) {
/*
* Case 1: node to erase has no more than 1 child (easy!)
rb_erase_augmented(struct rb_node *node, struct rb_root *root,
const struct rb_augment_callbacks *augment)
{
- struct rb_node *rebalance = __rb_erase_augmented(node, root,
- NULL, augment);
+ struct rb_node *rebalance = __rb_erase_augmented(node, root, augment);
if (rebalance)
__rb_erase_color(rebalance, root, augment->rotate);
}
rb_erase_augmented_cached(struct rb_node *node, struct rb_root_cached *root,
const struct rb_augment_callbacks *augment)
{
- struct rb_node *rebalance = __rb_erase_augmented(node, &root->rb_root,
- &root->rb_leftmost,
- augment);
- if (rebalance)
- __rb_erase_color(rebalance, &root->rb_root, augment->rotate);
+ if (root->rb_leftmost == node)
+ root->rb_leftmost = rb_next(node);
+ rb_erase_augmented(node, &root->rb_root, augment);
}
#endif /* _LINUX_RBTREE_AUGMENTED_H */
* the remote processor will be writing logs.
* @RSC_VDEV: declare support for a virtio device, and serve as its
* virtio header.
- * @RSC_LAST: just keep this one at the end
+ * @RSC_LAST: just keep this one at the end of standard resources
+ * @RSC_VENDOR_START: start of the vendor specific resource types range
+ * @RSC_VENDOR_END: end of the vendor specific resource types range
*
* For more details regarding a specific resource type, please see its
* dedicated structure below.
* please update it as needed.
*/
enum fw_resource_type {
- RSC_CARVEOUT = 0,
- RSC_DEVMEM = 1,
- RSC_TRACE = 2,
- RSC_VDEV = 3,
- RSC_LAST = 4,
+ RSC_CARVEOUT = 0,
+ RSC_DEVMEM = 1,
+ RSC_TRACE = 2,
+ RSC_VDEV = 3,
+ RSC_LAST = 4,
+ RSC_VENDOR_START = 128,
+ RSC_VENDOR_END = 512,
};
#define FW_RSC_ADDR_ANY (-1)
struct firmware;
/**
+ * enum rsc_handling_status - return status of rproc_ops handle_rsc hook
+ * @RSC_HANDLED: resource was handled
+ * @RSC_IGNORED: resource was ignored
+ */
+enum rsc_handling_status {
+ RSC_HANDLED = 0,
+ RSC_IGNORED = 1,
+};
+
+/**
* struct rproc_ops - platform-specific device handlers
* @start: power on the device and boot it
* @stop: power off the device
* @kick: kick a virtqueue (virtqueue id given as a parameter)
* @da_to_va: optional platform hook to perform address translations
* @parse_fw: parse firmware to extract information (e.g. resource table)
+ * @handle_rsc: optional platform hook to handle vendor resources. Should return
+ * RSC_HANDLED if resource was handled, RSC_IGNORED if not handled and a
+ * negative value on error
+ * @load_rsc_table: load resource table from firmware image
* @find_loaded_rsc_table: find the loaded resouce table
* @load: load firmware to memory, where the remote processor
* expects to find it
void (*kick)(struct rproc *rproc, int vqid);
void * (*da_to_va)(struct rproc *rproc, u64 da, int len);
int (*parse_fw)(struct rproc *rproc, const struct firmware *fw);
+ int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, void *rsc,
+ int offset, int avail);
struct resource_table *(*find_loaded_rsc_table)(
struct rproc *rproc, const struct firmware *fw);
int (*load)(struct rproc *rproc, const struct firmware *fw);
* static then another method for expressing nested locking is
* the explicit definition of lock class keys and the use of
* lockdep_set_class() at lock initialization time.
- * See Documentation/locking/lockdep-design.txt for more details.)
+ * See Documentation/locking/lockdep-design.rst for more details.)
*/
extern void down_read_nested(struct rw_semaphore *sem, int subclass);
extern void down_write_nested(struct rw_semaphore *sem, int subclass);
*/
struct sighand_struct {
- refcount_t count;
- struct k_sigaction action[_NSIG];
spinlock_t siglock;
+ refcount_t count;
wait_queue_head_t signalfd_wqh;
+ struct k_sigaction action[_NSIG];
};
/*
static inline void set_restore_sigmask(void)
{
set_thread_flag(TIF_RESTORE_SIGMASK);
- WARN_ON(!test_thread_flag(TIF_SIGPENDING));
}
static inline void clear_tsk_restore_sigmask(struct task_struct *task)
static inline void set_restore_sigmask(void)
{
current->restore_sigmask = true;
- WARN_ON(!test_thread_flag(TIF_SIGPENDING));
}
static inline void clear_tsk_restore_sigmask(struct task_struct *task)
{
__set_current_blocked(¤t->saved_sigmask);
}
+extern int set_user_sigmask(const sigset_t __user *umask, size_t sigsetsize);
+
+static inline void restore_saved_sigmask_unless(bool interrupted)
+{
+ if (interrupted)
+ WARN_ON(!test_thread_flag(TIF_SIGPENDING));
+ else
+ restore_saved_sigmask();
+}
+
static inline sigset_t *sigmask_to_save(void)
{
sigset_t *res = ¤t->blocked;
extern void exit_itimers(struct signal_struct *);
extern long _do_fork(struct kernel_clone_args *kargs);
+extern bool legacy_clone_args_valid(const struct kernel_clone_args *kargs);
extern long do_fork(unsigned long, unsigned long, unsigned long, int __user *, int __user *);
struct task_struct *fork_idle(int);
struct mm_struct *copy_init_mm(void);
/*
* This structure describes all the operations that can be done on the
- * physical hardware. See Documentation/serial/driver.rst for details.
+ * physical hardware. See Documentation/driver-api/serial/driver.rst for details.
*/
struct uart_ops {
unsigned int (*tx_empty)(struct uart_port *);
struct task_struct *p, enum pid_type type);
extern int __group_send_sig_info(int, struct kernel_siginfo *, struct task_struct *);
extern int sigprocmask(int, sigset_t *, sigset_t *);
-extern int set_user_sigmask(const sigset_t __user *usigmask, sigset_t *set,
- sigset_t *oldset, size_t sigsetsize);
-extern void restore_user_sigmask(const void __user *usigmask,
- sigset_t *sigsaved, bool interrupted);
extern void set_current_blocked(sigset_t *);
extern void __set_current_blocked(const sigset_t *);
extern int show_unhandled_signals;
const char *fw_name, int pas_id, void *mem_region,
phys_addr_t mem_phys, size_t mem_size,
phys_addr_t *reloc_base);
+void *qcom_mdt_read_metadata(const struct firmware *fw, size_t *data_len);
+
#endif
* managed by driver for that purpose.
*/
struct ti_sci_clk_ops {
- int (*get_clock)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*get_clock)(const struct ti_sci_handle *handle, u32 did, u32 cid,
bool needs_ssc, bool can_change_freq,
bool enable_input_term);
- int (*idle_clock)(const struct ti_sci_handle *handle, u32 did, u8 cid);
- int (*put_clock)(const struct ti_sci_handle *handle, u32 did, u8 cid);
- int (*is_auto)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*idle_clock)(const struct ti_sci_handle *handle, u32 did, u32 cid);
+ int (*put_clock)(const struct ti_sci_handle *handle, u32 did, u32 cid);
+ int (*is_auto)(const struct ti_sci_handle *handle, u32 did, u32 cid,
bool *req_state);
- int (*is_on)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*is_on)(const struct ti_sci_handle *handle, u32 did, u32 cid,
bool *req_state, bool *current_state);
- int (*is_off)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*is_off)(const struct ti_sci_handle *handle, u32 did, u32 cid,
bool *req_state, bool *current_state);
- int (*set_parent)(const struct ti_sci_handle *handle, u32 did, u8 cid,
- u8 parent_id);
- int (*get_parent)(const struct ti_sci_handle *handle, u32 did, u8 cid,
- u8 *parent_id);
+ int (*set_parent)(const struct ti_sci_handle *handle, u32 did, u32 cid,
+ u32 parent_id);
+ int (*get_parent)(const struct ti_sci_handle *handle, u32 did, u32 cid,
+ u32 *parent_id);
int (*get_num_parents)(const struct ti_sci_handle *handle, u32 did,
- u8 cid, u8 *num_parents);
+ u32 cid, u32 *num_parents);
int (*get_best_match_freq)(const struct ti_sci_handle *handle, u32 did,
- u8 cid, u64 min_freq, u64 target_freq,
+ u32 cid, u64 min_freq, u64 target_freq,
u64 max_freq, u64 *match_freq);
- int (*set_freq)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*set_freq)(const struct ti_sci_handle *handle, u32 did, u32 cid,
u64 min_freq, u64 target_freq, u64 max_freq);
- int (*get_freq)(const struct ti_sci_handle *handle, u32 did, u8 cid,
+ int (*get_freq)(const struct ti_sci_handle *handle, u32 did, u32 cid,
u64 *current_freq);
};
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0-only */
-/*
- * Header for the SUDMAC driver
- *
- * Copyright (C) 2013 Renesas Solutions Corp.
- */
-#ifndef SUDMAC_H
-#define SUDMAC_H
-
-#include <linux/dmaengine.h>
-#include <linux/shdma-base.h>
-#include <linux/types.h>
-
-/* Used by slave DMA clients to request DMA to/from a specific peripheral */
-struct sudmac_slave {
- struct shdma_slave shdma_slave; /* Set by the platform */
-};
-
-/*
- * Supplied by platforms to specify, how a DMA channel has to be configured for
- * a certain peripheral
- */
-struct sudmac_slave_config {
- int slave_id;
-};
-
-struct sudmac_channel {
- unsigned long offset;
- unsigned long config;
- unsigned long wait; /* The configuable range is 0 to 3 */
- unsigned long dint_end_bit;
-};
-
-struct sudmac_pdata {
- const struct sudmac_slave_config *slave;
- int slave_num;
- const struct sudmac_channel *channel;
- int channel_num;
-};
-
-/* Definitions for the sudmac_channel.config */
-#define SUDMAC_TX_BUFFER_MODE BIT(0)
-#define SUDMAC_RX_END_MODE BIT(1)
-
-/* Definitions for the sudmac_channel.dint_end_bit */
-#define SUDMAC_DMA_BIT_CH0 BIT(0)
-#define SUDMAC_DMA_BIT_CH1 BIT(1)
-
-#endif
int xprt_setup_bc(struct rpc_xprt *xprt, unsigned int min_reqs);
void xprt_destroy_bc(struct rpc_xprt *xprt, unsigned int max_reqs);
void xprt_free_bc_rqst(struct rpc_rqst *req);
+unsigned int xprt_bc_max_slots(struct rpc_xprt *xprt);
/*
* Determine if a shared backchannel is in use
u32 prognumber; /* overrides program->number */
u32 version;
rpc_authflavor_t authflavor;
+ u32 nconnect;
unsigned long flags;
char *client_name;
struct svc_xprt *bc_xprt; /* NFSv4.1 backchannel */
void rpc_release_client(struct rpc_clnt *);
void rpc_task_release_transport(struct rpc_task *);
void rpc_task_release_client(struct rpc_task *);
+struct rpc_xprt *rpc_task_get_xprt(struct rpc_clnt *clnt,
+ struct rpc_xprt *xprt);
int rpcb_create_local(struct net *);
void rpcb_put_local(struct net *);
struct net * rpc_net_ns(struct rpc_clnt *);
size_t rpc_max_payload(struct rpc_clnt *);
size_t rpc_max_bc_payload(struct rpc_clnt *);
+unsigned int rpc_num_bc_slots(struct rpc_clnt *);
void rpc_force_rebind(struct rpc_clnt *);
size_t rpc_peeraddr(struct rpc_clnt *, struct sockaddr *, size_t);
const char *rpc_peeraddr2str(struct rpc_clnt *, enum rpc_display_format_t);
#include <linux/ktime.h>
#include <linux/spinlock.h>
-#define RPC_IOSTATS_VERS "1.0"
+#define RPC_IOSTATS_VERS "1.1"
struct rpc_iostats {
spinlock_t om_lock;
ktime_t om_queue, /* queued for xmit */
om_rtt, /* RPC RTT */
om_execute; /* RPC execution */
+ /*
+ * The count of operations that complete with tk_status < 0.
+ * These statuses usually indicate error conditions.
+ */
+ unsigned long om_error_status;
} ____cacheline_aligned;
struct rpc_task;
#define RPC_CALL_MAJORSEEN 0x0020 /* major timeout seen */
#define RPC_TASK_ROOTCREDS 0x0040 /* force root creds */
#define RPC_TASK_DYNAMIC 0x0080 /* task was kmalloc'ed */
+#define RPC_TASK_NO_ROUND_ROBIN 0x0100 /* send requests on "main" xprt */
#define RPC_TASK_SOFT 0x0200 /* Use soft timeouts */
#define RPC_TASK_SOFTCONN 0x0400 /* Fail if can't connect */
#define RPC_TASK_SENT 0x0800 /* message was sent */
#define RPC_NR_PRIORITY (1 + RPC_PRIORITY_PRIVILEGED - RPC_PRIORITY_LOW)
struct rpc_timer {
- struct timer_list timer;
struct list_head list;
+ unsigned long expires;
+ struct delayed_work dwork;
};
/*
int (*bc_setup)(struct rpc_xprt *xprt,
unsigned int min_reqs);
size_t (*bc_maxpayload)(struct rpc_xprt *xprt);
+ unsigned int (*bc_num_slots)(struct rpc_xprt *xprt);
void (*bc_free_rqst)(struct rpc_rqst *rqst);
void (*bc_destroy)(struct rpc_xprt *xprt,
unsigned int max_reqs);
/*
* Send stuff
*/
+ atomic_long_t queuelen;
spinlock_t transport_lock; /* lock transport info */
spinlock_t reserve_lock; /* lock slot table */
spinlock_t queue_lock; /* send/receive queue lock */
#if defined(CONFIG_SUNRPC_BACKCHANNEL)
struct svc_serv *bc_serv; /* The RPC service which will */
/* process the callback */
- int bc_alloc_count; /* Total number of preallocs */
- atomic_t bc_free_slots;
+ unsigned int bc_alloc_max;
+ unsigned int bc_alloc_count; /* Total number of preallocs */
+ atomic_t bc_slot_count; /* Number of allocated slots */
spinlock_t bc_pa_lock; /* Protects the preallocated
* items */
struct list_head bc_pa_list; /* List of preallocated
*/
struct rpc_xprt *xprt_create_transport(struct xprt_create *args);
void xprt_connect(struct rpc_task *task);
+unsigned long xprt_reconnect_delay(const struct rpc_xprt *xprt);
+void xprt_reconnect_backoff(struct rpc_xprt *xprt,
+ unsigned long init_to);
void xprt_reserve(struct rpc_task *task);
void xprt_retry_reserve(struct rpc_task *task);
int xprt_reserve_xprt(struct rpc_xprt *xprt, struct rpc_task *task);
struct kref xps_kref;
unsigned int xps_nxprts;
+ unsigned int xps_nactive;
+ atomic_long_t xps_queuelen;
struct list_head xps_xprt_list;
struct net * xps_net;
*/
unsigned long sock_state;
struct delayed_work connect_worker;
+ struct work_struct error_worker;
struct work_struct recv_worker;
struct mutex recv_mutex;
struct sockaddr_storage srcaddr;
#define XPRT_SOCK_CONNECTING 1U
#define XPRT_SOCK_DATA_READY (2)
#define XPRT_SOCK_UPD_TIMEOUT (3)
+#define XPRT_SOCK_WAKE_ERROR (4)
+#define XPRT_SOCK_WAKE_WRITE (5)
+#define XPRT_SOCK_WAKE_PENDING (6)
+#define XPRT_SOCK_WAKE_DISCONNECT (7)
#endif /* __KERNEL__ */
#include <linux/bug.h>
#include <linux/mm_types.h>
+#ifdef CONFIG_MMU
+
/*
* swapcache pages are stored in the swapper_space radix tree. We want to
* get good packing density in that tree, so the index should be dense in
return entry.val & SWP_OFFSET_MASK;
}
-#ifdef CONFIG_MMU
/* check whether a pte points to a swap entry */
static inline int is_swap_pte(pte_t pte)
{
return !pte_none(pte) && !pte_present(pte);
}
-#endif
/*
* Convert the arch-dependent pte representation of a swp_entry_t into an
}
#endif
+#endif /* CONFIG_MMU */
#endif /* _LINUX_SWAPOPS_H */
* the ksys_xyzyyz() functions prototyped below.
*/
-int ksys_mount(char __user *dev_name, char __user *dir_name, char __user *type,
- unsigned long flags, void __user *data);
+int ksys_mount(const char __user *dev_name, const char __user *dir_name,
+ const char __user *type, unsigned long flags, void __user *data);
int ksys_umount(char __user *name, int flags);
int ksys_dup(unsigned int fildes);
int ksys_chroot(const char __user *filename);
struct ctl_table_header;
struct ctl_dir;
+/* Keep the same order as in fs/proc/proc_sysctl.c */
+#define SYSCTL_ZERO ((void *)&sysctl_vals[0])
+#define SYSCTL_ONE ((void *)&sysctl_vals[1])
+#define SYSCTL_INT_MAX ((void *)&sysctl_vals[2])
+
+extern const int sysctl_vals[];
+
typedef int proc_handler (struct ctl_table *ctl, int write,
void __user *buffer, size_t *lenp, loff_t *ppos);
* platform characterization. This value is relative to the
* rest of the weights so a cooling device whose weight is
* double that of another cooling device is twice as
- * effective. See Documentation/thermal/sysfs-api.txt for more
+ * effective. See Documentation/thermal/sysfs-api.rst for more
* information.
*/
int weight;
/*
* This is a bit mask that gives the binding relation between this
* thermal zone and cdev, for a particular trip point.
- * See Documentation/thermal/sysfs-api.txt for more information.
+ * See Documentation/thermal/sysfs-api.rst for more information.
*/
int trip_mask;
enum print_line_t trace_handle_return(struct trace_seq *s);
void tracing_generic_entry_update(struct trace_entry *entry,
+ unsigned short type,
unsigned long flags,
int pc);
struct trace_event_file;
return call->name;
}
+static inline struct list_head *
+trace_get_fields(struct trace_event_call *event_call)
+{
+ if (!event_call->class->get_fields)
+ return &event_call->class->fields;
+ return event_call->class->get_fields(event_call);
+}
+
struct trace_array;
struct trace_subsystem_dir;
/*
* ptrace report for syscall entry and exit looks identical.
*/
-static inline int ptrace_report_syscall(struct pt_regs *regs)
+static inline int ptrace_report_syscall(struct pt_regs *regs,
+ unsigned long message)
{
int ptrace = current->ptrace;
if (!(ptrace & PT_PTRACED))
return 0;
+ current->ptrace_message = message;
ptrace_notify(SIGTRAP | ((ptrace & PT_TRACESYSGOOD) ? 0x80 : 0));
/*
current->exit_code = 0;
}
+ current->ptrace_message = 0;
return fatal_signal_pending(current);
}
static inline __must_check int tracehook_report_syscall_entry(
struct pt_regs *regs)
{
- return ptrace_report_syscall(regs);
+ return ptrace_report_syscall(regs, PTRACE_EVENTMSG_SYSCALL_ENTRY);
}
/**
if (step)
user_single_step_report(regs);
else
- ptrace_report_syscall(regs);
+ ptrace_report_syscall(regs, PTRACE_EVENTMSG_SYSCALL_EXIT);
}
/**
/*
* Is the pagefault handler disabled? If so, user access methods will not sleep.
*/
-#define pagefault_disabled() (current->pagefault_disabled != 0)
+static inline bool pagefault_disabled(void)
+{
+ return current->pagefault_disabled != 0;
+}
/*
* The pagefault handler is in general disabled by pagefault_disable() or
extern long __probe_kernel_read(void *dst, const void *src, size_t size);
/*
+ * probe_user_read(): safely attempt to read from a location in user space
+ * @dst: pointer to the buffer that shall take the data
+ * @src: address to read from
+ * @size: size of the data chunk
+ *
+ * Safely read from address @src to the buffer at @dst. If a kernel fault
+ * happens, handle that and return -EFAULT.
+ */
+extern long probe_user_read(void *dst, const void __user *src, size_t size);
+extern long __probe_user_read(void *dst, const void __user *src, size_t size);
+
+/*
* probe_kernel_write(): safely attempt to write to a location
* @dst: address to write to
* @src: pointer to the data that shall be written
extern long notrace __probe_kernel_write(void *dst, const void *src, size_t size);
extern long strncpy_from_unsafe(char *dst, const void *unsafe_addr, long count);
+extern long strncpy_from_unsafe_user(char *dst, const void __user *unsafe_addr,
+ long count);
+extern long strnlen_unsafe_user(const void __user *unsafe_addr, long count);
/**
* probe_kernel_address(): safely attempt to read from a location
void tcp_get_available_congestion_control(char *buf, size_t len);
void tcp_get_allowed_congestion_control(char *buf, size_t len);
int tcp_set_allowed_congestion_control(char *allowed);
-int tcp_set_congestion_control(struct sock *sk, const char *name, bool load, bool reinit);
+int tcp_set_congestion_control(struct sock *sk, const char *name, bool load,
+ bool reinit, bool cap_net_admin);
u32 tcp_slow_start(struct tcp_sock *tp, u32 acked);
void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w, u32 acked);
unsigned int auto_runtime_pm:1; /* enable automatic codec runtime pm */
unsigned int force_pin_prefix:1; /* Add location prefix */
unsigned int link_down_at_suspend:1; /* link down at runtime suspend */
+ unsigned int relaxed_resume:1; /* don't resume forcibly for jack */
+
#ifdef CONFIG_PM
unsigned long power_on_acct;
unsigned long power_off_acct;
struct btrfs_qgroup;
struct extent_io_tree;
struct prelim_ref;
+struct btrfs_space_info;
TRACE_DEFINE_ENUM(FLUSH_DELAYED_ITEMS_NR);
TRACE_DEFINE_ENUM(FLUSH_DELAYED_ITEMS);
DEFINE_BTRFS_LOCK_EVENT(btrfs_try_tree_write_lock);
DEFINE_BTRFS_LOCK_EVENT(btrfs_tree_read_lock_atomic);
+DECLARE_EVENT_CLASS(btrfs__space_info_update,
+
+ TP_PROTO(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *sinfo, u64 old, s64 diff),
+
+ TP_ARGS(fs_info, sinfo, old, diff),
+
+ TP_STRUCT__entry_btrfs(
+ __field( u64, type )
+ __field( u64, old )
+ __field( s64, diff )
+ ),
+
+ TP_fast_assign_btrfs(fs_info,
+ __entry->type = sinfo->flags;
+ __entry->old = old;
+ __entry->diff = diff;
+ ),
+ TP_printk_btrfs("type=%s old=%llu diff=%lld",
+ __print_flags(__entry->type, "|", BTRFS_GROUP_FLAGS),
+ __entry->old, __entry->diff)
+);
+
+DEFINE_EVENT(btrfs__space_info_update, update_bytes_may_use,
+
+ TP_PROTO(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *sinfo, u64 old, s64 diff),
+
+ TP_ARGS(fs_info, sinfo, old, diff)
+);
+
+DEFINE_EVENT(btrfs__space_info_update, update_bytes_pinned,
+
+ TP_PROTO(struct btrfs_fs_info *fs_info,
+ struct btrfs_space_info *sinfo, u64 old, s64 diff),
+
+ TP_ARGS(fs_info, sinfo, old, diff)
+);
+
#endif /* _TRACE_BTRFS_H */
/* This part must be outside protection */
), \
TP_ARGS(task, mr, nsegs))
-TRACE_DEFINE_ENUM(FRWR_IS_INVALID);
-TRACE_DEFINE_ENUM(FRWR_IS_VALID);
-TRACE_DEFINE_ENUM(FRWR_FLUSHED_FR);
-TRACE_DEFINE_ENUM(FRWR_FLUSHED_LI);
-
-#define xprtrdma_show_frwr_state(x) \
- __print_symbolic(x, \
- { FRWR_IS_INVALID, "INVALID" }, \
- { FRWR_IS_VALID, "VALID" }, \
- { FRWR_FLUSHED_FR, "FLUSHED_FR" }, \
- { FRWR_FLUSHED_LI, "FLUSHED_LI" })
-
DECLARE_EVENT_CLASS(xprtrdma_frwr_done,
TP_PROTO(
const struct ib_wc *wc,
TP_STRUCT__entry(
__field(const void *, mr)
- __field(unsigned int, state)
__field(unsigned int, status)
__field(unsigned int, vendor_err)
),
TP_fast_assign(
__entry->mr = container_of(frwr, struct rpcrdma_mr, frwr);
- __entry->state = frwr->fr_state;
__entry->status = wc->status;
__entry->vendor_err = __entry->status ? wc->vendor_err : 0;
),
TP_printk(
- "mr=%p state=%s: %s (%u/0x%x)",
- __entry->mr, xprtrdma_show_frwr_state(__entry->state),
- rdma_show_wc_status(__entry->status),
+ "mr=%p: %s (%u/0x%x)",
+ __entry->mr, rdma_show_wc_status(__entry->status),
__entry->status, __entry->vendor_err
)
);
DEFINE_RXPRT_EVENT(xprtrdma_op_close);
DEFINE_RXPRT_EVENT(xprtrdma_op_connect);
+TRACE_EVENT(xprtrdma_op_set_cto,
+ TP_PROTO(
+ const struct rpcrdma_xprt *r_xprt,
+ unsigned long connect,
+ unsigned long reconnect
+ ),
+
+ TP_ARGS(r_xprt, connect, reconnect),
+
+ TP_STRUCT__entry(
+ __field(const void *, r_xprt)
+ __field(unsigned long, connect)
+ __field(unsigned long, reconnect)
+ __string(addr, rpcrdma_addrstr(r_xprt))
+ __string(port, rpcrdma_portstr(r_xprt))
+ ),
+
+ TP_fast_assign(
+ __entry->r_xprt = r_xprt;
+ __entry->connect = connect;
+ __entry->reconnect = reconnect;
+ __assign_str(addr, rpcrdma_addrstr(r_xprt));
+ __assign_str(port, rpcrdma_portstr(r_xprt));
+ ),
+
+ TP_printk("peer=[%s]:%s r_xprt=%p: connect=%lu reconnect=%lu",
+ __get_str(addr), __get_str(port), __entry->r_xprt,
+ __entry->connect / HZ, __entry->reconnect / HZ
+ )
+);
+
TRACE_EVENT(xprtrdma_qp_event,
TP_PROTO(
const struct rpcrdma_xprt *r_xprt,
TRACE_EVENT(xprtrdma_marshal,
TP_PROTO(
- const struct rpc_rqst *rqst,
- unsigned int hdrlen,
+ const struct rpcrdma_req *req,
unsigned int rtype,
unsigned int wtype
),
- TP_ARGS(rqst, hdrlen, rtype, wtype),
+ TP_ARGS(req, rtype, wtype),
TP_STRUCT__entry(
__field(unsigned int, task_id)
),
TP_fast_assign(
+ const struct rpc_rqst *rqst = &req->rl_slot;
+
__entry->task_id = rqst->rq_task->tk_pid;
__entry->client_id = rqst->rq_task->tk_client->cl_clid;
__entry->xid = be32_to_cpu(rqst->rq_xid);
- __entry->hdrlen = hdrlen;
+ __entry->hdrlen = req->rl_hdrbuf.len;
__entry->headlen = rqst->rq_snd_buf.head[0].iov_len;
__entry->pagelen = rqst->rq_snd_buf.page_len;
__entry->taillen = rqst->rq_snd_buf.tail[0].iov_len;
)
);
+TRACE_EVENT(xprtrdma_prepsend_failed,
+ TP_PROTO(const struct rpc_rqst *rqst,
+ int ret
+ ),
+
+ TP_ARGS(rqst, ret),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, task_id)
+ __field(unsigned int, client_id)
+ __field(u32, xid)
+ __field(int, ret)
+ ),
+
+ TP_fast_assign(
+ __entry->task_id = rqst->rq_task->tk_pid;
+ __entry->client_id = rqst->rq_task->tk_client->cl_clid;
+ __entry->xid = be32_to_cpu(rqst->rq_xid);
+ __entry->ret = ret;
+ ),
+
+ TP_printk("task:%u@%u xid=0x%08x: ret=%d",
+ __entry->task_id, __entry->client_id, __entry->xid,
+ __entry->ret
+ )
+);
+
TRACE_EVENT(xprtrdma_post_send,
TP_PROTO(
const struct rpcrdma_req *req,
const struct rpc_rqst *rqst = &req->rl_slot;
__entry->task_id = rqst->rq_task->tk_pid;
- __entry->client_id = rqst->rq_task->tk_client->cl_clid;
+ __entry->client_id = rqst->rq_task->tk_client ?
+ rqst->rq_task->tk_client->cl_clid : -1;
__entry->req = req;
__entry->num_sge = req->rl_sendctx->sc_wr.num_sge;
__entry->signaled = req->rl_sendctx->sc_wr.send_flags &
DEFINE_FRWR_DONE_EVENT(xprtrdma_wc_fastreg);
DEFINE_FRWR_DONE_EVENT(xprtrdma_wc_li);
DEFINE_FRWR_DONE_EVENT(xprtrdma_wc_li_wake);
+DEFINE_FRWR_DONE_EVENT(xprtrdma_wc_li_done);
TRACE_EVENT(xprtrdma_frwr_alloc,
TP_PROTO(
#define MAP_TYPE 0x0f /* Mask for type of mapping */
#define MAP_FIXED 0x10 /* Interpret addr exactly */
#define MAP_ANONYMOUS 0x20 /* don't use a file */
-#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED
-# define MAP_UNINITIALIZED 0x4000000 /* For anonymous mmap, memory could be uninitialized */
-#else
-# define MAP_UNINITIALIZED 0x0 /* Don't support this flag */
-#endif
-/* 0x0100 - 0x80000 flags are defined in asm-generic/mman.h */
+/* 0x0100 - 0x4000 flags are defined in asm-generic/mman.h */
+#define MAP_POPULATE 0x008000 /* populate (prefault) pagetables */
+#define MAP_NONBLOCK 0x010000 /* do not block on IO */
+#define MAP_STACK 0x020000 /* give out an address that is best suited for process/thread stacks */
+#define MAP_HUGETLB 0x040000 /* create a huge page mapping */
+#define MAP_SYNC 0x080000 /* perform synchronous page faults for the mapping */
#define MAP_FIXED_NOREPLACE 0x100000 /* MAP_FIXED which doesn't unmap underlying mapping */
+#define MAP_UNINITIALIZED 0x4000000 /* For anonymous mmap, memory could be
+ * uninitialized */
+
/*
* Flags for mlock
*/
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
#define MAP_LOCKED 0x2000 /* pages are locked */
#define MAP_NORESERVE 0x4000 /* don't check for reservations */
-#define MAP_POPULATE 0x8000 /* populate (prefault) pagetables */
-#define MAP_NONBLOCK 0x10000 /* do not block on IO */
-#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
-#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
-#define MAP_SYNC 0x80000 /* perform synchronous page faults for the mapping */
-/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
+/*
+ * Bits [26:31] are reserved, see asm-generic/hugetlb_encode.h
+ * for MAP_HUGETLB usage
+ */
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
__SYSCALL(__NR_fspick, sys_fspick)
#define __NR_pidfd_open 434
__SYSCALL(__NR_pidfd_open, sys_pidfd_open)
+#ifdef __ARCH_WANT_SYS_CLONE3
#define __NR_clone3 435
__SYSCALL(__NR_clone3, sys_clone3)
+#endif
#undef __NR_syscalls
#define __NR_syscalls 436
__u8 log2sharesize:4;
__u8 unused40:4;
__u8 big_flag:1;
- __u8 unused41:1;
+ __u8 unused41:7;
__u8 nzones_high;
+ __u8 reserved43;
__le32 format_version;
__le32 root_size;
__u8 unused52[60 - 52];
-};
+} __attribute__((packed, aligned(4)));
#define ADFS_DISCRECORD (0xc00)
#define ADFS_DR_OFFSET (0x1c0)
#define ADFS_DR_SIZE 60
#define ADFS_DR_SIZE_BITS (ADFS_DR_SIZE << 3)
-
#endif /* _UAPI_ADFS_FS_H */
* based on a user-provided identifier for all traffic coming from
* the tasks belonging to the related cgroup. See also the related
* kernel documentation, available from the Linux sources in file
- * *Documentation/cgroup-v1/net_cls.rst*.
+ * *Documentation/admin-guide/cgroup-v1/net_cls.rst*.
*
* The Linux kernel has two versions for cgroups: there are
* cgroups v1 and cgroups v2. Both are available to users, who can
__u32 user_ip4; /* Allows 1,2,4-byte read and 4-byte write.
* Stored in network byte order.
*/
- __u32 user_ip6[4]; /* Allows 1,2,4-byte read and 4,8-byte write.
+ __u32 user_ip6[4]; /* Allows 1,2,4,8-byte read and 4,8-byte write.
* Stored in network byte order.
*/
__u32 user_port; /* Allows 4-byte read and write.
__u32 msg_src_ip4; /* Allows 1,2,4-byte read and 4-byte write.
* Stored in network byte order.
*/
- __u32 msg_src_ip6[4]; /* Allows 1,2,4-byte read and 4,8-byte write.
+ __u32 msg_src_ip6[4]; /* Allows 1,2,4,8-byte read and 4,8-byte write.
* Stored in network byte order.
*/
__bpf_md_ptr(struct bpf_sock *, sk);
#define BTRFS_BLOCK_GROUP_RAID56_MASK (BTRFS_BLOCK_GROUP_RAID5 | \
BTRFS_BLOCK_GROUP_RAID6)
+#define BTRFS_BLOCK_GROUP_RAID1_MASK (BTRFS_BLOCK_GROUP_RAID1)
+
/*
* We need a bit for restriper to be able to tell when chunks of type
* SINGLE are available. This "extended" profile format is used in
#define inline
-struct timespec {
- long ts_sec;
- long ts_nsec;
-};
#else /* DJGPP but not KERNEL */
#include <sys/time.h>
typedef unsigned long long u_quad_t;
#define cdev_t dev_t
#endif
-#ifdef __CYGWIN32__
-struct timespec {
- time_t tv_sec; /* seconds */
- long tv_nsec; /* nanoseconds */
-};
-#endif
-
#ifndef __BIT_TYPES_DEFINED__
#define __BIT_TYPES_DEFINED__
typedef signed char int8_t;
*/
enum coda_vtype { C_VNON, C_VREG, C_VDIR, C_VBLK, C_VCHR, C_VLNK, C_VSOCK, C_VFIFO, C_VBAD };
+struct coda_timespec {
+ int64_t tv_sec; /* seconds */
+ long tv_nsec; /* nanoseconds */
+};
+
struct coda_vattr {
long va_type; /* vnode type (for create) */
u_short va_mode; /* files access mode and type */
long va_fileid; /* file id */
u_quad_t va_size; /* file size in bytes */
long va_blocksize; /* blocksize preferred for i/o */
- struct timespec va_atime; /* time of last access */
- struct timespec va_mtime; /* time of last modification */
- struct timespec va_ctime; /* time file changed */
+ struct coda_timespec va_atime; /* time of last access */
+ struct coda_timespec va_mtime; /* time of last modification */
+ struct coda_timespec va_ctime; /* time file changed */
u_long va_gen; /* generation number of file */
u_long va_flags; /* flags defined for file */
cdev_t va_rdev; /* device special file represents */
#define CODA_STATFS 34
#define CODA_STORE 35
#define CODA_RELEASE 36
-#define CODA_NCALLS 37
+#define CODA_ACCESS_INTENT 37
+#define CODA_NCALLS 38
#define DOWNCALL(opcode) (opcode >= CODA_REPLACE && opcode <= CODA_PURGEFID)
#define CIOC_KERNEL_VERSION _IOWR('c', 10, size_t)
-#define CODA_KERNEL_VERSION 3 /* 128-bit file identifiers */
+// CODA_KERNEL_VERSION 0 /* don't care about kernel version number */
+// CODA_KERNEL_VERSION 1 /* The old venus 4.6 compatible interface */
+// CODA_KERNEL_VERSION 2 /* venus_lookup gets an extra parameter */
+// CODA_KERNEL_VERSION 3 /* 128-bit file identifiers */
+// CODA_KERNEL_VERSION 4 /* 64-bit timespec */
+#define CODA_KERNEL_VERSION 5 /* access intent support */
/*
* Venus <-> Coda RPC arguments
struct coda_in_hdr {
u_int32_t opcode;
u_int32_t unique; /* Keep multiple outstanding msgs distinct */
- pid_t pid;
- pid_t pgid;
+ __kernel_pid_t pid;
+ __kernel_pid_t pgid;
vuid_t uid;
};
struct coda_statfs stat;
};
+#define CODA_ACCESS_TYPE_READ 1
+#define CODA_ACCESS_TYPE_WRITE 2
+#define CODA_ACCESS_TYPE_MMAP 3
+#define CODA_ACCESS_TYPE_READ_FINISH 4
+#define CODA_ACCESS_TYPE_WRITE_FINISH 5
+
+/* coda_access_intent: NO_OUT */
+struct coda_access_intent_in {
+ struct coda_in_hdr ih;
+ struct CodaFid VFid;
+ int count;
+ int pos;
+ int type;
+};
+
+struct coda_access_intent_out {
+ struct coda_out_hdr out;
+};
+
/*
* Occasionally, we don't cache the fid returned by CODA_LOOKUP.
* For instance, if the fid is inconsistent.
struct coda_open_by_fd_in coda_open_by_fd;
struct coda_open_by_path_in coda_open_by_path;
struct coda_statfs_in coda_statfs;
+ struct coda_access_intent_in coda_access_intent;
};
union outputArgs {
+++ /dev/null
-/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _UAPI__CODA_PSDEV_H
-#define _UAPI__CODA_PSDEV_H
-
-#include <linux/magic.h>
-
-#define CODA_PSDEV_MAJOR 67
-#define MAX_CODADEVS 5 /* how many do we allow */
-
-
-/* messages between coda filesystem in kernel and Venus */
-struct upc_req {
- struct list_head uc_chain;
- caddr_t uc_data;
- u_short uc_flags;
- u_short uc_inSize; /* Size is at most 5000 bytes */
- u_short uc_outSize;
- u_short uc_opcode; /* copied from data to save lookup */
- int uc_unique;
- wait_queue_head_t uc_sleep; /* process' wait queue */
-};
-
-#define CODA_REQ_ASYNC 0x1
-#define CODA_REQ_READ 0x2
-#define CODA_REQ_WRITE 0x4
-#define CODA_REQ_ABORT 0x8
-
-#endif /* _UAPI__CODA_PSDEV_H */
#define BALLOON_KVM_MAGIC 0x13661366
#define ZSMALLOC_MAGIC 0x58295829
#define DMA_BUF_MAGIC 0x444d4142 /* "DMAB" */
+#define Z3FOLD_MAGIC 0x33
#endif /* __LINUX_MAGIC_H__ */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME, /* s64 */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME_EXTENSION, /* s64 */
TCA_TAPRIO_ATTR_FLAGS, /* u32 */
- TCA_TAPRIO_ATTR_TXTIME_DELAY, /* s32 */
+ TCA_TAPRIO_ATTR_TXTIME_DELAY, /* u32 */
__TCA_TAPRIO_ATTR_MAX,
};
__u64 flags; /* Output: filter's flags */
};
+#define PTRACE_GET_SYSCALL_INFO 0x420e
+#define PTRACE_SYSCALL_INFO_NONE 0
+#define PTRACE_SYSCALL_INFO_ENTRY 1
+#define PTRACE_SYSCALL_INFO_EXIT 2
+#define PTRACE_SYSCALL_INFO_SECCOMP 3
+
+struct ptrace_syscall_info {
+ __u8 op; /* PTRACE_SYSCALL_INFO_* */
+ __u32 arch __attribute__((__aligned__(sizeof(__u32))));
+ __u64 instruction_pointer;
+ __u64 stack_pointer;
+ union {
+ struct {
+ __u64 nr;
+ __u64 args[6];
+ } entry;
+ struct {
+ __s64 rval;
+ __u8 is_error;
+ } exit;
+ struct {
+ __u64 nr;
+ __u64 args[6];
+ __u32 ret_data;
+ } seccomp;
+ };
+};
+
+/*
+ * These values are stored in task->ptrace_message
+ * by tracehook_report_syscall_* to describe the current syscall-stop.
+ */
+#define PTRACE_EVENTMSG_SYSCALL_ENTRY 1
+#define PTRACE_EVENTMSG_SYSCALL_EXIT 2
+
/* Read signals from a shared (process wide) queue */
#define PTRACE_PEEKSIGINFO_SHARED (1 << 0)
#define VIRTIO_ID_INPUT 18 /* virtio input */
#define VIRTIO_ID_VSOCK 19 /* virtio vsock transport */
#define VIRTIO_ID_CRYPTO 20 /* virtio crypto */
+#define VIRTIO_ID_IOMMU 23 /* virtio IOMMU */
+#define VIRTIO_ID_PMEM 27 /* virtio pmem */
#endif /* _LINUX_VIRTIO_IDS_H */
--- /dev/null
+/* SPDX-License-Identifier: BSD-3-Clause */
+/*
+ * Virtio-iommu definition v0.9
+ *
+ * Copyright (C) 2018 Arm Ltd.
+ */
+#ifndef _UAPI_LINUX_VIRTIO_IOMMU_H
+#define _UAPI_LINUX_VIRTIO_IOMMU_H
+
+#include <linux/types.h>
+
+/* Feature bits */
+#define VIRTIO_IOMMU_F_INPUT_RANGE 0
+#define VIRTIO_IOMMU_F_DOMAIN_BITS 1
+#define VIRTIO_IOMMU_F_MAP_UNMAP 2
+#define VIRTIO_IOMMU_F_BYPASS 3
+#define VIRTIO_IOMMU_F_PROBE 4
+
+struct virtio_iommu_range {
+ __u64 start;
+ __u64 end;
+};
+
+struct virtio_iommu_config {
+ /* Supported page sizes */
+ __u64 page_size_mask;
+ /* Supported IOVA range */
+ struct virtio_iommu_range input_range;
+ /* Max domain ID size */
+ __u8 domain_bits;
+ __u8 padding[3];
+ /* Probe buffer size */
+ __u32 probe_size;
+};
+
+/* Request types */
+#define VIRTIO_IOMMU_T_ATTACH 0x01
+#define VIRTIO_IOMMU_T_DETACH 0x02
+#define VIRTIO_IOMMU_T_MAP 0x03
+#define VIRTIO_IOMMU_T_UNMAP 0x04
+#define VIRTIO_IOMMU_T_PROBE 0x05
+
+/* Status types */
+#define VIRTIO_IOMMU_S_OK 0x00
+#define VIRTIO_IOMMU_S_IOERR 0x01
+#define VIRTIO_IOMMU_S_UNSUPP 0x02
+#define VIRTIO_IOMMU_S_DEVERR 0x03
+#define VIRTIO_IOMMU_S_INVAL 0x04
+#define VIRTIO_IOMMU_S_RANGE 0x05
+#define VIRTIO_IOMMU_S_NOENT 0x06
+#define VIRTIO_IOMMU_S_FAULT 0x07
+
+struct virtio_iommu_req_head {
+ __u8 type;
+ __u8 reserved[3];
+};
+
+struct virtio_iommu_req_tail {
+ __u8 status;
+ __u8 reserved[3];
+};
+
+struct virtio_iommu_req_attach {
+ struct virtio_iommu_req_head head;
+ __le32 domain;
+ __le32 endpoint;
+ __u8 reserved[8];
+ struct virtio_iommu_req_tail tail;
+};
+
+struct virtio_iommu_req_detach {
+ struct virtio_iommu_req_head head;
+ __le32 domain;
+ __le32 endpoint;
+ __u8 reserved[8];
+ struct virtio_iommu_req_tail tail;
+};
+
+#define VIRTIO_IOMMU_MAP_F_READ (1 << 0)
+#define VIRTIO_IOMMU_MAP_F_WRITE (1 << 1)
+#define VIRTIO_IOMMU_MAP_F_EXEC (1 << 2)
+#define VIRTIO_IOMMU_MAP_F_MMIO (1 << 3)
+
+#define VIRTIO_IOMMU_MAP_F_MASK (VIRTIO_IOMMU_MAP_F_READ | \
+ VIRTIO_IOMMU_MAP_F_WRITE | \
+ VIRTIO_IOMMU_MAP_F_EXEC | \
+ VIRTIO_IOMMU_MAP_F_MMIO)
+
+struct virtio_iommu_req_map {
+ struct virtio_iommu_req_head head;
+ __le32 domain;
+ __le64 virt_start;
+ __le64 virt_end;
+ __le64 phys_start;
+ __le32 flags;
+ struct virtio_iommu_req_tail tail;
+};
+
+struct virtio_iommu_req_unmap {
+ struct virtio_iommu_req_head head;
+ __le32 domain;
+ __le64 virt_start;
+ __le64 virt_end;
+ __u8 reserved[4];
+ struct virtio_iommu_req_tail tail;
+};
+
+#define VIRTIO_IOMMU_PROBE_T_NONE 0
+#define VIRTIO_IOMMU_PROBE_T_RESV_MEM 1
+
+#define VIRTIO_IOMMU_PROBE_T_MASK 0xfff
+
+struct virtio_iommu_probe_property {
+ __le16 type;
+ __le16 length;
+};
+
+#define VIRTIO_IOMMU_RESV_MEM_T_RESERVED 0
+#define VIRTIO_IOMMU_RESV_MEM_T_MSI 1
+
+struct virtio_iommu_probe_resv_mem {
+ struct virtio_iommu_probe_property head;
+ __u8 subtype;
+ __u8 reserved[3];
+ __le64 start;
+ __le64 end;
+};
+
+struct virtio_iommu_req_probe {
+ struct virtio_iommu_req_head head;
+ __le32 endpoint;
+ __u8 reserved[64];
+
+ __u8 properties[];
+
+ /*
+ * Tail follows the variable-length properties array. No padding,
+ * property lengths are all aligned on 8 bytes.
+ */
+};
+
+/* Fault types */
+#define VIRTIO_IOMMU_FAULT_R_UNKNOWN 0
+#define VIRTIO_IOMMU_FAULT_R_DOMAIN 1
+#define VIRTIO_IOMMU_FAULT_R_MAPPING 2
+
+#define VIRTIO_IOMMU_FAULT_F_READ (1 << 0)
+#define VIRTIO_IOMMU_FAULT_F_WRITE (1 << 1)
+#define VIRTIO_IOMMU_FAULT_F_EXEC (1 << 2)
+#define VIRTIO_IOMMU_FAULT_F_ADDRESS (1 << 8)
+
+struct virtio_iommu_fault {
+ __u8 reason;
+ __u8 reserved[3];
+ __le32 flags;
+ __le32 endpoint;
+ __u8 reserved2[4];
+ __le64 address;
+};
+
+#endif
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause */
+/*
+ * Definitions for virtio-pmem devices.
+ *
+ * Copyright (C) 2019 Red Hat, Inc.
+ *
+ * Author(s): Pankaj Gupta <pagupta@redhat.com>
+ */
+
+#ifndef _UAPI_LINUX_VIRTIO_PMEM_H
+#define _UAPI_LINUX_VIRTIO_PMEM_H
+
+#include <linux/types.h>
+#include <linux/virtio_ids.h>
+#include <linux/virtio_config.h>
+
+struct virtio_pmem_config {
+ __u64 start;
+ __u64 size;
+};
+
+#define VIRTIO_PMEM_REQ_TYPE_FLUSH 0
+
+struct virtio_pmem_resp {
+ /* Host return status corresponding to flush request */
+ __le32 ret;
+};
+
+struct virtio_pmem_req {
+ /* command type */
+ __le32 type;
+};
+
+#endif
#include <linux/types.h>
#include <linux/ioctl.h>
-/* Documentation/ioctl/ioctl-number.txt */
+/* Documentation/ioctl/ioctl-number.rst */
#define RDMA_IOCTL_MAGIC 0x1b
#define RDMA_VERBS_IOCTL \
_IOWR(RDMA_IOCTL_MAGIC, 1, struct ib_uverbs_ioctl_hdr)
have cpu.pressure, memory.pressure, and io.pressure files,
which aggregate pressure stalls for the grouped tasks only.
- For more details see Documentation/accounting/psi.txt.
+ For more details see Documentation/accounting/psi.rst.
Say N if unsure.
controls or device isolation.
See
- Documentation/scheduler/sched-design-CFS.rst (CFS)
- - Documentation/cgroup-v1/ (features for grouping, isolation
+ - Documentation/admin-guide/cgroup-v1/ (features for grouping, isolation
and resource control)
Say N if unsure.
CONFIG_CFQ_GROUP_IOSCHED=y; for enabling throttling policy, set
CONFIG_BLK_DEV_THROTTLING=y.
- See Documentation/cgroup-v1/blkio-controller.rst for more information.
+ See Documentation/admin-guide/cgroup-v1/blkio-controller.rst for more information.
config CGROUP_WRITEBACK
bool
help
Many kernel heap attacks try to target slab cache metadata and
other infrastructure. This options makes minor performance
- sacrifies to harden the kernel slab allocator against common
+ sacrifices to harden the kernel slab allocator against common
freelist exploit methods.
config SHUFFLE_PAGE_ALLOCATOR
depends on SLUB && SMP
bool "SLUB per cpu partial cache"
help
- Per cpu partial caches accellerate objects allocation and freeing
+ Per cpu partial caches accelerate objects allocation and freeing
that is local to a processor at the price of more indeterminism
in the latency of the free. On overflow these caches will be cleared
which requires the taking of locks that may cause latency spikes.
static struct dentry *rootfs_mount(struct file_system_type *fs_type,
int flags, const char *dev_name, void *data)
{
- static unsigned long once;
void *fill = ramfs_fill_super;
- if (test_and_set_bit(0, &once))
- return ERR_PTR(-ENODEV);
-
if (IS_ENABLED(CONFIG_TMPFS) && is_tmpfs)
fill = shmem_fill_super;
return mount_nodev(fs_type, flags, data, fill);
}
-static struct file_system_type rootfs_fs_type = {
+struct file_system_type rootfs_fs_type = {
.name = "rootfs",
.mount = rootfs_mount,
.kill_sb = kill_litter_super,
};
-int __init init_rootfs(void)
+void __init init_rootfs(void)
{
- int err = register_filesystem(&rootfs_fs_type);
-
- if (err)
- return err;
-
if (IS_ENABLED(CONFIG_TMPFS) && !saved_root_name[0] &&
- (!root_fs_names || strstr(root_fs_names, "tmpfs"))) {
- err = shmem_init();
+ (!root_fs_names || strstr(root_fs_names, "tmpfs")))
is_tmpfs = true;
- } else {
- err = init_ramfs_fs();
- }
-
- if (err)
- unregister_filesystem(&rootfs_fs_type);
-
- return err;
}
static void __init do_basic_setup(void)
{
cpuset_init_smp();
- shmem_init();
driver_init();
init_irq_proc();
do_ctors();
#define proc_ipc_sem_dointvec NULL
#endif
-static int zero;
-static int one = 1;
-static int int_max = INT_MAX;
int ipc_mni = IPCMNI;
int ipc_mni_shift = IPCMNI_SHIFT;
int ipc_min_cycle = RADIX_TREE_MAP_SIZE;
.maxlen = sizeof(init_ipc_ns.shm_ctlmni),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &ipc_mni,
},
{
.maxlen = sizeof(init_ipc_ns.shm_rmid_forced),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax_orphans,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "msgmax",
.maxlen = sizeof(init_ipc_ns.msg_ctlmax),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "msgmni",
.maxlen = sizeof(init_ipc_ns.msg_ctlmni),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &ipc_mni,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_ipc_auto_msgmni,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "msgmnb",
.maxlen = sizeof(init_ipc_ns.msg_ctlmnb),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "sem",
.maxlen = sizeof(init_ipc_ns.ids[IPC_SEM_IDS].next_id),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "msg_next_id",
.maxlen = sizeof(init_ipc_ns.ids[IPC_MSG_IDS].next_id),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "shm_next_id",
.maxlen = sizeof(init_ipc_ns.ids[IPC_SHM_IDS].next_id),
.mode = 0644,
.proc_handler = proc_ipc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
#endif
{}
{
struct mqueue_fs_context *ctx = fc->fs_private;
- put_user_ns(fc->user_ns);
- fc->user_ns = get_user_ns(ctx->ipc_ns->user_ns);
fc->s_fs_info = ctx->ipc_ns;
return vfs_get_super(fc, vfs_get_keyed_super, mqueue_fill_super);
}
{
struct mqueue_fs_context *ctx = fc->fs_private;
- if (ctx->ipc_ns)
- put_ipc_ns(ctx->ipc_ns);
+ put_ipc_ns(ctx->ipc_ns);
kfree(ctx);
}
return -ENOMEM;
ctx->ipc_ns = get_ipc_ns(current->nsproxy->ipc_ns);
+ put_user_ns(fc->user_ns);
+ fc->user_ns = get_user_ns(ctx->ipc_ns->user_ns);
fc->fs_private = ctx;
fc->ops = &mqueue_fs_context_ops;
return 0;
ctx = fc->fs_private;
put_ipc_ns(ctx->ipc_ns);
ctx->ipc_ns = get_ipc_ns(ns);
+ put_user_ns(fc->user_ns);
+ fc->user_ns = get_user_ns(ctx->ipc_ns->user_ns);
mnt = fc_mount(fc);
put_fs_context(fc);
{
struct mqueue_inode_info *info;
struct user_struct *user;
- unsigned long mq_bytes, mq_treesize;
struct ipc_namespace *ipc_ns;
struct msg_msg *msg, *nmsg;
LIST_HEAD(tmp_msg);
free_msg(msg);
}
- /* Total amount of bytes accounted for the mqueue */
- mq_treesize = info->attr.mq_maxmsg * sizeof(struct msg_msg) +
- min_t(unsigned int, info->attr.mq_maxmsg, MQ_PRIO_MAX) *
- sizeof(struct posix_msg_tree_node);
-
- mq_bytes = mq_treesize + (info->attr.mq_maxmsg *
- info->attr.mq_msgsize);
-
user = info->user;
if (user) {
+ unsigned long mq_bytes, mq_treesize;
+
+ /* Total amount of bytes accounted for the mqueue */
+ mq_treesize = info->attr.mq_maxmsg * sizeof(struct msg_msg) +
+ min_t(unsigned int, info->attr.mq_maxmsg, MQ_PRIO_MAX) *
+ sizeof(struct posix_msg_tree_node);
+
+ mq_bytes = mq_treesize + (info->attr.mq_maxmsg *
+ info->attr.mq_msgsize);
+
spin_lock(&mq_lock);
user->mq_bytes -= mq_bytes;
/*
!btf_type_is_var(size_type)))
return NULL;
- size = btf->resolved_sizes[size_type_id];
size_type_id = btf->resolved_ids[size_type_id];
size_type = btf_type_by_id(btf, size_type_id);
if (btf_type_nosize_or_null(size_type))
return NULL;
+ else if (btf_type_has_size(size_type))
+ size = size_type->size;
+ else if (btf_type_is_array(size_type))
+ size = btf->resolved_sizes[size_type_id];
+ else if (btf_type_is_ptr(size_type))
+ size = sizeof(void *);
+ else
+ return NULL;
}
*type_id = size_type_id;
const struct btf_type *next_type;
u32 next_type_id = t->type;
struct btf *btf = env->btf;
- u32 next_type_size = 0;
next_type = btf_type_by_id(btf, next_type_id);
if (!next_type || btf_type_is_resolve_source_only(next_type)) {
* save us a few type-following when we use it later (e.g. in
* pretty print).
*/
- if (!btf_type_id_size(btf, &next_type_id, &next_type_size)) {
+ if (!btf_type_id_size(btf, &next_type_id, NULL)) {
if (env_type_is_resolved(env, next_type_id))
next_type = btf_type_id_resolve(btf, &next_type_id);
}
}
- env_stack_pop_resolved(env, next_type_id, next_type_size);
+ env_stack_pop_resolved(env, next_type_id, 0);
return 0;
}
const struct btf_type *t = v->t;
u32 next_type_id = t->type;
struct btf *btf = env->btf;
- u32 next_type_size;
next_type = btf_type_by_id(btf, next_type_id);
if (!next_type || btf_type_is_resolve_source_only(next_type)) {
* forward types or similar that would resolve to size of
* zero is allowed.
*/
- if (!btf_type_id_size(btf, &next_type_id, &next_type_size)) {
+ if (!btf_type_id_size(btf, &next_type_id, NULL)) {
btf_verifier_log_type(env, v->t, "Invalid type_id");
return -EINVAL;
}
- env_stack_pop_resolved(env, next_type_id, next_type_size);
+ env_stack_pop_resolved(env, next_type_id, 0);
return 0;
}
return -EFAULT;
}
*stack_mask |= 1ull << spi;
- } else if (class == BPF_STX) {
+ } else if (class == BPF_STX || class == BPF_ST) {
if (*reg_mask & dreg)
- /* stx shouldn't be using _scalar_ dst_reg
+ /* stx & st shouldn't be using _scalar_ dst_reg
* to access memory. It means backtracking
* encountered a case of pointer subtraction.
*/
if (!(*stack_mask & (1ull << spi)))
return 0;
*stack_mask &= ~(1ull << spi);
- *reg_mask |= sreg;
+ if (class == BPF_STX)
+ *reg_mask |= sreg;
} else if (class == BPF_JMP || class == BPF_JMP32) {
if (opcode == BPF_CALL) {
if (insn->src_reg == BPF_PSEUDO_CALL)
if (mode == BPF_IND || mode == BPF_ABS)
/* to be analyzed */
return -ENOTSUPP;
- } else if (class == BPF_ST) {
- if (*reg_mask & dreg)
- /* likely pointer subtraction */
- return -ENOTSUPP;
}
return 0;
}
if (env->prog->expected_attach_type == BPF_CGROUP_UDP4_RECVMSG ||
env->prog->expected_attach_type == BPF_CGROUP_UDP6_RECVMSG)
range = tnum_range(1, 1);
+ break;
case BPF_PROG_TYPE_CGROUP_SKB:
if (env->prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) {
range = tnum_range(0, 3);
enforce_attach_type_range = tnum_range(2, 3);
}
+ break;
case BPF_PROG_TYPE_CGROUP_SOCK:
case BPF_PROG_TYPE_SOCK_OPS:
case BPF_PROG_TYPE_CGROUP_DEVICE:
fc->ops = &cgroup_fs_context_ops;
else
fc->ops = &cgroup1_fs_context_ops;
- if (fc->user_ns)
- put_user_ns(fc->user_ns);
+ put_user_ns(fc->user_ns);
fc->user_ns = get_user_ns(ctx->ns->user_ns);
fc->global = true;
return 0;
.fs_flags = FS_USERNS_MOUNT,
};
+#ifdef CONFIG_CPUSETS
+static const struct fs_context_operations cpuset_fs_context_ops = {
+ .get_tree = cgroup1_get_tree,
+ .free = cgroup_fs_context_free,
+};
+
+/*
+ * This is ugly, but preserves the userspace API for existing cpuset
+ * users. If someone tries to mount the "cpuset" filesystem, we
+ * silently switch it to mount "cgroup" instead
+ */
+static int cpuset_init_fs_context(struct fs_context *fc)
+{
+ char *agent = kstrdup("/sbin/cpuset_release_agent", GFP_USER);
+ struct cgroup_fs_context *ctx;
+ int err;
+
+ err = cgroup_init_fs_context(fc);
+ if (err) {
+ kfree(agent);
+ return err;
+ }
+
+ fc->ops = &cpuset_fs_context_ops;
+
+ ctx = cgroup_fc2context(fc);
+ ctx->subsys_mask = 1 << cpuset_cgrp_id;
+ ctx->flags |= CGRP_ROOT_NOPREFIX;
+ ctx->release_agent = agent;
+
+ get_filesystem(&cgroup_fs_type);
+ put_filesystem(fc->fs_type);
+ fc->fs_type = &cgroup_fs_type;
+
+ return 0;
+}
+
+static struct file_system_type cpuset_fs_type = {
+ .name = "cpuset",
+ .init_fs_context = cpuset_init_fs_context,
+ .fs_flags = FS_USERNS_MOUNT,
+};
+#endif
+
int cgroup_path_ns_locked(struct cgroup *cgrp, char *buf, size_t buflen,
struct cgroup_namespace *ns)
{
WARN_ON(register_filesystem(&cgroup_fs_type));
WARN_ON(register_filesystem(&cgroup2_fs_type));
WARN_ON(!proc_create_single("cgroups", 0, NULL, proc_cgroupstats_show));
+#ifdef CONFIG_CPUSETS
+ WARN_ON(register_filesystem(&cpuset_fs_type));
+#endif
return 0;
}
}
/*
- * This is ugly, but preserves the userspace API for existing cpuset
- * users. If someone tries to mount the "cpuset" filesystem, we
- * silently switch it to mount "cgroup" instead
- */
-static int cpuset_get_tree(struct fs_context *fc)
-{
- struct file_system_type *cgroup_fs;
- struct fs_context *new_fc;
- int ret;
-
- cgroup_fs = get_fs_type("cgroup");
- if (!cgroup_fs)
- return -ENODEV;
-
- new_fc = fs_context_for_mount(cgroup_fs, fc->sb_flags);
- if (IS_ERR(new_fc)) {
- ret = PTR_ERR(new_fc);
- } else {
- static const char agent_path[] = "/sbin/cpuset_release_agent";
- ret = vfs_parse_fs_string(new_fc, "cpuset", NULL, 0);
- if (!ret)
- ret = vfs_parse_fs_string(new_fc, "noprefix", NULL, 0);
- if (!ret)
- ret = vfs_parse_fs_string(new_fc, "release_agent",
- agent_path, sizeof(agent_path) - 1);
- if (!ret)
- ret = vfs_get_tree(new_fc);
- if (!ret) { /* steal the result */
- fc->root = new_fc->root;
- new_fc->root = NULL;
- }
- put_fs_context(new_fc);
- }
- put_filesystem(cgroup_fs);
- return ret;
-}
-
-static const struct fs_context_operations cpuset_fs_context_ops = {
- .get_tree = cpuset_get_tree,
-};
-
-static int cpuset_init_fs_context(struct fs_context *fc)
-{
- fc->ops = &cpuset_fs_context_ops;
- return 0;
-}
-
-static struct file_system_type cpuset_fs_type = {
- .name = "cpuset",
- .init_fs_context = cpuset_init_fs_context,
-};
-
-/*
* Return in pmask the portion of a cpusets's cpus_allowed that
* are online. If none are online, walk up the cpuset hierarchy
* until we find one that does have some online cpus.
* load balancing domains (sched domains) as specified by that partial
* partition.
*
- * See "What is sched_load_balance" in Documentation/cgroup-v1/cpusets.rst
+ * See "What is sched_load_balance" in Documentation/admin-guide/cgroup-v1/cpusets.rst
* for a background explanation of this.
*
* Does not return errors, on the theory that the callers of this
/**
* cpuset_init - initialize cpusets at system boot
*
- * Description: Initialize top_cpuset and the cpuset internal file system,
+ * Description: Initialize top_cpuset
**/
int __init cpuset_init(void)
{
- int err = 0;
-
BUG_ON(!alloc_cpumask_var(&top_cpuset.cpus_allowed, GFP_KERNEL));
BUG_ON(!alloc_cpumask_var(&top_cpuset.effective_cpus, GFP_KERNEL));
BUG_ON(!zalloc_cpumask_var(&top_cpuset.subparts_cpus, GFP_KERNEL));
set_bit(CS_SCHED_LOAD_BALANCE, &top_cpuset.flags);
top_cpuset.relax_domain_level = -1;
- err = register_filesystem(&cpuset_fs_type);
- if (err < 0)
- return err;
-
BUG_ON(!alloc_cpumask_var(&cpus_attach, GFP_KERNEL));
return 0;
}
early_param("swiotlb", setup_io_tlb_npages);
+static bool no_iotlb_memory;
+
unsigned long swiotlb_nr_tbl(void)
{
- return io_tlb_nslabs;
+ return unlikely(no_iotlb_memory) ? 0 : io_tlb_nslabs;
}
EXPORT_SYMBOL_GPL(swiotlb_nr_tbl);
unsigned int swiotlb_max_segment(void)
{
- return max_segment;
+ return unlikely(no_iotlb_memory) ? 0 : max_segment;
}
EXPORT_SYMBOL_GPL(swiotlb_max_segment);
return size ? size : (IO_TLB_DEFAULT_SIZE);
}
-static bool no_iotlb_memory;
-
void swiotlb_print_info(void)
{
unsigned long bytes = io_tlb_nslabs << IO_TLB_SHIFT;
return rc;
}
+static void swiotlb_cleanup(void)
+{
+ io_tlb_end = 0;
+ io_tlb_start = 0;
+ io_tlb_nslabs = 0;
+ max_segment = 0;
+}
+
int
swiotlb_late_init_with_tbl(char *tlb, unsigned long nslabs)
{
sizeof(int)));
io_tlb_list = NULL;
cleanup3:
- io_tlb_end = 0;
- io_tlb_start = 0;
- io_tlb_nslabs = 0;
- max_segment = 0;
+ swiotlb_cleanup();
return -ENOMEM;
}
memblock_free_late(io_tlb_start,
PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
}
- io_tlb_start = 0;
- io_tlb_end = 0;
- io_tlb_nslabs = 0;
- max_segment = 0;
+ swiotlb_cleanup();
}
/*
if (!(attrs & DMA_ATTR_NO_WARN) && printk_ratelimit())
dev_warn(hwdev, "swiotlb buffer is full (sz: %zd bytes), total %lu (slots), used %lu (slots)\n",
size, io_tlb_nslabs, tmp_io_tlb_used);
- return DMA_MAPPING_ERROR;
+ return (phys_addr_t)DMA_MAPPING_ERROR;
found:
io_tlb_used += nslots;
spin_unlock_irqrestore(&io_tlb_lock, flags);
/* Oh well, have to allocate and map a bounce buffer. */
*phys = swiotlb_tbl_map_single(dev, __phys_to_dma(dev, io_tlb_start),
*phys, size, dir, attrs);
- if (*phys == DMA_MAPPING_ERROR)
+ if (*phys == (phys_addr_t)DMA_MAPPING_ERROR)
return false;
/* Ensure that the address returned is DMA'ble */
struct file *perf_event_get(unsigned int fd)
{
- struct file *file;
-
- file = fget_raw(fd);
+ struct file *file = fget(fd);
if (!file)
return ERR_PTR(-EBADF);
return nr;
}
+bool legacy_clone_args_valid(const struct kernel_clone_args *kargs)
+{
+ /* clone(CLONE_PIDFD) uses parent_tidptr to return a pidfd */
+ if ((kargs->flags & CLONE_PIDFD) &&
+ (kargs->flags & CLONE_PARENT_SETTID))
+ return false;
+
+ return true;
+}
+
#ifndef CONFIG_HAVE_COPY_THREAD_TLS
/* For compatibility with architectures that call do_fork directly rather than
* using the syscall entry points below. */
{
struct kernel_clone_args args = {
.flags = (clone_flags & ~CSIGNAL),
+ .pidfd = parent_tidptr,
.child_tid = child_tidptr,
.parent_tid = parent_tidptr,
.exit_signal = (clone_flags & CSIGNAL),
.stack_size = stack_size,
};
+ if (!legacy_clone_args_valid(&args))
+ return -EINVAL;
+
return _do_fork(&args);
}
#endif
.tls = tls,
};
- /* clone(CLONE_PIDFD) uses parent_tidptr to return a pidfd */
- if ((clone_flags & CLONE_PIDFD) && (clone_flags & CLONE_PARENT_SETTID))
+ if (!legacy_clone_args_valid(&args))
return -EINVAL;
return _do_fork(&args);
init_test_probes();
return err;
}
+subsys_initcall(init_kprobes);
#ifdef CONFIG_DEBUG_FS
static void report_probe(struct seq_file *pi, struct kprobe *p,
late_initcall(debugfs_kprobe_init);
#endif /* CONFIG_DEBUG_FS */
-
-module_init(init_kprobes);
* by Steven Rostedt, based on work by Gregory Haskins, Peter Morreale
* and Sven Dietrich.
*
- * Also see Documentation/locking/mutex-design.txt.
+ * Also see Documentation/locking/mutex-design.rst.
*/
#include <linux/mutex.h>
#include <linux/ww_mutex.h>
* Copyright (C) 2005 Kihon Technologies Inc., Steven Rostedt
* Copyright (C) 2006 Esben Nielsen
*
- * See Documentation/locking/rt-mutex-design.txt for details.
+ * See Documentation/locking/rt-mutex-design.rst for details.
*/
#include <linux/spinlock.h>
#include <linux/export.h>
static unsigned long pfn_first(struct dev_pagemap *pgmap)
{
- return (pgmap->res.start >> PAGE_SHIFT) +
+ return PHYS_PFN(pgmap->res.start) +
vmem_altmap_offset(pgmap_altmap(pgmap));
}
struct dev_pagemap *pgmap = data;
struct device *dev = pgmap->dev;
struct resource *res = &pgmap->res;
- resource_size_t align_start, align_size;
unsigned long pfn;
int nid;
dev_pagemap_cleanup(pgmap);
/* pages are dead and unused, undo the arch mapping */
- align_start = res->start & ~(SECTION_SIZE - 1);
- align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE)
- - align_start;
-
- nid = page_to_nid(pfn_to_page(align_start >> PAGE_SHIFT));
+ nid = page_to_nid(pfn_to_page(PHYS_PFN(res->start)));
mem_hotplug_begin();
if (pgmap->type == MEMORY_DEVICE_PRIVATE) {
- pfn = align_start >> PAGE_SHIFT;
+ pfn = PHYS_PFN(res->start);
__remove_pages(page_zone(pfn_to_page(pfn)), pfn,
- align_size >> PAGE_SHIFT, NULL);
+ PHYS_PFN(resource_size(res)), NULL);
} else {
- arch_remove_memory(nid, align_start, align_size,
+ arch_remove_memory(nid, res->start, resource_size(res),
pgmap_altmap(pgmap));
- kasan_remove_zero_shadow(__va(align_start), align_size);
+ kasan_remove_zero_shadow(__va(res->start), resource_size(res));
}
mem_hotplug_done();
- untrack_pfn(NULL, PHYS_PFN(align_start), align_size);
+ untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
pgmap_array_delete(res);
dev_WARN_ONCE(dev, pgmap->altmap.alloc,
"%s: failed to free all reserved pages\n", __func__);
*/
void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
{
- resource_size_t align_start, align_size, align_end;
struct resource *res = &pgmap->res;
struct dev_pagemap *conflict_pgmap;
struct mhp_restrictions restrictions = {
/*
* We do not want any optional features only our own memmap
- */
+ */
.altmap = pgmap_altmap(pgmap),
};
pgprot_t pgprot = PAGE_KERNEL;
return ERR_PTR(error);
}
- align_start = res->start & ~(SECTION_SIZE - 1);
- align_size = ALIGN(res->start + resource_size(res), SECTION_SIZE)
- - align_start;
- align_end = align_start + align_size - 1;
-
- conflict_pgmap = get_dev_pagemap(PHYS_PFN(align_start), NULL);
+ conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->start), NULL);
if (conflict_pgmap) {
dev_WARN(dev, "Conflicting mapping in same section\n");
put_dev_pagemap(conflict_pgmap);
goto err_array;
}
- conflict_pgmap = get_dev_pagemap(PHYS_PFN(align_end), NULL);
+ conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->end), NULL);
if (conflict_pgmap) {
dev_WARN(dev, "Conflicting mapping in same section\n");
put_dev_pagemap(conflict_pgmap);
goto err_array;
}
- is_ram = region_intersects(align_start, align_size,
+ is_ram = region_intersects(res->start, resource_size(res),
IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE);
if (is_ram != REGION_DISJOINT) {
if (nid < 0)
nid = numa_mem_id();
- error = track_pfn_remap(NULL, &pgprot, PHYS_PFN(align_start), 0,
- align_size);
+ error = track_pfn_remap(NULL, &pgprot, PHYS_PFN(res->start), 0,
+ resource_size(res));
if (error)
goto err_pfn_remap;
* arch_add_memory().
*/
if (pgmap->type == MEMORY_DEVICE_PRIVATE) {
- error = add_pages(nid, align_start >> PAGE_SHIFT,
- align_size >> PAGE_SHIFT, &restrictions);
+ error = add_pages(nid, PHYS_PFN(res->start),
+ PHYS_PFN(resource_size(res)), &restrictions);
} else {
- error = kasan_add_zero_shadow(__va(align_start), align_size);
+ error = kasan_add_zero_shadow(__va(res->start), resource_size(res));
if (error) {
mem_hotplug_done();
goto err_kasan;
}
- error = arch_add_memory(nid, align_start, align_size,
+ error = arch_add_memory(nid, res->start, resource_size(res),
&restrictions);
}
struct zone *zone;
zone = &NODE_DATA(nid)->node_zones[ZONE_DEVICE];
- move_pfn_range_to_zone(zone, align_start >> PAGE_SHIFT,
- align_size >> PAGE_SHIFT, pgmap_altmap(pgmap));
+ move_pfn_range_to_zone(zone, PHYS_PFN(res->start),
+ PHYS_PFN(resource_size(res)), restrictions.altmap);
}
mem_hotplug_done();
* to allow us to do the work while not holding the hotplug lock.
*/
memmap_init_zone_device(&NODE_DATA(nid)->node_zones[ZONE_DEVICE],
- align_start >> PAGE_SHIFT,
- align_size >> PAGE_SHIFT, pgmap);
+ PHYS_PFN(res->start),
+ PHYS_PFN(resource_size(res)), pgmap);
percpu_ref_get_many(pgmap->ref, pfn_end(pgmap) - pfn_first(pgmap));
error = devm_add_action_or_reset(dev, devm_memremap_pages_release,
return __va(res->start);
err_add_memory:
- kasan_remove_zero_shadow(__va(align_start), align_size);
+ kasan_remove_zero_shadow(__va(res->start), resource_size(res));
err_kasan:
- untrack_pfn(NULL, PHYS_PFN(align_start), align_size);
+ untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
err_pfn_remap:
pgmap_array_delete(res);
err_array:
for (i = 0; i < info->hdr->e_shnum; i++)
if (!sect_empty(&info->sechdrs[i]))
nloaded++;
- size[0] = ALIGN(sizeof(*sect_attrs)
- + nloaded * sizeof(sect_attrs->attrs[0]),
+ size[0] = ALIGN(struct_size(sect_attrs, attrs, nloaded),
sizeof(sect_attrs->grp.attrs[0]));
size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.attrs[0]);
sect_attrs = kzalloc(size[0] + size[1], GFP_KERNEL);
return ret;
}
+static void module_remove_modinfo_attrs(struct module *mod, int end);
+
static int module_add_modinfo_attrs(struct module *mod)
{
struct module_attribute *attr;
return -ENOMEM;
temp_attr = mod->modinfo_attrs;
- for (i = 0; (attr = modinfo_attrs[i]) && !error; i++) {
+ for (i = 0; (attr = modinfo_attrs[i]); i++) {
if (!attr->test || attr->test(mod)) {
memcpy(temp_attr, attr, sizeof(*temp_attr));
sysfs_attr_init(&temp_attr->attr);
error = sysfs_create_file(&mod->mkobj.kobj,
&temp_attr->attr);
+ if (error)
+ goto error_out;
++temp_attr;
}
}
+
+ return 0;
+
+error_out:
+ if (i > 0)
+ module_remove_modinfo_attrs(mod, --i);
return error;
}
-static void module_remove_modinfo_attrs(struct module *mod)
+static void module_remove_modinfo_attrs(struct module *mod, int end)
{
struct module_attribute *attr;
int i;
for (i = 0; (attr = &mod->modinfo_attrs[i]); i++) {
+ if (end >= 0 && i > end)
+ break;
/* pick a field to test for end of list */
if (!attr->attr.name)
break;
return 0;
out_unreg_modinfo_attrs:
- module_remove_modinfo_attrs(mod);
+ module_remove_modinfo_attrs(mod, -1);
out_unreg_param:
module_param_sysfs_remove(mod);
out_unreg_holders:
{
}
-static void module_remove_modinfo_attrs(struct module *mod)
+static void module_remove_modinfo_attrs(struct module *mod, int end)
{
}
static void mod_sysfs_teardown(struct module *mod)
{
del_usage_links(mod);
- module_remove_modinfo_attrs(mod);
+ module_remove_modinfo_attrs(mod, -1);
module_param_sysfs_remove(mod);
kobject_put(mod->mkobj.drivers_dir);
kobject_put(mod->holders_dir);
mod_sysfs_fini(mod);
}
-#ifdef CONFIG_STRICT_MODULE_RWX
+#ifdef CONFIG_ARCH_HAS_STRICT_MODULE_RWX
/*
* LKM RO/NX protection: protect module's text/ro-data
* from modification and any data from execution.
layout->text_size >> PAGE_SHIFT);
}
+#ifdef CONFIG_STRICT_MODULE_RWX
static void frob_rodata(const struct module_layout *layout,
int (*set_memory)(unsigned long start, int num_pages))
{
set_vm_flush_reset_perms(mod->core_layout.base);
set_vm_flush_reset_perms(mod->init_layout.base);
frob_text(&mod->core_layout, set_memory_ro);
- frob_text(&mod->core_layout, set_memory_x);
frob_rodata(&mod->core_layout, set_memory_ro);
-
frob_text(&mod->init_layout, set_memory_ro);
- frob_text(&mod->init_layout, set_memory_x);
-
frob_rodata(&mod->init_layout, set_memory_ro);
if (after_init)
}
mutex_unlock(&module_mutex);
}
-#else
+#else /* !CONFIG_STRICT_MODULE_RWX */
static void module_enable_nx(const struct module *mod) { }
-#endif
+#endif /* CONFIG_STRICT_MODULE_RWX */
+static void module_enable_x(const struct module *mod)
+{
+ frob_text(&mod->core_layout, set_memory_x);
+ frob_text(&mod->init_layout, set_memory_x);
+}
+#else /* !CONFIG_ARCH_HAS_STRICT_MODULE_RWX */
+static void module_enable_nx(const struct module *mod) { }
+static void module_enable_x(const struct module *mod) { }
+#endif /* CONFIG_ARCH_HAS_STRICT_MODULE_RWX */
+
#ifdef CONFIG_LIVEPATCH
/*
return vmalloc_exec(size);
}
+bool __weak module_exit_section(const char *name)
+{
+ return strstarts(name, ".exit");
+}
+
#ifdef CONFIG_DEBUG_KMEMLEAK
static void kmemleak_load_module(const struct module *mod,
const struct load_info *info)
#ifndef CONFIG_MODULE_UNLOAD
/* Don't load .exit sections */
- if (strstarts(info->secstrings+shdr->sh_name, ".exit"))
+ if (module_exit_section(info->secstrings+shdr->sh_name))
shdr->sh_flags &= ~(unsigned long)SHF_ALLOC;
#endif
}
sched_annotate_sleep();
mutex_lock(&module_mutex);
mod = find_module_all(name, strlen(name), true);
- ret = !mod || mod->state == MODULE_STATE_LIVE
- || mod->state == MODULE_STATE_GOING;
+ ret = !mod || mod->state == MODULE_STATE_LIVE;
mutex_unlock(&module_mutex);
return ret;
mutex_lock(&module_mutex);
old = find_module_all(mod->name, strlen(mod->name), true);
if (old != NULL) {
- if (old->state == MODULE_STATE_COMING
- || old->state == MODULE_STATE_UNFORMED) {
+ if (old->state != MODULE_STATE_LIVE) {
/* Wait in case it fails to load. */
mutex_unlock(&module_mutex);
err = wait_event_interruptible(module_wq,
module_enable_ro(mod, false);
module_enable_nx(mod);
+ module_enable_x(mod);
/* Mark state as coming so strong_try_module_get() ignores us,
* but kallsyms etc. can see us. */
/**
* print_tainted - return a string to represent the kernel taint state.
*
- * For individual taint flag meanings, see Documentation/sysctl/kernel.txt
+ * For individual taint flag meanings, see Documentation/admin-guide/sysctl/kernel.rst
*
* The string is overwritten by the next call to print_tainted(),
* but is always NULL terminated.
#include <linux/init_task.h>
#include <linux/syscalls.h>
#include <linux/proc_ns.h>
-#include <linux/proc_fs.h>
+#include <linux/refcount.h>
#include <linux/anon_inodes.h>
#include <linux/sched/signal.h>
#include <linux/sched/task.h>
#include <linux/idr.h>
struct pid init_struct_pid = {
- .count = ATOMIC_INIT(1),
+ .count = REFCOUNT_INIT(1),
.tasks = {
{ .first = NULL },
{ .first = NULL },
return;
ns = pid->numbers[pid->level].ns;
- if ((atomic_read(&pid->count) == 1) ||
- atomic_dec_and_test(&pid->count)) {
+ if (refcount_dec_and_test(&pid->count)) {
kmem_cache_free(ns->pid_cachep, pid);
put_pid_ns(ns);
}
}
get_pid_ns(ns);
- atomic_set(&pid->count, 1);
+ refcount_set(&pid->count, 1);
for (type = 0; type < PIDTYPE_MAX; ++type)
INIT_HLIST_HEAD(&pid->tasks[type]);
}
extern int pid_max;
-static int zero = 0;
static struct ctl_table pid_ns_ctl_table[] = {
{
.procname = "ns_last_pid",
.maxlen = sizeof(int),
.mode = 0666, /* permissions are checked in the handler */
.proc_handler = pid_ns_ctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &pid_max,
},
{ }
#include <linux/compat.h>
#include <linux/sched/signal.h>
+#include <asm/syscall.h> /* for syscall_get_* */
+
/*
* Access another process' address space via ptrace.
* Source/target buffer must be kernel space,
* to ensure no machine forgets it.
*/
EXPORT_SYMBOL_GPL(task_user_regset_view);
-#endif
+
+static unsigned long
+ptrace_get_syscall_info_entry(struct task_struct *child, struct pt_regs *regs,
+ struct ptrace_syscall_info *info)
+{
+ unsigned long args[ARRAY_SIZE(info->entry.args)];
+ int i;
+
+ info->op = PTRACE_SYSCALL_INFO_ENTRY;
+ info->entry.nr = syscall_get_nr(child, regs);
+ syscall_get_arguments(child, regs, args);
+ for (i = 0; i < ARRAY_SIZE(args); i++)
+ info->entry.args[i] = args[i];
+
+ /* args is the last field in struct ptrace_syscall_info.entry */
+ return offsetofend(struct ptrace_syscall_info, entry.args);
+}
+
+static unsigned long
+ptrace_get_syscall_info_seccomp(struct task_struct *child, struct pt_regs *regs,
+ struct ptrace_syscall_info *info)
+{
+ /*
+ * As struct ptrace_syscall_info.entry is currently a subset
+ * of struct ptrace_syscall_info.seccomp, it makes sense to
+ * initialize that subset using ptrace_get_syscall_info_entry().
+ * This can be reconsidered in the future if these structures
+ * diverge significantly enough.
+ */
+ ptrace_get_syscall_info_entry(child, regs, info);
+ info->op = PTRACE_SYSCALL_INFO_SECCOMP;
+ info->seccomp.ret_data = child->ptrace_message;
+
+ /* ret_data is the last field in struct ptrace_syscall_info.seccomp */
+ return offsetofend(struct ptrace_syscall_info, seccomp.ret_data);
+}
+
+static unsigned long
+ptrace_get_syscall_info_exit(struct task_struct *child, struct pt_regs *regs,
+ struct ptrace_syscall_info *info)
+{
+ info->op = PTRACE_SYSCALL_INFO_EXIT;
+ info->exit.rval = syscall_get_error(child, regs);
+ info->exit.is_error = !!info->exit.rval;
+ if (!info->exit.is_error)
+ info->exit.rval = syscall_get_return_value(child, regs);
+
+ /* is_error is the last field in struct ptrace_syscall_info.exit */
+ return offsetofend(struct ptrace_syscall_info, exit.is_error);
+}
+
+static int
+ptrace_get_syscall_info(struct task_struct *child, unsigned long user_size,
+ void __user *datavp)
+{
+ struct pt_regs *regs = task_pt_regs(child);
+ struct ptrace_syscall_info info = {
+ .op = PTRACE_SYSCALL_INFO_NONE,
+ .arch = syscall_get_arch(child),
+ .instruction_pointer = instruction_pointer(regs),
+ .stack_pointer = user_stack_pointer(regs),
+ };
+ unsigned long actual_size = offsetof(struct ptrace_syscall_info, entry);
+ unsigned long write_size;
+
+ /*
+ * This does not need lock_task_sighand() to access
+ * child->last_siginfo because ptrace_freeze_traced()
+ * called earlier by ptrace_check_attach() ensures that
+ * the tracee cannot go away and clear its last_siginfo.
+ */
+ switch (child->last_siginfo ? child->last_siginfo->si_code : 0) {
+ case SIGTRAP | 0x80:
+ switch (child->ptrace_message) {
+ case PTRACE_EVENTMSG_SYSCALL_ENTRY:
+ actual_size = ptrace_get_syscall_info_entry(child, regs,
+ &info);
+ break;
+ case PTRACE_EVENTMSG_SYSCALL_EXIT:
+ actual_size = ptrace_get_syscall_info_exit(child, regs,
+ &info);
+ break;
+ }
+ break;
+ case SIGTRAP | (PTRACE_EVENT_SECCOMP << 8):
+ actual_size = ptrace_get_syscall_info_seccomp(child, regs,
+ &info);
+ break;
+ }
+
+ write_size = min(actual_size, user_size);
+ return copy_to_user(datavp, &info, write_size) ? -EFAULT : actual_size;
+}
+#endif /* CONFIG_HAVE_ARCH_TRACEHOOK */
int ptrace_request(struct task_struct *child, long request,
unsigned long addr, unsigned long data)
ret = __put_user(kiov.iov_len, &uiov->iov_len);
break;
}
+
+ case PTRACE_GET_SYSCALL_INFO:
+ ret = ptrace_get_syscall_info(child, addr, datavp);
+ break;
#endif
case PTRACE_SECCOMP_GET_FILTER:
*
* If a resource is found, returns 0 and @*res is overwritten with the part
* of the resource that's within [@start..@end]; if none is found, returns
- * -1 or -EINVAL for other invalid parameters.
+ * -ENODEV. Returns -EINVAL for invalid parameters.
*
* This function walks the whole tree and not just first level children
* unless @first_lvl is true.
unsigned long flags, unsigned long desc,
bool first_lvl, struct resource *res)
{
+ bool siblings_only = true;
struct resource *p;
if (!res)
read_lock(&resource_lock);
- for (p = iomem_resource.child; p; p = next_resource(p, first_lvl)) {
- if ((p->flags & flags) != flags)
- continue;
- if ((desc != IORES_DESC_NONE) && (desc != p->desc))
- continue;
+ for (p = iomem_resource.child; p; p = next_resource(p, siblings_only)) {
+ /* If we passed the resource we are looking for, stop */
if (p->start > end) {
p = NULL;
break;
}
- if ((p->end >= start) && (p->start <= end))
- break;
+
+ /* Skip until we find a range that matches what we look for */
+ if (p->end < start)
+ continue;
+
+ /*
+ * Now that we found a range that matches what we look for,
+ * check the flags and the descriptor. If we were not asked to
+ * use only the first level, start looking at children as well.
+ */
+ siblings_only = first_lvl;
+
+ if ((p->flags & flags) != flags)
+ continue;
+ if ((desc != IORES_DESC_NONE) && (desc != p->desc))
+ continue;
+
+ /* Found a match, break */
+ break;
+ }
+
+ if (p) {
+ /* copy data */
+ res->start = max(start, p->start);
+ res->end = min(end, p->end);
+ res->flags = p->flags;
+ res->desc = p->desc;
}
read_unlock(&resource_lock);
- if (!p)
- return -1;
-
- /* copy data */
- res->start = max(start, p->start);
- res->end = min(end, p->end);
- res->flags = p->flags;
- res->desc = p->desc;
- return 0;
+ return p ? 0 : -ENODEV;
}
static int __walk_iomem_res_desc(resource_size_t start, resource_size_t end,
*
* This is useful for syscalls such as ppoll, pselect, io_pgetevents and
* epoll_pwait where a new sigmask is passed from userland for the syscalls.
+ *
+ * Note that it does set_restore_sigmask() in advance, so it must be always
+ * paired with restore_saved_sigmask_unless() before return from syscall.
*/
-int set_user_sigmask(const sigset_t __user *usigmask, sigset_t *set,
- sigset_t *oldset, size_t sigsetsize)
+int set_user_sigmask(const sigset_t __user *umask, size_t sigsetsize)
{
- if (!usigmask)
- return 0;
+ sigset_t kmask;
+ if (!umask)
+ return 0;
if (sigsetsize != sizeof(sigset_t))
return -EINVAL;
- if (copy_from_user(set, usigmask, sizeof(sigset_t)))
+ if (copy_from_user(&kmask, umask, sizeof(sigset_t)))
return -EFAULT;
- *oldset = current->blocked;
- set_current_blocked(set);
+ set_restore_sigmask();
+ current->saved_sigmask = current->blocked;
+ set_current_blocked(&kmask);
return 0;
}
-EXPORT_SYMBOL(set_user_sigmask);
#ifdef CONFIG_COMPAT
-int set_compat_user_sigmask(const compat_sigset_t __user *usigmask,
- sigset_t *set, sigset_t *oldset,
+int set_compat_user_sigmask(const compat_sigset_t __user *umask,
size_t sigsetsize)
{
- if (!usigmask)
- return 0;
+ sigset_t kmask;
+ if (!umask)
+ return 0;
if (sigsetsize != sizeof(compat_sigset_t))
return -EINVAL;
- if (get_compat_sigset(set, usigmask))
+ if (get_compat_sigset(&kmask, umask))
return -EFAULT;
- *oldset = current->blocked;
- set_current_blocked(set);
+ set_restore_sigmask();
+ current->saved_sigmask = current->blocked;
+ set_current_blocked(&kmask);
return 0;
}
-EXPORT_SYMBOL(set_compat_user_sigmask);
#endif
-/*
- * restore_user_sigmask:
- * usigmask: sigmask passed in from userland.
- * sigsaved: saved sigmask when the syscall started and changed the sigmask to
- * usigmask.
- *
- * This is useful for syscalls such as ppoll, pselect, io_pgetevents and
- * epoll_pwait where a new sigmask is passed in from userland for the syscalls.
- */
-void restore_user_sigmask(const void __user *usigmask, sigset_t *sigsaved,
- bool interrupted)
-{
-
- if (!usigmask)
- return;
- /*
- * When signals are pending, do not restore them here.
- * Restoring sigmask here can lead to delivering signals that the above
- * syscalls are intended to block because of the sigmask passed in.
- */
- if (interrupted) {
- current->saved_sigmask = *sigsaved;
- set_restore_sigmask();
- return;
- }
-
- /*
- * This is needed because the fast syscall return path does not restore
- * saved_sigmask when signals are not pending.
- */
- set_current_blocked(sigsaved);
-}
-EXPORT_SYMBOL(restore_user_sigmask);
-
/**
* sys_rt_sigprocmask - change the list of currently blocked signals
* @how: whether to add, remove, or set signals
#endif
static int __maybe_unused neg_one = -1;
-
-static int zero;
-static int __maybe_unused one = 1;
static int __maybe_unused two = 2;
static int __maybe_unused four = 4;
static unsigned long zero_ul;
* enum sysctl_writes_mode - supported sysctl write modes
*
* @SYSCTL_WRITES_LEGACY: each write syscall must fully contain the sysctl value
- * to be written, and multiple writes on the same sysctl file descriptor
- * will rewrite the sysctl value, regardless of file position. No warning
- * is issued when the initial position is not 0.
+ * to be written, and multiple writes on the same sysctl file descriptor
+ * will rewrite the sysctl value, regardless of file position. No warning
+ * is issued when the initial position is not 0.
* @SYSCTL_WRITES_WARN: same as above but warn when the initial file position is
- * not 0.
+ * not 0.
* @SYSCTL_WRITES_STRICT: writes to numeric sysctl entries must always be at
- * file position 0 and the value must be fully contained in the buffer
- * sent to the write syscall. If dealing with strings respect the file
- * position, but restrict this to the max length of the buffer, anything
- * passed the max lenght will be ignored. Multiple writes will append
- * to the buffer.
+ * file position 0 and the value must be fully contained in the buffer
+ * sent to the write syscall. If dealing with strings respect the file
+ * position, but restrict this to the max length of the buffer, anything
+ * passed the max length will be ignored. Multiple writes will append
+ * to the buffer.
*
* These write modes control how current file position affects the behavior of
* updating sysctl values through the proc interface on each write.
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = sysctl_schedstats,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif /* CONFIG_SCHEDSTATS */
#endif /* CONFIG_SMP */
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{
.procname = "numa_balancing",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = sysctl_numa_balancing,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif /* CONFIG_NUMA_BALANCING */
#endif /* CONFIG_SCHED_DEBUG */
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_CFS_BANDWIDTH
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
#endif
#if defined(CONFIG_ENERGY_MODEL) && defined(CONFIG_CPU_FREQ_GOV_SCHEDUTIL)
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = sched_energy_aware_handler,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_PROVE_LOCKING
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
.extra1 = &neg_one,
- .extra2 = &one,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_LATENCYTOP
.mode = 0644,
/* only handle a transition from default "0" to "1" */
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &one,
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_MODULES
.mode = 0644,
/* only handle a transition from default "0" to "1" */
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &one,
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_UEVENT_HELPER
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &ten_thousand,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax_sysadmin,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "kptr_restrict",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax_sysadmin,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
#endif
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_watchdog,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "watchdog_thresh",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_watchdog_thresh,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &sixty,
},
{
.maxlen = sizeof(int),
.mode = NMI_WATCHDOG_SYSCTL_PERM,
.proc_handler = proc_nmi_watchdog,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "watchdog_cpumask",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_soft_watchdog,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "softlockup_panic",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#ifdef CONFIG_SMP
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif /* CONFIG_SMP */
#endif
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#ifdef CONFIG_SMP
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif /* CONFIG_SMP */
#endif
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "hung_task_check_count",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "hung_task_timeout_secs",
.maxlen = sizeof(sysctl_perf_event_sample_rate),
.mode = 0644,
.proc_handler = perf_proc_update_handler,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{
.procname = "perf_cpu_time_max_percent",
.maxlen = sizeof(sysctl_perf_cpu_time_max_percent),
.mode = 0644,
.proc_handler = perf_cpu_time_max_percent_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
{
.maxlen = sizeof(sysctl_perf_event_max_stack),
.mode = 0644,
.proc_handler = perf_event_max_stack_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &six_hundred_forty_kb,
},
{
.maxlen = sizeof(sysctl_perf_event_max_contexts_per_stack),
.mode = 0644,
.proc_handler = perf_event_max_stack_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_thousand,
},
#endif
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#if defined(CONFIG_SMP) && defined(CONFIG_NO_HZ_COMMON)
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = timer_migration_handler,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_BPF_SYSCALL
.mode = 0644,
/* only handle a transition from default "0" to "1" */
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &one,
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "bpf_stats_enabled",
.maxlen = sizeof(sysctl_panic_on_rcu_stall),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_STACKLEAK_RUNTIME_DISABLE
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = stack_erasing_sysctl,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{ }
.maxlen = sizeof(sysctl_overcommit_memory),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(sysctl_panic_on_oom),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "dirty_background_ratio",
.maxlen = sizeof(dirty_background_ratio),
.mode = 0644,
.proc_handler = dirty_background_ratio_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
{
.maxlen = sizeof(vm_dirty_ratio),
.mode = 0644,
.proc_handler = dirty_ratio_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
{
.maxlen = sizeof(dirty_expire_interval),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "dirtytime_expire_seconds",
.maxlen = sizeof(dirtytime_expire_interval),
.mode = 0644,
.proc_handler = dirtytime_interval_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "swappiness",
.maxlen = sizeof(vm_swappiness),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
#ifdef CONFIG_HUGETLB_PAGE
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = sysctl_vm_numa_stat_handler,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = drop_caches_sysctl_handler,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &four,
},
#ifdef CONFIG_COMPACTION
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif /* CONFIG_COMPACTION */
.maxlen = sizeof(min_free_kbytes),
.mode = 0644,
.proc_handler = min_free_kbytes_sysctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "watermark_boost_factor",
.maxlen = sizeof(watermark_boost_factor),
.mode = 0644,
.proc_handler = watermark_boost_factor_sysctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "watermark_scale_factor",
.maxlen = sizeof(watermark_scale_factor),
.mode = 0644,
.proc_handler = watermark_scale_factor_sysctl_handler,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &one_thousand,
},
{
.maxlen = sizeof(percpu_pagelist_fraction),
.mode = 0644,
.proc_handler = percpu_pagelist_fraction_sysctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#ifdef CONFIG_MMU
{
.maxlen = sizeof(sysctl_max_map_count),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#else
{
.maxlen = sizeof(sysctl_nr_trim_pages),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#endif
{
.maxlen = sizeof(block_dump),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "vfs_cache_pressure",
.maxlen = sizeof(sysctl_vfs_cache_pressure),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#ifdef HAVE_ARCH_PICK_MMAP_LAYOUT
{
.maxlen = sizeof(sysctl_legacy_va_layout),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#endif
#ifdef CONFIG_NUMA
.maxlen = sizeof(node_reclaim_mode),
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "min_unmapped_ratio",
.maxlen = sizeof(sysctl_min_unmapped_ratio),
.mode = 0644,
.proc_handler = sysctl_min_unmapped_ratio_sysctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
{
.maxlen = sizeof(sysctl_min_slab_ratio),
.mode = 0644,
.proc_handler = sysctl_min_slab_ratio_sysctl_handler,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_hundred,
},
#endif
#endif
.mode = 0644,
.proc_handler = proc_dointvec,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#endif
#ifdef CONFIG_HIGHMEM
.maxlen = sizeof(vm_highmem_is_dirtyable),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
#ifdef CONFIG_MEMORY_FAILURE
.maxlen = sizeof(sysctl_memory_failure_early_kill),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "memory_failure_recovery",
.maxlen = sizeof(sysctl_memory_failure_recovery),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(sysctl_unprivileged_userfaultfd),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{ }
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "protected_hardlinks",
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "protected_fifos",
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax_coredump,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
#if defined(CONFIG_BINFMT_MISC) || defined(CONFIG_BINFMT_MISC_MODULE)
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{ }
};
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_kprobes_optimization_handler,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{ }
.data = &val,
.maxlen = sizeof(val),
.mode = table->mode,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
};
if (write && !capable(CAP_SYS_ADMIN))
functioning properly. It will do tests on all the configured
tracers of ftrace.
+config EVENT_TRACE_STARTUP_TEST
+ bool "Run selftest on trace events"
+ depends on FTRACE_STARTUP_TEST
+ default y
+ help
+ This option performs a test on all trace events in the system.
+ It basically just enables each event and runs some code that
+ will trigger events (not necessarily the event it enables)
+ This may take some time run as there are a lot of events.
+
config EVENT_TRACE_TEST_SYSCALLS
bool "Run selftest on syscall events"
- depends on FTRACE_STARTUP_TEST
+ depends on EVENT_TRACE_STARTUP_TEST
help
This option will also enable testing every syscall event.
It only enables the event and disables it and runs various loads
return keep_regs;
}
+static struct ftrace_ops *
+ftrace_find_tramp_ops_any(struct dyn_ftrace *rec);
+static struct ftrace_ops *
+ftrace_find_tramp_ops_next(struct dyn_ftrace *rec, struct ftrace_ops *ops);
+
static bool __ftrace_hash_rec_update(struct ftrace_ops *ops,
int filter_hash,
bool inc)
}
/*
- * If the rec had TRAMP enabled, then it needs to
- * be cleared. As TRAMP can only be enabled iff
- * there is only a single ops attached to it.
- * In otherwords, always disable it on decrementing.
- * In the future, we may set it if rec count is
- * decremented to one, and the ops that is left
- * has a trampoline.
+ * The TRAMP needs to be set only if rec count
+ * is decremented to one, and the ops that is
+ * left has a trampoline. As TRAMP can only be
+ * enabled if there is only a single ops attached
+ * to it.
*/
- rec->flags &= ~FTRACE_FL_TRAMP;
+ if (ftrace_rec_count(rec) == 1 &&
+ ftrace_find_tramp_ops_any(rec))
+ rec->flags |= FTRACE_FL_TRAMP;
+ else
+ rec->flags &= ~FTRACE_FL_TRAMP;
/*
* flags will be cleared in ftrace_check_record()
count++;
/* Must match FTRACE_UPDATE_CALLS in ftrace_modify_all_code() */
- update |= ftrace_test_record(rec, 1) != FTRACE_UPDATE_IGNORE;
+ update |= ftrace_test_record(rec, true) != FTRACE_UPDATE_IGNORE;
/* Shortcut, if we handled all records, we are done. */
if (!all && count == hash->count)
printk(KERN_CONT "%s%02x", i ? ":" : "", p[i]);
}
-static struct ftrace_ops *
-ftrace_find_tramp_ops_any(struct dyn_ftrace *rec);
-static struct ftrace_ops *
-ftrace_find_tramp_ops_next(struct dyn_ftrace *rec, struct ftrace_ops *ops);
-
enum ftrace_bug_type ftrace_bug_type;
const void *ftrace_expected;
}
}
-static int ftrace_check_record(struct dyn_ftrace *rec, int enable, int update)
+static int ftrace_check_record(struct dyn_ftrace *rec, bool enable, bool update)
{
unsigned long flag = 0UL;
/**
* ftrace_update_record, set a record that now is tracing or not
* @rec: the record to update
- * @enable: set to 1 if the record is tracing, zero to force disable
+ * @enable: set to true if the record is tracing, false to force disable
*
* The records that represent all functions that can be traced need
* to be updated when tracing has been enabled.
*/
-int ftrace_update_record(struct dyn_ftrace *rec, int enable)
+int ftrace_update_record(struct dyn_ftrace *rec, bool enable)
{
- return ftrace_check_record(rec, enable, 1);
+ return ftrace_check_record(rec, enable, true);
}
/**
* ftrace_test_record, check if the record has been enabled or not
* @rec: the record to test
- * @enable: set to 1 to check if enabled, 0 if it is disabled
+ * @enable: set to true to check if enabled, false if it is disabled
*
* The arch code may need to test if a record is already set to
* tracing to determine how to modify the function code that it
* represents.
*/
-int ftrace_test_record(struct dyn_ftrace *rec, int enable)
+int ftrace_test_record(struct dyn_ftrace *rec, bool enable)
{
- return ftrace_check_record(rec, enable, 0);
+ return ftrace_check_record(rec, enable, false);
}
static struct ftrace_ops *
}
static int
-__ftrace_replace_code(struct dyn_ftrace *rec, int enable)
+__ftrace_replace_code(struct dyn_ftrace *rec, bool enable)
{
unsigned long ftrace_old_addr;
unsigned long ftrace_addr;
{
struct dyn_ftrace *rec;
struct ftrace_page *pg;
- int enable = mod_flags & FTRACE_MODIFY_ENABLE_FL;
+ bool enable = mod_flags & FTRACE_MODIFY_ENABLE_FL;
int schedulable = mod_flags & FTRACE_MODIFY_MAY_SLEEP_FL;
int failed;
#define RB_ALIGNMENT 4U
#define RB_MAX_SMALL_DATA (RB_ALIGNMENT * RINGBUF_TYPE_DATA_TYPE_LEN_MAX)
#define RB_EVNT_MIN_SIZE 8U /* two 32bit words */
-
-#ifndef CONFIG_HAVE_64BIT_ALIGNED_ACCESS
-# define RB_FORCE_8BYTE_ALIGNMENT 0
-# define RB_ARCH_ALIGNMENT RB_ALIGNMENT
-#else
-# define RB_FORCE_8BYTE_ALIGNMENT 1
-# define RB_ARCH_ALIGNMENT 8U
-#endif
-
-#define RB_ALIGN_DATA __aligned(RB_ARCH_ALIGNMENT)
+#define RB_ALIGN_DATA __aligned(RB_ALIGNMENT)
/* define RINGBUF_TYPE_DATA for 'case RINGBUF_TYPE_DATA:' */
#define RINGBUF_TYPE_DATA 0 ... RINGBUF_TYPE_DATA_TYPE_LEN_MAX
event->time_delta = delta;
length -= RB_EVNT_HDR_SIZE;
- if (length > RB_MAX_SMALL_DATA || RB_FORCE_8BYTE_ALIGNMENT) {
+ if (length > RB_MAX_SMALL_DATA) {
event->type_len = 0;
event->array[0] = length;
} else
if (!length)
length++;
- if (length > RB_MAX_SMALL_DATA || RB_FORCE_8BYTE_ALIGNMENT)
+ if (length > RB_MAX_SMALL_DATA)
length += sizeof(event.array[0]);
length += RB_EVNT_HDR_SIZE;
- length = ALIGN(length, RB_ARCH_ALIGNMENT);
+ length = ALIGN(length, RB_ALIGNMENT);
/*
* In case the time delta is larger than the 27 bits for it
}
/**
- * trace_pid_filter_add_remove_task - Add or remove a task from a pid_list
+ * trace_filter_add_remove_task - Add or remove a task from a pid_list
* @pid_list: The list to modify
* @self: The current task for fork or NULL for exit
* @task: The task to add or remove
{
struct trace_entry *ent = ring_buffer_event_data(event);
- tracing_generic_entry_update(ent, flags, pc);
- ent->type = type;
+ tracing_generic_entry_update(ent, type, flags, pc);
}
static __always_inline struct ring_buffer_event *
EXPORT_SYMBOL_GPL(trace_handle_return);
void
-tracing_generic_entry_update(struct trace_entry *entry, unsigned long flags,
- int pc)
+tracing_generic_entry_update(struct trace_entry *entry, unsigned short type,
+ unsigned long flags, int pc)
{
struct task_struct *tsk = current;
entry->preempt_count = pc & 0xff;
entry->pid = (tsk) ? tsk->pid : 0;
+ entry->type = type;
entry->flags =
#ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT
(irqs_disabled_flags(flags) ? TRACE_FLAG_IRQS_OFF : 0) |
"\t args: <name>=fetcharg[:type]\n"
"\t fetcharg: %<register>, @<address>, @<symbol>[+|-<offset>],\n"
#ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API
- "\t $stack<index>, $stack, $retval, $comm, $arg<N>\n"
+ "\t $stack<index>, $stack, $retval, $comm, $arg<N>,\n"
#else
- "\t $stack<index>, $stack, $retval, $comm\n"
+ "\t $stack<index>, $stack, $retval, $comm,\n"
#endif
+ "\t +|-[u]<offset>(<fetcharg>)\n"
"\t type: s8/16/32/64, u8/16/32/64, x8/16/32/64, string, symbol,\n"
- "\t b<bit-width>@<bit-offset>/<container-size>,\n"
+ "\t b<bit-width>@<bit-offset>/<container-size>, ustring,\n"
"\t <type>\\[<array-size>\\]\n"
#ifdef CONFIG_HIST_TRIGGERS
"\t field: <stype> <name>;\n"
unsigned long flags;
local_save_flags(flags);
- tracing_generic_entry_update(entry, flags, pc);
- entry->type = type;
+ tracing_generic_entry_update(entry, type, flags, pc);
}
NOKPROBE_SYMBOL(perf_trace_buf_update);
#define while_for_each_event_file() \
}
-static struct list_head *
-trace_get_fields(struct trace_event_call *event_call)
-{
- if (!event_call->class->get_fields)
- return &event_call->class->fields;
- return event_call->class->get_fields(event_call);
-}
-
static struct ftrace_event_field *
__find_event_field(struct list_head *head, char *name)
{
event_trace_enable();
}
-#ifdef CONFIG_FTRACE_STARTUP_TEST
+#ifdef CONFIG_EVENT_TRACE_STARTUP_TEST
static DEFINE_SPINLOCK(test_spinlock);
static DEFINE_SPINLOCK(test_spinlock_irq);
if (strchr(type, '[') && strstr(type, "char"))
return FILTER_STATIC_STRING;
+ if (strcmp(type, "char *") == 0 || strcmp(type, "const char *") == 0)
+ return FILTER_PTR_STRING;
+
return FILTER_OTHER;
}
#include <linux/rculist.h>
#include <linux/error-injection.h>
+#include <asm/setup.h> /* for COMMAND_LINE_SIZE */
+
#include "trace_dynevent.h"
#include "trace_kprobe_selftest.h"
#include "trace_probe.h"
#define KPROBE_EVENT_SYSTEM "kprobes"
#define KRETPROBE_MAXACTIVE_MAX 4096
+#define MAX_KPROBE_CMDLINE_SIZE 1024
+
+/* Kprobe early definition from command line */
+static char kprobe_boot_events_buf[COMMAND_LINE_SIZE] __initdata;
+static bool kprobe_boot_events_enabled __initdata;
+
+static int __init set_kprobe_boot_events(char *str)
+{
+ strlcpy(kprobe_boot_events_buf, str, COMMAND_LINE_SIZE);
+ return 0;
+}
+__setup("kprobe_event=", set_kprobe_boot_events);
static int trace_kprobe_create(int argc, const char **argv);
static int trace_kprobe_show(struct seq_file *m, struct dyn_event *ev);
{
struct trace_kprobe *tk = to_trace_kprobe(ev);
- return strcmp(trace_event_name(&tk->tp.call), event) == 0 &&
- (!system || strcmp(tk->tp.call.class->system, system) == 0);
+ return strcmp(trace_probe_name(&tk->tp), event) == 0 &&
+ (!system || strcmp(trace_probe_group_name(&tk->tp), system) == 0);
}
static nokprobe_inline unsigned long trace_kprobe_nhit(struct trace_kprobe *tk)
return nhit;
}
+static nokprobe_inline bool trace_kprobe_is_registered(struct trace_kprobe *tk)
+{
+ return !(list_empty(&tk->rp.kp.list) &&
+ hlist_unhashed(&tk->rp.kp.hlist));
+}
+
/* Return 0 if it fails to find the symbol address */
static nokprobe_inline
unsigned long trace_kprobe_address(struct trace_kprobe *tk)
static int kretprobe_dispatcher(struct kretprobe_instance *ri,
struct pt_regs *regs);
+static void free_trace_kprobe(struct trace_kprobe *tk)
+{
+ if (tk) {
+ trace_probe_cleanup(&tk->tp);
+ kfree(tk->symbol);
+ free_percpu(tk->nhit);
+ kfree(tk);
+ }
+}
+
/*
* Allocate new trace_probe and initialize it (including kprobes).
*/
tk->rp.kp.pre_handler = kprobe_dispatcher;
tk->rp.maxactive = maxactive;
+ INIT_HLIST_NODE(&tk->rp.kp.hlist);
+ INIT_LIST_HEAD(&tk->rp.kp.list);
- if (!event || !group) {
- ret = -EINVAL;
- goto error;
- }
-
- tk->tp.call.class = &tk->tp.class;
- tk->tp.call.name = kstrdup(event, GFP_KERNEL);
- if (!tk->tp.call.name)
- goto error;
-
- tk->tp.class.system = kstrdup(group, GFP_KERNEL);
- if (!tk->tp.class.system)
+ ret = trace_probe_init(&tk->tp, event, group);
+ if (ret < 0)
goto error;
dyn_event_init(&tk->devent, &trace_kprobe_ops);
- INIT_LIST_HEAD(&tk->tp.files);
return tk;
error:
- kfree(tk->tp.call.name);
- kfree(tk->symbol);
- free_percpu(tk->nhit);
- kfree(tk);
+ free_trace_kprobe(tk);
return ERR_PTR(ret);
}
-static void free_trace_kprobe(struct trace_kprobe *tk)
-{
- int i;
-
- if (!tk)
- return;
-
- for (i = 0; i < tk->tp.nr_args; i++)
- traceprobe_free_probe_arg(&tk->tp.args[i]);
-
- kfree(tk->tp.call.class->system);
- kfree(tk->tp.call.name);
- kfree(tk->symbol);
- free_percpu(tk->nhit);
- kfree(tk);
-}
-
static struct trace_kprobe *find_trace_kprobe(const char *event,
const char *group)
{
struct trace_kprobe *tk;
for_each_trace_kprobe(tk, pos)
- if (strcmp(trace_event_name(&tk->tp.call), event) == 0 &&
- strcmp(tk->tp.call.class->system, group) == 0)
+ if (strcmp(trace_probe_name(&tk->tp), event) == 0 &&
+ strcmp(trace_probe_group_name(&tk->tp), group) == 0)
return tk;
return NULL;
}
{
int ret = 0;
- if (trace_probe_is_registered(&tk->tp) && !trace_kprobe_has_gone(tk)) {
+ if (trace_kprobe_is_registered(tk) && !trace_kprobe_has_gone(tk)) {
if (trace_kprobe_is_return(tk))
ret = enable_kretprobe(&tk->rp);
else
static int
enable_trace_kprobe(struct trace_kprobe *tk, struct trace_event_file *file)
{
- struct event_file_link *link;
+ bool enabled = trace_probe_is_enabled(&tk->tp);
int ret = 0;
if (file) {
- link = kmalloc(sizeof(*link), GFP_KERNEL);
- if (!link) {
- ret = -ENOMEM;
- goto out;
- }
-
- link->file = file;
- list_add_tail_rcu(&link->list, &tk->tp.files);
+ ret = trace_probe_add_file(&tk->tp, file);
+ if (ret)
+ return ret;
+ } else
+ trace_probe_set_flag(&tk->tp, TP_FLAG_PROFILE);
- tk->tp.flags |= TP_FLAG_TRACE;
- ret = __enable_trace_kprobe(tk);
- if (ret) {
- list_del_rcu(&link->list);
- kfree(link);
- tk->tp.flags &= ~TP_FLAG_TRACE;
- }
+ if (enabled)
+ return 0;
- } else {
- tk->tp.flags |= TP_FLAG_PROFILE;
- ret = __enable_trace_kprobe(tk);
- if (ret)
- tk->tp.flags &= ~TP_FLAG_PROFILE;
+ ret = __enable_trace_kprobe(tk);
+ if (ret) {
+ if (file)
+ trace_probe_remove_file(&tk->tp, file);
+ else
+ trace_probe_clear_flag(&tk->tp, TP_FLAG_PROFILE);
}
- out:
+
return ret;
}
static int
disable_trace_kprobe(struct trace_kprobe *tk, struct trace_event_file *file)
{
- struct event_file_link *link = NULL;
- int wait = 0;
+ struct trace_probe *tp = &tk->tp;
int ret = 0;
if (file) {
- link = find_event_file_link(&tk->tp, file);
- if (!link) {
- ret = -EINVAL;
- goto out;
- }
-
- list_del_rcu(&link->list);
- wait = 1;
- if (!list_empty(&tk->tp.files))
+ if (!trace_probe_get_file_link(tp, file))
+ return -ENOENT;
+ if (!trace_probe_has_single_file(tp))
goto out;
-
- tk->tp.flags &= ~TP_FLAG_TRACE;
+ trace_probe_clear_flag(tp, TP_FLAG_TRACE);
} else
- tk->tp.flags &= ~TP_FLAG_PROFILE;
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
- if (!trace_probe_is_enabled(&tk->tp) && trace_probe_is_registered(&tk->tp)) {
+ if (!trace_probe_is_enabled(tp) && trace_kprobe_is_registered(tk)) {
if (trace_kprobe_is_return(tk))
disable_kretprobe(&tk->rp);
else
disable_kprobe(&tk->rp.kp);
- wait = 1;
}
- /*
- * if tk is not added to any list, it must be a local trace_kprobe
- * created with perf_event_open. We don't need to wait for these
- * trace_kprobes
- */
- if (list_empty(&tk->devent.list))
- wait = 0;
out:
- if (wait) {
+ if (file)
/*
- * Synchronize with kprobe_trace_func/kretprobe_trace_func
- * to ensure disabled (all running handlers are finished).
- * This is not only for kfree(), but also the caller,
- * trace_remove_event_call() supposes it for releasing
- * event_call related objects, which will be accessed in
- * the kprobe_trace_func/kretprobe_trace_func.
+ * Synchronization is done in below function. For perf event,
+ * file == NULL and perf_trace_event_unreg() calls
+ * tracepoint_synchronize_unregister() to ensure synchronize
+ * event. We don't need to care about it.
*/
- synchronize_rcu();
- kfree(link); /* Ignored if link == NULL */
- }
+ trace_probe_remove_file(tp, file);
return ret;
}
{
int i, ret;
- if (trace_probe_is_registered(&tk->tp))
+ if (trace_kprobe_is_registered(tk))
return -EINVAL;
if (within_notrace_func(tk)) {
else
ret = register_kprobe(&tk->rp.kp);
- if (ret == 0)
- tk->tp.flags |= TP_FLAG_REGISTERED;
return ret;
}
/* Internal unregister function - just handle k*probes and flags */
static void __unregister_trace_kprobe(struct trace_kprobe *tk)
{
- if (trace_probe_is_registered(&tk->tp)) {
+ if (trace_kprobe_is_registered(tk)) {
if (trace_kprobe_is_return(tk))
unregister_kretprobe(&tk->rp);
else
unregister_kprobe(&tk->rp.kp);
- tk->tp.flags &= ~TP_FLAG_REGISTERED;
- /* Cleanup kprobe for reuse */
+ /* Cleanup kprobe for reuse and mark it unregistered */
+ INIT_HLIST_NODE(&tk->rp.kp.hlist);
+ INIT_LIST_HEAD(&tk->rp.kp.list);
if (tk->rp.kp.symbol_name)
tk->rp.kp.addr = NULL;
}
mutex_lock(&event_mutex);
/* Delete old (same name) event if exist */
- old_tk = find_trace_kprobe(trace_event_name(&tk->tp.call),
- tk->tp.call.class->system);
+ old_tk = find_trace_kprobe(trace_probe_name(&tk->tp),
+ trace_probe_group_name(&tk->tp));
if (old_tk) {
ret = unregister_trace_kprobe(old_tk);
if (ret < 0)
ret = __register_trace_kprobe(tk);
if (ret)
pr_warn("Failed to re-register probe %s on %s: %d\n",
- trace_event_name(&tk->tp.call),
+ trace_probe_name(&tk->tp),
mod->name, ret);
}
}
goto error; /* This can be -ENOMEM */
}
+ ret = traceprobe_set_print_fmt(&tk->tp, is_return);
+ if (ret < 0)
+ goto error;
+
ret = register_trace_kprobe(tk);
if (ret) {
trace_probe_log_set_index(1);
int i;
seq_putc(m, trace_kprobe_is_return(tk) ? 'r' : 'p');
- seq_printf(m, ":%s/%s", tk->tp.call.class->system,
- trace_event_name(&tk->tp.call));
+ seq_printf(m, ":%s/%s", trace_probe_group_name(&tk->tp),
+ trace_probe_name(&tk->tp));
if (!tk->symbol)
seq_printf(m, " 0x%p", tk->rp.kp.addr);
tk = to_trace_kprobe(ev);
seq_printf(m, " %-44s %15lu %15lu\n",
- trace_event_name(&tk->tp.call),
+ trace_probe_name(&tk->tp),
trace_kprobe_nhit(tk),
tk->rp.kp.nmissed);
return (ret < 0) ? ret : len;
}
+/* Return the length of string -- including null terminal byte */
+static nokprobe_inline int
+fetch_store_strlen_user(unsigned long addr)
+{
+ const void __user *uaddr = (__force const void __user *)addr;
+
+ return strnlen_unsafe_user(uaddr, MAX_STRING_SIZE);
+}
+
/*
* Fetch a null-terminated string. Caller MUST set *(u32 *)buf with max
* length and relative data location.
fetch_store_string(unsigned long addr, void *dest, void *base)
{
int maxlen = get_loc_len(*(u32 *)dest);
- u8 *dst = get_loc_data(dest, base);
+ void *__dest;
long ret;
if (unlikely(!maxlen))
return -ENOMEM;
+
+ __dest = get_loc_data(dest, base);
+
/*
* Try to get string again, since the string can be changed while
* probing.
*/
- ret = strncpy_from_unsafe(dst, (void *)addr, maxlen);
+ ret = strncpy_from_unsafe(__dest, (void *)addr, maxlen);
+ if (ret >= 0)
+ *(u32 *)dest = make_data_loc(ret, __dest - base);
+
+ return ret;
+}
+/*
+ * Fetch a null-terminated string from user. Caller MUST set *(u32 *)buf
+ * with max length and relative data location.
+ */
+static nokprobe_inline int
+fetch_store_string_user(unsigned long addr, void *dest, void *base)
+{
+ const void __user *uaddr = (__force const void __user *)addr;
+ int maxlen = get_loc_len(*(u32 *)dest);
+ void *__dest;
+ long ret;
+
+ if (unlikely(!maxlen))
+ return -ENOMEM;
+
+ __dest = get_loc_data(dest, base);
+
+ ret = strncpy_from_unsafe_user(__dest, uaddr, maxlen);
if (ret >= 0)
- *(u32 *)dest = make_data_loc(ret, (void *)dst - base);
+ *(u32 *)dest = make_data_loc(ret, __dest - base);
+
return ret;
}
return probe_kernel_read(dest, src, size);
}
+static nokprobe_inline int
+probe_mem_read_user(void *dest, void *src, size_t size)
+{
+ const void __user *uaddr = (__force const void __user *)src;
+
+ return probe_user_read(dest, uaddr, size);
+}
+
/* Note that we don't verify it, since the code does not come from user space */
static int
process_fetch_insn(struct fetch_insn *code, struct pt_regs *regs, void *dest,
struct ring_buffer *buffer;
int size, dsize, pc;
unsigned long irq_flags;
- struct trace_event_call *call = &tk->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tk->tp);
WARN_ON(call != trace_file->event_call);
{
struct event_file_link *link;
- list_for_each_entry_rcu(link, &tk->tp.files, list)
+ trace_probe_for_each_link_rcu(link, &tk->tp)
__kprobe_trace_func(tk, regs, link->file);
}
NOKPROBE_SYMBOL(kprobe_trace_func);
struct ring_buffer *buffer;
int size, pc, dsize;
unsigned long irq_flags;
- struct trace_event_call *call = &tk->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tk->tp);
WARN_ON(call != trace_file->event_call);
{
struct event_file_link *link;
- list_for_each_entry_rcu(link, &tk->tp.files, list)
+ trace_probe_for_each_link_rcu(link, &tk->tp)
__kretprobe_trace_func(tk, ri, regs, link->file);
}
NOKPROBE_SYMBOL(kretprobe_trace_func);
field = (struct kprobe_trace_entry_head *)iter->ent;
tp = container_of(event, struct trace_probe, call.event);
- trace_seq_printf(s, "%s: (", trace_event_name(&tp->call));
+ trace_seq_printf(s, "%s: (", trace_probe_name(tp));
if (!seq_print_ip_sym(s, field->ip, flags | TRACE_ITER_SYM_OFFSET))
goto out;
field = (struct kretprobe_trace_entry_head *)iter->ent;
tp = container_of(event, struct trace_probe, call.event);
- trace_seq_printf(s, "%s: (", trace_event_name(&tp->call));
+ trace_seq_printf(s, "%s: (", trace_probe_name(tp));
if (!seq_print_ip_sym(s, field->ret_ip, flags | TRACE_ITER_SYM_OFFSET))
goto out;
static int
kprobe_perf_func(struct trace_kprobe *tk, struct pt_regs *regs)
{
- struct trace_event_call *call = &tk->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tk->tp);
struct kprobe_trace_entry_head *entry;
struct hlist_head *head;
int size, __size, dsize;
kretprobe_perf_func(struct trace_kprobe *tk, struct kretprobe_instance *ri,
struct pt_regs *regs)
{
- struct trace_event_call *call = &tk->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tk->tp);
struct kretprobe_trace_entry_head *entry;
struct hlist_head *head;
int size, __size, dsize;
raw_cpu_inc(*tk->nhit);
- if (tk->tp.flags & TP_FLAG_TRACE)
+ if (trace_probe_test_flag(&tk->tp, TP_FLAG_TRACE))
kprobe_trace_func(tk, regs);
#ifdef CONFIG_PERF_EVENTS
- if (tk->tp.flags & TP_FLAG_PROFILE)
+ if (trace_probe_test_flag(&tk->tp, TP_FLAG_PROFILE))
ret = kprobe_perf_func(tk, regs);
#endif
return ret;
raw_cpu_inc(*tk->nhit);
- if (tk->tp.flags & TP_FLAG_TRACE)
+ if (trace_probe_test_flag(&tk->tp, TP_FLAG_TRACE))
kretprobe_trace_func(tk, ri, regs);
#ifdef CONFIG_PERF_EVENTS
- if (tk->tp.flags & TP_FLAG_PROFILE)
+ if (trace_probe_test_flag(&tk->tp, TP_FLAG_PROFILE))
kretprobe_perf_func(tk, ri, regs);
#endif
return 0; /* We don't tweek kernel, so just return 0 */
.trace = print_kprobe_event
};
-static inline void init_trace_event_call(struct trace_kprobe *tk,
- struct trace_event_call *call)
+static inline void init_trace_event_call(struct trace_kprobe *tk)
{
- INIT_LIST_HEAD(&call->class->fields);
+ struct trace_event_call *call = trace_probe_event_call(&tk->tp);
+
if (trace_kprobe_is_return(tk)) {
call->event.funcs = &kretprobe_funcs;
call->class->define_fields = kretprobe_event_define_fields;
static int register_kprobe_event(struct trace_kprobe *tk)
{
- struct trace_event_call *call = &tk->tp.call;
- int ret = 0;
-
- init_trace_event_call(tk, call);
+ init_trace_event_call(tk);
- if (traceprobe_set_print_fmt(&tk->tp, trace_kprobe_is_return(tk)) < 0)
- return -ENOMEM;
- ret = register_trace_event(&call->event);
- if (!ret) {
- kfree(call->print_fmt);
- return -ENODEV;
- }
- ret = trace_add_event_call(call);
- if (ret) {
- pr_info("Failed to register kprobe event: %s\n",
- trace_event_name(call));
- kfree(call->print_fmt);
- unregister_trace_event(&call->event);
- }
- return ret;
+ return trace_probe_register_event_call(&tk->tp);
}
static int unregister_kprobe_event(struct trace_kprobe *tk)
{
- int ret;
-
- /* tp->event is unregistered in trace_remove_event_call() */
- ret = trace_remove_event_call(&tk->tp.call);
- if (!ret)
- kfree(tk->tp.call.print_fmt);
- return ret;
+ return trace_probe_unregister_event_call(&tk->tp);
}
#ifdef CONFIG_PERF_EVENTS
return ERR_CAST(tk);
}
- init_trace_event_call(tk, &tk->tp.call);
+ init_trace_event_call(tk);
if (traceprobe_set_print_fmt(&tk->tp, trace_kprobe_is_return(tk)) < 0) {
ret = -ENOMEM;
}
ret = __register_trace_kprobe(tk);
- if (ret < 0) {
- kfree(tk->tp.call.print_fmt);
+ if (ret < 0)
goto error;
- }
- return &tk->tp.call;
+ return trace_probe_event_call(&tk->tp);
error:
free_trace_kprobe(tk);
return ERR_PTR(ret);
__unregister_trace_kprobe(tk);
- kfree(tk->tp.call.print_fmt);
free_trace_kprobe(tk);
}
#endif /* CONFIG_PERF_EVENTS */
+static __init void enable_boot_kprobe_events(void)
+{
+ struct trace_array *tr = top_trace_array();
+ struct trace_event_file *file;
+ struct trace_kprobe *tk;
+ struct dyn_event *pos;
+
+ mutex_lock(&event_mutex);
+ for_each_trace_kprobe(tk, pos) {
+ list_for_each_entry(file, &tr->events, list)
+ if (file->event_call == trace_probe_event_call(&tk->tp))
+ trace_event_enable_disable(file, 1, 0);
+ }
+ mutex_unlock(&event_mutex);
+}
+
+static __init void setup_boot_kprobe_events(void)
+{
+ char *p, *cmd = kprobe_boot_events_buf;
+ int ret;
+
+ strreplace(kprobe_boot_events_buf, ',', ' ');
+
+ while (cmd && *cmd != '\0') {
+ p = strchr(cmd, ';');
+ if (p)
+ *p++ = '\0';
+
+ ret = trace_run_command(cmd, create_or_delete_trace_kprobe);
+ if (ret)
+ pr_warn("Failed to add event(%d): %s\n", ret, cmd);
+ else
+ kprobe_boot_events_enabled = true;
+
+ cmd = p;
+ }
+
+ enable_boot_kprobe_events();
+}
+
/* Make a tracefs interface for controlling probe points */
static __init int init_kprobe_trace(void)
{
if (!entry)
pr_warn("Could not create tracefs 'kprobe_profile' entry\n");
+
+ setup_boot_kprobe_events();
+
return 0;
}
fs_initcall(init_kprobe_trace);
struct trace_event_file *file;
list_for_each_entry(file, &tr->events, list)
- if (file->event_call == &tk->tp.call)
+ if (file->event_call == trace_probe_event_call(&tk->tp))
return file;
return NULL;
if (tracing_is_disabled())
return -ENODEV;
+ if (kprobe_boot_events_enabled) {
+ pr_info("Skipping kprobe tests due to kprobe_event on cmdline\n");
+ return 0;
+ }
+
target = kprobe_trace_selftest_target;
pr_info("Testing kprobe tracing: ");
/* Special types */
__ASSIGN_FETCH_TYPE("string", string, string, sizeof(u32), 1,
"__data_loc char[]"),
+ __ASSIGN_FETCH_TYPE("ustring", string, string, sizeof(u32), 1,
+ "__data_loc char[]"),
/* Basic types */
ASSIGN_FETCH_TYPE(u8, u8, 0),
ASSIGN_FETCH_TYPE(u16, u16, 0),
{
struct fetch_insn *code = *pcode;
unsigned long param;
+ int deref = FETCH_OP_DEREF;
long offset = 0;
char *tmp;
int ret = 0;
break;
case '+': /* deref memory */
- arg++; /* Skip '+', because kstrtol() rejects it. */
- /* fall through */
case '-':
+ if (arg[1] == 'u') {
+ deref = FETCH_OP_UDEREF;
+ arg[1] = arg[0];
+ arg++;
+ }
+ if (arg[0] == '+')
+ arg++; /* Skip '+', because kstrtol() rejects it. */
tmp = strchr(arg, '(');
if (!tmp) {
trace_probe_log_err(offs, DEREF_NEED_BRACE);
}
*pcode = code;
- code->op = FETCH_OP_DEREF;
+ code->op = deref;
code->offset = offset;
}
break;
goto fail;
/* Store operation */
- if (!strcmp(parg->type->name, "string")) {
- if (code->op != FETCH_OP_DEREF && code->op != FETCH_OP_IMM &&
- code->op != FETCH_OP_COMM) {
+ if (!strcmp(parg->type->name, "string") ||
+ !strcmp(parg->type->name, "ustring")) {
+ if (code->op != FETCH_OP_DEREF && code->op != FETCH_OP_UDEREF &&
+ code->op != FETCH_OP_IMM && code->op != FETCH_OP_COMM) {
trace_probe_log_err(offset + (t ? (t - arg) : 0),
BAD_STRING);
ret = -EINVAL;
goto fail;
}
- if (code->op != FETCH_OP_DEREF || parg->count) {
+ if ((code->op == FETCH_OP_IMM || code->op == FETCH_OP_COMM) ||
+ parg->count) {
/*
* IMM and COMM is pointing actual address, those must
* be kept, and if parg->count != 0, this is an array
goto fail;
}
}
- code->op = FETCH_OP_ST_STRING; /* In DEREF case, replace it */
+ /* If op == DEREF, replace it with STRING */
+ if (!strcmp(parg->type->name, "ustring") ||
+ code->op == FETCH_OP_UDEREF)
+ code->op = FETCH_OP_ST_USTRING;
+ else
+ code->op = FETCH_OP_ST_STRING;
code->size = parg->type->size;
parg->dynamic = true;
} else if (code->op == FETCH_OP_DEREF) {
code->op = FETCH_OP_ST_MEM;
code->size = parg->type->size;
+ } else if (code->op == FETCH_OP_UDEREF) {
+ code->op = FETCH_OP_ST_UMEM;
+ code->size = parg->type->size;
} else {
code++;
if (code->op != FETCH_OP_NOP) {
/* Loop(Array) operation */
if (parg->count) {
if (scode->op != FETCH_OP_ST_MEM &&
- scode->op != FETCH_OP_ST_STRING) {
+ scode->op != FETCH_OP_ST_STRING &&
+ scode->op != FETCH_OP_ST_USTRING) {
trace_probe_log_err(offset + (t ? (t - arg) : 0),
BAD_STRING);
ret = -EINVAL;
int traceprobe_set_print_fmt(struct trace_probe *tp, bool is_return)
{
+ struct trace_event_call *call = trace_probe_event_call(tp);
int len;
char *print_fmt;
/* Second: actually write the @print_fmt */
__set_print_fmt(tp, print_fmt, len + 1, is_return);
- tp->call.print_fmt = print_fmt;
+ call->print_fmt = print_fmt;
return 0;
}
}
return 0;
}
+
+
+void trace_probe_cleanup(struct trace_probe *tp)
+{
+ struct trace_event_call *call = trace_probe_event_call(tp);
+ int i;
+
+ for (i = 0; i < tp->nr_args; i++)
+ traceprobe_free_probe_arg(&tp->args[i]);
+
+ kfree(call->class->system);
+ kfree(call->name);
+ kfree(call->print_fmt);
+}
+
+int trace_probe_init(struct trace_probe *tp, const char *event,
+ const char *group)
+{
+ struct trace_event_call *call = trace_probe_event_call(tp);
+
+ if (!event || !group)
+ return -EINVAL;
+
+ call->class = &tp->class;
+ call->name = kstrdup(event, GFP_KERNEL);
+ if (!call->name)
+ return -ENOMEM;
+
+ tp->class.system = kstrdup(group, GFP_KERNEL);
+ if (!tp->class.system) {
+ kfree(call->name);
+ call->name = NULL;
+ return -ENOMEM;
+ }
+ INIT_LIST_HEAD(&tp->files);
+ INIT_LIST_HEAD(&tp->class.fields);
+
+ return 0;
+}
+
+int trace_probe_register_event_call(struct trace_probe *tp)
+{
+ struct trace_event_call *call = trace_probe_event_call(tp);
+ int ret;
+
+ ret = register_trace_event(&call->event);
+ if (!ret)
+ return -ENODEV;
+
+ ret = trace_add_event_call(call);
+ if (ret)
+ unregister_trace_event(&call->event);
+
+ return ret;
+}
+
+int trace_probe_add_file(struct trace_probe *tp, struct trace_event_file *file)
+{
+ struct event_file_link *link;
+
+ link = kmalloc(sizeof(*link), GFP_KERNEL);
+ if (!link)
+ return -ENOMEM;
+
+ link->file = file;
+ INIT_LIST_HEAD(&link->list);
+ list_add_tail_rcu(&link->list, &tp->files);
+ trace_probe_set_flag(tp, TP_FLAG_TRACE);
+ return 0;
+}
+
+struct event_file_link *trace_probe_get_file_link(struct trace_probe *tp,
+ struct trace_event_file *file)
+{
+ struct event_file_link *link;
+
+ trace_probe_for_each_link(link, tp) {
+ if (link->file == file)
+ return link;
+ }
+
+ return NULL;
+}
+
+int trace_probe_remove_file(struct trace_probe *tp,
+ struct trace_event_file *file)
+{
+ struct event_file_link *link;
+
+ link = trace_probe_get_file_link(tp, file);
+ if (!link)
+ return -ENOENT;
+
+ list_del_rcu(&link->list);
+ synchronize_rcu();
+ kfree(link);
+
+ if (list_empty(&tp->files))
+ trace_probe_clear_flag(tp, TP_FLAG_TRACE);
+
+ return 0;
+}
/* Flags for trace_probe */
#define TP_FLAG_TRACE 1
#define TP_FLAG_PROFILE 2
-#define TP_FLAG_REGISTERED 4
/* data_loc: data location, compatible with u32 */
#define make_data_loc(len, offs) \
FETCH_OP_FOFFS, /* File offset: .immediate */
// Stage 2 (dereference) op
FETCH_OP_DEREF, /* Dereference: .offset */
+ FETCH_OP_UDEREF, /* User-space Dereference: .offset */
// Stage 3 (store) ops
FETCH_OP_ST_RAW, /* Raw: .size */
FETCH_OP_ST_MEM, /* Mem: .offset, .size */
+ FETCH_OP_ST_UMEM, /* Mem: .offset, .size */
FETCH_OP_ST_STRING, /* String: .offset, .size */
+ FETCH_OP_ST_USTRING, /* User String: .offset, .size */
// Stage 4 (modify) op
FETCH_OP_MOD_BF, /* Bitfield: .basesize, .lshift, .rshift */
// Stage 5 (loop) op
struct list_head list;
};
+static inline bool trace_probe_test_flag(struct trace_probe *tp,
+ unsigned int flag)
+{
+ return !!(tp->flags & flag);
+}
+
+static inline void trace_probe_set_flag(struct trace_probe *tp,
+ unsigned int flag)
+{
+ tp->flags |= flag;
+}
+
+static inline void trace_probe_clear_flag(struct trace_probe *tp,
+ unsigned int flag)
+{
+ tp->flags &= ~flag;
+}
+
static inline bool trace_probe_is_enabled(struct trace_probe *tp)
{
- return !!(tp->flags & (TP_FLAG_TRACE | TP_FLAG_PROFILE));
+ return trace_probe_test_flag(tp, TP_FLAG_TRACE | TP_FLAG_PROFILE);
}
-static inline bool trace_probe_is_registered(struct trace_probe *tp)
+static inline const char *trace_probe_name(struct trace_probe *tp)
{
- return !!(tp->flags & TP_FLAG_REGISTERED);
+ return trace_event_name(&tp->call);
}
+static inline const char *trace_probe_group_name(struct trace_probe *tp)
+{
+ return tp->call.class->system;
+}
+
+static inline struct trace_event_call *
+ trace_probe_event_call(struct trace_probe *tp)
+{
+ return &tp->call;
+}
+
+static inline int trace_probe_unregister_event_call(struct trace_probe *tp)
+{
+ /* tp->event is unregistered in trace_remove_event_call() */
+ return trace_remove_event_call(&tp->call);
+}
+
+static inline bool trace_probe_has_single_file(struct trace_probe *tp)
+{
+ return !!list_is_singular(&tp->files);
+}
+
+int trace_probe_init(struct trace_probe *tp, const char *event,
+ const char *group);
+void trace_probe_cleanup(struct trace_probe *tp);
+int trace_probe_register_event_call(struct trace_probe *tp);
+int trace_probe_add_file(struct trace_probe *tp, struct trace_event_file *file);
+int trace_probe_remove_file(struct trace_probe *tp,
+ struct trace_event_file *file);
+struct event_file_link *trace_probe_get_file_link(struct trace_probe *tp,
+ struct trace_event_file *file);
+
+#define trace_probe_for_each_link(pos, tp) \
+ list_for_each_entry(pos, &(tp)->files, list)
+#define trace_probe_for_each_link_rcu(pos, tp) \
+ list_for_each_entry_rcu(pos, &(tp)->files, list)
+
/* Check the name is good for event/group/fields */
static inline bool is_good_name(const char *name)
{
return true;
}
-static inline struct event_file_link *
-find_event_file_link(struct trace_probe *tp, struct trace_event_file *file)
-{
- struct event_file_link *link;
-
- list_for_each_entry(link, &tp->files, list)
- if (link->file == file)
- return link;
-
- return NULL;
-}
-
#define TPARG_FL_RETURN BIT(0)
#define TPARG_FL_KERNEL BIT(1)
#define TPARG_FL_FENTRY BIT(2)
static nokprobe_inline int fetch_store_strlen(unsigned long addr);
static nokprobe_inline int
fetch_store_string(unsigned long addr, void *dest, void *base);
+static nokprobe_inline int fetch_store_strlen_user(unsigned long addr);
+static nokprobe_inline int
+fetch_store_string_user(unsigned long addr, void *dest, void *base);
static nokprobe_inline int
probe_mem_read(void *dest, void *src, size_t size);
+static nokprobe_inline int
+probe_mem_read_user(void *dest, void *src, size_t size);
/* From the 2nd stage, routine is same */
static nokprobe_inline int
stage2:
/* 2nd stage: dereference memory if needed */
- while (code->op == FETCH_OP_DEREF) {
- lval = val;
- ret = probe_mem_read(&val, (void *)val + code->offset,
- sizeof(val));
+ do {
+ if (code->op == FETCH_OP_DEREF) {
+ lval = val;
+ ret = probe_mem_read(&val, (void *)val + code->offset,
+ sizeof(val));
+ } else if (code->op == FETCH_OP_UDEREF) {
+ lval = val;
+ ret = probe_mem_read_user(&val,
+ (void *)val + code->offset, sizeof(val));
+ } else
+ break;
if (ret)
return ret;
code++;
- }
+ } while (1);
s3 = code;
stage3:
ret = fetch_store_strlen(val + code->offset);
code++;
goto array;
+ } else if (code->op == FETCH_OP_ST_USTRING) {
+ ret += fetch_store_strlen_user(val + code->offset);
+ code++;
+ goto array;
} else
return -EILSEQ;
}
case FETCH_OP_ST_MEM:
probe_mem_read(dest, (void *)val + code->offset, code->size);
break;
+ case FETCH_OP_ST_UMEM:
+ probe_mem_read_user(dest, (void *)val + code->offset, code->size);
+ break;
case FETCH_OP_ST_STRING:
loc = *(u32 *)dest;
ret = fetch_store_string(val + code->offset, dest, base);
break;
+ case FETCH_OP_ST_USTRING:
+ loc = *(u32 *)dest;
+ ret = fetch_store_string_user(val + code->offset, dest, base);
+ break;
default:
return -EILSEQ;
}
total += ret;
if (++i < code->param) {
code = s3;
- if (s3->op != FETCH_OP_ST_STRING) {
+ if (s3->op != FETCH_OP_ST_STRING &&
+ s3->op != FETCH_OP_ST_USTRING) {
dest += s3->size;
val += s3->size;
goto stage3;
return copy_from_user(dest, vaddr, size) ? -EFAULT : 0;
}
+
+static nokprobe_inline int
+probe_mem_read_user(void *dest, void *src, size_t size)
+{
+ return probe_mem_read(dest, src, size);
+}
+
/*
* Fetch a null-terminated string. Caller MUST set *(u32 *)dest with max
* length and relative data location.
return ret;
}
+static nokprobe_inline int
+fetch_store_string_user(unsigned long addr, void *dest, void *base)
+{
+ return fetch_store_string(addr, dest, base);
+}
+
/* Return the length of string -- including null terminal byte */
static nokprobe_inline int
fetch_store_strlen(unsigned long addr)
return (len > MAX_STRING_SIZE) ? 0 : len;
}
+static nokprobe_inline int
+fetch_store_strlen_user(unsigned long addr)
+{
+ return fetch_store_strlen(addr);
+}
+
static unsigned long translate_user_vaddr(unsigned long file_offset)
{
unsigned long base_addr;
{
struct trace_uprobe *tu = to_trace_uprobe(ev);
- return strcmp(trace_event_name(&tu->tp.call), event) == 0 &&
- (!system || strcmp(tu->tp.call.class->system, system) == 0);
+ return strcmp(trace_probe_name(&tu->tp), event) == 0 &&
+ (!system || strcmp(trace_probe_group_name(&tu->tp), system) == 0);
}
/*
alloc_trace_uprobe(const char *group, const char *event, int nargs, bool is_ret)
{
struct trace_uprobe *tu;
-
- if (!event || !group)
- return ERR_PTR(-EINVAL);
+ int ret;
tu = kzalloc(SIZEOF_TRACE_UPROBE(nargs), GFP_KERNEL);
if (!tu)
return ERR_PTR(-ENOMEM);
- tu->tp.call.class = &tu->tp.class;
- tu->tp.call.name = kstrdup(event, GFP_KERNEL);
- if (!tu->tp.call.name)
- goto error;
-
- tu->tp.class.system = kstrdup(group, GFP_KERNEL);
- if (!tu->tp.class.system)
+ ret = trace_probe_init(&tu->tp, event, group);
+ if (ret < 0)
goto error;
dyn_event_init(&tu->devent, &trace_uprobe_ops);
- INIT_LIST_HEAD(&tu->tp.files);
tu->consumer.handler = uprobe_dispatcher;
if (is_ret)
tu->consumer.ret_handler = uretprobe_dispatcher;
return tu;
error:
- kfree(tu->tp.call.name);
kfree(tu);
- return ERR_PTR(-ENOMEM);
+ return ERR_PTR(ret);
}
static void free_trace_uprobe(struct trace_uprobe *tu)
{
- int i;
-
if (!tu)
return;
- for (i = 0; i < tu->tp.nr_args; i++)
- traceprobe_free_probe_arg(&tu->tp.args[i]);
-
path_put(&tu->path);
- kfree(tu->tp.call.class->system);
- kfree(tu->tp.call.name);
+ trace_probe_cleanup(&tu->tp);
kfree(tu->filename);
kfree(tu);
}
struct trace_uprobe *tu;
for_each_trace_uprobe(tu, pos)
- if (strcmp(trace_event_name(&tu->tp.call), event) == 0 &&
- strcmp(tu->tp.call.class->system, group) == 0)
+ if (strcmp(trace_probe_name(&tu->tp), event) == 0 &&
+ strcmp(trace_probe_group_name(&tu->tp), group) == 0)
return tu;
return NULL;
struct trace_uprobe *tmp, *old = NULL;
struct inode *new_inode = d_real_inode(new->path.dentry);
- old = find_probe_event(trace_event_name(&new->tp.call),
- new->tp.call.class->system);
+ old = find_probe_event(trace_probe_name(&new->tp),
+ trace_probe_group_name(&new->tp));
for_each_trace_uprobe(tmp, pos) {
if ((old ? old != tmp : true) &&
goto error;
}
+ ret = traceprobe_set_print_fmt(&tu->tp, is_ret_probe(tu));
+ if (ret < 0)
+ goto error;
+
ret = register_trace_uprobe(tu);
if (!ret)
goto out;
char c = is_ret_probe(tu) ? 'r' : 'p';
int i;
- seq_printf(m, "%c:%s/%s %s:0x%0*lx", c, tu->tp.call.class->system,
- trace_event_name(&tu->tp.call), tu->filename,
+ seq_printf(m, "%c:%s/%s %s:0x%0*lx", c, trace_probe_group_name(&tu->tp),
+ trace_probe_name(&tu->tp), tu->filename,
(int)(sizeof(void *) * 2), tu->offset);
if (tu->ref_ctr_offset)
tu = to_trace_uprobe(ev);
seq_printf(m, " %s %-44s %15lu\n", tu->filename,
- trace_event_name(&tu->tp.call), tu->nhit);
+ trace_probe_name(&tu->tp), tu->nhit);
return 0;
}
struct ring_buffer *buffer;
void *data;
int size, esize;
- struct trace_event_call *call = &tu->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tu->tp);
WARN_ON(call != trace_file->event_call);
return 0;
rcu_read_lock();
- list_for_each_entry_rcu(link, &tu->tp.files, list)
+ trace_probe_for_each_link_rcu(link, &tu->tp)
__uprobe_trace_func(tu, 0, regs, ucb, dsize, link->file);
rcu_read_unlock();
struct event_file_link *link;
rcu_read_lock();
- list_for_each_entry_rcu(link, &tu->tp.files, list)
+ trace_probe_for_each_link_rcu(link, &tu->tp)
__uprobe_trace_func(tu, func, regs, ucb, dsize, link->file);
rcu_read_unlock();
}
if (is_ret_probe(tu)) {
trace_seq_printf(s, "%s: (0x%lx <- 0x%lx)",
- trace_event_name(&tu->tp.call),
+ trace_probe_name(&tu->tp),
entry->vaddr[1], entry->vaddr[0]);
data = DATAOF_TRACE_ENTRY(entry, true);
} else {
trace_seq_printf(s, "%s: (0x%lx)",
- trace_event_name(&tu->tp.call),
+ trace_probe_name(&tu->tp),
entry->vaddr[0]);
data = DATAOF_TRACE_ENTRY(entry, false);
}
filter_func_t filter)
{
bool enabled = trace_probe_is_enabled(&tu->tp);
- struct event_file_link *link = NULL;
int ret;
if (file) {
- if (tu->tp.flags & TP_FLAG_PROFILE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_PROFILE))
return -EINTR;
- link = kmalloc(sizeof(*link), GFP_KERNEL);
- if (!link)
- return -ENOMEM;
-
- link->file = file;
- list_add_tail_rcu(&link->list, &tu->tp.files);
-
- tu->tp.flags |= TP_FLAG_TRACE;
+ ret = trace_probe_add_file(&tu->tp, file);
+ if (ret < 0)
+ return ret;
} else {
- if (tu->tp.flags & TP_FLAG_TRACE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_TRACE))
return -EINTR;
- tu->tp.flags |= TP_FLAG_PROFILE;
+ trace_probe_set_flag(&tu->tp, TP_FLAG_PROFILE);
}
WARN_ON(!uprobe_filter_is_empty(&tu->filter));
uprobe_buffer_disable();
err_flags:
- if (file) {
- list_del(&link->list);
- kfree(link);
- tu->tp.flags &= ~TP_FLAG_TRACE;
- } else {
- tu->tp.flags &= ~TP_FLAG_PROFILE;
- }
+ if (file)
+ trace_probe_remove_file(&tu->tp, file);
+ else
+ trace_probe_clear_flag(&tu->tp, TP_FLAG_PROFILE);
+
return ret;
}
return;
if (file) {
- struct event_file_link *link;
-
- link = find_event_file_link(&tu->tp, file);
- if (!link)
+ if (trace_probe_remove_file(&tu->tp, file) < 0)
return;
- list_del_rcu(&link->list);
- /* synchronize with u{,ret}probe_trace_func */
- synchronize_rcu();
- kfree(link);
-
- if (!list_empty(&tu->tp.files))
+ if (trace_probe_is_enabled(&tu->tp))
return;
- }
+ } else
+ trace_probe_clear_flag(&tu->tp, TP_FLAG_PROFILE);
WARN_ON(!uprobe_filter_is_empty(&tu->filter));
uprobe_unregister(tu->inode, tu->offset, &tu->consumer);
tu->inode = NULL;
- tu->tp.flags &= file ? ~TP_FLAG_TRACE : ~TP_FLAG_PROFILE;
uprobe_buffer_disable();
}
unsigned long func, struct pt_regs *regs,
struct uprobe_cpu_buffer *ucb, int dsize)
{
- struct trace_event_call *call = &tu->tp.call;
+ struct trace_event_call *call = trace_probe_event_call(&tu->tp);
struct uprobe_trace_entry_head *entry;
struct hlist_head *head;
void *data;
ucb = uprobe_buffer_get();
store_trace_args(ucb->buf, &tu->tp, regs, esize, dsize);
- if (tu->tp.flags & TP_FLAG_TRACE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_TRACE))
ret |= uprobe_trace_func(tu, regs, ucb, dsize);
#ifdef CONFIG_PERF_EVENTS
- if (tu->tp.flags & TP_FLAG_PROFILE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_PROFILE))
ret |= uprobe_perf_func(tu, regs, ucb, dsize);
#endif
uprobe_buffer_put(ucb);
ucb = uprobe_buffer_get();
store_trace_args(ucb->buf, &tu->tp, regs, esize, dsize);
- if (tu->tp.flags & TP_FLAG_TRACE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_TRACE))
uretprobe_trace_func(tu, func, regs, ucb, dsize);
#ifdef CONFIG_PERF_EVENTS
- if (tu->tp.flags & TP_FLAG_PROFILE)
+ if (trace_probe_test_flag(&tu->tp, TP_FLAG_PROFILE))
uretprobe_perf_func(tu, func, regs, ucb, dsize);
#endif
uprobe_buffer_put(ucb);
.trace = print_uprobe_event
};
-static inline void init_trace_event_call(struct trace_uprobe *tu,
- struct trace_event_call *call)
+static inline void init_trace_event_call(struct trace_uprobe *tu)
{
- INIT_LIST_HEAD(&call->class->fields);
+ struct trace_event_call *call = trace_probe_event_call(&tu->tp);
+
call->event.funcs = &uprobe_funcs;
call->class->define_fields = uprobe_event_define_fields;
static int register_uprobe_event(struct trace_uprobe *tu)
{
- struct trace_event_call *call = &tu->tp.call;
- int ret = 0;
-
- init_trace_event_call(tu, call);
-
- if (traceprobe_set_print_fmt(&tu->tp, is_ret_probe(tu)) < 0)
- return -ENOMEM;
+ init_trace_event_call(tu);
- ret = register_trace_event(&call->event);
- if (!ret) {
- kfree(call->print_fmt);
- return -ENODEV;
- }
-
- ret = trace_add_event_call(call);
-
- if (ret) {
- pr_info("Failed to register uprobe event: %s\n",
- trace_event_name(call));
- kfree(call->print_fmt);
- unregister_trace_event(&call->event);
- }
-
- return ret;
+ return trace_probe_register_event_call(&tu->tp);
}
static int unregister_uprobe_event(struct trace_uprobe *tu)
{
- int ret;
-
- /* tu->event is unregistered in trace_remove_event_call() */
- ret = trace_remove_event_call(&tu->tp.call);
- if (ret)
- return ret;
- kfree(tu->tp.call.print_fmt);
- tu->tp.call.print_fmt = NULL;
- return 0;
+ return trace_probe_unregister_event_call(&tu->tp);
}
#ifdef CONFIG_PERF_EVENTS
tu->path = path;
tu->ref_ctr_offset = ref_ctr_offset;
tu->filename = kstrdup(name, GFP_KERNEL);
- init_trace_event_call(tu, &tu->tp.call);
+ init_trace_event_call(tu);
if (traceprobe_set_print_fmt(&tu->tp, is_ret_probe(tu)) < 0) {
ret = -ENOMEM;
goto error;
}
- return &tu->tp.call;
+ return trace_probe_event_call(&tu->tp);
error:
free_trace_uprobe(tu);
return ERR_PTR(ret);
tu = container_of(event_call, struct trace_uprobe, tp.call);
- kfree(tu->tp.call.print_fmt);
- tu->tp.call.print_fmt = NULL;
-
free_trace_uprobe(tu);
}
#endif /* CONFIG_PERF_EVENTS */
static inline void *allocate_probes(int count)
{
- struct tp_probes *p = kmalloc(count * sizeof(struct tracepoint_func)
- + sizeof(struct tp_probes), GFP_KERNEL);
+ struct tp_probes *p = kmalloc(struct_size(p, probes, count),
+ GFP_KERNEL);
return p == NULL ? NULL : p->probes;
}
.permissions = set_permissions,
};
-static int zero = 0;
-static int int_max = INT_MAX;
#define UCOUNT_ENTRY(name) \
{ \
.procname = name, \
.maxlen = sizeof(int), \
.mode = 0644, \
.proc_handler = proc_dointvec_minmax, \
- .extra1 = &zero, \
- .extra2 = &int_max, \
+ .extra1 = SYSCTL_ZERO, \
+ .extra2 = SYSCTL_INT_MAX, \
}
static struct ctl_table user_table[] = {
UCOUNT_ENTRY("max_user_namespaces"),
the proof of observed correctness is also maintained for an
arbitrary combination of these separate locking variants.
- For more details, see Documentation/locking/lockdep-design.txt.
+ For more details, see Documentation/locking/lockdep-design.rst.
config LOCK_STAT
bool "Lock usage statistics"
help
This feature enables tracking lock contention points
- For more details, see Documentation/locking/lockstat.txt
+ For more details, see Documentation/locking/lockstat.rst
This also enables lock events required by "perf lock",
subcommand of perf.
If unsure, say N.
+config TEST_MEMINIT
+ tristate "Test heap/page initialization"
+ help
+ Test if the kernel is zero-initializing heap and page allocations.
+ This can be useful to test init_on_alloc and init_on_free features.
+
+ If unsure, say N.
+
endif # RUNTIME_TESTING_MENU
config MEMTEST
obj-$(CONFIG_TEST_OBJAGG) += test_objagg.o
obj-$(CONFIG_TEST_STACKINIT) += test_stackinit.o
obj-$(CONFIG_TEST_BLACKHOLE_DEV) += test_blackhole_dev.o
+obj-$(CONFIG_TEST_MEMINIT) += test_meminit.o
obj-$(CONFIG_TEST_LIVEPATCH) += livepatch/
void __init ioremap_huge_init(void)
{
if (!ioremap_huge_disabled) {
+ if (arch_ioremap_p4d_supported())
+ ioremap_p4d_capable = 1;
if (arch_ioremap_pud_supported())
ioremap_pud_capable = 1;
if (arch_ioremap_pmd_supported())
if ((end - addr) != PMD_SIZE)
return 0;
+ if (!IS_ALIGNED(addr, PMD_SIZE))
+ return 0;
+
if (!IS_ALIGNED(phys_addr, PMD_SIZE))
return 0;
if ((end - addr) != PUD_SIZE)
return 0;
+ if (!IS_ALIGNED(addr, PUD_SIZE))
+ return 0;
+
if (!IS_ALIGNED(phys_addr, PUD_SIZE))
return 0;
if ((end - addr) != P4D_SIZE)
return 0;
+ if (!IS_ALIGNED(addr, P4D_SIZE))
+ return 0;
+
if (!IS_ALIGNED(phys_addr, P4D_SIZE))
return 0;
#define add_ssaaaa(sh, sl, ah, al, bh, bl) \
__asm__ ("addl %5,%1\n" \
"adcl %3,%0" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "%0" ((USItype)(ah)), \
"g" ((USItype)(bh)), \
"%1" ((USItype)(al)), \
#define sub_ddmmss(sh, sl, ah, al, bh, bl) \
__asm__ ("subl %5,%1\n" \
"sbbl %3,%0" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "0" ((USItype)(ah)), \
"g" ((USItype)(bh)), \
"1" ((USItype)(al)), \
"g" ((USItype)(bl)))
#define umul_ppmm(w1, w0, u, v) \
__asm__ ("mull %3" \
- : "=a" ((USItype)(w0)), \
- "=d" ((USItype)(w1)) \
+ : "=a" (w0), \
+ "=d" (w1) \
: "%0" ((USItype)(u)), \
"rm" ((USItype)(v)))
#define udiv_qrnnd(q, r, n1, n0, d) \
__asm__ ("divl %4" \
- : "=a" ((USItype)(q)), \
- "=d" ((USItype)(r)) \
+ : "=a" (q), \
+ "=d" (r) \
: "0" ((USItype)(n0)), \
"1" ((USItype)(n1)), \
"rm" ((USItype)(d)))
static __always_inline void
__rb_insert(struct rb_node *node, struct rb_root *root,
- bool newleft, struct rb_node **leftmost,
void (*augment_rotate)(struct rb_node *old, struct rb_node *new))
{
struct rb_node *parent = rb_red_parent(node), *gparent, *tmp;
- if (newleft)
- *leftmost = node;
-
while (true) {
/*
* Loop invariant: node is red.
void rb_insert_color(struct rb_node *node, struct rb_root *root)
{
- __rb_insert(node, root, false, NULL, dummy_rotate);
+ __rb_insert(node, root, dummy_rotate);
}
EXPORT_SYMBOL(rb_insert_color);
void rb_erase(struct rb_node *node, struct rb_root *root)
{
struct rb_node *rebalance;
- rebalance = __rb_erase_augmented(node, root,
- NULL, &dummy_callbacks);
+ rebalance = __rb_erase_augmented(node, root, &dummy_callbacks);
if (rebalance)
____rb_erase_color(rebalance, root, dummy_rotate);
}
EXPORT_SYMBOL(rb_erase);
-void rb_insert_color_cached(struct rb_node *node,
- struct rb_root_cached *root, bool leftmost)
-{
- __rb_insert(node, &root->rb_root, leftmost,
- &root->rb_leftmost, dummy_rotate);
-}
-EXPORT_SYMBOL(rb_insert_color_cached);
-
-void rb_erase_cached(struct rb_node *node, struct rb_root_cached *root)
-{
- struct rb_node *rebalance;
- rebalance = __rb_erase_augmented(node, &root->rb_root,
- &root->rb_leftmost, &dummy_callbacks);
- if (rebalance)
- ____rb_erase_color(rebalance, &root->rb_root, dummy_rotate);
-}
-EXPORT_SYMBOL(rb_erase_cached);
-
/*
* Augmented rbtree manipulation functions.
*
*/
void __rb_insert_augmented(struct rb_node *node, struct rb_root *root,
- bool newleft, struct rb_node **leftmost,
void (*augment_rotate)(struct rb_node *old, struct rb_node *new))
{
- __rb_insert(node, root, newleft, leftmost, augment_rotate);
+ __rb_insert(node, root, augment_rotate);
}
EXPORT_SYMBOL(__rb_insert_augmented);
}
EXPORT_SYMBOL(rb_replace_node);
-void rb_replace_node_cached(struct rb_node *victim, struct rb_node *new,
- struct rb_root_cached *root)
-{
- rb_replace_node(victim, new, &root->rb_root);
-
- if (root->rb_leftmost == victim)
- root->rb_leftmost = new;
-}
-EXPORT_SYMBOL(rb_replace_node_cached);
-
void rb_replace_node_rcu(struct rb_node *victim, struct rb_node *new,
struct rb_root *root)
{
* strchr - Find the first occurrence of a character in a string
* @s: The string to be searched
* @c: The character to search for
+ *
+ * Note that the %NUL-terminator is considered part of the string, and can
+ * be searched for.
*/
char *strchr(const char *s, int c)
{
* @s: The string to be searched
* @count: The number of characters to be searched
* @c: The character to search for
+ *
+ * Note that the %NUL-terminator is considered part of the string, and can
+ * be searched for.
*/
char *strnchr(const char *s, size_t count, int c)
{
- for (; count-- && *s != '\0'; ++s)
+ while (count--) {
if (*s == (char)c)
return (char *)s;
+ if (*s++ == '\0')
+ break;
+ }
return NULL;
}
EXPORT_SYMBOL(strnchr);
* @src: source buffer (escaped)
* @dst: destination buffer (unescaped)
* @size: size of the destination buffer (0 to unlimit)
- * @flags: combination of the flags (bitwise OR):
- * %UNESCAPE_SPACE:
+ * @flags: combination of the flags.
+ *
+ * Description:
+ * The function unquotes characters in the given string.
+ *
+ * Because the size of the output will be the same as or less than the size of
+ * the input, the transformation may be performed in place.
+ *
+ * Caller must provide valid source and destination pointers. Be aware that
+ * destination buffer will always be NULL-terminated. Source string must be
+ * NULL-terminated as well. The supported flags are::
+ *
+ * UNESCAPE_SPACE:
* '\f' - form feed
* '\n' - new line
* '\r' - carriage return
* '\t' - horizontal tab
* '\v' - vertical tab
- * %UNESCAPE_OCTAL:
+ * UNESCAPE_OCTAL:
* '\NNN' - byte with octal value NNN (1 to 3 digits)
- * %UNESCAPE_HEX:
+ * UNESCAPE_HEX:
* '\xHH' - byte with hexadecimal value HH (1 to 2 digits)
- * %UNESCAPE_SPECIAL:
+ * UNESCAPE_SPECIAL:
* '\"' - double quote
* '\\' - backslash
* '\a' - alert (BEL)
* '\e' - escape
- * %UNESCAPE_ANY:
+ * UNESCAPE_ANY:
* all previous together
*
- * Description:
- * The function unquotes characters in the given string.
- *
- * Because the size of the output will be the same as or less than the size of
- * the input, the transformation may be performed in place.
- *
- * Caller must provide valid source and destination pointers. Be aware that
- * destination buffer will always be NULL-terminated. Source string must be
- * NULL-terminated as well.
- *
* Return:
* The amount of the characters processed to the destination buffer excluding
* trailing '\0' is returned.
* @isz: source buffer size
* @dst: destination buffer (escaped)
* @osz: destination buffer size
- * @flags: combination of the flags (bitwise OR):
+ * @flags: combination of the flags
+ * @only: NULL-terminated string containing characters used to limit
+ * the selected escape class. If characters are included in @only
+ * that would not normally be escaped by the classes selected
+ * in @flags, they will be copied to @dst unescaped.
+ *
+ * Description:
+ * The process of escaping byte buffer includes several parts. They are applied
+ * in the following sequence.
+ *
+ * 1. The character is matched to the printable class, if asked, and in
+ * case of match it passes through to the output.
+ * 2. The character is not matched to the one from @only string and thus
+ * must go as-is to the output.
+ * 3. The character is checked if it falls into the class given by @flags.
+ * %ESCAPE_OCTAL and %ESCAPE_HEX are going last since they cover any
+ * character. Note that they actually can't go together, otherwise
+ * %ESCAPE_HEX will be ignored.
+ *
+ * Caller must provide valid source and destination pointers. Be aware that
+ * destination buffer will not be NULL-terminated, thus caller have to append
+ * it if needs. The supported flags are::
+ *
* %ESCAPE_SPACE: (special white space, not space itself)
* '\f' - form feed
* '\n' - new line
* all previous together
* %ESCAPE_HEX:
* '\xHH' - byte with hexadecimal value HH (2 digits)
- * @only: NULL-terminated string containing characters used to limit
- * the selected escape class. If characters are included in @only
- * that would not normally be escaped by the classes selected
- * in @flags, they will be copied to @dst unescaped.
- *
- * Description:
- * The process of escaping byte buffer includes several parts. They are applied
- * in the following sequence.
- * 1. The character is matched to the printable class, if asked, and in
- * case of match it passes through to the output.
- * 2. The character is not matched to the one from @only string and thus
- * must go as-is to the output.
- * 3. The character is checked if it falls into the class given by @flags.
- * %ESCAPE_OCTAL and %ESCAPE_HEX are going last since they cover any
- * character. Note that they actually can't go together, otherwise
- * %ESCAPE_HEX will be ignored.
- *
- * Caller must provide valid source and destination pointers. Be aware that
- * destination buffer will not be NULL-terminated, thus caller have to append
- * it if needs.
*
* Return:
* The total size of the escaped output that would be generated for
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test cases for SL[AOU]B/page initialization at alloc/free time.
+ */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/vmalloc.h>
+
+#define GARBAGE_INT (0x09A7BA9E)
+#define GARBAGE_BYTE (0x9E)
+
+#define REPORT_FAILURES_IN_FN() \
+ do { \
+ if (failures) \
+ pr_info("%s failed %d out of %d times\n", \
+ __func__, failures, num_tests); \
+ else \
+ pr_info("all %d tests in %s passed\n", \
+ num_tests, __func__); \
+ } while (0)
+
+/* Calculate the number of uninitialized bytes in the buffer. */
+static int __init count_nonzero_bytes(void *ptr, size_t size)
+{
+ int i, ret = 0;
+ unsigned char *p = (unsigned char *)ptr;
+
+ for (i = 0; i < size; i++)
+ if (p[i])
+ ret++;
+ return ret;
+}
+
+/* Fill a buffer with garbage, skipping |skip| first bytes. */
+static void __init fill_with_garbage_skip(void *ptr, int size, size_t skip)
+{
+ unsigned int *p = (unsigned int *)((char *)ptr + skip);
+ int i = 0;
+
+ WARN_ON(skip > size);
+ size -= skip;
+
+ while (size >= sizeof(*p)) {
+ p[i] = GARBAGE_INT;
+ i++;
+ size -= sizeof(*p);
+ }
+ if (size)
+ memset(&p[i], GARBAGE_BYTE, size);
+}
+
+static void __init fill_with_garbage(void *ptr, size_t size)
+{
+ fill_with_garbage_skip(ptr, size, 0);
+}
+
+static int __init do_alloc_pages_order(int order, int *total_failures)
+{
+ struct page *page;
+ void *buf;
+ size_t size = PAGE_SIZE << order;
+
+ page = alloc_pages(GFP_KERNEL, order);
+ buf = page_address(page);
+ fill_with_garbage(buf, size);
+ __free_pages(page, order);
+
+ page = alloc_pages(GFP_KERNEL, order);
+ buf = page_address(page);
+ if (count_nonzero_bytes(buf, size))
+ (*total_failures)++;
+ fill_with_garbage(buf, size);
+ __free_pages(page, order);
+ return 1;
+}
+
+/* Test the page allocator by calling alloc_pages with different orders. */
+static int __init test_pages(int *total_failures)
+{
+ int failures = 0, num_tests = 0;
+ int i;
+
+ for (i = 0; i < 10; i++)
+ num_tests += do_alloc_pages_order(i, &failures);
+
+ REPORT_FAILURES_IN_FN();
+ *total_failures += failures;
+ return num_tests;
+}
+
+/* Test kmalloc() with given parameters. */
+static int __init do_kmalloc_size(size_t size, int *total_failures)
+{
+ void *buf;
+
+ buf = kmalloc(size, GFP_KERNEL);
+ fill_with_garbage(buf, size);
+ kfree(buf);
+
+ buf = kmalloc(size, GFP_KERNEL);
+ if (count_nonzero_bytes(buf, size))
+ (*total_failures)++;
+ fill_with_garbage(buf, size);
+ kfree(buf);
+ return 1;
+}
+
+/* Test vmalloc() with given parameters. */
+static int __init do_vmalloc_size(size_t size, int *total_failures)
+{
+ void *buf;
+
+ buf = vmalloc(size);
+ fill_with_garbage(buf, size);
+ vfree(buf);
+
+ buf = vmalloc(size);
+ if (count_nonzero_bytes(buf, size))
+ (*total_failures)++;
+ fill_with_garbage(buf, size);
+ vfree(buf);
+ return 1;
+}
+
+/* Test kmalloc()/vmalloc() by allocating objects of different sizes. */
+static int __init test_kvmalloc(int *total_failures)
+{
+ int failures = 0, num_tests = 0;
+ int i, size;
+
+ for (i = 0; i < 20; i++) {
+ size = 1 << i;
+ num_tests += do_kmalloc_size(size, &failures);
+ num_tests += do_vmalloc_size(size, &failures);
+ }
+
+ REPORT_FAILURES_IN_FN();
+ *total_failures += failures;
+ return num_tests;
+}
+
+#define CTOR_BYTES (sizeof(unsigned int))
+#define CTOR_PATTERN (0x41414141)
+/* Initialize the first 4 bytes of the object. */
+static void test_ctor(void *obj)
+{
+ *(unsigned int *)obj = CTOR_PATTERN;
+}
+
+/*
+ * Check the invariants for the buffer allocated from a slab cache.
+ * If the cache has a test constructor, the first 4 bytes of the object must
+ * always remain equal to CTOR_PATTERN.
+ * If the cache isn't an RCU-typesafe one, or if the allocation is done with
+ * __GFP_ZERO, then the object contents must be zeroed after allocation.
+ * If the cache is an RCU-typesafe one, the object contents must never be
+ * zeroed after the first use. This is checked by memcmp() in
+ * do_kmem_cache_size().
+ */
+static bool __init check_buf(void *buf, int size, bool want_ctor,
+ bool want_rcu, bool want_zero)
+{
+ int bytes;
+ bool fail = false;
+
+ bytes = count_nonzero_bytes(buf, size);
+ WARN_ON(want_ctor && want_zero);
+ if (want_zero)
+ return bytes;
+ if (want_ctor) {
+ if (*(unsigned int *)buf != CTOR_PATTERN)
+ fail = 1;
+ } else {
+ if (bytes)
+ fail = !want_rcu;
+ }
+ return fail;
+}
+
+/*
+ * Test kmem_cache with given parameters:
+ * want_ctor - use a constructor;
+ * want_rcu - use SLAB_TYPESAFE_BY_RCU;
+ * want_zero - use __GFP_ZERO.
+ */
+static int __init do_kmem_cache_size(size_t size, bool want_ctor,
+ bool want_rcu, bool want_zero,
+ int *total_failures)
+{
+ struct kmem_cache *c;
+ int iter;
+ bool fail = false;
+ gfp_t alloc_mask = GFP_KERNEL | (want_zero ? __GFP_ZERO : 0);
+ void *buf, *buf_copy;
+
+ c = kmem_cache_create("test_cache", size, 1,
+ want_rcu ? SLAB_TYPESAFE_BY_RCU : 0,
+ want_ctor ? test_ctor : NULL);
+ for (iter = 0; iter < 10; iter++) {
+ buf = kmem_cache_alloc(c, alloc_mask);
+ /* Check that buf is zeroed, if it must be. */
+ fail = check_buf(buf, size, want_ctor, want_rcu, want_zero);
+ fill_with_garbage_skip(buf, size, want_ctor ? CTOR_BYTES : 0);
+
+ if (!want_rcu) {
+ kmem_cache_free(c, buf);
+ continue;
+ }
+
+ /*
+ * If this is an RCU cache, use a critical section to ensure we
+ * can touch objects after they're freed.
+ */
+ rcu_read_lock();
+ /*
+ * Copy the buffer to check that it's not wiped on
+ * free().
+ */
+ buf_copy = kmalloc(size, GFP_KERNEL);
+ if (buf_copy)
+ memcpy(buf_copy, buf, size);
+
+ kmem_cache_free(c, buf);
+ /*
+ * Check that |buf| is intact after kmem_cache_free().
+ * |want_zero| is false, because we wrote garbage to
+ * the buffer already.
+ */
+ fail |= check_buf(buf, size, want_ctor, want_rcu,
+ false);
+ if (buf_copy) {
+ fail |= (bool)memcmp(buf, buf_copy, size);
+ kfree(buf_copy);
+ }
+ rcu_read_unlock();
+ }
+ kmem_cache_destroy(c);
+
+ *total_failures += fail;
+ return 1;
+}
+
+/*
+ * Check that the data written to an RCU-allocated object survives
+ * reallocation.
+ */
+static int __init do_kmem_cache_rcu_persistent(int size, int *total_failures)
+{
+ struct kmem_cache *c;
+ void *buf, *buf_contents, *saved_ptr;
+ void **used_objects;
+ int i, iter, maxiter = 1024;
+ bool fail = false;
+
+ c = kmem_cache_create("test_cache", size, size, SLAB_TYPESAFE_BY_RCU,
+ NULL);
+ buf = kmem_cache_alloc(c, GFP_KERNEL);
+ saved_ptr = buf;
+ fill_with_garbage(buf, size);
+ buf_contents = kmalloc(size, GFP_KERNEL);
+ if (!buf_contents)
+ goto out;
+ used_objects = kmalloc_array(maxiter, sizeof(void *), GFP_KERNEL);
+ if (!used_objects) {
+ kfree(buf_contents);
+ goto out;
+ }
+ memcpy(buf_contents, buf, size);
+ kmem_cache_free(c, buf);
+ /*
+ * Run for a fixed number of iterations. If we never hit saved_ptr,
+ * assume the test passes.
+ */
+ for (iter = 0; iter < maxiter; iter++) {
+ buf = kmem_cache_alloc(c, GFP_KERNEL);
+ used_objects[iter] = buf;
+ if (buf == saved_ptr) {
+ fail = memcmp(buf_contents, buf, size);
+ for (i = 0; i <= iter; i++)
+ kmem_cache_free(c, used_objects[i]);
+ goto free_out;
+ }
+ }
+
+free_out:
+ kmem_cache_destroy(c);
+ kfree(buf_contents);
+ kfree(used_objects);
+out:
+ *total_failures += fail;
+ return 1;
+}
+
+/*
+ * Test kmem_cache allocation by creating caches of different sizes, with and
+ * without constructors, with and without SLAB_TYPESAFE_BY_RCU.
+ */
+static int __init test_kmemcache(int *total_failures)
+{
+ int failures = 0, num_tests = 0;
+ int i, flags, size;
+ bool ctor, rcu, zero;
+
+ for (i = 0; i < 10; i++) {
+ size = 8 << i;
+ for (flags = 0; flags < 8; flags++) {
+ ctor = flags & 1;
+ rcu = flags & 2;
+ zero = flags & 4;
+ if (ctor & zero)
+ continue;
+ num_tests += do_kmem_cache_size(size, ctor, rcu, zero,
+ &failures);
+ }
+ }
+ REPORT_FAILURES_IN_FN();
+ *total_failures += failures;
+ return num_tests;
+}
+
+/* Test the behavior of SLAB_TYPESAFE_BY_RCU caches of different sizes. */
+static int __init test_rcu_persistent(int *total_failures)
+{
+ int failures = 0, num_tests = 0;
+ int i, size;
+
+ for (i = 0; i < 10; i++) {
+ size = 8 << i;
+ num_tests += do_kmem_cache_rcu_persistent(size, &failures);
+ }
+ REPORT_FAILURES_IN_FN();
+ *total_failures += failures;
+ return num_tests;
+}
+
+/*
+ * Run the tests. Each test function returns the number of executed tests and
+ * updates |failures| with the number of failed tests.
+ */
+static int __init test_meminit_init(void)
+{
+ int failures = 0, num_tests = 0;
+
+ num_tests += test_pages(&failures);
+ num_tests += test_kvmalloc(&failures);
+ num_tests += test_kmemcache(&failures);
+ num_tests += test_rcu_persistent(&failures);
+
+ if (failures == 0)
+ pr_info("all %d tests passed!\n", num_tests);
+ else
+ pr_info("failures: %d out of %d\n", failures, num_tests);
+
+ return failures ? -EINVAL : 0;
+}
+module_init(test_meminit_init);
+
+MODULE_LICENSE("GPL");
* Deal with the various forms of allocator arguments. See comments above
* the DEFINE_TEST_ALLOC() instances for mapping of the "bits".
*/
-#define alloc010(alloc, arg, sz) alloc(sz, GFP_KERNEL)
-#define alloc011(alloc, arg, sz) alloc(sz, GFP_KERNEL, NUMA_NO_NODE)
+#define alloc_GFP (GFP_KERNEL | __GFP_NOWARN)
+#define alloc010(alloc, arg, sz) alloc(sz, alloc_GFP)
+#define alloc011(alloc, arg, sz) alloc(sz, alloc_GFP, NUMA_NO_NODE)
#define alloc000(alloc, arg, sz) alloc(sz)
#define alloc001(alloc, arg, sz) alloc(sz, NUMA_NO_NODE)
-#define alloc110(alloc, arg, sz) alloc(arg, sz, GFP_KERNEL)
+#define alloc110(alloc, arg, sz) alloc(arg, sz, alloc_GFP)
#define free0(free, arg, ptr) free(ptr)
#define free1(free, arg, ptr) free(arg, ptr)
-/* Wrap around to 8K */
-#define TEST_SIZE (9 << PAGE_SHIFT)
+/* Wrap around to 16K */
+#define TEST_SIZE (5 * 4096)
#define DEFINE_TEST_ALLOC(func, free_func, want_arg, want_gfp, want_node)\
static int __init test_ ## func (void *arg) \
fail:
kfree(p);
if (i < 256)
- return (i << 24) | (j << 16) | k;
+ return (i << 24) | (j << 16) | k | 0x8000;
return 0;
}
fail:
kfree(p);
if (i < 256)
- return (i << 24) | (j << 16) | k;
+ return (i << 24) | (j << 16) | k | 0x8000;
return 0;
}
fail:
kfree(p);
if (i < 256)
- return (i << 24) | (j << 16) | k;
+ return (i << 24) | (j << 16) | k | 0x8000;
+ return 0;
+}
+
+static __init int strchr_selftest(void)
+{
+ const char *test_string = "abcdefghijkl";
+ const char *empty_string = "";
+ char *result;
+ int i;
+
+ for (i = 0; i < strlen(test_string) + 1; i++) {
+ result = strchr(test_string, test_string[i]);
+ if (result - test_string != i)
+ return i + 'a';
+ }
+
+ result = strchr(empty_string, '\0');
+ if (result != empty_string)
+ return 0x101;
+
+ result = strchr(empty_string, 'a');
+ if (result)
+ return 0x102;
+
+ result = strchr(test_string, 'z');
+ if (result)
+ return 0x103;
+
+ return 0;
+}
+
+static __init int strnchr_selftest(void)
+{
+ const char *test_string = "abcdefghijkl";
+ const char *empty_string = "";
+ char *result;
+ int i, j;
+
+ for (i = 0; i < strlen(test_string) + 1; i++) {
+ for (j = 0; j < strlen(test_string) + 2; j++) {
+ result = strnchr(test_string, j, test_string[i]);
+ if (j <= i) {
+ if (!result)
+ continue;
+ return ((i + 'a') << 8) | j;
+ }
+ if (result - test_string != i)
+ return ((i + 'a') << 8) | j;
+ }
+ }
+
+ result = strnchr(empty_string, 0, '\0');
+ if (result)
+ return 0x10001;
+
+ result = strnchr(empty_string, 1, '\0');
+ if (result != empty_string)
+ return 0x10002;
+
+ result = strnchr(empty_string, 1, 'a');
+ if (result)
+ return 0x10003;
+
+ result = strnchr(NULL, 0, '\0');
+ if (result)
+ return 0x10004;
+
return 0;
}
if (subtest)
goto fail;
+ test = 4;
+ subtest = strchr_selftest();
+ if (subtest)
+ goto fail;
+
+ test = 5;
+ subtest = strnchr_selftest();
+ if (subtest)
+ goto fail;
+
pr_info("String selftests succeeded\n");
return 0;
fail:
See Documentation/admin-guide/mm/idle_page_tracking.rst for
more details.
-# arch_add_memory() comprehends device memory
-config ARCH_HAS_ZONE_DEVICE
+config ARCH_HAS_PTE_DEVMAP
bool
config ZONE_DEVICE
depends on MEMORY_HOTPLUG
depends on MEMORY_HOTREMOVE
depends on SPARSEMEM_VMEMMAP
- depends on ARCH_HAS_ZONE_DEVICE
+ depends on ARCH_HAS_PTE_DEVMAP
select XARRAY_MULTI
help
*/
alignment = max(alignment, (phys_addr_t)PAGE_SIZE <<
max_t(unsigned long, MAX_ORDER - 1, pageblock_order));
+ if (fixed && base & (alignment - 1)) {
+ ret = -EINVAL;
+ pr_err("Region at %pa must be aligned to %pa bytes\n",
+ &base, &alignment);
+ goto err;
+ }
base = ALIGN(base, alignment);
size = ALIGN(size, alignment);
limit &= ~(alignment - 1);
if (limit == 0 || limit > memblock_end)
limit = memblock_end;
+ if (base + size > limit) {
+ ret = -EINVAL;
+ pr_err("Size (%pa) of region at %pa exceeds limit (%pa)\n",
+ &size, &base, &limit);
+ goto err;
+ }
+
/* Reserve memory */
if (fixed) {
if (memblock_is_region_reserved(base, size) ||
* @pages: Allocated pages.
* @count: Number of allocated pages.
*
- * This function releases memory allocated by alloc_cma().
+ * This function releases memory allocated by cma_alloc().
* It returns false when provided pages do not belong to contiguous area and
* true otherwise.
*/
}
#endif /* CONFIG_ARCH_HAS_PTE_SPECIAL */
-#if defined(__HAVE_ARCH_PTE_DEVMAP) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
+#if defined(CONFIG_ARCH_HAS_PTE_DEVMAP) && defined(CONFIG_TRANSPARENT_HUGEPAGE)
static int __gup_device_huge(unsigned long pfn, unsigned long addr,
unsigned long end, struct page **pages, int *nr)
{
bool transparent_hugepage_enabled(struct vm_area_struct *vma)
{
+ /* The addr is used to check if the vma size fits */
+ unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE;
+
+ if (!transhuge_vma_suitable(vma, addr))
+ return false;
if (vma_is_anonymous(vma))
return __transparent_hugepage_enabled(vma);
- if (vma_is_shmem(vma) && shmem_huge_enabled(vma))
- return __transparent_hugepage_enabled(vma);
+ if (vma_is_shmem(vma))
+ return shmem_huge_enabled(vma);
return false;
}
struct page *page;
unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
- if (haddr < vma->vm_start || haddr + HPAGE_PMD_SIZE > vma->vm_end)
+ if (!transhuge_vma_suitable(vma, haddr))
return VM_FAULT_FALLBACK;
if (unlikely(anon_vma_prepare(vma)))
return VM_FAULT_OOM;
#include <linux/mm.h>
#include <linux/uaccess.h>
+static __always_inline long
+probe_read_common(void *dst, const void __user *src, size_t size)
+{
+ long ret;
+
+ pagefault_disable();
+ ret = __copy_from_user_inatomic(dst, src, size);
+ pagefault_enable();
+
+ return ret ? -EFAULT : 0;
+}
+
/**
- * probe_kernel_read(): safely attempt to read from a location
+ * probe_kernel_read(): safely attempt to read from a kernel-space location
* @dst: pointer to the buffer that shall take the data
* @src: address to read from
* @size: size of the data chunk
mm_segment_t old_fs = get_fs();
set_fs(KERNEL_DS);
- pagefault_disable();
- ret = __copy_from_user_inatomic(dst,
- (__force const void __user *)src, size);
- pagefault_enable();
+ ret = probe_read_common(dst, (__force const void __user *)src, size);
set_fs(old_fs);
- return ret ? -EFAULT : 0;
+ return ret;
}
EXPORT_SYMBOL_GPL(probe_kernel_read);
/**
+ * probe_user_read(): safely attempt to read from a user-space location
+ * @dst: pointer to the buffer that shall take the data
+ * @src: address to read from. This must be a user address.
+ * @size: size of the data chunk
+ *
+ * Safely read from user address @src to the buffer at @dst. If a kernel fault
+ * happens, handle that and return -EFAULT.
+ */
+
+long __weak probe_user_read(void *dst, const void __user *src, size_t size)
+ __attribute__((alias("__probe_user_read")));
+
+long __probe_user_read(void *dst, const void __user *src, size_t size)
+{
+ long ret = -EFAULT;
+ mm_segment_t old_fs = get_fs();
+
+ set_fs(USER_DS);
+ if (access_ok(src, size))
+ ret = probe_read_common(dst, src, size);
+ set_fs(old_fs);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(probe_user_read);
+
+/**
* probe_kernel_write(): safely attempt to write to a location
* @dst: address to write to
* @src: pointer to the data that shall be written
}
EXPORT_SYMBOL_GPL(probe_kernel_write);
+
/**
* strncpy_from_unsafe: - Copy a NUL terminated string from unsafe address.
* @dst: Destination address, in kernel space. This buffer must be at
return ret ? -EFAULT : src - unsafe_addr;
}
+
+/**
+ * strncpy_from_unsafe_user: - Copy a NUL terminated string from unsafe user
+ * address.
+ * @dst: Destination address, in kernel space. This buffer must be at
+ * least @count bytes long.
+ * @unsafe_addr: Unsafe user address.
+ * @count: Maximum number of bytes to copy, including the trailing NUL.
+ *
+ * Copies a NUL-terminated string from unsafe user address to kernel buffer.
+ *
+ * On success, returns the length of the string INCLUDING the trailing NUL.
+ *
+ * If access fails, returns -EFAULT (some data may have been copied
+ * and the trailing NUL added).
+ *
+ * If @count is smaller than the length of the string, copies @count-1 bytes,
+ * sets the last byte of @dst buffer to NUL and returns @count.
+ */
+long strncpy_from_unsafe_user(char *dst, const void __user *unsafe_addr,
+ long count)
+{
+ mm_segment_t old_fs = get_fs();
+ long ret;
+
+ if (unlikely(count <= 0))
+ return 0;
+
+ set_fs(USER_DS);
+ pagefault_disable();
+ ret = strncpy_from_user(dst, unsafe_addr, count);
+ pagefault_enable();
+ set_fs(old_fs);
+
+ if (ret >= count) {
+ ret = count;
+ dst[ret - 1] = '\0';
+ } else if (ret > 0) {
+ ret++;
+ }
+
+ return ret;
+}
+
+/**
+ * strnlen_unsafe_user: - Get the size of a user string INCLUDING final NUL.
+ * @unsafe_addr: The string to measure.
+ * @count: Maximum count (including NUL)
+ *
+ * Get the size of a NUL-terminated string in user space without pagefault.
+ *
+ * Returns the size of the string INCLUDING the terminating NUL.
+ *
+ * If the string is too long, returns a number larger than @count. User
+ * has to check the return value against "> count".
+ * On exception (or invalid count), returns 0.
+ *
+ * Unlike strnlen_user, this can be used from IRQ handler etc. because
+ * it disables pagefaults.
+ */
+long strnlen_unsafe_user(const void __user *unsafe_addr, long count)
+{
+ mm_segment_t old_fs = get_fs();
+ int ret;
+
+ set_fs(USER_DS);
+ pagefault_disable();
+ ret = strnlen_user(unsafe_addr, count);
+ pagefault_enable();
+ set_fs(old_fs);
+
+ return ret;
+}
if (mem_cgroup_disabled())
return;
- __this_cpu_add(memcg->vmstats_local->stat[idx], val);
-
x = val + __this_cpu_read(memcg->vmstats_percpu->stat[idx]);
if (unlikely(abs(x) > MEMCG_CHARGE_BATCH)) {
struct mem_cgroup *mi;
+ /*
+ * Batch local counters to keep them in sync with
+ * the hierarchical ones.
+ */
+ __this_cpu_add(memcg->vmstats_local->stat[idx], x);
for (mi = memcg; mi; mi = parent_mem_cgroup(mi))
atomic_long_add(x, &mi->vmstats[idx]);
x = 0;
/* Update memcg */
__mod_memcg_state(memcg, idx, val);
- /* Update lruvec */
- __this_cpu_add(pn->lruvec_stat_local->count[idx], val);
-
x = val + __this_cpu_read(pn->lruvec_stat_cpu->count[idx]);
if (unlikely(abs(x) > MEMCG_CHARGE_BATCH)) {
struct mem_cgroup_per_node *pi;
+ /*
+ * Batch local counters to keep them in sync with
+ * the hierarchical ones.
+ */
+ __this_cpu_add(pn->lruvec_stat_local->count[idx], x);
for (pi = pn; pi; pi = parent_nodeinfo(pi, pgdat->node_id))
atomic_long_add(x, &pi->lruvec_stat[idx]);
x = 0;
if (mem_cgroup_disabled())
return;
- __this_cpu_add(memcg->vmstats_local->events[idx], count);
-
x = count + __this_cpu_read(memcg->vmstats_percpu->events[idx]);
if (unlikely(x > MEMCG_CHARGE_BATCH)) {
struct mem_cgroup *mi;
+ /*
+ * Batch local counters to keep them in sync with
+ * the hierarchical ones.
+ */
+ __this_cpu_add(memcg->vmstats_local->events[idx], x);
for (mi = memcg; mi; mi = parent_mem_cgroup(mi))
atomic_long_add(x, &mi->vmevents[idx]);
x = 0;
}
#ifdef CONFIG_TRANSPARENT_HUGE_PAGECACHE
-
-#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1)
-static inline bool transhuge_vma_suitable(struct vm_area_struct *vma,
- unsigned long haddr)
-{
- if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) !=
- (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK))
- return false;
- if (haddr < vma->vm_start || haddr + HPAGE_PMD_SIZE > vma->vm_end)
- return false;
- return true;
-}
-
static void deposit_prealloc_pte(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
#ifndef CONFIG_SPARSEMEM_VMEMMAP
static void register_page_bootmem_info_section(unsigned long start_pfn)
{
- unsigned long *usemap, mapsize, section_nr, i;
+ unsigned long mapsize, section_nr, i;
struct mem_section *ms;
struct page *page, *memmap;
+ struct mem_section_usage *usage;
section_nr = pfn_to_section_nr(start_pfn);
ms = __nr_to_section(section_nr);
for (i = 0; i < mapsize; i++, page++)
get_page_bootmem(section_nr, page, SECTION_INFO);
- usemap = ms->pageblock_flags;
- page = virt_to_page(usemap);
+ usage = ms->usage;
+ page = virt_to_page(usage);
- mapsize = PAGE_ALIGN(usemap_size()) >> PAGE_SHIFT;
+ mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT;
for (i = 0; i < mapsize; i++, page++)
get_page_bootmem(section_nr, page, MIX_SECTION_INFO);
#else /* CONFIG_SPARSEMEM_VMEMMAP */
static void register_page_bootmem_info_section(unsigned long start_pfn)
{
- unsigned long *usemap, mapsize, section_nr, i;
+ unsigned long mapsize, section_nr, i;
struct mem_section *ms;
struct page *page, *memmap;
+ struct mem_section_usage *usage;
section_nr = pfn_to_section_nr(start_pfn);
ms = __nr_to_section(section_nr);
register_page_bootmem_memmap(section_nr, memmap, PAGES_PER_SECTION);
- usemap = ms->pageblock_flags;
- page = virt_to_page(usemap);
+ usage = ms->usage;
+ page = virt_to_page(usage);
- mapsize = PAGE_ALIGN(usemap_size()) >> PAGE_SHIFT;
+ mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT;
for (i = 0; i < mapsize; i++, page++)
get_page_bootmem(section_nr, page, MIX_SECTION_INFO);
}
#endif /* CONFIG_HAVE_BOOTMEM_INFO_NODE */
-static int __meminit __add_section(int nid, unsigned long phys_start_pfn,
- struct vmem_altmap *altmap, bool want_memblock)
+static int check_pfn_span(unsigned long pfn, unsigned long nr_pages,
+ const char *reason)
{
- int ret;
-
- if (pfn_valid(phys_start_pfn))
- return -EEXIST;
-
- ret = sparse_add_one_section(nid, phys_start_pfn, altmap);
- if (ret < 0)
- return ret;
-
- if (!want_memblock)
- return 0;
-
- return hotplug_memory_register(nid, __pfn_to_section(phys_start_pfn));
+ /*
+ * Disallow all operations smaller than a sub-section and only
+ * allow operations smaller than a section for
+ * SPARSEMEM_VMEMMAP. Note that check_hotplug_memory_range()
+ * enforces a larger memory_block_size_bytes() granularity for
+ * memory that will be marked online, so this check should only
+ * fire for direct arch_{add,remove}_memory() users outside of
+ * add_memory_resource().
+ */
+ unsigned long min_align;
+
+ if (IS_ENABLED(CONFIG_SPARSEMEM_VMEMMAP))
+ min_align = PAGES_PER_SUBSECTION;
+ else
+ min_align = PAGES_PER_SECTION;
+ if (!IS_ALIGNED(pfn, min_align)
+ || !IS_ALIGNED(nr_pages, min_align)) {
+ WARN(1, "Misaligned __%s_pages start: %#lx end: #%lx\n",
+ reason, pfn, pfn + nr_pages - 1);
+ return -EINVAL;
+ }
+ return 0;
}
/*
* call this function after deciding the zone to which to
* add the new pages.
*/
-int __ref __add_pages(int nid, unsigned long phys_start_pfn,
- unsigned long nr_pages, struct mhp_restrictions *restrictions)
+int __ref __add_pages(int nid, unsigned long pfn, unsigned long nr_pages,
+ struct mhp_restrictions *restrictions)
{
- unsigned long i;
- int err = 0;
- int start_sec, end_sec;
+ int err;
+ unsigned long nr, start_sec, end_sec;
struct vmem_altmap *altmap = restrictions->altmap;
- /* during initialize mem_map, align hot-added range to section */
- start_sec = pfn_to_section_nr(phys_start_pfn);
- end_sec = pfn_to_section_nr(phys_start_pfn + nr_pages - 1);
-
if (altmap) {
/*
* Validate altmap is within bounds of the total request
*/
- if (altmap->base_pfn != phys_start_pfn
+ if (altmap->base_pfn != pfn
|| vmem_altmap_offset(altmap) > nr_pages) {
pr_warn_once("memory add fail, invalid altmap\n");
- err = -EINVAL;
- goto out;
+ return -EINVAL;
}
altmap->alloc = 0;
}
- for (i = start_sec; i <= end_sec; i++) {
- err = __add_section(nid, section_nr_to_pfn(i), altmap,
- restrictions->flags & MHP_MEMBLOCK_API);
+ err = check_pfn_span(pfn, nr_pages, "add");
+ if (err)
+ return err;
- /*
- * EEXIST is finally dealt with by ioresource collision
- * check. see add_memory() => register_memory_resource()
- * Warning will be printed if there is collision.
- */
- if (err && (err != -EEXIST))
+ start_sec = pfn_to_section_nr(pfn);
+ end_sec = pfn_to_section_nr(pfn + nr_pages - 1);
+ for (nr = start_sec; nr <= end_sec; nr++) {
+ unsigned long pfns;
+
+ pfns = min(nr_pages, PAGES_PER_SECTION
+ - (pfn & ~PAGE_SECTION_MASK));
+ err = sparse_add_section(nid, pfn, pfns, altmap);
+ if (err)
break;
- err = 0;
+ pfn += pfns;
+ nr_pages -= pfns;
cond_resched();
}
vmemmap_populate_print_last();
-out:
return err;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
/* find the smallest valid pfn in the range [start_pfn, end_pfn) */
static unsigned long find_smallest_section_pfn(int nid, struct zone *zone,
unsigned long start_pfn,
unsigned long end_pfn)
{
- struct mem_section *ms;
-
- for (; start_pfn < end_pfn; start_pfn += PAGES_PER_SECTION) {
- ms = __pfn_to_section(start_pfn);
-
- if (unlikely(!valid_section(ms)))
+ for (; start_pfn < end_pfn; start_pfn += PAGES_PER_SUBSECTION) {
+ if (unlikely(!pfn_valid(start_pfn)))
continue;
if (unlikely(pfn_to_nid(start_pfn) != nid))
unsigned long start_pfn,
unsigned long end_pfn)
{
- struct mem_section *ms;
unsigned long pfn;
/* pfn is the end pfn of a memory section. */
pfn = end_pfn - 1;
- for (; pfn >= start_pfn; pfn -= PAGES_PER_SECTION) {
- ms = __pfn_to_section(pfn);
-
- if (unlikely(!valid_section(ms)))
+ for (; pfn >= start_pfn; pfn -= PAGES_PER_SUBSECTION) {
+ if (unlikely(!pfn_valid(pfn)))
continue;
if (unlikely(pfn_to_nid(pfn) != nid))
unsigned long z = zone_end_pfn(zone); /* zone_end_pfn namespace clash */
unsigned long zone_end_pfn = z;
unsigned long pfn;
- struct mem_section *ms;
int nid = zone_to_nid(zone);
zone_span_writelock(zone);
* it check the zone has only hole or not.
*/
pfn = zone_start_pfn;
- for (; pfn < zone_end_pfn; pfn += PAGES_PER_SECTION) {
- ms = __pfn_to_section(pfn);
-
- if (unlikely(!valid_section(ms)))
+ for (; pfn < zone_end_pfn; pfn += PAGES_PER_SUBSECTION) {
+ if (unlikely(!pfn_valid(pfn)))
continue;
if (page_zone(pfn_to_page(pfn)) != zone)
continue;
- /* If the section is current section, it continues the loop */
- if (start_pfn == pfn)
+ /* Skip range to be removed */
+ if (pfn >= start_pfn && pfn < end_pfn)
continue;
/* If we find valid section, we have nothing to do */
unsigned long p = pgdat_end_pfn(pgdat); /* pgdat_end_pfn namespace clash */
unsigned long pgdat_end_pfn = p;
unsigned long pfn;
- struct mem_section *ms;
int nid = pgdat->node_id;
if (pgdat_start_pfn == start_pfn) {
* has only hole or not.
*/
pfn = pgdat_start_pfn;
- for (; pfn < pgdat_end_pfn; pfn += PAGES_PER_SECTION) {
- ms = __pfn_to_section(pfn);
-
- if (unlikely(!valid_section(ms)))
+ for (; pfn < pgdat_end_pfn; pfn += PAGES_PER_SUBSECTION) {
+ if (unlikely(!pfn_valid(pfn)))
continue;
if (pfn_to_nid(pfn) != nid)
continue;
- /* If the section is current section, it continues the loop */
- if (start_pfn == pfn)
+ /* Skip range to be removed */
+ if (pfn >= start_pfn && pfn < end_pfn)
continue;
/* If we find valid section, we have nothing to do */
pgdat->node_spanned_pages = 0;
}
-static void __remove_zone(struct zone *zone, unsigned long start_pfn)
+static void __remove_zone(struct zone *zone, unsigned long start_pfn,
+ unsigned long nr_pages)
{
struct pglist_data *pgdat = zone->zone_pgdat;
- int nr_pages = PAGES_PER_SECTION;
unsigned long flags;
pgdat_resize_lock(zone->zone_pgdat, &flags);
pgdat_resize_unlock(zone->zone_pgdat, &flags);
}
-static void __remove_section(struct zone *zone, struct mem_section *ms,
- unsigned long map_offset,
- struct vmem_altmap *altmap)
+static void __remove_section(struct zone *zone, unsigned long pfn,
+ unsigned long nr_pages, unsigned long map_offset,
+ struct vmem_altmap *altmap)
{
- unsigned long start_pfn;
- int scn_nr;
+ struct mem_section *ms = __nr_to_section(pfn_to_section_nr(pfn));
if (WARN_ON_ONCE(!valid_section(ms)))
return;
- unregister_memory_section(ms);
-
- scn_nr = __section_nr(ms);
- start_pfn = section_nr_to_pfn((unsigned long)scn_nr);
- __remove_zone(zone, start_pfn);
-
- sparse_remove_one_section(zone, ms, map_offset, altmap);
+ __remove_zone(zone, pfn, nr_pages);
+ sparse_remove_section(ms, pfn, nr_pages, map_offset, altmap);
}
/**
* __remove_pages() - remove sections of pages from a zone
* @zone: zone from which pages need to be removed
- * @phys_start_pfn: starting pageframe (must be aligned to start of a section)
+ * @pfn: starting pageframe (must be aligned to start of a section)
* @nr_pages: number of pages to remove (must be multiple of section size)
* @altmap: alternative device page map or %NULL if default memmap is used
*
* sure that pages are marked reserved and zones are adjust properly by
* calling offline_pages().
*/
-void __remove_pages(struct zone *zone, unsigned long phys_start_pfn,
+void __remove_pages(struct zone *zone, unsigned long pfn,
unsigned long nr_pages, struct vmem_altmap *altmap)
{
- unsigned long i;
unsigned long map_offset = 0;
- int sections_to_remove;
+ unsigned long nr, start_sec, end_sec;
- /* In the ZONE_DEVICE case device driver owns the memory region */
- if (is_dev_zone(zone))
- map_offset = vmem_altmap_offset(altmap);
+ map_offset = vmem_altmap_offset(altmap);
clear_zone_contiguous(zone);
- /*
- * We can only remove entire sections
- */
- BUG_ON(phys_start_pfn & ~PAGE_SECTION_MASK);
- BUG_ON(nr_pages % PAGES_PER_SECTION);
+ if (check_pfn_span(pfn, nr_pages, "remove"))
+ return;
- sections_to_remove = nr_pages / PAGES_PER_SECTION;
- for (i = 0; i < sections_to_remove; i++) {
- unsigned long pfn = phys_start_pfn + i*PAGES_PER_SECTION;
+ start_sec = pfn_to_section_nr(pfn);
+ end_sec = pfn_to_section_nr(pfn + nr_pages - 1);
+ for (nr = start_sec; nr <= end_sec; nr++) {
+ unsigned long pfns;
cond_resched();
- __remove_section(zone, __pfn_to_section(pfn), map_offset,
- altmap);
+ pfns = min(nr_pages, PAGES_PER_SECTION
+ - (pfn & ~PAGE_SECTION_MASK));
+ __remove_section(zone, pfn, pfns, map_offset, altmap);
+ pfn += pfns;
+ nr_pages -= pfns;
map_offset = 0;
}
set_zone_contiguous(zone);
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
int set_online_page_callback(online_page_callback_t callback)
{
static int check_hotplug_memory_range(u64 start, u64 size)
{
- unsigned long block_sz = memory_block_size_bytes();
- u64 block_nr_pages = block_sz >> PAGE_SHIFT;
- u64 nr_pages = size >> PAGE_SHIFT;
- u64 start_pfn = PFN_DOWN(start);
-
/* memory range must be block size aligned */
- if (!nr_pages || !IS_ALIGNED(start_pfn, block_nr_pages) ||
- !IS_ALIGNED(nr_pages, block_nr_pages)) {
+ if (!size || !IS_ALIGNED(start, memory_block_size_bytes()) ||
+ !IS_ALIGNED(size, memory_block_size_bytes())) {
pr_err("Block size [%#lx] unaligned hotplug range: start %#llx, size %#llx",
- block_sz, start, size);
+ memory_block_size_bytes(), start, size);
return -EINVAL;
}
*/
int __ref add_memory_resource(int nid, struct resource *res)
{
- struct mhp_restrictions restrictions = {
- .flags = MHP_MEMBLOCK_API,
- };
+ struct mhp_restrictions restrictions = {};
u64 start, size;
bool new_node = false;
int ret;
if (ret < 0)
goto error;
+ /* create memory block devices after memory was added */
+ ret = create_memory_block_devices(start, size);
+ if (ret) {
+ arch_remove_memory(nid, start, size, NULL);
+ goto error;
+ }
+
if (new_node) {
/* If sysfs file of new node can't be created, cpu on the node
* can't be hot-added. There is no rollback way now.
/* online pages if requested */
if (memhp_auto_online)
- walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1),
- NULL, online_memory_block);
+ walk_memory_blocks(start, size, NULL, online_memory_block);
return ret;
error:
{
return __offline_pages(start_pfn, start_pfn + nr_pages);
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
-
-/**
- * walk_memory_range - walks through all mem sections in [start_pfn, end_pfn)
- * @start_pfn: start pfn of the memory range
- * @end_pfn: end pfn of the memory range
- * @arg: argument passed to func
- * @func: callback for each memory section walked
- *
- * This function walks through all present mem sections in range
- * [start_pfn, end_pfn) and call func on each mem section.
- *
- * Returns the return value of func.
- */
-int walk_memory_range(unsigned long start_pfn, unsigned long end_pfn,
- void *arg, int (*func)(struct memory_block *, void *))
-{
- struct memory_block *mem = NULL;
- struct mem_section *section;
- unsigned long pfn, section_nr;
- int ret;
-
- for (pfn = start_pfn; pfn < end_pfn; pfn += PAGES_PER_SECTION) {
- section_nr = pfn_to_section_nr(pfn);
- if (!present_section_nr(section_nr))
- continue;
-
- section = __nr_to_section(section_nr);
- /* same memblock? */
- if (mem)
- if ((section_nr >= mem->start_section_nr) &&
- (section_nr <= mem->end_section_nr))
- continue;
- mem = find_memory_block_hinted(section, mem);
- if (!mem)
- continue;
-
- ret = func(mem, arg);
- if (ret) {
- kobject_put(&mem->dev.kobj);
- return ret;
- }
- }
-
- if (mem)
- kobject_put(&mem->dev.kobj);
-
- return 0;
-}
-
-#ifdef CONFIG_MEMORY_HOTREMOVE
static int check_memblock_offlined_cb(struct memory_block *mem, void *arg)
{
int ret = !is_memblock_offlined(mem);
endpa = PFN_PHYS(section_nr_to_pfn(mem->end_section_nr + 1))-1;
pr_warn("removing memory fails, because memory [%pa-%pa] is onlined\n",
&beginpa, &endpa);
- }
- return ret;
+ return -EBUSY;
+ }
+ return 0;
}
static int check_cpu_on_node(pg_data_t *pgdat)
}
}
-/**
- * remove_memory
- * @nid: the node ID
- * @start: physical address of the region to remove
- * @size: size of the region to remove
- *
- * NOTE: The caller must call lock_device_hotplug() to serialize hotplug
- * and online/offline operations before this call, as required by
- * try_offline_node().
- */
-void __ref __remove_memory(int nid, u64 start, u64 size)
+static int __ref try_remove_memory(int nid, u64 start, u64 size)
{
- int ret;
+ int rc = 0;
BUG_ON(check_hotplug_memory_range(start, size));
/*
* All memory blocks must be offlined before removing memory. Check
- * whether all memory blocks in question are offline and trigger a BUG()
+ * whether all memory blocks in question are offline and return error
* if this is not the case.
*/
- ret = walk_memory_range(PFN_DOWN(start), PFN_UP(start + size - 1), NULL,
- check_memblock_offlined_cb);
- if (ret)
- BUG();
+ rc = walk_memory_blocks(start, size, NULL, check_memblock_offlined_cb);
+ if (rc)
+ goto done;
/* remove memmap entry */
firmware_map_remove(start, start + size, "System RAM");
memblock_free(start, size);
memblock_remove(start, size);
+ /* remove memory block devices before removing memory */
+ remove_memory_block_devices(start, size);
+
arch_remove_memory(nid, start, size, NULL);
__release_memory_resource(start, size);
try_offline_node(nid);
+done:
mem_hotplug_done();
+ return rc;
+}
+
+/**
+ * remove_memory
+ * @nid: the node ID
+ * @start: physical address of the region to remove
+ * @size: size of the region to remove
+ *
+ * NOTE: The caller must call lock_device_hotplug() to serialize hotplug
+ * and online/offline operations before this call, as required by
+ * try_offline_node().
+ */
+void __remove_memory(int nid, u64 start, u64 size)
+{
+
+ /*
+ * trigger BUG() is some memory is not offlined prior to calling this
+ * function
+ */
+ if (try_remove_memory(nid, start, size))
+ BUG();
}
-void remove_memory(int nid, u64 start, u64 size)
+/*
+ * Remove memory if every memory block is offline, otherwise return -EBUSY is
+ * some memory is not offline
+ */
+int remove_memory(int nid, u64 start, u64 size)
{
+ int rc;
+
lock_device_hotplug();
- __remove_memory(nid, start, size);
+ rc = try_remove_memory(nid, start, size);
unlock_device_hotplug();
+
+ return rc;
}
EXPORT_SYMBOL_GPL(remove_memory);
#endif /* CONFIG_MEMORY_HOTREMOVE */
* 3 for pages with a mapping and PagePrivate/PagePrivate2 set.
*/
int migrate_page_move_mapping(struct address_space *mapping,
- struct page *newpage, struct page *page, enum migrate_mode mode,
- int extra_count)
+ struct page *newpage, struct page *page, int extra_count)
{
XA_STATE(xas, &mapping->i_pages, page_index(page));
struct zone *oldzone, *newzone;
BUG_ON(PageWriteback(page)); /* Writeback must be complete */
- rc = migrate_page_move_mapping(mapping, newpage, page, mode, 0);
+ rc = migrate_page_move_mapping(mapping, newpage, page, 0);
if (rc != MIGRATEPAGE_SUCCESS)
return rc;
}
}
- rc = migrate_page_move_mapping(mapping, newpage, page, mode, 0);
+ rc = migrate_page_move_mapping(mapping, newpage, page, 0);
if (rc != MIGRATEPAGE_SUCCESS)
goto unlock_buffers;
add_nommu_region(region);
/* clear anonymous mappings that don't ask for uninitialized data */
- if (!vma->vm_file && !(flags & MAP_UNINITIALIZED))
+ if (!vma->vm_file &&
+ (!IS_ENABLED(CONFIG_MMAP_ALLOW_UNINITIALIZED) ||
+ !(flags & MAP_UNINITIALIZED)))
memset((void *)region->vm_start, 0,
region->vm_end - region->vm_start);
unsigned long pfn)
{
#ifdef CONFIG_SPARSEMEM
- return __pfn_to_section(pfn)->pageblock_flags;
+ return section_to_usemap(__pfn_to_section(pfn));
#else
return page_zone(page)->pageblock_flags;
#endif /* CONFIG_SPARSEMEM */
__perform_reclaim(gfp_t gfp_mask, unsigned int order,
const struct alloc_context *ac)
{
- struct reclaim_state reclaim_state;
int progress;
unsigned int noreclaim_flag;
unsigned long pflags;
psi_memstall_enter(&pflags);
fs_reclaim_acquire(gfp_mask);
noreclaim_flag = memalloc_noreclaim_save();
- reclaim_state.reclaimed_slab = 0;
- current->reclaim_state = &reclaim_state;
progress = try_to_free_pages(ac->zonelist, order, gfp_mask,
ac->nodemask);
- current->reclaim_state = NULL;
memalloc_noreclaim_restore(noreclaim_flag);
fs_reclaim_release(gfp_mask);
psi_memstall_leave(&pflags);
unsigned long start = jiffies;
int nid = pgdat->node_id;
- if (WARN_ON_ONCE(!pgmap || !is_dev_zone(zone)))
+ if (WARN_ON_ONCE(!pgmap || zone_idx(zone) != ZONE_DEVICE))
return;
/*
* pfn out of zone.
*
* Please note that MEMMAP_HOTPLUG path doesn't clear memmap
- * because this is done early in sparse_add_one_section
+ * because this is done early in section_activate()
*/
if (!(pfn & (pageblock_nr_pages - 1))) {
set_pageblock_migratetype(page, MIGRATE_MOVABLE);
(u64)zone_movable_pfn[i] << PAGE_SHIFT);
}
- /* Print out the early node map */
+ /*
+ * Print out the early node map, and initialize the
+ * subsection-map relative to active online memory ranges to
+ * enable future "sub-section" extensions of the memory map.
+ */
pr_info("Early memory node ranges\n");
- for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid)
+ for_each_mem_pfn_range(i, MAX_NUMNODES, &start_pfn, &end_pfn, &nid) {
pr_info(" node %3d: [mem %#018Lx-%#018Lx]\n", nid,
(u64)start_pfn << PAGE_SHIFT,
((u64)end_pfn << PAGE_SHIFT) - 1);
+ subsection_map_init(start_pfn, end_pfn - start_pfn);
+ }
/* Initialise every node */
mminit_verify_pageflags_layout();
static int shmem_huge __read_mostly;
-#if defined(CONFIG_SYSFS) || defined(CONFIG_TMPFS)
+#if defined(CONFIG_SYSFS)
static int shmem_parse_huge(const char *str)
{
if (!strcmp(str, "never"))
return SHMEM_HUGE_FORCE;
return -EINVAL;
}
+#endif
+#if defined(CONFIG_SYSFS) || defined(CONFIG_TMPFS)
static const char *shmem_format_huge(int huge)
{
switch (huge) {
{
int error;
- /* If rootfs called this, don't re-init */
- if (shmem_inode_cachep)
- return 0;
-
shmem_init_inodecache();
error = register_filesystem(&shmem_fs_type);
loff_t i_size;
pgoff_t off;
+ if ((vma->vm_flags & VM_NOHUGEPAGE) ||
+ test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags))
+ return false;
if (shmem_huge == SHMEM_HUGE_FORCE)
return true;
if (shmem_huge == SHMEM_HUGE_DENY)
}
struct kmem_cache *
-kmalloc_caches[NR_KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1] __ro_after_init;
+kmalloc_caches[NR_KMALLOC_TYPES][KMALLOC_SHIFT_HIGH + 1] __ro_after_init =
+{ /* initialization for https://bugs.llvm.org/show_bug.cgi?id=42570 */ };
EXPORT_SYMBOL(kmalloc_caches);
/*
return 0;
}
-struct page * __meminit sparse_mem_map_populate(unsigned long pnum, int nid,
- struct vmem_altmap *altmap)
+struct page * __meminit __populate_section_memmap(unsigned long pfn,
+ unsigned long nr_pages, int nid, struct vmem_altmap *altmap)
{
unsigned long start;
unsigned long end;
- struct page *map;
- map = pfn_to_page(pnum * PAGES_PER_SECTION);
- start = (unsigned long)map;
- end = (unsigned long)(map + PAGES_PER_SECTION);
+ /*
+ * The minimum granularity of memmap extensions is
+ * PAGES_PER_SUBSECTION as allocations are tracked in the
+ * 'subsection_map' bitmap of the section.
+ */
+ end = ALIGN(pfn + nr_pages, PAGES_PER_SUBSECTION);
+ pfn &= PAGE_SUBSECTION_MASK;
+ nr_pages = end - pfn;
+
+ start = (unsigned long) pfn_to_page(pfn);
+ end = start + nr_pages * sizeof(struct page);
if (vmemmap_populate(start, end, nid, altmap))
return NULL;
- return map;
+ return pfn_to_page(pfn);
}
unsigned long root = SECTION_NR_TO_ROOT(section_nr);
struct mem_section *section;
+ /*
+ * An existing section is possible in the sub-section hotplug
+ * case. First hot-add instantiates, follow-on hot-add reuses
+ * the existing section.
+ *
+ * The mem_hotplug_lock resolves the apparent race below.
+ */
if (mem_section[root])
- return -EEXIST;
+ return 0;
section = sparse_index_alloc(nid);
if (!section)
#endif
#ifdef CONFIG_SPARSEMEM_EXTREME
-int __section_nr(struct mem_section* ms)
+unsigned long __section_nr(struct mem_section *ms)
{
unsigned long root_nr;
struct mem_section *root = NULL;
return (root_nr * SECTIONS_PER_ROOT) + (ms - root);
}
#else
-int __section_nr(struct mem_section* ms)
+unsigned long __section_nr(struct mem_section *ms)
{
- return (int)(ms - mem_section[0]);
+ return (unsigned long)(ms - mem_section[0]);
}
#endif
* Keeping track of this gives us an easy way to break out of
* those loops early.
*/
-int __highest_present_section_nr;
+unsigned long __highest_present_section_nr;
static void section_mark_present(struct mem_section *ms)
{
- int section_nr = __section_nr(ms);
+ unsigned long section_nr = __section_nr(ms);
if (section_nr > __highest_present_section_nr)
__highest_present_section_nr = section_nr;
ms->section_mem_map |= SECTION_MARKED_PRESENT;
}
-static inline int next_present_section_nr(int section_nr)
+static inline unsigned long next_present_section_nr(unsigned long section_nr)
{
do {
section_nr++;
return next_present_section_nr(-1);
}
+void subsection_mask_set(unsigned long *map, unsigned long pfn,
+ unsigned long nr_pages)
+{
+ int idx = subsection_map_index(pfn);
+ int end = subsection_map_index(pfn + nr_pages - 1);
+
+ bitmap_set(map, idx, end - idx + 1);
+}
+
+void __init subsection_map_init(unsigned long pfn, unsigned long nr_pages)
+{
+ int end_sec = pfn_to_section_nr(pfn + nr_pages - 1);
+ unsigned long nr, start_sec = pfn_to_section_nr(pfn);
+
+ if (!nr_pages)
+ return;
+
+ for (nr = start_sec; nr <= end_sec; nr++) {
+ struct mem_section *ms;
+ unsigned long pfns;
+
+ pfns = min(nr_pages, PAGES_PER_SECTION
+ - (pfn & ~PAGE_SECTION_MASK));
+ ms = __nr_to_section(nr);
+ subsection_mask_set(ms->usage->subsection_map, pfn, pfns);
+
+ pr_debug("%s: sec: %lu pfns: %lu set(%d, %d)\n", __func__, nr,
+ pfns, subsection_map_index(pfn),
+ subsection_map_index(pfn + pfns - 1));
+
+ pfn += pfns;
+ nr_pages -= pfns;
+ }
+}
+
/* Record a memory area against a node. */
void __init memory_present(int nid, unsigned long start, unsigned long end)
{
static void __meminit sparse_init_one_section(struct mem_section *ms,
unsigned long pnum, struct page *mem_map,
- unsigned long *pageblock_bitmap)
+ struct mem_section_usage *usage, unsigned long flags)
{
ms->section_mem_map &= ~SECTION_MAP_MASK;
- ms->section_mem_map |= sparse_encode_mem_map(mem_map, pnum) |
- SECTION_HAS_MEM_MAP;
- ms->pageblock_flags = pageblock_bitmap;
+ ms->section_mem_map |= sparse_encode_mem_map(mem_map, pnum)
+ | SECTION_HAS_MEM_MAP | flags;
+ ms->usage = usage;
}
-unsigned long usemap_size(void)
+static unsigned long usemap_size(void)
{
return BITS_TO_LONGS(SECTION_BLOCKFLAGS_BITS) * sizeof(unsigned long);
}
-#ifdef CONFIG_MEMORY_HOTPLUG
-static unsigned long *__kmalloc_section_usemap(void)
+size_t mem_section_usage_size(void)
{
- return kmalloc(usemap_size(), GFP_KERNEL);
+ return sizeof(struct mem_section_usage) + usemap_size();
}
-#endif /* CONFIG_MEMORY_HOTPLUG */
#ifdef CONFIG_MEMORY_HOTREMOVE
-static unsigned long * __init
+static struct mem_section_usage * __init
sparse_early_usemaps_alloc_pgdat_section(struct pglist_data *pgdat,
unsigned long size)
{
+ struct mem_section_usage *usage;
unsigned long goal, limit;
- unsigned long *p;
int nid;
/*
* A page may contain usemaps for other sections preventing the
limit = goal + (1UL << PA_SECTION_SHIFT);
nid = early_pfn_to_nid(goal >> PAGE_SHIFT);
again:
- p = memblock_alloc_try_nid(size, SMP_CACHE_BYTES, goal, limit, nid);
- if (!p && limit) {
+ usage = memblock_alloc_try_nid(size, SMP_CACHE_BYTES, goal, limit, nid);
+ if (!usage && limit) {
limit = 0;
goto again;
}
- return p;
+ return usage;
}
-static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
+static void __init check_usemap_section_nr(int nid,
+ struct mem_section_usage *usage)
{
unsigned long usemap_snr, pgdat_snr;
static unsigned long old_usemap_snr;
old_pgdat_snr = NR_MEM_SECTIONS;
}
- usemap_snr = pfn_to_section_nr(__pa(usemap) >> PAGE_SHIFT);
+ usemap_snr = pfn_to_section_nr(__pa(usage) >> PAGE_SHIFT);
pgdat_snr = pfn_to_section_nr(__pa(pgdat) >> PAGE_SHIFT);
if (usemap_snr == pgdat_snr)
return;
usemap_snr, pgdat_snr, nid);
}
#else
-static unsigned long * __init
+static struct mem_section_usage * __init
sparse_early_usemaps_alloc_pgdat_section(struct pglist_data *pgdat,
unsigned long size)
{
return memblock_alloc_node(size, SMP_CACHE_BYTES, pgdat->node_id);
}
-static void __init check_usemap_section_nr(int nid, unsigned long *usemap)
+static void __init check_usemap_section_nr(int nid,
+ struct mem_section_usage *usage)
{
}
#endif /* CONFIG_MEMORY_HOTREMOVE */
return PAGE_ALIGN(sizeof(struct page) * PAGES_PER_SECTION);
}
-struct page __init *sparse_mem_map_populate(unsigned long pnum, int nid,
- struct vmem_altmap *altmap)
+struct page __init *__populate_section_memmap(unsigned long pfn,
+ unsigned long nr_pages, int nid, struct vmem_altmap *altmap)
{
unsigned long size = section_map_size();
struct page *map = sparse_buffer_alloc(size);
unsigned long pnum_end,
unsigned long map_count)
{
- unsigned long pnum, usemap_longs, *usemap;
+ struct mem_section_usage *usage;
+ unsigned long pnum;
struct page *map;
- usemap_longs = BITS_TO_LONGS(SECTION_BLOCKFLAGS_BITS);
- usemap = sparse_early_usemaps_alloc_pgdat_section(NODE_DATA(nid),
- usemap_size() *
- map_count);
- if (!usemap) {
+ usage = sparse_early_usemaps_alloc_pgdat_section(NODE_DATA(nid),
+ mem_section_usage_size() * map_count);
+ if (!usage) {
pr_err("%s: node[%d] usemap allocation failed", __func__, nid);
goto failed;
}
sparse_buffer_init(map_count * section_map_size(), nid);
for_each_present_section_nr(pnum_begin, pnum) {
+ unsigned long pfn = section_nr_to_pfn(pnum);
+
if (pnum >= pnum_end)
break;
- map = sparse_mem_map_populate(pnum, nid, NULL);
+ map = __populate_section_memmap(pfn, PAGES_PER_SECTION,
+ nid, NULL);
if (!map) {
pr_err("%s: node[%d] memory map backing failed. Some memory will not be available.",
__func__, nid);
pnum_begin = pnum;
goto failed;
}
- check_usemap_section_nr(nid, usemap);
- sparse_init_one_section(__nr_to_section(pnum), pnum, map, usemap);
- usemap += usemap_longs;
+ check_usemap_section_nr(nid, usage);
+ sparse_init_one_section(__nr_to_section(pnum), pnum, map, usage,
+ SECTION_IS_EARLY);
+ usage = (void *) usage + mem_section_usage_size();
}
sparse_buffer_fini();
return;
#endif
#ifdef CONFIG_SPARSEMEM_VMEMMAP
-static inline struct page *kmalloc_section_memmap(unsigned long pnum, int nid,
- struct vmem_altmap *altmap)
+static struct page *populate_section_memmap(unsigned long pfn,
+ unsigned long nr_pages, int nid, struct vmem_altmap *altmap)
{
- /* This will make the necessary allocations eventually. */
- return sparse_mem_map_populate(pnum, nid, altmap);
+ return __populate_section_memmap(pfn, nr_pages, nid, altmap);
}
-static void __kfree_section_memmap(struct page *memmap,
+
+static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,
struct vmem_altmap *altmap)
{
- unsigned long start = (unsigned long)memmap;
- unsigned long end = (unsigned long)(memmap + PAGES_PER_SECTION);
+ unsigned long start = (unsigned long) pfn_to_page(pfn);
+ unsigned long end = start + nr_pages * sizeof(struct page);
vmemmap_free(start, end, altmap);
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
static void free_map_bootmem(struct page *memmap)
{
unsigned long start = (unsigned long)memmap;
vmemmap_free(start, end, NULL);
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
#else
-static struct page *__kmalloc_section_memmap(void)
+struct page *populate_section_memmap(unsigned long pfn,
+ unsigned long nr_pages, int nid, struct vmem_altmap *altmap)
{
struct page *page, *ret;
unsigned long memmap_size = sizeof(struct page) * PAGES_PER_SECTION;
return ret;
}
-static inline struct page *kmalloc_section_memmap(unsigned long pnum, int nid,
+static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,
struct vmem_altmap *altmap)
{
- return __kmalloc_section_memmap();
-}
+ struct page *memmap = pfn_to_page(pfn);
-static void __kfree_section_memmap(struct page *memmap,
- struct vmem_altmap *altmap)
-{
if (is_vmalloc_addr(memmap))
vfree(memmap);
else
get_order(sizeof(struct page) * PAGES_PER_SECTION));
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
static void free_map_bootmem(struct page *memmap)
{
unsigned long maps_section_nr, removing_section_nr, i;
put_page_bootmem(page);
}
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
#endif /* CONFIG_SPARSEMEM_VMEMMAP */
+static void section_deactivate(unsigned long pfn, unsigned long nr_pages,
+ struct vmem_altmap *altmap)
+{
+ DECLARE_BITMAP(map, SUBSECTIONS_PER_SECTION) = { 0 };
+ DECLARE_BITMAP(tmp, SUBSECTIONS_PER_SECTION) = { 0 };
+ struct mem_section *ms = __pfn_to_section(pfn);
+ bool section_is_early = early_section(ms);
+ struct page *memmap = NULL;
+ unsigned long *subsection_map = ms->usage
+ ? &ms->usage->subsection_map[0] : NULL;
+
+ subsection_mask_set(map, pfn, nr_pages);
+ if (subsection_map)
+ bitmap_and(tmp, map, subsection_map, SUBSECTIONS_PER_SECTION);
+
+ if (WARN(!subsection_map || !bitmap_equal(tmp, map, SUBSECTIONS_PER_SECTION),
+ "section already deactivated (%#lx + %ld)\n",
+ pfn, nr_pages))
+ return;
+
+ /*
+ * There are 3 cases to handle across two configurations
+ * (SPARSEMEM_VMEMMAP={y,n}):
+ *
+ * 1/ deactivation of a partial hot-added section (only possible
+ * in the SPARSEMEM_VMEMMAP=y case).
+ * a/ section was present at memory init
+ * b/ section was hot-added post memory init
+ * 2/ deactivation of a complete hot-added section
+ * 3/ deactivation of a complete section from memory init
+ *
+ * For 1/, when subsection_map does not empty we will not be
+ * freeing the usage map, but still need to free the vmemmap
+ * range.
+ *
+ * For 2/ and 3/ the SPARSEMEM_VMEMMAP={y,n} cases are unified
+ */
+ bitmap_xor(subsection_map, map, subsection_map, SUBSECTIONS_PER_SECTION);
+ if (bitmap_empty(subsection_map, SUBSECTIONS_PER_SECTION)) {
+ unsigned long section_nr = pfn_to_section_nr(pfn);
+
+ if (!section_is_early) {
+ kfree(ms->usage);
+ ms->usage = NULL;
+ }
+ memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr);
+ ms->section_mem_map = sparse_encode_mem_map(NULL, section_nr);
+ }
+
+ if (section_is_early && memmap)
+ free_map_bootmem(memmap);
+ else
+ depopulate_section_memmap(pfn, nr_pages, altmap);
+}
+
+static struct page * __meminit section_activate(int nid, unsigned long pfn,
+ unsigned long nr_pages, struct vmem_altmap *altmap)
+{
+ DECLARE_BITMAP(map, SUBSECTIONS_PER_SECTION) = { 0 };
+ struct mem_section *ms = __pfn_to_section(pfn);
+ struct mem_section_usage *usage = NULL;
+ unsigned long *subsection_map;
+ struct page *memmap;
+ int rc = 0;
+
+ subsection_mask_set(map, pfn, nr_pages);
+
+ if (!ms->usage) {
+ usage = kzalloc(mem_section_usage_size(), GFP_KERNEL);
+ if (!usage)
+ return ERR_PTR(-ENOMEM);
+ ms->usage = usage;
+ }
+ subsection_map = &ms->usage->subsection_map[0];
+
+ if (bitmap_empty(map, SUBSECTIONS_PER_SECTION))
+ rc = -EINVAL;
+ else if (bitmap_intersects(map, subsection_map, SUBSECTIONS_PER_SECTION))
+ rc = -EEXIST;
+ else
+ bitmap_or(subsection_map, map, subsection_map,
+ SUBSECTIONS_PER_SECTION);
+
+ if (rc) {
+ if (usage)
+ ms->usage = NULL;
+ kfree(usage);
+ return ERR_PTR(rc);
+ }
+
+ /*
+ * The early init code does not consider partially populated
+ * initial sections, it simply assumes that memory will never be
+ * referenced. If we hot-add memory into such a section then we
+ * do not need to populate the memmap and can simply reuse what
+ * is already there.
+ */
+ if (nr_pages < PAGES_PER_SECTION && early_section(ms))
+ return pfn_to_page(pfn);
+
+ memmap = populate_section_memmap(pfn, nr_pages, nid, altmap);
+ if (!memmap) {
+ section_deactivate(pfn, nr_pages, altmap);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ return memmap;
+}
+
/**
- * sparse_add_one_section - add a memory section
+ * sparse_add_section - add a memory section, or populate an existing one
* @nid: The node to add section on
* @start_pfn: start pfn of the memory range
+ * @nr_pages: number of pfns to add in the section
* @altmap: device page map
*
* This is only intended for hotplug.
* * -EEXIST - Section has been present.
* * -ENOMEM - Out of memory.
*/
-int __meminit sparse_add_one_section(int nid, unsigned long start_pfn,
- struct vmem_altmap *altmap)
+int __meminit sparse_add_section(int nid, unsigned long start_pfn,
+ unsigned long nr_pages, struct vmem_altmap *altmap)
{
unsigned long section_nr = pfn_to_section_nr(start_pfn);
struct mem_section *ms;
struct page *memmap;
- unsigned long *usemap;
int ret;
- /*
- * no locking for this, because it does its own
- * plus, it does a kmalloc
- */
ret = sparse_index_init(section_nr, nid);
- if (ret < 0 && ret != -EEXIST)
+ if (ret < 0)
return ret;
- ret = 0;
- memmap = kmalloc_section_memmap(section_nr, nid, altmap);
- if (!memmap)
- return -ENOMEM;
- usemap = __kmalloc_section_usemap();
- if (!usemap) {
- __kfree_section_memmap(memmap, altmap);
- return -ENOMEM;
- }
- ms = __pfn_to_section(start_pfn);
- if (ms->section_mem_map & SECTION_MARKED_PRESENT) {
- ret = -EEXIST;
- goto out;
- }
+ memmap = section_activate(nid, start_pfn, nr_pages, altmap);
+ if (IS_ERR(memmap))
+ return PTR_ERR(memmap);
/*
* Poison uninitialized struct pages in order to catch invalid flags
* combinations.
*/
- page_init_poison(memmap, sizeof(struct page) * PAGES_PER_SECTION);
+ page_init_poison(pfn_to_page(start_pfn), sizeof(struct page) * nr_pages);
+ ms = __pfn_to_section(start_pfn);
+ set_section_nid(section_nr, nid);
section_mark_present(ms);
- sparse_init_one_section(ms, section_nr, memmap, usemap);
-out:
- if (ret < 0) {
- kfree(usemap);
- __kfree_section_memmap(memmap, altmap);
- }
- return ret;
+ /* Align memmap to section boundary in the subsection case */
+ if (section_nr_to_pfn(section_nr) != start_pfn)
+ memmap = pfn_to_kaddr(section_nr_to_pfn(section_nr));
+ sparse_init_one_section(ms, section_nr, memmap, ms->usage, 0);
+
+ return 0;
}
-#ifdef CONFIG_MEMORY_HOTREMOVE
#ifdef CONFIG_MEMORY_FAILURE
static void clear_hwpoisoned_pages(struct page *memmap, int nr_pages)
{
}
#endif
-static void free_section_usemap(struct page *memmap, unsigned long *usemap,
+void sparse_remove_section(struct mem_section *ms, unsigned long pfn,
+ unsigned long nr_pages, unsigned long map_offset,
struct vmem_altmap *altmap)
{
- struct page *usemap_page;
-
- if (!usemap)
- return;
-
- usemap_page = virt_to_page(usemap);
- /*
- * Check to see if allocation came from hot-plug-add
- */
- if (PageSlab(usemap_page) || PageCompound(usemap_page)) {
- kfree(usemap);
- if (memmap)
- __kfree_section_memmap(memmap, altmap);
- return;
- }
-
- /*
- * The usemap came from bootmem. This is packed with other usemaps
- * on the section which has pgdat at boot time. Just keep it as is now.
- */
-
- if (memmap)
- free_map_bootmem(memmap);
-}
-
-void sparse_remove_one_section(struct zone *zone, struct mem_section *ms,
- unsigned long map_offset, struct vmem_altmap *altmap)
-{
- struct page *memmap = NULL;
- unsigned long *usemap = NULL;
-
- if (ms->section_mem_map) {
- usemap = ms->pageblock_flags;
- memmap = sparse_decode_mem_map(ms->section_mem_map,
- __section_nr(ms));
- ms->section_mem_map = 0;
- ms->pageblock_flags = NULL;
- }
-
- clear_hwpoisoned_pages(memmap + map_offset,
- PAGES_PER_SECTION - map_offset);
- free_section_usemap(memmap, usemap, altmap);
+ clear_hwpoisoned_pages(pfn_to_page(pfn) + map_offset,
+ nr_pages - map_offset);
+ section_deactivate(pfn, nr_pages, altmap);
}
-#endif /* CONFIG_MEMORY_HOTREMOVE */
#endif /* CONFIG_MEMORY_HOTPLUG */
/*
* This file contains the default values for the operation of the
* Linux VM subsystem. Fine-tuning documentation can be found in
- * Documentation/sysctl/vm.txt.
+ * Documentation/admin-guide/sysctl/vm.rst.
* Started 18.12.91
* Swap aging added 23.2.95, Stephen Tweedie.
* Buffermem limits added 12.3.98, Rik van Riel.
#include <linux/err.h>
#include <linux/sched.h>
#include <linux/sched/mm.h>
+#include <linux/sched/signal.h>
#include <linux/sched/task_stack.h>
#include <linux/security.h>
#include <linux/swap.h>
}
#endif
+/**
+ * __account_locked_vm - account locked pages to an mm's locked_vm
+ * @mm: mm to account against
+ * @pages: number of pages to account
+ * @inc: %true if @pages should be considered positive, %false if not
+ * @task: task used to check RLIMIT_MEMLOCK
+ * @bypass_rlim: %true if checking RLIMIT_MEMLOCK should be skipped
+ *
+ * Assumes @task and @mm are valid (i.e. at least one reference on each), and
+ * that mmap_sem is held as writer.
+ *
+ * Return:
+ * * 0 on success
+ * * -ENOMEM if RLIMIT_MEMLOCK would be exceeded.
+ */
+int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc,
+ struct task_struct *task, bool bypass_rlim)
+{
+ unsigned long locked_vm, limit;
+ int ret = 0;
+
+ lockdep_assert_held_write(&mm->mmap_sem);
+
+ locked_vm = mm->locked_vm;
+ if (inc) {
+ if (!bypass_rlim) {
+ limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT;
+ if (locked_vm + pages > limit)
+ ret = -ENOMEM;
+ }
+ if (!ret)
+ mm->locked_vm = locked_vm + pages;
+ } else {
+ WARN_ON_ONCE(pages > locked_vm);
+ mm->locked_vm = locked_vm - pages;
+ }
+
+ pr_debug("%s: [%d] caller %ps %c%lu %lu/%lu%s\n", __func__, task->pid,
+ (void *)_RET_IP_, (inc) ? '+' : '-', pages << PAGE_SHIFT,
+ locked_vm << PAGE_SHIFT, task_rlimit(task, RLIMIT_MEMLOCK),
+ ret ? " - exceeded" : "");
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(__account_locked_vm);
+
+/**
+ * account_locked_vm - account locked pages to an mm's locked_vm
+ * @mm: mm to account against, may be NULL
+ * @pages: number of pages to account
+ * @inc: %true if @pages should be considered positive, %false if not
+ *
+ * Assumes a non-NULL @mm is valid (i.e. at least one reference on it).
+ *
+ * Return:
+ * * 0 on success, or if mm is NULL
+ * * -ENOMEM if RLIMIT_MEMLOCK would be exceeded.
+ */
+int account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc)
+{
+ int ret;
+
+ if (pages == 0 || !mm)
+ return 0;
+
+ down_write(&mm->mmap_sem);
+ ret = __account_locked_vm(mm, pages, inc, current,
+ capable(CAP_IPC_LOCK));
+ up_write(&mm->mmap_sem);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(account_locked_vm);
+
unsigned long vm_mmap_pgoff(struct file *file, unsigned long addr,
unsigned long len, unsigned long prot,
unsigned long flag, unsigned long pgoff)
unsigned int file_taken;
unsigned int taken;
} nr;
+
+ /* for recording the reclaimed slab by now */
+ struct reclaim_state reclaim_state;
};
#ifdef ARCH_HAS_PREFETCH
}
#endif /* CONFIG_MEMCG_KMEM */
+static void set_task_reclaim_state(struct task_struct *task,
+ struct reclaim_state *rs)
+{
+ /* Check for an overwrite */
+ WARN_ON_ONCE(rs && task->reclaim_state);
+
+ /* Check for the nulling of an already-nulled member */
+ WARN_ON_ONCE(!rs && !task->reclaim_state);
+
+ task->reclaim_state = rs;
+}
+
#ifdef CONFIG_MEMCG
static bool global_reclaim(struct scan_control *sc)
{
if (throttle_direct_reclaim(sc.gfp_mask, zonelist, nodemask))
return 1;
+ set_task_reclaim_state(current, &sc.reclaim_state);
trace_mm_vmscan_direct_reclaim_begin(order, sc.gfp_mask);
nr_reclaimed = do_try_to_free_pages(zonelist, &sc);
trace_mm_vmscan_direct_reclaim_end(nr_reclaimed);
+ set_task_reclaim_state(current, NULL);
return nr_reclaimed;
}
};
unsigned long lru_pages;
+ set_task_reclaim_state(current, &sc.reclaim_state);
sc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
(GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
trace_mm_vmscan_memcg_softlimit_reclaim_end(sc.nr_reclaimed);
+ set_task_reclaim_state(current, NULL);
*nr_scanned = sc.nr_scanned;
+
return sc.nr_reclaimed;
}
.may_shrinkslab = 1,
};
+ set_task_reclaim_state(current, &sc.reclaim_state);
/*
* Unlike direct reclaim via alloc_pages(), memcg's reclaim doesn't
* take care of from where we get pages. So the node where we start the
psi_memstall_leave(&pflags);
trace_mm_vmscan_memcg_reclaim_end(nr_reclaimed);
+ set_task_reclaim_state(current, NULL);
return nr_reclaimed;
}
.may_unmap = 1,
};
+ set_task_reclaim_state(current, &sc.reclaim_state);
psi_memstall_enter(&pflags);
__fs_reclaim_acquire();
snapshot_refaults(NULL, pgdat);
__fs_reclaim_release();
psi_memstall_leave(&pflags);
+ set_task_reclaim_state(current, NULL);
+
/*
* Return the order kswapd stopped reclaiming at as
* prepare_kswapd_sleep() takes it into account. If another caller
unsigned int classzone_idx = MAX_NR_ZONES - 1;
pg_data_t *pgdat = (pg_data_t*)p;
struct task_struct *tsk = current;
-
- struct reclaim_state reclaim_state = {
- .reclaimed_slab = 0,
- };
const struct cpumask *cpumask = cpumask_of_node(pgdat->node_id);
if (!cpumask_empty(cpumask))
set_cpus_allowed_ptr(tsk, cpumask);
- current->reclaim_state = &reclaim_state;
/*
* Tell the memory management that we're a "memory allocator",
}
tsk->flags &= ~(PF_MEMALLOC | PF_SWAPWRITE | PF_KSWAPD);
- current->reclaim_state = NULL;
return 0;
}
*/
unsigned long shrink_all_memory(unsigned long nr_to_reclaim)
{
- struct reclaim_state reclaim_state;
struct scan_control sc = {
.nr_to_reclaim = nr_to_reclaim,
.gfp_mask = GFP_HIGHUSER_MOVABLE,
.hibernation_mode = 1,
};
struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask);
- struct task_struct *p = current;
unsigned long nr_reclaimed;
unsigned int noreclaim_flag;
fs_reclaim_acquire(sc.gfp_mask);
noreclaim_flag = memalloc_noreclaim_save();
- reclaim_state.reclaimed_slab = 0;
- p->reclaim_state = &reclaim_state;
+ set_task_reclaim_state(current, &sc.reclaim_state);
nr_reclaimed = do_try_to_free_pages(zonelist, &sc);
- p->reclaim_state = NULL;
+ set_task_reclaim_state(current, NULL);
memalloc_noreclaim_restore(noreclaim_flag);
fs_reclaim_release(sc.gfp_mask);
/* Minimum pages needed in order to stay on node */
const unsigned long nr_pages = 1 << order;
struct task_struct *p = current;
- struct reclaim_state reclaim_state;
unsigned int noreclaim_flag;
struct scan_control sc = {
.nr_to_reclaim = max(nr_pages, SWAP_CLUSTER_MAX),
*/
noreclaim_flag = memalloc_noreclaim_save();
p->flags |= PF_SWAPWRITE;
- reclaim_state.reclaimed_slab = 0;
- p->reclaim_state = &reclaim_state;
+ set_task_reclaim_state(p, &sc.reclaim_state);
if (node_pagecache_reclaimable(pgdat) > pgdat->min_unmapped_pages) {
/*
} while (sc.nr_reclaimed < nr_pages && --sc.priority >= 0);
}
- p->reclaim_state = NULL;
+ set_task_reclaim_state(p, NULL);
current->flags &= ~PF_SWAPWRITE;
memalloc_noreclaim_restore(noreclaim_flag);
fs_reclaim_release(sc.gfp_mask);
#include <linux/atomic.h>
#include <linux/sched.h>
#include <linux/cpumask.h>
-#include <linux/dcache.h>
#include <linux/list.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/compaction.h>
#include <linux/percpu.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/fs.h>
#include <linux/preempt.h>
#include <linux/workqueue.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/zpool.h>
+#include <linux/magic.h>
/*
* NCHUNKS_ORDER determines the internal allocation granularity, effectively
* @refcount: reference count for the z3fold page
* @work: work_struct for page layout optimization
* @slots: pointer to the structure holding buddy slots
+ * @pool: pointer to the containing pool
* @cpu: CPU which this page "belongs" to
* @first_chunks: the size of the first buddy in chunks, 0 if free
* @middle_chunks: the size of the middle buddy in chunks, 0 if free
struct kref refcount;
struct work_struct work;
struct z3fold_buddy_slots *slots;
+ struct z3fold_pool *pool;
short cpu;
unsigned short first_chunks;
unsigned short middle_chunks;
static inline struct z3fold_buddy_slots *alloc_slots(struct z3fold_pool *pool,
gfp_t gfp)
{
- struct z3fold_buddy_slots *slots = kmem_cache_alloc(pool->c_handle,
- gfp);
+ struct z3fold_buddy_slots *slots;
+
+ slots = kmem_cache_alloc(pool->c_handle,
+ (gfp & ~(__GFP_HIGHMEM | __GFP_MOVABLE)));
if (slots) {
memset(slots->slot, 0, sizeof(slots->slot));
}
}
-static struct dentry *z3fold_do_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int z3fold_init_fs_context(struct fs_context *fc)
{
- static const struct dentry_operations ops = {
- .d_dname = simple_dname,
- };
-
- return mount_pseudo(fs_type, "z3fold:", NULL, &ops, 0x33);
+ return init_pseudo(fc, Z3FOLD_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type z3fold_fs = {
.name = "z3fold",
- .mount = z3fold_do_mount,
+ .init_fs_context = z3fold_init_fs_context,
.kill_sb = kill_anon_super,
};
zhdr->start_middle = 0;
zhdr->cpu = -1;
zhdr->slots = slots;
+ zhdr->pool = pool;
INIT_LIST_HEAD(&zhdr->buddy);
INIT_WORK(&zhdr->work, compact_page_work);
return zhdr;
static inline struct z3fold_pool *zhdr_to_pool(struct z3fold_header *zhdr)
{
- return slots_to_pool(zhdr->slots);
+ return zhdr->pool;
}
static void __release_z3fold_page(struct z3fold_header *zhdr, bool locked)
enum buddy bud;
bool can_sleep = gfpflags_allow_blocking(gfp);
- if (!size || (gfp & __GFP_HIGHMEM))
+ if (!size)
return -EINVAL;
if (size > PAGE_SIZE)
zhdr = page_address(page);
pool = zhdr_to_pool(zhdr);
- if (!trylock_page(page))
- return -EAGAIN;
-
if (!z3fold_page_trylock(zhdr)) {
- unlock_page(page);
return -EAGAIN;
}
if (zhdr->mapped_count != 0) {
z3fold_page_unlock(zhdr);
- unlock_page(page);
return -EBUSY;
}
+ if (work_pending(&zhdr->work)) {
+ z3fold_page_unlock(zhdr);
+ return -EAGAIN;
+ }
new_zhdr = page_address(newpage);
memcpy(new_zhdr, zhdr, PAGE_SIZE);
newpage->private = page->private;
page->private = 0;
z3fold_page_unlock(zhdr);
spin_lock_init(&new_zhdr->page_lock);
+ INIT_WORK(&new_zhdr->work, compact_page_work);
+ /*
+ * z3fold_page_isolate() ensures that new_zhdr->buddy is empty,
+ * so we only have to reinitialize it.
+ */
+ INIT_LIST_HEAD(&new_zhdr->buddy);
new_mapping = page_mapping(page);
__ClearPageMovable(page);
ClearPagePrivate(page);
queue_work_on(new_zhdr->cpu, pool->compact_wq, &new_zhdr->work);
page_mapcount_reset(page);
- unlock_page(page);
put_page(page);
return 0;
}
#include <linux/zsmalloc.h>
#include <linux/zpool.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/migrate.h>
#include <linux/pagemap.h>
#include <linux/fs.h>
} while ((page = get_next_page(page)) != NULL);
}
-static struct dentry *zs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int zs_init_fs_context(struct fs_context *fc)
{
- static const struct dentry_operations ops = {
- .d_dname = simple_dname,
- };
-
- return mount_pseudo(fs_type, "zsmalloc:", NULL, &ops, ZSMALLOC_MAGIC);
+ return init_pseudo(fc, ZSMALLOC_MAGIC) ? 0 : -ENOMEM;
}
static struct file_system_type zsmalloc_fs = {
.name = "zsmalloc",
- .mount = zs_mount,
+ .init_fs_context = zs_init_fs_context,
.kill_sb = kill_anon_super,
};
obj-$(CONFIG_CEPH_LIB) += libceph.o
libceph-y := ceph_common.o messenger.o msgpool.o buffer.o pagelist.o \
- mon_client.o \
+ mon_client.o decode.o \
cls_lock_client.o \
osd_client.o osdmap.o crush/crush.o crush/mapper.o crush/hash.o \
striper.o \
#include <linux/ceph/cls_lock_client.h>
#include <linux/ceph/decode.h>
+#include <linux/ceph/libceph.h>
/**
* ceph_cls_lock - grab rados lock for object
return ret;
*p += sizeof(struct ceph_timespec); /* skip expiration */
- ceph_decode_copy(p, &locker->info.addr, sizeof(locker->info.addr));
- ceph_decode_addr(&locker->info.addr);
+
+ ret = ceph_decode_entity_addr(p, end, &locker->info.addr);
+ if (ret)
+ return ret;
+
len = ceph_decode_32(p);
*p += len; /* skip description */
dout("%s lock_name %s\n", __func__, lock_name);
ret = ceph_osdc_call(osdc, oid, oloc, "lock", "get_info",
CEPH_OSD_FLAG_READ, get_info_op_page,
- get_info_op_buf_size, reply_page, &reply_len);
+ get_info_op_buf_size, &reply_page, &reply_len);
dout("%s: status %d\n", __func__, ret);
if (ret >= 0) {
return ret;
}
EXPORT_SYMBOL(ceph_cls_lock_info);
+
+int ceph_cls_assert_locked(struct ceph_osd_request *req, int which,
+ char *lock_name, u8 type, char *cookie, char *tag)
+{
+ int assert_op_buf_size;
+ int name_len = strlen(lock_name);
+ int cookie_len = strlen(cookie);
+ int tag_len = strlen(tag);
+ struct page **pages;
+ void *p, *end;
+ int ret;
+
+ assert_op_buf_size = name_len + sizeof(__le32) +
+ cookie_len + sizeof(__le32) +
+ tag_len + sizeof(__le32) +
+ sizeof(u8) + CEPH_ENCODING_START_BLK_LEN;
+ if (assert_op_buf_size > PAGE_SIZE)
+ return -E2BIG;
+
+ ret = osd_req_op_cls_init(req, which, "lock", "assert_locked");
+ if (ret)
+ return ret;
+
+ pages = ceph_alloc_page_vector(1, GFP_NOIO);
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
+
+ p = page_address(pages[0]);
+ end = p + assert_op_buf_size;
+
+ /* encode cls_lock_assert_op struct */
+ ceph_start_encoding(&p, 1, 1,
+ assert_op_buf_size - CEPH_ENCODING_START_BLK_LEN);
+ ceph_encode_string(&p, end, lock_name, name_len);
+ ceph_encode_8(&p, type);
+ ceph_encode_string(&p, end, cookie, cookie_len);
+ ceph_encode_string(&p, end, tag, tag_len);
+ WARN_ON(p != end);
+
+ osd_req_op_cls_request_data_pages(req, which, pages, assert_op_buf_size,
+ 0, false, true);
+ return 0;
+}
+EXPORT_SYMBOL(ceph_cls_assert_locked);
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/ceph/decode.h>
+
+static int
+ceph_decode_entity_addr_versioned(void **p, void *end,
+ struct ceph_entity_addr *addr)
+{
+ int ret;
+ u8 struct_v;
+ u32 struct_len, addr_len;
+ void *struct_end;
+
+ ret = ceph_start_decoding(p, end, 1, "entity_addr_t", &struct_v,
+ &struct_len);
+ if (ret)
+ goto bad;
+
+ ret = -EINVAL;
+ struct_end = *p + struct_len;
+
+ ceph_decode_copy_safe(p, end, &addr->type, sizeof(addr->type), bad);
+
+ ceph_decode_copy_safe(p, end, &addr->nonce, sizeof(addr->nonce), bad);
+
+ ceph_decode_32_safe(p, end, addr_len, bad);
+ if (addr_len > sizeof(addr->in_addr))
+ goto bad;
+
+ memset(&addr->in_addr, 0, sizeof(addr->in_addr));
+ if (addr_len) {
+ ceph_decode_copy_safe(p, end, &addr->in_addr, addr_len, bad);
+
+ addr->in_addr.ss_family =
+ le16_to_cpu((__force __le16)addr->in_addr.ss_family);
+ }
+
+ /* Advance past anything the client doesn't yet understand */
+ *p = struct_end;
+ ret = 0;
+bad:
+ return ret;
+}
+
+static int
+ceph_decode_entity_addr_legacy(void **p, void *end,
+ struct ceph_entity_addr *addr)
+{
+ int ret = -EINVAL;
+
+ /* Skip rest of type field */
+ ceph_decode_skip_n(p, end, 3, bad);
+
+ /*
+ * Clients that don't support ADDR2 always send TYPE_NONE, change it
+ * to TYPE_LEGACY for forward compatibility.
+ */
+ addr->type = CEPH_ENTITY_ADDR_TYPE_LEGACY;
+ ceph_decode_copy_safe(p, end, &addr->nonce, sizeof(addr->nonce), bad);
+ memset(&addr->in_addr, 0, sizeof(addr->in_addr));
+ ceph_decode_copy_safe(p, end, &addr->in_addr,
+ sizeof(addr->in_addr), bad);
+ addr->in_addr.ss_family =
+ be16_to_cpu((__force __be16)addr->in_addr.ss_family);
+ ret = 0;
+bad:
+ return ret;
+}
+
+int
+ceph_decode_entity_addr(void **p, void *end, struct ceph_entity_addr *addr)
+{
+ u8 marker;
+
+ ceph_decode_8_safe(p, end, marker, bad);
+ if (marker == 1)
+ return ceph_decode_entity_addr_versioned(p, end, addr);
+ else if (marker == 0)
+ return ceph_decode_entity_addr_legacy(p, end, addr);
+bad:
+ return -EINVAL;
+}
+EXPORT_SYMBOL(ceph_decode_entity_addr);
+
switch (ss.ss_family) {
case AF_INET:
- snprintf(s, MAX_ADDR_STR_LEN, "%pI4:%hu", &in4->sin_addr,
+ snprintf(s, MAX_ADDR_STR_LEN, "(%d)%pI4:%hu",
+ le32_to_cpu(addr->type), &in4->sin_addr,
ntohs(in4->sin_port));
break;
case AF_INET6:
- snprintf(s, MAX_ADDR_STR_LEN, "[%pI6c]:%hu", &in6->sin6_addr,
+ snprintf(s, MAX_ADDR_STR_LEN, "(%d)[%pI6c]:%hu",
+ le32_to_cpu(addr->type), &in6->sin6_addr,
ntohs(in6->sin6_port));
break;
static void encode_my_addr(struct ceph_messenger *msgr)
{
memcpy(&msgr->my_enc_addr, &msgr->inst.addr, sizeof(msgr->my_enc_addr));
- ceph_encode_addr(&msgr->my_enc_addr);
+ ceph_encode_banner_addr(&msgr->my_enc_addr);
}
/*
ret = read_partial(con, end, size, &con->actual_peer_addr);
if (ret <= 0)
goto out;
+ ceph_decode_banner_addr(&con->actual_peer_addr);
size = sizeof (con->peer_addr_for_me);
end += size;
ret = read_partial(con, end, size, &con->peer_addr_for_me);
if (ret <= 0)
goto out;
+ ceph_decode_banner_addr(&con->peer_addr_for_me);
out:
return ret;
}
addr_set_port(&addr[i], port);
+ addr[i].type = CEPH_ENTITY_ADDR_TYPE_LEGACY;
dout("parse_ips got %s\n", ceph_pr_addr(&addr[i]));
if (verify_hello(con) < 0)
return -1;
- ceph_decode_addr(&con->actual_peer_addr);
- ceph_decode_addr(&con->peer_addr_for_me);
-
/*
* Make sure the other end is who we wanted. note that the other
* end may not yet know their ip address, so if it's 0.0.0.0, give
/*
* Decode a monmap blob (e.g., during mount).
*/
-struct ceph_monmap *ceph_monmap_decode(void *p, void *end)
+static struct ceph_monmap *ceph_monmap_decode(void *p, void *end)
{
struct ceph_monmap *m = NULL;
int i, err = -EINVAL;
ceph_decode_32_safe(&p, end, len, bad);
ceph_decode_need(&p, end, len, bad);
- dout("monmap_decode %p %p len %d\n", p, end, (int)(end-p));
+ dout("monmap_decode %p %p len %d (%d)\n", p, end, len, (int)(end-p));
p += sizeof(u16); /* skip version */
ceph_decode_need(&p, end, sizeof(fsid) + 2*sizeof(u32), bad);
epoch = ceph_decode_32(&p);
num_mon = ceph_decode_32(&p);
- ceph_decode_need(&p, end, num_mon*sizeof(m->mon_inst[0]), bad);
if (num_mon > CEPH_MAX_MON)
goto bad;
m->fsid = fsid;
m->epoch = epoch;
m->num_mon = num_mon;
- ceph_decode_copy(&p, m->mon_inst, num_mon*sizeof(m->mon_inst[0]));
- for (i = 0; i < num_mon; i++)
- ceph_decode_addr(&m->mon_inst[i].addr);
-
+ for (i = 0; i < num_mon; ++i) {
+ struct ceph_entity_inst *inst = &m->mon_inst[i];
+
+ /* copy name portion */
+ ceph_decode_copy_safe(&p, end, &inst->name,
+ sizeof(inst->name), bad);
+ err = ceph_decode_entity_addr(&p, end, &inst->addr);
+ if (err)
+ goto bad;
+ }
dout("monmap_decode epoch %d, num_mon %d\n", m->epoch,
m->num_mon);
for (i = 0; i < m->num_mon; i++)
dout("monmap_decode mon%d is %s\n", i,
ceph_pr_addr(&m->mon_inst[i].addr));
return m;
-
bad:
dout("monmap_decode failed with %d\n", err);
kfree(m);
if (IS_ERR(monmap)) {
pr_err("problem decoding monmap, %d\n",
(int)PTR_ERR(monmap));
+ ceph_msg_dump(msg);
goto out;
}
osd_data->num_bvecs = num_bvecs;
}
-#define osd_req_op_data(oreq, whch, typ, fld) \
-({ \
- struct ceph_osd_request *__oreq = (oreq); \
- unsigned int __whch = (whch); \
- BUG_ON(__whch >= __oreq->r_num_ops); \
- &__oreq->r_ops[__whch].typ.fld; \
-})
-
static struct ceph_osd_data *
osd_req_op_raw_data_in(struct ceph_osd_request *osd_req, unsigned int which)
{
{
WARN_ON(!RB_EMPTY_NODE(&req->r_node));
WARN_ON(!RB_EMPTY_NODE(&req->r_mc_node));
- WARN_ON(!list_empty(&req->r_unsafe_item));
+ WARN_ON(!list_empty(&req->r_private_item));
WARN_ON(req->r_osd);
}
init_completion(&req->r_completion);
RB_CLEAR_NODE(&req->r_node);
RB_CLEAR_NODE(&req->r_mc_node);
- INIT_LIST_HEAD(&req->r_unsafe_item);
+ INIT_LIST_HEAD(&req->r_private_item);
target_init(&req->r_t);
}
ret = ceph_start_decoding(p, end, 2, "watch_item_t",
&struct_v, &struct_len);
if (ret)
- return ret;
+ goto bad;
+
+ ret = -EINVAL;
+ ceph_decode_copy_safe(p, end, &item->name, sizeof(item->name), bad);
+ ceph_decode_64_safe(p, end, item->cookie, bad);
+ ceph_decode_skip_32(p, end, bad); /* skip timeout seconds */
- ceph_decode_copy(p, &item->name, sizeof(item->name));
- item->cookie = ceph_decode_64(p);
- *p += 4; /* skip timeout_seconds */
if (struct_v >= 2) {
- ceph_decode_copy(p, &item->addr, sizeof(item->addr));
- ceph_decode_addr(&item->addr);
+ ret = ceph_decode_entity_addr(p, end, &item->addr);
+ if (ret)
+ goto bad;
+ } else {
+ ret = 0;
}
dout("%s %s%llu cookie %llu addr %s\n", __func__,
ENTITY_NAME(item->name), item->cookie,
ceph_pr_addr(&item->addr));
- return 0;
+bad:
+ return ret;
}
static int decode_watchers(void **p, void *end,
const char *class, const char *method,
unsigned int flags,
struct page *req_page, size_t req_len,
- struct page *resp_page, size_t *resp_len)
+ struct page **resp_pages, size_t *resp_len)
{
struct ceph_osd_request *req;
int ret;
- if (req_len > PAGE_SIZE || (resp_page && *resp_len > PAGE_SIZE))
+ if (req_len > PAGE_SIZE)
return -E2BIG;
req = ceph_osdc_alloc_request(osdc, NULL, 1, false, GFP_NOIO);
if (req_page)
osd_req_op_cls_request_data_pages(req, 0, &req_page, req_len,
0, false, false);
- if (resp_page)
- osd_req_op_cls_response_data_pages(req, 0, &resp_page,
+ if (resp_pages)
+ osd_req_op_cls_response_data_pages(req, 0, resp_pages,
*resp_len, 0, false, false);
ret = ceph_osdc_alloc_messages(req, GFP_NOIO);
ret = ceph_osdc_wait_request(osdc, req);
if (ret >= 0) {
ret = req->r_ops[0].rval;
- if (resp_page)
+ if (resp_pages)
*resp_len = req->r_ops[0].outdata_len;
}
/* osd_state, osd_weight, osd_addrs->client_addr */
ceph_decode_need(p, end, 3*sizeof(u32) +
- map->max_osd*((struct_v >= 5 ? sizeof(u32) :
- sizeof(u8)) +
- sizeof(*map->osd_weight) +
- sizeof(*map->osd_addr)), e_inval);
-
+ map->max_osd*(struct_v >= 5 ? sizeof(u32) :
+ sizeof(u8)) +
+ sizeof(*map->osd_weight), e_inval);
if (ceph_decode_32(p) != map->max_osd)
goto e_inval;
if (ceph_decode_32(p) != map->max_osd)
goto e_inval;
- ceph_decode_copy(p, map->osd_addr, map->max_osd*sizeof(*map->osd_addr));
- for (i = 0; i < map->max_osd; i++)
- ceph_decode_addr(&map->osd_addr[i]);
+ for (i = 0; i < map->max_osd; i++) {
+ err = ceph_decode_entity_addr(p, end, &map->osd_addr[i]);
+ if (err)
+ goto bad;
+ }
/* pg_temp */
err = decode_pg_temp(p, end, map);
void *new_state;
void *new_weight_end;
u32 len;
+ int i;
new_up_client = *p;
ceph_decode_32_safe(p, end, len, e_inval);
- len *= sizeof(u32) + sizeof(struct ceph_entity_addr);
- ceph_decode_need(p, end, len, e_inval);
- *p += len;
+ for (i = 0; i < len; ++i) {
+ struct ceph_entity_addr addr;
+
+ ceph_decode_skip_32(p, end, e_inval);
+ if (ceph_decode_entity_addr(p, end, &addr))
+ goto e_inval;
+ }
new_state = *p;
ceph_decode_32_safe(p, end, len, e_inval);
struct ceph_entity_addr addr;
osd = ceph_decode_32(p);
- ceph_decode_copy(p, &addr, sizeof(addr));
- ceph_decode_addr(&addr);
BUG_ON(osd >= map->max_osd);
+ if (ceph_decode_entity_addr(p, end, &addr))
+ goto e_inval;
pr_info("osd%d up\n", osd);
map->osd_state[osd] |= CEPH_OSD_EXISTS | CEPH_OSD_UP;
map->osd_addr[osd] = addr;
#include <linux/ceph/libceph.h>
-/*
- * build a vector of user pages
- */
-struct page **ceph_get_direct_page_vector(const void __user *data,
- int num_pages, bool write_page)
-{
- struct page **pages;
- int got = 0;
- int rc = 0;
-
- pages = kmalloc_array(num_pages, sizeof(*pages), GFP_NOFS);
- if (!pages)
- return ERR_PTR(-ENOMEM);
-
- while (got < num_pages) {
- rc = get_user_pages_fast(
- (unsigned long)data + ((unsigned long)got * PAGE_SIZE),
- num_pages - got, write_page ? FOLL_WRITE : 0, pages + got);
- if (rc < 0)
- break;
- BUG_ON(rc == 0);
- got += rc;
- }
- if (rc < 0)
- goto fail;
- return pages;
-
-fail:
- ceph_put_page_vector(pages, got, false);
- return ERR_PTR(rc);
-}
-EXPORT_SYMBOL(ceph_get_direct_page_vector);
-
void ceph_put_page_vector(struct page **pages, int num_pages, bool dirty)
{
int i;
return 0;
}
EXPORT_SYMBOL(ceph_extent_to_file);
+
+u64 ceph_get_num_objects(struct ceph_file_layout *l, u64 size)
+{
+ u64 period = (u64)l->stripe_count * l->object_size;
+ u64 num_periods = DIV64_U64_ROUND_UP(size, period);
+ u64 remainder_bytes;
+ u64 remainder_objs = 0;
+
+ div64_u64_rem(size, period, &remainder_bytes);
+ if (remainder_bytes > 0 &&
+ remainder_bytes < (u64)l->stripe_count * l->stripe_unit)
+ remainder_objs = l->stripe_count -
+ DIV_ROUND_UP_ULL(remainder_bytes, l->stripe_unit);
+
+ return num_periods * l->stripe_count - remainder_objs;
+}
+EXPORT_SYMBOL(ceph_get_num_objects);
TCP_CA_NAME_MAX-1));
name[TCP_CA_NAME_MAX-1] = 0;
ret = tcp_set_congestion_control(sk, name, false,
- reinit);
+ reinit, true);
} else {
struct tcp_sock *tp = tcp_sk(sk);
case bpf_ctx_range(struct bpf_sock_addr, msg_src_ip4):
case bpf_ctx_range_till(struct bpf_sock_addr, msg_src_ip6[0],
msg_src_ip6[3]):
- /* Only narrow read access allowed for now. */
if (type == BPF_READ) {
bpf_ctx_record_field_size(info, size_default);
+
+ if (bpf_ctx_wide_access_ok(off, size,
+ struct bpf_sock_addr,
+ user_ip6))
+ return true;
+
+ if (bpf_ctx_wide_access_ok(off, size,
+ struct bpf_sock_addr,
+ msg_src_ip6))
+ return true;
+
if (!bpf_ctx_narrow_access_ok(off, size, size_default))
return false;
} else {
- if (bpf_ctx_wide_store_ok(off, size,
- struct bpf_sock_addr,
- user_ip6))
+ if (bpf_ctx_wide_access_ok(off, size,
+ struct bpf_sock_addr,
+ user_ip6))
return true;
- if (bpf_ctx_wide_store_ok(off, size,
- struct bpf_sock_addr,
- msg_src_ip6))
+ if (bpf_ctx_wide_access_ok(off, size,
+ struct bpf_sock_addr,
+ msg_src_ip6))
return true;
if (size != size_default)
atomic_set(&neigh->probes,
NEIGH_VAR(neigh->parms, UCAST_PROBES));
+ neigh_del_timer(neigh);
neigh->nud_state = NUD_INCOMPLETE;
neigh->updated = now;
next = now + max(NEIGH_VAR(neigh->parms, RETRANS_TIME),
}
} else if (neigh->nud_state & NUD_STALE) {
neigh_dbg(2, "neigh %p is delayed\n", neigh);
+ neigh_del_timer(neigh);
neigh->nud_state = NUD_DELAY;
neigh->updated = jiffies;
neigh_add_timer(neigh, jiffies +
EXPORT_SYMBOL(neigh_app_ns);
#ifdef CONFIG_SYSCTL
-static int zero;
-static int int_max = INT_MAX;
static int unres_qlen_max = INT_MAX / SKB_TRUESIZE(ETH_FRAME_LEN);
static int proc_unres_qlen(struct ctl_table *ctl, int write,
int size, ret;
struct ctl_table tmp = *ctl;
- tmp.extra1 = &zero;
+ tmp.extra1 = SYSCTL_ZERO;
tmp.extra2 = &unres_qlen_max;
tmp.data = &size;
struct ctl_table tmp = *ctl;
int ret;
- tmp.extra1 = &zero;
- tmp.extra2 = &int_max;
+ tmp.extra1 = SYSCTL_ZERO;
+ tmp.extra2 = SYSCTL_INT_MAX;
ret = proc_dointvec_minmax(&tmp, write, buffer, lenp, ppos);
neigh_proc_update(ctl, write);
.procname = "gc_thresh1",
.maxlen = sizeof(int),
.mode = 0644,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
.proc_handler = proc_dointvec_minmax,
},
[NEIGH_VAR_GC_THRESH2] = {
.procname = "gc_thresh2",
.maxlen = sizeof(int),
.mode = 0644,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
.proc_handler = proc_dointvec_minmax,
},
[NEIGH_VAR_GC_THRESH3] = {
.procname = "gc_thresh3",
.maxlen = sizeof(int),
.mode = 0644,
- .extra1 = &zero,
- .extra2 = &int_max,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
.proc_handler = proc_dointvec_minmax,
},
{},
printk("%sdev name=%s feat=0x%pNF\n",
level, dev->name, &dev->features);
if (sk)
- printk("%ssk family=%hu type=%hu proto=%hu\n",
+ printk("%ssk family=%hu type=%u proto=%u\n",
level, sk->sk_family, sk->sk_type, sk->sk_protocol);
if (full_pkt && headroom)
#include <net/busy_poll.h>
#include <net/pkt_sched.h>
-static int zero = 0;
-static int one = 1;
static int two __maybe_unused = 2;
static int min_sndbuf = SOCK_MIN_SNDBUF;
static int min_rcvbuf = SOCK_MIN_RCVBUF;
.mode = 0644,
.proc_handler = proc_dointvec_minmax_bpf_enable,
# ifdef CONFIG_BPF_JIT_ALWAYS_ON
- .extra1 = &one,
- .extra2 = &one,
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_ONE,
# else
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
# endif
},
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax_bpf_restricted,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(int),
.mode = 0600,
.proc_handler = proc_dointvec_minmax_bpf_restricted,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
# endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE
},
#ifdef CONFIG_RPS
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "busy_read",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#endif
#ifdef CONFIG_NET_SCHED
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &max_skb_frags,
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "fb_tunnels_only_for_init_net",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "devconf_inherit_init_net",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.data = &init_net.core.sysctl_somaxconn,
.maxlen = sizeof(int),
.mode = 0644,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.proc_handler = proc_dointvec_minmax
},
{ }
#endif
/* Boundary values */
-static int zero = 0,
- one = 1,
- u8_max = 0xFF;
+static int u8_max = 0xFF;
static unsigned long seqw_min = DCCPF_SEQ_WMIN,
seqw_max = 0xFFFFFFFF; /* maximum on 32 bit */
.maxlen = sizeof(sysctl_dccp_rx_ccid),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &u8_max, /* RFC 4340, 10. */
},
{
.maxlen = sizeof(sysctl_dccp_tx_ccid),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &u8_max, /* RFC 4340, 10. */
},
{
.maxlen = sizeof(sysctl_dccp_request_retries),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &u8_max,
},
{
.maxlen = sizeof(sysctl_dccp_retries1),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &u8_max,
},
{
.maxlen = sizeof(sysctl_dccp_retries2),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &u8_max,
},
{
.maxlen = sizeof(sysctl_dccp_tx_qlen),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "sync_ratelimit",
if (!skb) {
dev_err_ratelimited(dp->ds->dev,
"Failed to copy stampable skb\n");
+ spin_unlock(&sp->data->meta_lock);
return NULL;
}
sja1105_transfer_meta(skb, meta);
fib_combine_itag(itag, &res);
dev_match = fib_info_nh_uses_dev(res.fi, dev);
+ /* This is not common, loopback packets retain skb_dst so normally they
+ * would not even hit this slow path.
+ */
+ dev_match = dev_match || (res.type == RTN_LOCAL &&
+ dev == net->loopback_dev);
if (dev_match) {
ret = FIB_RES_NHC(res)->nhc_scope >= RT_SCOPE_HOST;
return ret;
#include <net/protocol.h>
#include <net/netevent.h>
-static int zero;
-static int one = 1;
static int two = 2;
static int four = 4;
static int thousand = 1000;
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "icmp_msgs_burst",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "udp_mem",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = ipv4_fwd_update_priority,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "ip_nonlocal_bind",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one
+ .extra1 = SYSCTL_ONE
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &two,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_tfo_blackhole_detect_timeout,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
#ifdef CONFIG_IP_ROUTE_MULTIPATH
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "fib_multipath_hash_policy",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_fib_multipath_hash_policy,
- .extra1 = &zero,
- .extra2 = &two,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
#endif
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &four,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &gso_max_segs,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &one_day_secs
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "tcp_invalid_ratelimit",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &thousand,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &thousand,
},
{
.maxlen = sizeof(init_net.ipv4.sysctl_tcp_wmem),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{
.procname = "tcp_rmem",
.maxlen = sizeof(init_net.ipv4.sysctl_tcp_rmem),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{
.procname = "tcp_comp_sack_delay_ns",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &comp_sack_nr_max,
},
{
.maxlen = sizeof(init_net.ipv4.sysctl_udp_rmem_min),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one
+ .extra1 = SYSCTL_ONE
},
{
.procname = "udp_wmem_min",
.maxlen = sizeof(init_net.ipv4.sysctl_udp_wmem_min),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one
+ .extra1 = SYSCTL_ONE
},
{ }
};
name[val] = 0;
lock_sock(sk);
- err = tcp_set_congestion_control(sk, name, true, true);
+ err = tcp_set_congestion_control(sk, name, true, true,
+ ns_capable(sock_net(sk)->user_ns,
+ CAP_NET_ADMIN));
release_sock(sk);
return err;
}
* tcp_reinit_congestion_control (if the current congestion control was
* already initialized.
*/
-int tcp_set_congestion_control(struct sock *sk, const char *name, bool load, bool reinit)
+int tcp_set_congestion_control(struct sock *sk, const char *name, bool load,
+ bool reinit, bool cap_net_admin)
{
struct inet_connection_sock *icsk = inet_csk(sk);
const struct tcp_congestion_ops *ca;
} else {
err = -EBUSY;
}
- } else if (!((ca->flags & TCP_CONG_NON_RESTRICTED) ||
- ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN))) {
+ } else if (!((ca->flags & TCP_CONG_NON_RESTRICTED) || cap_net_admin)) {
err = -EPERM;
} else if (!try_module_get(ca->owner)) {
err = -EBUSY;
/* Initialize UDP checksum. If exited with zero value (success),
* CHECKSUM_UNNECESSARY means, that no more checks are required.
- * Otherwise, csum completion requires chacksumming packet body,
+ * Otherwise, csum completion requires checksumming packet body,
* including udp header and folding it to skb->csum.
*/
static inline int udp4_csum_init(struct sk_buff *skb, struct udphdr *uh,
}
static int minus_one = -1;
-static const int zero = 0;
-static const int one = 1;
static const int two_five_five = 255;
static const struct ctl_table addrconf_sysctl[] = {
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&one,
+ .extra1 = (void *)SYSCTL_ONE,
.extra2 = (void *)&two_five_five,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&zero,
+ .extra1 = (void *)SYSCTL_ZERO,
.extra2 = (void *)&two_five_five,
},
{
err = call_fib6_entry_notifiers(info->nl_net,
FIB_EVENT_ENTRY_ADD,
rt, extack);
- if (err)
+ if (err) {
+ struct fib6_info *sibling, *next_sibling;
+
+ /* If the route has siblings, then it first
+ * needs to be unlinked from them.
+ */
+ if (!rt->fib6_nsiblings)
+ return err;
+
+ list_for_each_entry_safe(sibling, next_sibling,
+ &rt->fib6_siblings,
+ fib6_siblings)
+ sibling->fib6_nsiblings--;
+ rt->fib6_nsiblings = 0;
+ list_del_init(&rt->fib6_siblings);
+ rt6_multipath_rebalance(next_sibling);
return err;
+ }
}
rcu_assign_pointer(rt->fib6_next, iter);
{
u32 rt_cookie = 0;
- if ((from && !fib6_get_cookie_safe(from, &rt_cookie)) ||
+ if (!from || !fib6_get_cookie_safe(from, &rt_cookie) ||
rt_cookie != cookie)
return NULL;
return 0;
}
-static int zero;
-static int one = 1;
-
static struct ctl_table ipv6_route_table_template[] = {
{
.procname = "flush",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{ }
};
RT_TOS(tos), RT_SCOPE_UNIVERSE, IPPROTO_IPV6,
0, dst, tiph->saddr, 0, 0,
sock_net_uid(tunnel->net, NULL));
- rt = ip_route_output_flow(tunnel->net, &fl4, NULL);
- if (IS_ERR(rt)) {
- dev->stats.tx_carrier_errors++;
- goto tx_error_icmp;
+ rt = dst_cache_get_ip4(&tunnel->dst_cache, &fl4.saddr);
+ if (!rt) {
+ rt = ip_route_output_flow(tunnel->net, &fl4, NULL);
+ if (IS_ERR(rt)) {
+ dev->stats.tx_carrier_errors++;
+ goto tx_error_icmp;
+ }
+ dst_cache_set_ip4(&tunnel->dst_cache, &rt->dst, fl4.saddr);
}
+
if (rt->rt_type != RTN_UNICAST) {
ip_rt_put(rt);
dev->stats.tx_carrier_errors++;
#include <net/calipso.h>
#endif
-static int zero;
-static int one = 1;
static int flowlabel_reflect_max = 0x7;
static int auto_flowlabels_min;
static int auto_flowlabels_max = IP6_AUTO_FLOW_LABEL_MAX;
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &flowlabel_reflect_max,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_rt6_multipath_hash_policy,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "seg6_flowlabel",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one
+ .extra1 = SYSCTL_ONE
},
#ifdef CONFIG_NETLABEL
{
#define MPLS_NEIGH_TABLE_UNSPEC (NEIGH_LINK_TABLE + 1)
-static int zero = 0;
-static int one = 1;
static int label_limit = (1 << 20) - 1;
static int ttl_max = 255;
.data = &platform_labels,
.maxlen = sizeof(int),
.mode = table->mode,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &label_limit,
};
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{
.procname = "default_ttl",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &ttl_max,
},
{ }
#ifdef CONFIG_SYSCTL
-static int zero;
static int three = 3;
static int
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &three,
},
{
/* To control the number of wrs from fastreg */
atomic_t i_fastreg_wrs;
+ atomic_t i_fastreg_inuse_count;
/* interrupt handling */
struct tasklet_struct i_send_tasklet;
#include "rds_single_path.h"
#include "rds.h"
#include "ib.h"
+#include "ib_mr.h"
/*
* Set the selected protocol version
attr.qp_type = IB_QPT_RC;
attr.send_cq = ic->i_send_cq;
attr.recv_cq = ic->i_recv_cq;
- atomic_set(&ic->i_fastreg_wrs, RDS_IB_DEFAULT_FR_WR);
/*
* XXX this can fail if max_*_wr is too large? Are we supposed
ic->i_cm_id, err);
}
+ /* kick off "flush_worker" for all pools in order to reap
+ * all FRMR registrations that are still marked "FRMR_IS_INUSE"
+ */
+ rds_ib_flush_mrs();
+
/*
* We want to wait for tx and rx completion to finish
* before we tear down the connection, but we have to be
wait_event(rds_ib_ring_empty_wait,
rds_ib_ring_empty(&ic->i_recv_ring) &&
(atomic_read(&ic->i_signaled_sends) == 0) &&
+ (atomic_read(&ic->i_fastreg_inuse_count) == 0) &&
(atomic_read(&ic->i_fastreg_wrs) == RDS_IB_DEFAULT_FR_WR));
tasklet_kill(&ic->i_send_tasklet);
tasklet_kill(&ic->i_recv_tasklet);
spin_lock_init(&ic->i_ack_lock);
#endif
atomic_set(&ic->i_signaled_sends, 0);
+ atomic_set(&ic->i_fastreg_wrs, RDS_IB_DEFAULT_FR_WR);
/*
* rds_ib_conn_shutdown() waits for these to be emptied so they
#include "ib_mr.h"
+static inline void
+rds_transition_frwr_state(struct rds_ib_mr *ibmr,
+ enum rds_ib_fr_state old_state,
+ enum rds_ib_fr_state new_state)
+{
+ if (cmpxchg(&ibmr->u.frmr.fr_state,
+ old_state, new_state) == old_state &&
+ old_state == FRMR_IS_INUSE) {
+ /* enforce order of ibmr->u.frmr.fr_state update
+ * before decrementing i_fastreg_inuse_count
+ */
+ smp_mb__before_atomic();
+ atomic_dec(&ibmr->ic->i_fastreg_inuse_count);
+ if (waitqueue_active(&rds_ib_ring_empty_wait))
+ wake_up(&rds_ib_ring_empty_wait);
+ }
+}
+
static struct rds_ib_mr *rds_ib_alloc_frmr(struct rds_ib_device *rds_ibdev,
int npages)
{
pool->max_items_soft = pool->max_items;
frmr->fr_state = FRMR_IS_FREE;
+ init_waitqueue_head(&frmr->fr_inv_done);
+ init_waitqueue_head(&frmr->fr_reg_done);
return ibmr;
out_no_cigar:
if (unlikely(ret != ibmr->sg_len))
return ret < 0 ? ret : -EINVAL;
+ if (cmpxchg(&frmr->fr_state,
+ FRMR_IS_FREE, FRMR_IS_INUSE) != FRMR_IS_FREE)
+ return -EBUSY;
+
+ atomic_inc(&ibmr->ic->i_fastreg_inuse_count);
+
/* Perform a WR for the fast_reg_mr. Each individual page
* in the sg list is added to the fast reg page list and placed
* inside the fast_reg_mr WR. The key used is a rolling 8bit
* counter, which should guarantee uniqueness.
*/
ib_update_fast_reg_key(frmr->mr, ibmr->remap_count++);
- frmr->fr_state = FRMR_IS_INUSE;
+ frmr->fr_reg = true;
memset(®_wr, 0, sizeof(reg_wr));
reg_wr.wr.wr_id = (unsigned long)(void *)ibmr;
ret = ib_post_send(ibmr->ic->i_cm_id->qp, ®_wr.wr, NULL);
if (unlikely(ret)) {
/* Failure here can be because of -ENOMEM as well */
- frmr->fr_state = FRMR_IS_STALE;
+ rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_STALE);
+
atomic_inc(&ibmr->ic->i_fastreg_wrs);
if (printk_ratelimit())
pr_warn("RDS/IB: %s returned error(%d)\n",
__func__, ret);
+ goto out;
}
+
+ /* Wait for the registration to complete in order to prevent an invalid
+ * access error resulting from a race between the memory region already
+ * being accessed while registration is still pending.
+ */
+ wait_event(frmr->fr_reg_done, !frmr->fr_reg);
+
+out:
+
return ret;
}
ret = ib_post_send(i_cm_id->qp, s_wr, NULL);
if (unlikely(ret)) {
- frmr->fr_state = FRMR_IS_STALE;
+ rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_STALE);
frmr->fr_inv = false;
+ /* enforce order of frmr->fr_inv update
+ * before incrementing i_fastreg_wrs
+ */
+ smp_mb__before_atomic();
atomic_inc(&ibmr->ic->i_fastreg_wrs);
pr_err("RDS/IB: %s returned error(%d)\n", __func__, ret);
goto out;
}
+
+ /* Wait for the FRMR_IS_FREE (or FRMR_IS_STALE) transition in order to
+ * 1) avoid a silly bouncing between "clean_list" and "drop_list"
+ * triggered by function "rds_ib_reg_frmr" as it is releases frmr
+ * regions whose state is not "FRMR_IS_FREE" right away.
+ * 2) prevents an invalid access error in a race
+ * from a pending "IB_WR_LOCAL_INV" operation
+ * with a teardown ("dma_unmap_sg", "put_page")
+ * and de-registration ("ib_dereg_mr") of the corresponding
+ * memory region.
+ */
+ wait_event(frmr->fr_inv_done, frmr->fr_state != FRMR_IS_INUSE);
+
out:
return ret;
}
struct rds_ib_frmr *frmr = &ibmr->u.frmr;
if (wc->status != IB_WC_SUCCESS) {
- frmr->fr_state = FRMR_IS_STALE;
+ rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_STALE);
if (rds_conn_up(ic->conn))
rds_ib_conn_error(ic->conn,
"frmr completion <%pI4,%pI4> status %u(%s), vendor_err 0x%x, disconnecting and reconnecting\n",
}
if (frmr->fr_inv) {
- frmr->fr_state = FRMR_IS_FREE;
+ rds_transition_frwr_state(ibmr, FRMR_IS_INUSE, FRMR_IS_FREE);
frmr->fr_inv = false;
+ wake_up(&frmr->fr_inv_done);
}
+ if (frmr->fr_reg) {
+ frmr->fr_reg = false;
+ wake_up(&frmr->fr_reg_done);
+ }
+
+ /* enforce order of frmr->{fr_reg,fr_inv} update
+ * before incrementing i_fastreg_wrs
+ */
+ smp_mb__before_atomic();
atomic_inc(&ic->i_fastreg_wrs);
}
{
struct rds_ib_mr *ibmr, *next;
struct rds_ib_frmr *frmr;
- int ret = 0;
+ int ret = 0, ret2;
unsigned int freed = *nfreed;
/* String all ib_mr's onto one list and hand them to ib_unmap_fmr */
list_for_each_entry(ibmr, list, unmap_list) {
- if (ibmr->sg_dma_len)
- ret |= rds_ib_post_inv(ibmr);
+ if (ibmr->sg_dma_len) {
+ ret2 = rds_ib_post_inv(ibmr);
+ if (ret2 && !ret)
+ ret = ret2;
+ }
}
+
if (ret)
pr_warn("RDS/IB: %s failed (err=%d)\n", __func__, ret);
struct ib_mr *mr;
enum rds_ib_fr_state fr_state;
bool fr_inv;
+ wait_queue_head_t fr_inv_done;
+ bool fr_reg;
+ wait_queue_head_t fr_reg_done;
struct ib_send_wr fr_wr;
unsigned int dma_npages;
unsigned int sg_byte_len;
struct llist_head free_list; /* unused MRs */
struct llist_head clean_list; /* unused & unmapped MRs */
wait_queue_head_t flush_wait;
+ spinlock_t clean_lock; /* "clean_list" concurrency */
atomic_t free_pinned; /* memory pinned by free MRs */
unsigned long max_items;
struct workqueue_struct *rds_ib_mr_wq;
-static DEFINE_PER_CPU(unsigned long, clean_list_grace);
-#define CLEAN_LIST_BUSY_BIT 0
-
static struct rds_ib_device *rds_ib_get_device(__be32 ipaddr)
{
struct rds_ib_device *rds_ibdev;
{
struct rds_ib_mr *ibmr = NULL;
struct llist_node *ret;
- unsigned long *flag;
+ unsigned long flags;
- preempt_disable();
- flag = this_cpu_ptr(&clean_list_grace);
- set_bit(CLEAN_LIST_BUSY_BIT, flag);
+ spin_lock_irqsave(&pool->clean_lock, flags);
ret = llist_del_first(&pool->clean_list);
+ spin_unlock_irqrestore(&pool->clean_lock, flags);
if (ret) {
ibmr = llist_entry(ret, struct rds_ib_mr, llnode);
if (pool->pool_type == RDS_IB_MR_8K_POOL)
rds_ib_stats_inc(s_ib_rdma_mr_1m_reused);
}
- clear_bit(CLEAN_LIST_BUSY_BIT, flag);
- preempt_enable();
return ibmr;
}
-static inline void wait_clean_list_grace(void)
-{
- int cpu;
- unsigned long *flag;
-
- for_each_online_cpu(cpu) {
- flag = &per_cpu(clean_list_grace, cpu);
- while (test_bit(CLEAN_LIST_BUSY_BIT, flag))
- cpu_relax();
- }
-}
-
void rds_ib_sync_mr(void *trans_private, int direction)
{
struct rds_ib_mr *ibmr = trans_private;
* of clusters. Each cluster has linked llist nodes of
* MR_CLUSTER_SIZE mrs that are ready for reuse.
*/
-static void list_to_llist_nodes(struct rds_ib_mr_pool *pool,
- struct list_head *list,
+static void list_to_llist_nodes(struct list_head *list,
struct llist_node **nodes_head,
struct llist_node **nodes_tail)
{
*/
dirty_to_clean = llist_append_to_list(&pool->drop_list, &unmap_list);
dirty_to_clean += llist_append_to_list(&pool->free_list, &unmap_list);
- if (free_all)
+ if (free_all) {
+ unsigned long flags;
+
+ spin_lock_irqsave(&pool->clean_lock, flags);
llist_append_to_list(&pool->clean_list, &unmap_list);
+ spin_unlock_irqrestore(&pool->clean_lock, flags);
+ }
free_goal = rds_ib_flush_goal(pool, free_all);
rds_ib_unreg_fmr(&unmap_list, &nfreed, &unpinned, free_goal);
if (!list_empty(&unmap_list)) {
- /* we have to make sure that none of the things we're about
- * to put on the clean list would race with other cpus trying
- * to pull items off. The llist would explode if we managed to
- * remove something from the clean list and then add it back again
- * while another CPU was spinning on that same item in llist_del_first.
- *
- * This is pretty unlikely, but just in case wait for an llist grace period
- * here before adding anything back into the clean list.
- */
- wait_clean_list_grace();
-
- list_to_llist_nodes(pool, &unmap_list, &clean_nodes, &clean_tail);
+ unsigned long flags;
+
+ list_to_llist_nodes(&unmap_list, &clean_nodes, &clean_tail);
if (ibmr_ret) {
*ibmr_ret = llist_entry(clean_nodes, struct rds_ib_mr, llnode);
clean_nodes = clean_nodes->next;
}
/* more than one entry in llist nodes */
- if (clean_nodes)
+ if (clean_nodes) {
+ spin_lock_irqsave(&pool->clean_lock, flags);
llist_add_batch(clean_nodes, clean_tail,
&pool->clean_list);
-
+ spin_unlock_irqrestore(&pool->clean_lock, flags);
+ }
}
atomic_sub(unpinned, &pool->free_pinned);
rds_ib_stats_inc(s_ib_rdma_mr_8k_pool_depleted);
else
rds_ib_stats_inc(s_ib_rdma_mr_1m_pool_depleted);
- return ERR_PTR(-EAGAIN);
+ break;
}
/* We do have some empty MRs. Flush them out. */
return ibmr;
}
- return ibmr;
+ return NULL;
}
static void rds_ib_mr_pool_flush_worker(struct work_struct *work)
init_llist_head(&pool->free_list);
init_llist_head(&pool->drop_list);
init_llist_head(&pool->clean_list);
+ spin_lock_init(&pool->clean_lock);
mutex_init(&pool->flush_lock);
init_waitqueue_head(&pool->flush_wait);
INIT_DELAYED_WORK(&pool->flush_worker, rds_ib_mr_pool_flush_worker);
#include "ar-internal.h"
static struct ctl_table_header *rxrpc_sysctl_reg_table;
-static const unsigned int one = 1;
static const unsigned int four = 4;
static const unsigned int thirtytwo = 32;
static const unsigned int n_65535 = 65535;
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&one,
+ .extra1 = (void *)SYSCTL_ONE,
.extra2 = (void *)&rxrpc_max_client_connections,
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&one,
+ .extra1 = (void *)SYSCTL_ONE,
.extra2 = (void *)&n_max_acks,
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&one,
+ .extra1 = (void *)SYSCTL_ONE,
.extra2 = (void *)&n_65535,
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *)&one,
+ .extra1 = (void *)SYSCTL_ONE,
.extra2 = (void *)&four,
},
config NET_ACT_CT
tristate "connection tracking tc action"
- depends on NET_CLS_ACT && NF_CONNTRACK
+ depends on NET_CLS_ACT && NF_CONNTRACK && NF_NAT
help
Say Y here to allow sending the packets to conntrack module.
tfilter_notify(net, skb, n, tp, block, q, parent, fh,
RTM_NEWTFILTER, false, rtnl_held);
tfilter_put(tp, fh);
+ q->flags &= ~TCQ_F_CAN_BYPASS;
}
errout:
static unsigned long fq_codel_bind(struct Qdisc *sch, unsigned long parent,
u32 classid)
{
- /* we cannot bypass queue discipline anymore */
- sch->flags &= ~TCQ_F_CAN_BYPASS;
return 0;
}
static unsigned long sfq_bind(struct Qdisc *sch, unsigned long parent,
u32 classid)
{
- /* we cannot bypass queue discipline anymore */
- sch->flags &= ~TCQ_F_CAN_BYPASS;
return 0;
}
struct sched_gate_list __rcu *admin_sched;
struct hrtimer advance_timer;
struct list_head taprio_list;
- int txtime_delay;
+ u32 txtime_delay;
};
static ktime_t sched_base_time(const struct sched_gate_list *sched)
goto unlock;
}
- q->txtime_delay = nla_get_s32(tb[TCA_TAPRIO_ATTR_TXTIME_DELAY]);
+ q->txtime_delay = nla_get_u32(tb[TCA_TAPRIO_ATTR_TXTIME_DELAY]);
}
if (!TXTIME_ASSIST_IS_ENABLED(taprio_flags) &&
goto options_error;
if (q->txtime_delay &&
- nla_put_s32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
+ nla_put_u32(skb, TCA_TAPRIO_ATTR_TXTIME_DELAY, q->txtime_delay))
goto options_error;
if (oper && dump_schedule(skb, oper))
case SCTP_PARAM_STATE_COOKIE:
asoc->peer.cookie_len =
ntohs(param.p->length) - sizeof(struct sctp_paramhdr);
- if (asoc->peer.cookie)
- kfree(asoc->peer.cookie);
+ kfree(asoc->peer.cookie);
asoc->peer.cookie = kmemdup(param.cookie->body, asoc->peer.cookie_len, gfp);
if (!asoc->peer.cookie)
retval = 0;
goto fall_through;
/* Save peer's random parameter */
- if (asoc->peer.peer_random)
- kfree(asoc->peer.peer_random);
+ kfree(asoc->peer.peer_random);
asoc->peer.peer_random = kmemdup(param.p,
ntohs(param.p->length), gfp);
if (!asoc->peer.peer_random) {
goto fall_through;
/* Save peer's HMAC list */
- if (asoc->peer.peer_hmacs)
- kfree(asoc->peer.peer_hmacs);
+ kfree(asoc->peer.peer_hmacs);
asoc->peer.peer_hmacs = kmemdup(param.p,
ntohs(param.p->length), gfp);
if (!asoc->peer.peer_hmacs) {
if (!ep->auth_enable)
goto fall_through;
- if (asoc->peer.peer_chunks)
- kfree(asoc->peer.peer_chunks);
+ kfree(asoc->peer.peer_chunks);
asoc->peer.peer_chunks = kmemdup(param.p,
ntohs(param.p->length), gfp);
if (!asoc->peer.peer_chunks)
#include <net/sctp/sctp.h>
#include <linux/sysctl.h>
-static int zero = 0;
-static int one = 1;
static int timer_max = 86400000; /* ms in one day */
-static int int_max = INT_MAX;
static int sack_timer_min = 1;
static int sack_timer_max = 500;
static int addr_scope_max = SCTP_SCOPE_POLICY_MAX;
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &timer_max
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_sctp_do_rto_min,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &init_net.sctp.rto_max
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "cookie_preserve_enable",
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &timer_max
},
{
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &timer_max
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &int_max
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "path_max_retrans",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &int_max
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "max_init_retransmits",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
- .extra2 = &int_max
+ .extra1 = SYSCTL_ONE,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "pf_retrans",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &int_max
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_INT_MAX,
},
{
.procname = "sndbuf_policy",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &addr_scope_max,
},
{
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = &proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
.extra2 = &rwnd_scale_max,
},
{
#include <linux/module.h>
#include <linux/highmem.h>
#include <linux/mount.h>
+#include <linux/pseudo_fs.h>
#include <linux/security.h>
#include <linux/syscalls.h>
#include <linux/compat.h>
NULL
};
-static struct dentry *sockfs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int sockfs_init_fs_context(struct fs_context *fc)
{
- return mount_pseudo_xattr(fs_type, "socket:", &sockfs_ops,
- sockfs_xattr_handlers,
- &sockfs_dentry_operations, SOCKFS_MAGIC);
+ struct pseudo_fs_context *ctx = init_pseudo(fc, SOCKFS_MAGIC);
+ if (!ctx)
+ return -ENOMEM;
+ ctx->ops = &sockfs_ops;
+ ctx->dops = &sockfs_dentry_operations;
+ ctx->xattr = sockfs_xattr_handlers;
+ return 0;
}
static struct vfsmount *sock_mnt __read_mostly;
static struct file_system_type sock_fs_type = {
.name = "sockfs",
- .mount = sockfs_mount,
+ .init_fs_context = sockfs_init_fs_context,
.kill_sb = kill_anon_super,
};
If unsure, say Y.
-config CONFIG_SUNRPC_DISABLE_INSECURE_ENCTYPES
+config SUNRPC_DISABLE_INSECURE_ENCTYPES
bool "Secure RPC: Disable insecure Kerberos encryption types"
depends on RPCSEC_GSS_KRB5
default n
#define RPCDBG_FACILITY RPCDBG_TRANS
#endif
+#define BC_MAX_SLOTS 64U
+
+unsigned int xprt_bc_max_slots(struct rpc_xprt *xprt)
+{
+ return BC_MAX_SLOTS;
+}
+
/*
* Helper routines that track the number of preallocation elements
* on the transport.
*/
static inline int xprt_need_to_requeue(struct rpc_xprt *xprt)
{
- return xprt->bc_alloc_count < atomic_read(&xprt->bc_free_slots);
-}
-
-static inline void xprt_inc_alloc_count(struct rpc_xprt *xprt, unsigned int n)
-{
- atomic_add(n, &xprt->bc_free_slots);
- xprt->bc_alloc_count += n;
-}
-
-static inline int xprt_dec_alloc_count(struct rpc_xprt *xprt, unsigned int n)
-{
- atomic_sub(n, &xprt->bc_free_slots);
- return xprt->bc_alloc_count -= n;
+ return xprt->bc_alloc_count < xprt->bc_alloc_max;
}
/*
dprintk("RPC: setup backchannel transport\n");
+ if (min_reqs > BC_MAX_SLOTS)
+ min_reqs = BC_MAX_SLOTS;
+
/*
* We use a temporary list to keep track of the preallocated
* buffers. Once we're done building the list we splice it
*/
spin_lock(&xprt->bc_pa_lock);
list_splice(&tmp_list, &xprt->bc_pa_list);
- xprt_inc_alloc_count(xprt, min_reqs);
+ xprt->bc_alloc_count += min_reqs;
+ xprt->bc_alloc_max += min_reqs;
+ atomic_add(min_reqs, &xprt->bc_slot_count);
spin_unlock(&xprt->bc_pa_lock);
dprintk("RPC: setup backchannel transport done\n");
goto out;
spin_lock_bh(&xprt->bc_pa_lock);
- xprt_dec_alloc_count(xprt, max_reqs);
+ xprt->bc_alloc_max -= max_reqs;
list_for_each_entry_safe(req, tmp, &xprt->bc_pa_list, rq_bc_pa_list) {
dprintk("RPC: req=%p\n", req);
list_del(&req->rq_bc_pa_list);
xprt_free_allocation(req);
+ xprt->bc_alloc_count--;
+ atomic_dec(&xprt->bc_slot_count);
if (--max_reqs == 0)
break;
}
struct rpc_rqst *req = NULL;
dprintk("RPC: allocate a backchannel request\n");
- if (atomic_read(&xprt->bc_free_slots) <= 0)
- goto not_found;
if (list_empty(&xprt->bc_pa_list)) {
if (!new)
goto not_found;
+ if (atomic_read(&xprt->bc_slot_count) >= BC_MAX_SLOTS)
+ goto not_found;
list_add_tail(&new->rq_bc_pa_list, &xprt->bc_pa_list);
xprt->bc_alloc_count++;
+ atomic_inc(&xprt->bc_slot_count);
}
req = list_first_entry(&xprt->bc_pa_list, struct rpc_rqst,
rq_bc_pa_list);
if (xprt_need_to_requeue(xprt)) {
list_add_tail(&req->rq_bc_pa_list, &xprt->bc_pa_list);
xprt->bc_alloc_count++;
+ atomic_inc(&xprt->bc_slot_count);
req = NULL;
}
spin_unlock_bh(&xprt->bc_pa_lock);
spin_lock(&xprt->bc_pa_lock);
list_del(&req->rq_bc_pa_list);
- xprt_dec_alloc_count(xprt, 1);
+ xprt->bc_alloc_count--;
spin_unlock(&xprt->bc_pa_lock);
req->rq_private_buf.len = copied;
.bc_xprt = args->bc_xprt,
};
char servername[48];
+ struct rpc_clnt *clnt;
+ int i;
if (args->bc_xprt) {
WARN_ON_ONCE(!(args->protocol & XPRT_TRANSPORT_BC));
if (args->flags & RPC_CLNT_CREATE_NONPRIVPORT)
xprt->resvport = 0;
- return rpc_create_xprt(args, xprt);
+ clnt = rpc_create_xprt(args, xprt);
+ if (IS_ERR(clnt) || args->nconnect <= 1)
+ return clnt;
+
+ for (i = 0; i < args->nconnect - 1; i++) {
+ if (rpc_clnt_add_xprt(clnt, &xprtargs, NULL, NULL) < 0)
+ break;
+ }
+ return clnt;
}
EXPORT_SYMBOL_GPL(rpc_create);
}
EXPORT_SYMBOL_GPL(rpc_bind_new_program);
+struct rpc_xprt *
+rpc_task_get_xprt(struct rpc_clnt *clnt, struct rpc_xprt *xprt)
+{
+ struct rpc_xprt_switch *xps;
+
+ if (!xprt)
+ return NULL;
+ rcu_read_lock();
+ xps = rcu_dereference(clnt->cl_xpi.xpi_xpswitch);
+ atomic_long_inc(&xps->xps_queuelen);
+ rcu_read_unlock();
+ atomic_long_inc(&xprt->queuelen);
+
+ return xprt;
+}
+
+static void
+rpc_task_release_xprt(struct rpc_clnt *clnt, struct rpc_xprt *xprt)
+{
+ struct rpc_xprt_switch *xps;
+
+ atomic_long_dec(&xprt->queuelen);
+ rcu_read_lock();
+ xps = rcu_dereference(clnt->cl_xpi.xpi_xpswitch);
+ atomic_long_dec(&xps->xps_queuelen);
+ rcu_read_unlock();
+
+ xprt_put(xprt);
+}
+
void rpc_task_release_transport(struct rpc_task *task)
{
struct rpc_xprt *xprt = task->tk_xprt;
if (xprt) {
task->tk_xprt = NULL;
- xprt_put(xprt);
+ if (task->tk_client)
+ rpc_task_release_xprt(task->tk_client, xprt);
+ else
+ xprt_put(xprt);
}
}
EXPORT_SYMBOL_GPL(rpc_task_release_transport);
{
struct rpc_clnt *clnt = task->tk_client;
+ rpc_task_release_transport(task);
if (clnt != NULL) {
/* Remove from client task list */
spin_lock(&clnt->cl_lock);
rpc_release_client(clnt);
}
- rpc_task_release_transport(task);
+}
+
+static struct rpc_xprt *
+rpc_task_get_first_xprt(struct rpc_clnt *clnt)
+{
+ struct rpc_xprt *xprt;
+
+ rcu_read_lock();
+ xprt = xprt_get(rcu_dereference(clnt->cl_xprt));
+ rcu_read_unlock();
+ return rpc_task_get_xprt(clnt, xprt);
+}
+
+static struct rpc_xprt *
+rpc_task_get_next_xprt(struct rpc_clnt *clnt)
+{
+ return rpc_task_get_xprt(clnt, xprt_iter_get_next(&clnt->cl_xpi));
}
static
void rpc_task_set_transport(struct rpc_task *task, struct rpc_clnt *clnt)
{
- if (!task->tk_xprt)
- task->tk_xprt = xprt_iter_get_next(&clnt->cl_xpi);
+ if (task->tk_xprt)
+ return;
+ if (task->tk_flags & RPC_TASK_NO_ROUND_ROBIN)
+ task->tk_xprt = rpc_task_get_first_xprt(clnt);
+ else
+ task->tk_xprt = rpc_task_get_next_xprt(clnt);
}
static
}
EXPORT_SYMBOL_GPL(rpc_max_bc_payload);
+unsigned int rpc_num_bc_slots(struct rpc_clnt *clnt)
+{
+ struct rpc_xprt *xprt;
+ unsigned int ret;
+
+ rcu_read_lock();
+ xprt = rcu_dereference(clnt->cl_xprt);
+ ret = xprt->ops->bc_num_slots(xprt);
+ rcu_read_unlock();
+ return ret;
+}
+EXPORT_SYMBOL_GPL(rpc_num_bc_slots);
+
/**
* rpc_force_rebind - force transport to check that remote port is unchanged
* @clnt: client to rebind
req->rq_snd_buf.head[0].iov_len = 0;
xdr_init_encode(&xdr, &req->rq_snd_buf,
req->rq_snd_buf.head[0].iov_base, req);
+ xdr_free_bvec(&req->rq_snd_buf);
if (rpc_encode_header(task, &xdr))
return;
rpc_call_rpcerror(task, task->tk_status);
}
return;
- } else {
- xprt_request_prepare(task->tk_rqstp);
}
/* Add task to reply queue before transmission to avoid races */
return -ENOMEM;
data->xps = xprt_switch_get(xps);
data->xprt = xprt_get(xprt);
+ if (rpc_xprt_switch_has_addr(data->xps, (struct sockaddr *)&xprt->addr)) {
+ rpc_cb_add_xprt_release(data);
+ goto success;
+ }
task = rpc_call_null_helper(clnt, xprt, NULL,
RPC_TASK_SOFT|RPC_TASK_SOFTCONN|RPC_TASK_ASYNC|RPC_TASK_NULLCREDS,
if (IS_ERR(task))
return PTR_ERR(task);
rpc_put_task(task);
+success:
return 1;
}
EXPORT_SYMBOL_GPL(rpc_clnt_test_and_add_xprt);
// SPDX-License-Identifier: GPL-2.0
-/**
+/*
* debugfs interface for sunrpc
*
* (c) 2014 Jeff Layton <jlayton@primarydata.com>
.release = tasks_release,
};
+static int do_xprt_debugfs(struct rpc_clnt *clnt, struct rpc_xprt *xprt, void *numv)
+{
+ int len;
+ char name[24]; /* enough for "../../rpc_xprt/ + 8 hex digits + NULL */
+ char link[9]; /* enough for 8 hex digits + NULL */
+ int *nump = numv;
+
+ if (IS_ERR_OR_NULL(xprt->debugfs))
+ return 0;
+ len = snprintf(name, sizeof(name), "../../rpc_xprt/%s",
+ xprt->debugfs->d_name.name);
+ if (len > sizeof(name))
+ return -1;
+ if (*nump == 0)
+ strcpy(link, "xprt");
+ else {
+ len = snprintf(link, sizeof(link), "xprt%d", *nump);
+ if (len > sizeof(link))
+ return -1;
+ }
+ debugfs_create_symlink(link, clnt->cl_debugfs, name);
+ (*nump)++;
+ return 0;
+}
+
void
rpc_clnt_debugfs_register(struct rpc_clnt *clnt)
{
int len;
- char name[24]; /* enough for "../../rpc_xprt/ + 8 hex digits + NULL */
- struct rpc_xprt *xprt;
+ char name[9]; /* enough for 8 hex digits + NULL */
+ int xprtnum = 0;
len = snprintf(name, sizeof(name), "%x", clnt->cl_clid);
if (len >= sizeof(name))
debugfs_create_file("tasks", S_IFREG | 0400, clnt->cl_debugfs, clnt,
&tasks_fops);
- rcu_read_lock();
- xprt = rcu_dereference(clnt->cl_xprt);
- /* no "debugfs" dentry? Don't bother with the symlink. */
- if (IS_ERR_OR_NULL(xprt->debugfs)) {
- rcu_read_unlock();
- return;
- }
- len = snprintf(name, sizeof(name), "../../rpc_xprt/%s",
- xprt->debugfs->d_name.name);
- rcu_read_unlock();
-
- if (len >= sizeof(name))
- goto out_err;
-
- debugfs_create_symlink("xprt", clnt->cl_debugfs, name);
-
- return;
-out_err:
- debugfs_remove_recursive(clnt->cl_debugfs);
- clnt->cl_debugfs = NULL;
+ rpc_clnt_iterate_for_each_xprt(clnt, do_xprt_debugfs, &xprtnum);
}
void
#include <linux/string.h>
#include <linux/pagemap.h>
#include <linux/mount.h>
+#include <linux/fs_context.h>
#include <linux/namei.h>
#include <linux/fsnotify.h>
#include <linux/kernel.h>
}
static int
-rpc_fill_super(struct super_block *sb, void *data, int silent)
+rpc_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct inode *inode;
struct dentry *root, *gssd_dentry;
- struct net *net = get_net(sb->s_fs_info);
+ struct net *net = sb->s_fs_info;
struct sunrpc_net *sn = net_generic(net, sunrpc_net_id);
int err;
}
EXPORT_SYMBOL_GPL(gssd_running);
-static struct dentry *
-rpc_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int rpc_fs_get_tree(struct fs_context *fc)
+{
+ fc->s_fs_info = get_net(fc->net_ns);
+ return vfs_get_super(fc, vfs_get_keyed_super, rpc_fill_super);
+}
+
+static void rpc_fs_free_fc(struct fs_context *fc)
{
- struct net *net = current->nsproxy->net_ns;
- return mount_ns(fs_type, flags, data, net, net->user_ns, rpc_fill_super);
+ if (fc->s_fs_info)
+ put_net(fc->s_fs_info);
+}
+
+static const struct fs_context_operations rpc_fs_context_ops = {
+ .free = rpc_fs_free_fc,
+ .get_tree = rpc_fs_get_tree,
+};
+
+static int rpc_init_fs_context(struct fs_context *fc)
+{
+ put_user_ns(fc->user_ns);
+ fc->user_ns = get_user_ns(fc->net_ns->user_ns);
+ fc->ops = &rpc_fs_context_ops;
+ return 0;
}
static void rpc_kill_sb(struct super_block *sb)
static struct file_system_type rpc_pipe_fs_type = {
.owner = THIS_MODULE,
.name = "rpc_pipefs",
- .mount = rpc_mount,
+ .init_fs_context = rpc_init_fs_context,
.kill_sb = rpc_kill_sb,
};
MODULE_ALIAS_FS("rpc_pipefs");
#include <linux/sched/mm.h>
#include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/metrics.h>
#include "sunrpc.h"
static void rpc_async_schedule(struct work_struct *);
static void rpc_release_task(struct rpc_task *task);
-static void __rpc_queue_timer_fn(struct timer_list *t);
+static void __rpc_queue_timer_fn(struct work_struct *);
/*
* RPC tasks sit here while waiting for conditions to improve.
*/
struct workqueue_struct *rpciod_workqueue __read_mostly;
struct workqueue_struct *xprtiod_workqueue __read_mostly;
+EXPORT_SYMBOL_GPL(xprtiod_workqueue);
unsigned long
rpc_task_timeout(const struct rpc_task *task)
task->tk_timeout = 0;
list_del(&task->u.tk_wait.timer_list);
if (list_empty(&queue->timer_list.list))
- del_timer(&queue->timer_list.timer);
+ cancel_delayed_work(&queue->timer_list.dwork);
}
static void
rpc_set_queue_timer(struct rpc_wait_queue *queue, unsigned long expires)
{
- timer_reduce(&queue->timer_list.timer, expires);
+ unsigned long now = jiffies;
+ queue->timer_list.expires = expires;
+ if (time_before_eq(expires, now))
+ expires = 0;
+ else
+ expires -= now;
+ mod_delayed_work(rpciod_workqueue, &queue->timer_list.dwork, expires);
}
/*
task->tk_pid, jiffies_to_msecs(timeout - jiffies));
task->tk_timeout = timeout;
- rpc_set_queue_timer(queue, timeout);
+ if (list_empty(&queue->timer_list.list) || time_before(timeout, queue->timer_list.expires))
+ rpc_set_queue_timer(queue, timeout);
list_add(&task->u.tk_wait.timer_list, &queue->timer_list.list);
}
queue->maxpriority = nr_queues - 1;
rpc_reset_waitqueue_priority(queue);
queue->qlen = 0;
- timer_setup(&queue->timer_list.timer, __rpc_queue_timer_fn, 0);
+ queue->timer_list.expires = 0;
+ INIT_DEFERRABLE_WORK(&queue->timer_list.dwork, __rpc_queue_timer_fn);
INIT_LIST_HEAD(&queue->timer_list.list);
rpc_assign_waitqueue_name(queue, qname);
}
void rpc_destroy_wait_queue(struct rpc_wait_queue *queue)
{
- del_timer_sync(&queue->timer_list.timer);
+ cancel_delayed_work_sync(&queue->timer_list.dwork);
}
EXPORT_SYMBOL_GPL(rpc_destroy_wait_queue);
/*
* Protect the queue operations.
*/
- spin_lock_bh(&q->lock);
+ spin_lock(&q->lock);
__rpc_sleep_on_priority_timeout(q, task, timeout, task->tk_priority);
- spin_unlock_bh(&q->lock);
+ spin_unlock(&q->lock);
}
EXPORT_SYMBOL_GPL(rpc_sleep_on_timeout);
/*
* Protect the queue operations.
*/
- spin_lock_bh(&q->lock);
+ spin_lock(&q->lock);
__rpc_sleep_on_priority(q, task, task->tk_priority);
- spin_unlock_bh(&q->lock);
+ spin_unlock(&q->lock);
}
EXPORT_SYMBOL_GPL(rpc_sleep_on);
/*
* Protect the queue operations.
*/
- spin_lock_bh(&q->lock);
+ spin_lock(&q->lock);
__rpc_sleep_on_priority_timeout(q, task, timeout, priority);
- spin_unlock_bh(&q->lock);
+ spin_unlock(&q->lock);
}
EXPORT_SYMBOL_GPL(rpc_sleep_on_priority_timeout);
/*
* Protect the queue operations.
*/
- spin_lock_bh(&q->lock);
+ spin_lock(&q->lock);
__rpc_sleep_on_priority(q, task, priority);
- spin_unlock_bh(&q->lock);
+ spin_unlock(&q->lock);
}
EXPORT_SYMBOL_GPL(rpc_sleep_on_priority);
{
if (!RPC_IS_QUEUED(task))
return;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
rpc_wake_up_task_on_wq_queue_locked(wq, queue, task);
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
}
/*
{
if (!RPC_IS_QUEUED(task))
return;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
rpc_wake_up_task_queue_locked(queue, task);
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
}
EXPORT_SYMBOL_GPL(rpc_wake_up_queued_task);
{
if (!RPC_IS_QUEUED(task))
return;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
rpc_wake_up_task_queue_set_status_locked(queue, task, status);
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
}
/*
dprintk("RPC: wake_up_first(%p \"%s\")\n",
queue, rpc_qname(queue));
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
task = __rpc_find_next_queued(queue);
if (task != NULL)
task = rpc_wake_up_task_on_wq_queue_action_locked(wq, queue,
task, func, data);
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
return task;
}
{
struct list_head *head;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
head = &queue->tasks[queue->maxpriority];
for (;;) {
while (!list_empty(head)) {
break;
head--;
}
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
}
EXPORT_SYMBOL_GPL(rpc_wake_up);
{
struct list_head *head;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
head = &queue->tasks[queue->maxpriority];
for (;;) {
while (!list_empty(head)) {
break;
head--;
}
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
}
EXPORT_SYMBOL_GPL(rpc_wake_up_status);
-static void __rpc_queue_timer_fn(struct timer_list *t)
+static void __rpc_queue_timer_fn(struct work_struct *work)
{
- struct rpc_wait_queue *queue = from_timer(queue, t, timer_list.timer);
+ struct rpc_wait_queue *queue = container_of(work,
+ struct rpc_wait_queue,
+ timer_list.dwork.work);
struct rpc_task *task, *n;
unsigned long expires, now, timeo;
void rpc_exit_task(struct rpc_task *task)
{
task->tk_action = NULL;
+ if (task->tk_ops->rpc_count_stats)
+ task->tk_ops->rpc_count_stats(task, task->tk_calldata);
+ else if (task->tk_client)
+ rpc_count_iostats(task, task->tk_client->cl_metrics);
if (task->tk_ops->rpc_call_done != NULL) {
task->tk_ops->rpc_call_done(task, task->tk_calldata);
if (task->tk_action != NULL) {
* rpc_task pointer may still be dereferenced.
*/
queue = task->tk_waitqueue;
- spin_lock_bh(&queue->lock);
+ spin_lock(&queue->lock);
if (!RPC_IS_QUEUED(task)) {
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
continue;
}
rpc_clear_running(task);
- spin_unlock_bh(&queue->lock);
+ spin_unlock(&queue->lock);
if (task_is_async)
return;
/* Initialize workqueue for async tasks */
task->tk_workqueue = task_setup_data->workqueue;
- task->tk_xprt = xprt_get(task_setup_data->rpc_xprt);
+ task->tk_xprt = rpc_task_get_xprt(task_setup_data->rpc_client,
+ xprt_get(task_setup_data->rpc_xprt));
task->tk_op_cred = get_rpccred(task_setup_data->rpc_op_cred);
execute = ktime_sub(now, task->tk_start);
op_metrics->om_execute = ktime_add(op_metrics->om_execute, execute);
+ if (task->tk_status < 0)
+ op_metrics->om_error_status++;
spin_unlock(&op_metrics->om_lock);
a->om_queue = ktime_add(a->om_queue, b->om_queue);
a->om_rtt = ktime_add(a->om_rtt, b->om_rtt);
a->om_execute = ktime_add(a->om_execute, b->om_execute);
+ a->om_error_status += b->om_error_status;
}
static void _print_rpc_iostats(struct seq_file *seq, struct rpc_iostats *stats,
int op, const struct rpc_procinfo *procs)
{
_print_name(seq, op, procs);
- seq_printf(seq, "%lu %lu %lu %Lu %Lu %Lu %Lu %Lu\n",
+ seq_printf(seq, "%lu %lu %lu %llu %llu %llu %llu %llu %lu\n",
stats->om_ops,
stats->om_ntrans,
stats->om_timeouts,
stats->om_bytes_recv,
ktime_to_ms(stats->om_queue),
ktime_to_ms(stats->om_rtt),
- ktime_to_ms(stats->om_execute));
+ ktime_to_ms(stats->om_execute),
+ stats->om_error_status);
+}
+
+static int do_print_stats(struct rpc_clnt *clnt, struct rpc_xprt *xprt, void *seqv)
+{
+ struct seq_file *seq = seqv;
+
+ xprt->ops->print_stats(xprt, seq);
+ return 0;
}
void rpc_clnt_show_stats(struct seq_file *seq, struct rpc_clnt *clnt)
{
- struct rpc_xprt *xprt;
unsigned int op, maxproc = clnt->cl_maxproc;
if (!clnt->cl_metrics)
seq_printf(seq, "p/v: %u/%u (%s)\n",
clnt->cl_prog, clnt->cl_vers, clnt->cl_program->name);
- rcu_read_lock();
- xprt = rcu_dereference(clnt->cl_xprt);
- if (xprt)
- xprt->ops->print_stats(xprt, seq);
- rcu_read_unlock();
+ rpc_clnt_iterate_for_each_xprt(clnt, do_print_stats, seq);
seq_printf(seq, "\tper-op statistics\n");
for (op = 0; op < maxproc; op++) {
/* Parse and execute the bc call */
proc_error = svc_process_common(rqstp, argv, resv);
- atomic_inc(&req->rq_xprt->bc_free_slots);
+ atomic_dec(&req->rq_xprt->bc_slot_count);
if (!proc_error) {
/* Processing error: drop the request */
xprt_free_bc_request(req);
if (test_bit(XPRT_LOCKED, &xprt->state) && xprt->snd_task == task)
return 1;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
retval = xprt->ops->reserve_xprt(xprt, task);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
return retval;
}
{
if (xprt->snd_task != task)
return;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt->ops->release_xprt(xprt, task);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
/*
if (req->rq_cong)
return true;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
ret = __xprt_get_cong(xprt, req) != 0;
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
return ret;
}
EXPORT_SYMBOL_GPL(xprt_request_get_cong);
xprt_clear_congestion_window_wait(struct rpc_xprt *xprt)
{
if (test_and_clear_bit(XPRT_CWND_WAIT, &xprt->state)) {
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
__xprt_lock_write_next_cong(xprt);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
}
if (!test_bit(XPRT_WRITE_SPACE, &xprt->state))
return false;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
ret = xprt_clear_write_space_locked(xprt);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
return ret;
}
EXPORT_SYMBOL_GPL(xprt_write_space);
req->rq_retries = 0;
xprt_reset_majortimeo(req);
/* Reset the RTT counters == "slow start" */
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
rpc_init_rtt(req->rq_task->tk_client->cl_rtt, to->to_initval);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
status = -ETIMEDOUT;
}
void xprt_disconnect_done(struct rpc_xprt *xprt)
{
dprintk("RPC: disconnected transport %p\n", xprt);
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt_clear_connected(xprt);
xprt_clear_write_space_locked(xprt);
xprt_wake_pending_tasks(xprt, -ENOTCONN);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
EXPORT_SYMBOL_GPL(xprt_disconnect_done);
void xprt_force_disconnect(struct rpc_xprt *xprt)
{
/* Don't race with the test_bit() in xprt_clear_locked() */
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
set_bit(XPRT_CLOSE_WAIT, &xprt->state);
/* Try to schedule an autoclose RPC call */
if (test_and_set_bit(XPRT_LOCKED, &xprt->state) == 0)
else if (xprt->snd_task)
rpc_wake_up_queued_task_set_status(&xprt->pending,
xprt->snd_task, -ENOTCONN);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
EXPORT_SYMBOL_GPL(xprt_force_disconnect);
void xprt_conditional_disconnect(struct rpc_xprt *xprt, unsigned int cookie)
{
/* Don't race with the test_bit() in xprt_clear_locked() */
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
if (cookie != xprt->connect_cookie)
goto out;
if (test_bit(XPRT_CLOSING, &xprt->state))
queue_work(xprtiod_workqueue, &xprt->task_cleanup);
xprt_wake_pending_tasks(xprt, -EAGAIN);
out:
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
static bool
xprt_schedule_autodisconnect(struct rpc_xprt *xprt)
__must_hold(&xprt->transport_lock)
{
+ xprt->last_used = jiffies;
if (RB_EMPTY_ROOT(&xprt->recv_queue) && xprt_has_timer(xprt))
mod_timer(&xprt->timer, xprt->last_used + xprt->idle_timeout);
}
{
struct rpc_xprt *xprt = from_timer(xprt, t, timer);
- spin_lock(&xprt->transport_lock);
if (!RB_EMPTY_ROOT(&xprt->recv_queue))
- goto out_abort;
+ return;
/* Reset xprt->last_used to avoid connect/autodisconnect cycling */
xprt->last_used = jiffies;
if (test_and_set_bit(XPRT_LOCKED, &xprt->state))
- goto out_abort;
- spin_unlock(&xprt->transport_lock);
+ return;
queue_work(xprtiod_workqueue, &xprt->task_cleanup);
- return;
-out_abort:
- spin_unlock(&xprt->transport_lock);
}
bool xprt_lock_connect(struct rpc_xprt *xprt,
{
bool ret = false;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
if (!test_bit(XPRT_LOCKED, &xprt->state))
goto out;
if (xprt->snd_task != task)
xprt->snd_task = cookie;
ret = true;
out:
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
return ret;
}
void xprt_unlock_connect(struct rpc_xprt *xprt, void *cookie)
{
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
if (xprt->snd_task != cookie)
goto out;
if (!test_bit(XPRT_LOCKED, &xprt->state))
xprt->ops->release_xprt(xprt, NULL);
xprt_schedule_autodisconnect(xprt);
out:
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
wake_up_bit(&xprt->state, XPRT_LOCKED);
}
xprt_release_write(xprt, task);
}
+/**
+ * xprt_reconnect_delay - compute the wait before scheduling a connect
+ * @xprt: transport instance
+ *
+ */
+unsigned long xprt_reconnect_delay(const struct rpc_xprt *xprt)
+{
+ unsigned long start, now = jiffies;
+
+ start = xprt->stat.connect_start + xprt->reestablish_timeout;
+ if (time_after(start, now))
+ return start - now;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xprt_reconnect_delay);
+
+/**
+ * xprt_reconnect_backoff - compute the new re-establish timeout
+ * @xprt: transport instance
+ * @init_to: initial reestablish timeout
+ *
+ */
+void xprt_reconnect_backoff(struct rpc_xprt *xprt, unsigned long init_to)
+{
+ xprt->reestablish_timeout <<= 1;
+ if (xprt->reestablish_timeout > xprt->max_reconnect_timeout)
+ xprt->reestablish_timeout = xprt->max_reconnect_timeout;
+ if (xprt->reestablish_timeout < init_to)
+ xprt->reestablish_timeout = init_to;
+}
+EXPORT_SYMBOL_GPL(xprt_reconnect_backoff);
+
enum xprt_xid_rb_cmp {
XID_RB_EQUAL,
XID_RB_LEFT,
if (!xprt_request_need_enqueue_receive(task, req))
return;
+
+ xprt_request_prepare(task->tk_rqstp);
spin_lock(&xprt->queue_lock);
/* Update the softirq receive buffer */
xprt_inject_disconnect(xprt);
task->tk_flags |= RPC_TASK_SENT;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt->stat.sends++;
xprt->stat.req_u += xprt->stat.sends - xprt->stat.recvs;
xprt->stat.bklog_u += xprt->backlog.qlen;
xprt->stat.sending_u += xprt->sending.qlen;
xprt->stat.pending_u += xprt->pending.qlen;
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
req->rq_connect_cookie = connect_cookie;
out_dequeue:
}
xprt = req->rq_xprt;
- if (task->tk_ops->rpc_count_stats != NULL)
- task->tk_ops->rpc_count_stats(task, task->tk_calldata);
- else if (task->tk_client)
- rpc_count_iostats(task, task->tk_client->cl_metrics);
xprt_request_dequeue_all(task, req);
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt->ops->release_xprt(xprt, task);
if (xprt->ops->release_request)
xprt->ops->release_request(task);
- xprt->last_used = jiffies;
xprt_schedule_autodisconnect(xprt);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
if (req->rq_buffer)
xprt->ops->buf_free(task);
xprt_inject_disconnect(xprt);
#include <linux/sunrpc/addr.h>
#include <linux/sunrpc/xprtmultipath.h>
-typedef struct rpc_xprt *(*xprt_switch_find_xprt_t)(struct list_head *head,
+typedef struct rpc_xprt *(*xprt_switch_find_xprt_t)(struct rpc_xprt_switch *xps,
const struct rpc_xprt *cur);
static const struct rpc_xprt_iter_ops rpc_xprt_iter_singular;
if (xps->xps_nxprts == 0)
xps->xps_net = xprt->xprt_net;
xps->xps_nxprts++;
+ xps->xps_nactive++;
}
/**
if (xprt == NULL)
return;
spin_lock(&xps->xps_lock);
- if ((xps->xps_net == xprt->xprt_net || xps->xps_net == NULL) &&
- !rpc_xprt_switch_has_addr(xps, (struct sockaddr *)&xprt->addr))
+ if (xps->xps_net == xprt->xprt_net || xps->xps_net == NULL)
xprt_switch_add_xprt_locked(xps, xprt);
spin_unlock(&xps->xps_lock);
}
{
if (unlikely(xprt == NULL))
return;
+ xps->xps_nactive--;
xps->xps_nxprts--;
if (xps->xps_nxprts == 0)
xps->xps_net = NULL;
if (xps != NULL) {
spin_lock_init(&xps->xps_lock);
kref_init(&xps->xps_kref);
- xps->xps_nxprts = 0;
+ xps->xps_nxprts = xps->xps_nactive = 0;
+ atomic_long_set(&xps->xps_queuelen, 0);
+ xps->xps_net = NULL;
INIT_LIST_HEAD(&xps->xps_xprt_list);
xps->xps_iter_ops = &rpc_xprt_iter_singular;
xprt_switch_add_xprt_locked(xps, xprt);
}
static
+bool xprt_is_active(const struct rpc_xprt *xprt)
+{
+ return kref_read(&xprt->kref) != 0;
+}
+
+static
struct rpc_xprt *xprt_switch_find_first_entry(struct list_head *head)
{
- return list_first_or_null_rcu(head, struct rpc_xprt, xprt_switch);
+ struct rpc_xprt *pos;
+
+ list_for_each_entry_rcu(pos, head, xprt_switch) {
+ if (xprt_is_active(pos))
+ return pos;
+ }
+ return NULL;
}
static
const struct rpc_xprt *cur)
{
struct rpc_xprt *pos;
+ bool found = false;
list_for_each_entry_rcu(pos, head, xprt_switch) {
if (cur == pos)
+ found = true;
+ if (found && xprt_is_active(pos))
return pos;
}
return NULL;
const struct rpc_xprt *cur)
{
struct rpc_xprt *pos, *prev = NULL;
+ bool found = false;
list_for_each_entry_rcu(pos, head, xprt_switch) {
if (cur == prev)
+ found = true;
+ if (found && xprt_is_active(pos))
return pos;
prev = pos;
}
}
static
-struct rpc_xprt *xprt_switch_set_next_cursor(struct list_head *head,
+struct rpc_xprt *xprt_switch_set_next_cursor(struct rpc_xprt_switch *xps,
struct rpc_xprt **cursor,
xprt_switch_find_xprt_t find_next)
{
- struct rpc_xprt *cur, *pos, *old;
+ struct rpc_xprt *pos, *old;
- cur = READ_ONCE(*cursor);
- for (;;) {
- old = cur;
- pos = find_next(head, old);
- if (pos == NULL)
- break;
- cur = cmpxchg_relaxed(cursor, old, pos);
- if (cur == old)
- break;
- }
+ old = smp_load_acquire(cursor);
+ pos = find_next(xps, old);
+ smp_store_release(cursor, pos);
return pos;
}
if (xps == NULL)
return NULL;
- return xprt_switch_set_next_cursor(&xps->xps_xprt_list,
- &xpi->xpi_cursor,
- find_next);
+ return xprt_switch_set_next_cursor(xps, &xpi->xpi_cursor, find_next);
}
static
-struct rpc_xprt *xprt_switch_find_next_entry_roundrobin(struct list_head *head,
+struct rpc_xprt *__xprt_switch_find_next_entry_roundrobin(struct list_head *head,
const struct rpc_xprt *cur)
{
struct rpc_xprt *ret;
}
static
+struct rpc_xprt *xprt_switch_find_next_entry_roundrobin(struct rpc_xprt_switch *xps,
+ const struct rpc_xprt *cur)
+{
+ struct list_head *head = &xps->xps_xprt_list;
+ struct rpc_xprt *xprt;
+ unsigned int nactive;
+
+ for (;;) {
+ unsigned long xprt_queuelen, xps_queuelen;
+
+ xprt = __xprt_switch_find_next_entry_roundrobin(head, cur);
+ if (!xprt)
+ break;
+ xprt_queuelen = atomic_long_read(&xprt->queuelen);
+ xps_queuelen = atomic_long_read(&xps->xps_queuelen);
+ nactive = READ_ONCE(xps->xps_nactive);
+ /* Exit loop if xprt_queuelen <= average queue length */
+ if (xprt_queuelen * nactive <= xps_queuelen)
+ break;
+ cur = xprt;
+ }
+ return xprt;
+}
+
+static
struct rpc_xprt *xprt_iter_next_entry_roundrobin(struct rpc_xprt_iter *xpi)
{
return xprt_iter_next_entry_multiple(xpi,
}
static
+struct rpc_xprt *xprt_switch_find_next_entry_all(struct rpc_xprt_switch *xps,
+ const struct rpc_xprt *cur)
+{
+ return xprt_switch_find_next_entry(&xps->xps_xprt_list, cur);
+}
+
+static
struct rpc_xprt *xprt_iter_next_entry_all(struct rpc_xprt_iter *xpi)
{
- return xprt_iter_next_entry_multiple(xpi, xprt_switch_find_next_entry);
+ return xprt_iter_next_entry_multiple(xpi,
+ xprt_switch_find_next_entry_all);
}
/*
return maxmsg - RPCRDMA_HDRLEN_MIN;
}
+unsigned int xprt_rdma_bc_max_slots(struct rpc_xprt *xprt)
+{
+ struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
+
+ return r_xprt->rx_buf.rb_bc_srv_max_requests;
+}
+
static int rpcrdma_bc_marshal_reply(struct rpc_rqst *rqst)
{
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(rqst->rq_xprt);
frwr_release_mr(mr);
}
+/* frwr_reset - Place MRs back on the free list
+ * @req: request to reset
+ *
+ * Used after a failed marshal. For FRWR, this means the MRs
+ * don't have to be fully released and recreated.
+ *
+ * NB: This is safe only as long as none of @req's MRs are
+ * involved with an ongoing asynchronous FAST_REG or LOCAL_INV
+ * Work Request.
+ */
+void frwr_reset(struct rpcrdma_req *req)
+{
+ while (!list_empty(&req->rl_registered)) {
+ struct rpcrdma_mr *mr;
+
+ mr = rpcrdma_mr_pop(&req->rl_registered);
+ rpcrdma_mr_unmap_and_put(mr);
+ }
+}
+
/**
* frwr_init_mr - Initialize one MR
* @ia: interface adapter
goto out_list_err;
mr->frwr.fr_mr = frmr;
- mr->frwr.fr_state = FRWR_IS_INVALID;
mr->mr_dir = DMA_NONE;
INIT_LIST_HEAD(&mr->mr_list);
INIT_WORK(&mr->mr_recycle, frwr_mr_recycle_worker);
}
/**
- * frwr_wc_fastreg - Invoked by RDMA provider for a flushed FastReg WC
- * @cq: completion queue (ignored)
- * @wc: completed WR
- *
- */
-static void
-frwr_wc_fastreg(struct ib_cq *cq, struct ib_wc *wc)
-{
- struct ib_cqe *cqe = wc->wr_cqe;
- struct rpcrdma_frwr *frwr =
- container_of(cqe, struct rpcrdma_frwr, fr_cqe);
-
- /* WARNING: Only wr_cqe and status are reliable at this point */
- if (wc->status != IB_WC_SUCCESS)
- frwr->fr_state = FRWR_FLUSHED_FR;
- trace_xprtrdma_wc_fastreg(wc, frwr);
-}
-
-/**
- * frwr_wc_localinv - Invoked by RDMA provider for a flushed LocalInv WC
- * @cq: completion queue (ignored)
- * @wc: completed WR
- *
- */
-static void
-frwr_wc_localinv(struct ib_cq *cq, struct ib_wc *wc)
-{
- struct ib_cqe *cqe = wc->wr_cqe;
- struct rpcrdma_frwr *frwr = container_of(cqe, struct rpcrdma_frwr,
- fr_cqe);
-
- /* WARNING: Only wr_cqe and status are reliable at this point */
- if (wc->status != IB_WC_SUCCESS)
- frwr->fr_state = FRWR_FLUSHED_LI;
- trace_xprtrdma_wc_li(wc, frwr);
-}
-
-/**
- * frwr_wc_localinv_wake - Invoked by RDMA provider for a signaled LocalInv WC
- * @cq: completion queue (ignored)
- * @wc: completed WR
- *
- * Awaken anyone waiting for an MR to finish being fenced.
- */
-static void
-frwr_wc_localinv_wake(struct ib_cq *cq, struct ib_wc *wc)
-{
- struct ib_cqe *cqe = wc->wr_cqe;
- struct rpcrdma_frwr *frwr = container_of(cqe, struct rpcrdma_frwr,
- fr_cqe);
-
- /* WARNING: Only wr_cqe and status are reliable at this point */
- if (wc->status != IB_WC_SUCCESS)
- frwr->fr_state = FRWR_FLUSHED_LI;
- trace_xprtrdma_wc_li_wake(wc, frwr);
- complete(&frwr->fr_linv_done);
-}
-
-/**
* frwr_map - Register a memory region
* @r_xprt: controlling transport
* @seg: memory region co-ordinates
{
struct rpcrdma_ia *ia = &r_xprt->rx_ia;
bool holes_ok = ia->ri_mrtype == IB_MR_TYPE_SG_GAPS;
- struct rpcrdma_frwr *frwr;
struct rpcrdma_mr *mr;
struct ib_mr *ibmr;
struct ib_reg_wr *reg_wr;
int i, n;
u8 key;
- mr = NULL;
- do {
- if (mr)
- rpcrdma_mr_recycle(mr);
- mr = rpcrdma_mr_get(r_xprt);
- if (!mr)
- return ERR_PTR(-EAGAIN);
- } while (mr->frwr.fr_state != FRWR_IS_INVALID);
- frwr = &mr->frwr;
- frwr->fr_state = FRWR_IS_VALID;
+ mr = rpcrdma_mr_get(r_xprt);
+ if (!mr)
+ goto out_getmr_err;
if (nsegs > ia->ri_max_frwr_depth)
nsegs = ia->ri_max_frwr_depth;
if (!mr->mr_nents)
goto out_dmamap_err;
- ibmr = frwr->fr_mr;
+ ibmr = mr->frwr.fr_mr;
n = ib_map_mr_sg(ibmr, mr->mr_sg, mr->mr_nents, NULL, PAGE_SIZE);
if (unlikely(n != mr->mr_nents))
goto out_mapmr_err;
key = (u8)(ibmr->rkey & 0x000000FF);
ib_update_fast_reg_key(ibmr, ++key);
- reg_wr = &frwr->fr_regwr;
+ reg_wr = &mr->frwr.fr_regwr;
reg_wr->mr = ibmr;
reg_wr->key = ibmr->rkey;
reg_wr->access = writing ?
*out = mr;
return seg;
+out_getmr_err:
+ xprt_wait_for_buffer_space(&r_xprt->rx_xprt);
+ return ERR_PTR(-EAGAIN);
+
out_dmamap_err:
mr->mr_dir = DMA_NONE;
trace_xprtrdma_frwr_sgerr(mr, i);
}
/**
+ * frwr_wc_fastreg - Invoked by RDMA provider for a flushed FastReg WC
+ * @cq: completion queue (ignored)
+ * @wc: completed WR
+ *
+ */
+static void frwr_wc_fastreg(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct ib_cqe *cqe = wc->wr_cqe;
+ struct rpcrdma_frwr *frwr =
+ container_of(cqe, struct rpcrdma_frwr, fr_cqe);
+
+ /* WARNING: Only wr_cqe and status are reliable at this point */
+ trace_xprtrdma_wc_fastreg(wc, frwr);
+ /* The MR will get recycled when the associated req is retransmitted */
+}
+
+/**
* frwr_send - post Send WR containing the RPC Call message
* @ia: interface adapter
* @req: Prepared RPC Call
if (mr->mr_handle == rep->rr_inv_rkey) {
list_del_init(&mr->mr_list);
trace_xprtrdma_mr_remoteinv(mr);
- mr->frwr.fr_state = FRWR_IS_INVALID;
rpcrdma_mr_unmap_and_put(mr);
break; /* only one invalidated MR per RPC */
}
}
+static void __frwr_release_mr(struct ib_wc *wc, struct rpcrdma_mr *mr)
+{
+ if (wc->status != IB_WC_SUCCESS)
+ rpcrdma_mr_recycle(mr);
+ else
+ rpcrdma_mr_unmap_and_put(mr);
+}
+
/**
- * frwr_unmap_sync - invalidate memory regions that were registered for @req
- * @r_xprt: controlling transport
- * @mrs: list of MRs to process
+ * frwr_wc_localinv - Invoked by RDMA provider for a LOCAL_INV WC
+ * @cq: completion queue (ignored)
+ * @wc: completed WR
+ *
+ */
+static void frwr_wc_localinv(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct ib_cqe *cqe = wc->wr_cqe;
+ struct rpcrdma_frwr *frwr =
+ container_of(cqe, struct rpcrdma_frwr, fr_cqe);
+ struct rpcrdma_mr *mr = container_of(frwr, struct rpcrdma_mr, frwr);
+
+ /* WARNING: Only wr_cqe and status are reliable at this point */
+ trace_xprtrdma_wc_li(wc, frwr);
+ __frwr_release_mr(wc, mr);
+}
+
+/**
+ * frwr_wc_localinv_wake - Invoked by RDMA provider for a LOCAL_INV WC
+ * @cq: completion queue (ignored)
+ * @wc: completed WR
*
- * Sleeps until it is safe for the host CPU to access the
- * previously mapped memory regions.
+ * Awaken anyone waiting for an MR to finish being fenced.
+ */
+static void frwr_wc_localinv_wake(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct ib_cqe *cqe = wc->wr_cqe;
+ struct rpcrdma_frwr *frwr =
+ container_of(cqe, struct rpcrdma_frwr, fr_cqe);
+ struct rpcrdma_mr *mr = container_of(frwr, struct rpcrdma_mr, frwr);
+
+ /* WARNING: Only wr_cqe and status are reliable at this point */
+ trace_xprtrdma_wc_li_wake(wc, frwr);
+ complete(&frwr->fr_linv_done);
+ __frwr_release_mr(wc, mr);
+}
+
+/**
+ * frwr_unmap_sync - invalidate memory regions that were registered for @req
+ * @r_xprt: controlling transport instance
+ * @req: rpcrdma_req with a non-empty list of MRs to process
*
- * Caller ensures that @mrs is not empty before the call. This
- * function empties the list.
+ * Sleeps until it is safe for the host CPU to access the previously mapped
+ * memory regions. This guarantees that registered MRs are properly fenced
+ * from the server before the RPC consumer accesses the data in them. It
+ * also ensures proper Send flow control: waking the next RPC waits until
+ * this RPC has relinquished all its Send Queue entries.
*/
-void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt, struct list_head *mrs)
+void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
{
struct ib_send_wr *first, **prev, *last;
const struct ib_send_wr *bad_wr;
- struct rpcrdma_ia *ia = &r_xprt->rx_ia;
struct rpcrdma_frwr *frwr;
struct rpcrdma_mr *mr;
- int count, rc;
+ int rc;
/* ORDER: Invalidate all of the MRs first
*
* a single ib_post_send() call.
*/
frwr = NULL;
- count = 0;
prev = &first;
- list_for_each_entry(mr, mrs, mr_list) {
- mr->frwr.fr_state = FRWR_IS_INVALID;
+ while (!list_empty(&req->rl_registered)) {
+ mr = rpcrdma_mr_pop(&req->rl_registered);
- frwr = &mr->frwr;
trace_xprtrdma_mr_localinv(mr);
+ r_xprt->rx_stats.local_inv_needed++;
+ frwr = &mr->frwr;
frwr->fr_cqe.done = frwr_wc_localinv;
last = &frwr->fr_invwr;
- memset(last, 0, sizeof(*last));
+ last->next = NULL;
last->wr_cqe = &frwr->fr_cqe;
+ last->sg_list = NULL;
+ last->num_sge = 0;
last->opcode = IB_WR_LOCAL_INV;
+ last->send_flags = IB_SEND_SIGNALED;
last->ex.invalidate_rkey = mr->mr_handle;
- count++;
*prev = last;
prev = &last->next;
}
- if (!frwr)
- goto unmap;
/* Strong send queue ordering guarantees that when the
* last WR in the chain completes, all WRs in the chain
* are complete.
*/
- last->send_flags = IB_SEND_SIGNALED;
frwr->fr_cqe.done = frwr_wc_localinv_wake;
reinit_completion(&frwr->fr_linv_done);
* replaces the QP. The RPC reply handler won't call us
* unless ri_id->qp is a valid pointer.
*/
- r_xprt->rx_stats.local_inv_needed++;
bad_wr = NULL;
- rc = ib_post_send(ia->ri_id->qp, first, &bad_wr);
+ rc = ib_post_send(r_xprt->rx_ia.ri_id->qp, first, &bad_wr);
+ trace_xprtrdma_post_send(req, rc);
+
+ /* The final LOCAL_INV WR in the chain is supposed to
+ * do the wake. If it was never posted, the wake will
+ * not happen, so don't wait in that case.
+ */
if (bad_wr != first)
wait_for_completion(&frwr->fr_linv_done);
- if (rc)
- goto out_release;
+ if (!rc)
+ return;
- /* ORDER: Now DMA unmap all of the MRs, and return
- * them to the free MR list.
+ /* Recycle MRs in the LOCAL_INV chain that did not get posted.
*/
-unmap:
- while (!list_empty(mrs)) {
- mr = rpcrdma_mr_pop(mrs);
- rpcrdma_mr_unmap_and_put(mr);
+ while (bad_wr) {
+ frwr = container_of(bad_wr, struct rpcrdma_frwr,
+ fr_invwr);
+ mr = container_of(frwr, struct rpcrdma_mr, frwr);
+ bad_wr = bad_wr->next;
+
+ list_del_init(&mr->mr_list);
+ rpcrdma_mr_recycle(mr);
}
- return;
+}
-out_release:
- pr_err("rpcrdma: FRWR invalidate ib_post_send returned %i\n", rc);
+/**
+ * frwr_wc_localinv_done - Invoked by RDMA provider for a signaled LOCAL_INV WC
+ * @cq: completion queue (ignored)
+ * @wc: completed WR
+ *
+ */
+static void frwr_wc_localinv_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct ib_cqe *cqe = wc->wr_cqe;
+ struct rpcrdma_frwr *frwr =
+ container_of(cqe, struct rpcrdma_frwr, fr_cqe);
+ struct rpcrdma_mr *mr = container_of(frwr, struct rpcrdma_mr, frwr);
- /* Unmap and release the MRs in the LOCAL_INV WRs that did not
- * get posted.
+ /* WARNING: Only wr_cqe and status are reliable at this point */
+ trace_xprtrdma_wc_li_done(wc, frwr);
+ rpcrdma_complete_rqst(frwr->fr_req->rl_reply);
+ __frwr_release_mr(wc, mr);
+}
+
+/**
+ * frwr_unmap_async - invalidate memory regions that were registered for @req
+ * @r_xprt: controlling transport instance
+ * @req: rpcrdma_req with a non-empty list of MRs to process
+ *
+ * This guarantees that registered MRs are properly fenced from the
+ * server before the RPC consumer accesses the data in them. It also
+ * ensures proper Send flow control: waking the next RPC waits until
+ * this RPC has relinquished all its Send Queue entries.
+ */
+void frwr_unmap_async(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
+{
+ struct ib_send_wr *first, *last, **prev;
+ const struct ib_send_wr *bad_wr;
+ struct rpcrdma_frwr *frwr;
+ struct rpcrdma_mr *mr;
+ int rc;
+
+ /* Chain the LOCAL_INV Work Requests and post them with
+ * a single ib_post_send() call.
+ */
+ frwr = NULL;
+ prev = &first;
+ while (!list_empty(&req->rl_registered)) {
+ mr = rpcrdma_mr_pop(&req->rl_registered);
+
+ trace_xprtrdma_mr_localinv(mr);
+ r_xprt->rx_stats.local_inv_needed++;
+
+ frwr = &mr->frwr;
+ frwr->fr_cqe.done = frwr_wc_localinv;
+ frwr->fr_req = req;
+ last = &frwr->fr_invwr;
+ last->next = NULL;
+ last->wr_cqe = &frwr->fr_cqe;
+ last->sg_list = NULL;
+ last->num_sge = 0;
+ last->opcode = IB_WR_LOCAL_INV;
+ last->send_flags = IB_SEND_SIGNALED;
+ last->ex.invalidate_rkey = mr->mr_handle;
+
+ *prev = last;
+ prev = &last->next;
+ }
+
+ /* Strong send queue ordering guarantees that when the
+ * last WR in the chain completes, all WRs in the chain
+ * are complete. The last completion will wake up the
+ * RPC waiter.
+ */
+ frwr->fr_cqe.done = frwr_wc_localinv_done;
+
+ /* Transport disconnect drains the receive CQ before it
+ * replaces the QP. The RPC reply handler won't call us
+ * unless ri_id->qp is a valid pointer.
+ */
+ bad_wr = NULL;
+ rc = ib_post_send(r_xprt->rx_ia.ri_id->qp, first, &bad_wr);
+ trace_xprtrdma_post_send(req, rc);
+ if (!rc)
+ return;
+
+ /* Recycle MRs in the LOCAL_INV chain that did not get posted.
*/
while (bad_wr) {
- frwr = container_of(bad_wr, struct rpcrdma_frwr,
- fr_invwr);
+ frwr = container_of(bad_wr, struct rpcrdma_frwr, fr_invwr);
mr = container_of(frwr, struct rpcrdma_mr, frwr);
bad_wr = bad_wr->next;
- list_del_init(&mr->mr_list);
rpcrdma_mr_recycle(mr);
}
+
+ /* The final LOCAL_INV WR in the chain is supposed to
+ * do the wake. If it was never posted, the wake will
+ * not happen, so wake here in that case.
+ */
+ rpcrdma_complete_rqst(req->rl_reply);
}
unsigned int pos;
int nsegs;
+ if (rtype == rpcrdma_noch)
+ goto done;
+
pos = rqst->rq_snd_buf.head[0].iov_len;
if (rtype == rpcrdma_areadch)
pos = 0;
nsegs -= mr->mr_nents;
} while (nsegs);
- return 0;
+done:
+ return encode_item_not_present(xdr);
}
/* Register and XDR encode the Write list. Supports encoding a list
int nsegs, nchunks;
__be32 *segcount;
+ if (wtype != rpcrdma_writech)
+ goto done;
+
seg = req->rl_segments;
nsegs = rpcrdma_convert_iovs(r_xprt, &rqst->rq_rcv_buf,
rqst->rq_rcv_buf.head[0].iov_len,
/* Update count of segments in this Write chunk */
*segcount = cpu_to_be32(nchunks);
- return 0;
+done:
+ return encode_item_not_present(xdr);
}
/* Register and XDR encode the Reply chunk. Supports encoding an array
int nsegs, nchunks;
__be32 *segcount;
+ if (wtype != rpcrdma_replych)
+ return encode_item_not_present(xdr);
+
seg = req->rl_segments;
nsegs = rpcrdma_convert_iovs(r_xprt, &rqst->rq_rcv_buf, 0, wtype, seg);
if (nsegs < 0)
return 0;
}
+static void rpcrdma_sendctx_done(struct kref *kref)
+{
+ struct rpcrdma_req *req =
+ container_of(kref, struct rpcrdma_req, rl_kref);
+ struct rpcrdma_rep *rep = req->rl_reply;
+
+ rpcrdma_complete_rqst(rep);
+ rep->rr_rxprt->rx_stats.reply_waits_for_send++;
+}
+
/**
* rpcrdma_sendctx_unmap - DMA-unmap Send buffer
* @sc: sendctx containing SGEs to unmap
{
struct ib_sge *sge;
+ if (!sc->sc_unmap_count)
+ return;
+
/* The first two SGEs contain the transport header and
* the inline buffer. These are always left mapped so
* they can be cheaply re-used.
ib_dma_unmap_page(sc->sc_device, sge->addr, sge->length,
DMA_TO_DEVICE);
- if (test_and_clear_bit(RPCRDMA_REQ_F_TX_RESOURCES,
- &sc->sc_req->rl_flags))
- wake_up_bit(&sc->sc_req->rl_flags, RPCRDMA_REQ_F_TX_RESOURCES);
+ kref_put(&sc->sc_req->rl_kref, rpcrdma_sendctx_done);
}
/* Prepare an SGE for the RPC-over-RDMA transport header.
out:
sc->sc_wr.num_sge += sge_no;
if (sc->sc_unmap_count)
- __set_bit(RPCRDMA_REQ_F_TX_RESOURCES, &req->rl_flags);
+ kref_get(&req->rl_kref);
return true;
out_regbuf:
struct rpcrdma_req *req, u32 hdrlen,
struct xdr_buf *xdr, enum rpcrdma_chunktype rtype)
{
+ int ret;
+
+ ret = -EAGAIN;
req->rl_sendctx = rpcrdma_sendctx_get_locked(r_xprt);
if (!req->rl_sendctx)
- return -EAGAIN;
+ goto err;
req->rl_sendctx->sc_wr.num_sge = 0;
req->rl_sendctx->sc_unmap_count = 0;
req->rl_sendctx->sc_req = req;
- __clear_bit(RPCRDMA_REQ_F_TX_RESOURCES, &req->rl_flags);
+ kref_init(&req->rl_kref);
+ ret = -EIO;
if (!rpcrdma_prepare_hdr_sge(r_xprt, req, hdrlen))
- return -EIO;
-
+ goto err;
if (rtype != rpcrdma_areadch)
if (!rpcrdma_prepare_msg_sges(r_xprt, req, xdr, rtype))
- return -EIO;
-
+ goto err;
return 0;
+
+err:
+ trace_xprtrdma_prepsend_failed(&req->rl_slot, ret);
+ return ret;
}
/**
* send a Call message with a Position Zero Read chunk and a
* regular Read chunk at the same time.
*/
- if (rtype != rpcrdma_noch) {
- ret = rpcrdma_encode_read_list(r_xprt, req, rqst, rtype);
- if (ret)
- goto out_err;
- }
- ret = encode_item_not_present(xdr);
+ ret = rpcrdma_encode_read_list(r_xprt, req, rqst, rtype);
if (ret)
goto out_err;
-
- if (wtype == rpcrdma_writech) {
- ret = rpcrdma_encode_write_list(r_xprt, req, rqst, wtype);
- if (ret)
- goto out_err;
- }
- ret = encode_item_not_present(xdr);
+ ret = rpcrdma_encode_write_list(r_xprt, req, rqst, wtype);
if (ret)
goto out_err;
-
- if (wtype != rpcrdma_replych)
- ret = encode_item_not_present(xdr);
- else
- ret = rpcrdma_encode_reply_chunk(r_xprt, req, rqst, wtype);
+ ret = rpcrdma_encode_reply_chunk(r_xprt, req, rqst, wtype);
if (ret)
goto out_err;
- trace_xprtrdma_marshal(rqst, xdr_stream_pos(xdr), rtype, wtype);
-
- ret = rpcrdma_prepare_send_sges(r_xprt, req, xdr_stream_pos(xdr),
+ ret = rpcrdma_prepare_send_sges(r_xprt, req, req->rl_hdrbuf.len,
&rqst->rq_snd_buf, rtype);
if (ret)
goto out_err;
+
+ trace_xprtrdma_marshal(req, rtype, wtype);
return 0;
out_err:
trace_xprtrdma_marshal_failed(rqst, ret);
- switch (ret) {
- case -EAGAIN:
- xprt_wait_for_buffer_space(rqst->rq_xprt);
- break;
- case -ENOBUFS:
- break;
- default:
- r_xprt->rx_stats.failed_marshal_count++;
- }
+ r_xprt->rx_stats.failed_marshal_count++;
+ frwr_reset(req);
return ret;
}
goto out;
}
-void rpcrdma_release_rqst(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req)
-{
- /* Invalidate and unmap the data payloads before waking
- * the waiting application. This guarantees the memory
- * regions are properly fenced from the server before the
- * application accesses the data. It also ensures proper
- * send flow control: waking the next RPC waits until this
- * RPC has relinquished all its Send Queue entries.
- */
- if (!list_empty(&req->rl_registered))
- frwr_unmap_sync(r_xprt, &req->rl_registered);
-
- /* Ensure that any DMA mapped pages associated with
- * the Send of the RPC Call have been unmapped before
- * allowing the RPC to complete. This protects argument
- * memory not controlled by the RPC client from being
- * re-used before we're done with it.
- */
- if (test_bit(RPCRDMA_REQ_F_TX_RESOURCES, &req->rl_flags)) {
- r_xprt->rx_stats.reply_waits_for_send++;
- out_of_line_wait_on_bit(&req->rl_flags,
- RPCRDMA_REQ_F_TX_RESOURCES,
- bit_wait,
- TASK_UNINTERRUPTIBLE);
- }
-}
-
-/* Reply handling runs in the poll worker thread. Anything that
- * might wait is deferred to a separate workqueue.
- */
-void rpcrdma_deferred_completion(struct work_struct *work)
+static void rpcrdma_reply_done(struct kref *kref)
{
- struct rpcrdma_rep *rep =
- container_of(work, struct rpcrdma_rep, rr_work);
- struct rpcrdma_req *req = rpcr_to_rdmar(rep->rr_rqst);
- struct rpcrdma_xprt *r_xprt = rep->rr_rxprt;
+ struct rpcrdma_req *req =
+ container_of(kref, struct rpcrdma_req, rl_kref);
- trace_xprtrdma_defer_cmp(rep);
- if (rep->rr_wc_flags & IB_WC_WITH_INVALIDATE)
- frwr_reminv(rep, &req->rl_registered);
- rpcrdma_release_rqst(r_xprt, req);
- rpcrdma_complete_rqst(rep);
+ rpcrdma_complete_rqst(req->rl_reply);
}
-/* Process received RPC/RDMA messages.
+/**
+ * rpcrdma_reply_handler - Process received RPC/RDMA messages
+ * @rep: Incoming rpcrdma_rep object to process
*
* Errors must result in the RPC task either being awakened, or
* allowed to timeout, to discover the errors at that time.
else if (credits > buf->rb_max_requests)
credits = buf->rb_max_requests;
if (buf->rb_credits != credits) {
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
buf->rb_credits = credits;
xprt->cwnd = credits << RPC_CWNDSHIFT;
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
req = rpcr_to_rdmar(rqst);
}
req->rl_reply = rep;
rep->rr_rqst = rqst;
- clear_bit(RPCRDMA_REQ_F_PENDING, &req->rl_flags);
trace_xprtrdma_reply(rqst->rq_task, rep, req, credits);
- queue_work(buf->rb_completion_wq, &rep->rr_work);
+
+ if (rep->rr_wc_flags & IB_WC_WITH_INVALIDATE)
+ frwr_reminv(rep, &req->rl_registered);
+ if (!list_empty(&req->rl_registered))
+ frwr_unmap_async(r_xprt, req);
+ /* LocalInv completion will complete the RPC */
+ else
+ kref_put(&req->rl_kref, rpcrdma_reply_done);
return;
out_badversion:
else if (credits > r_xprt->rx_buf.rb_bc_max_requests)
credits = r_xprt->rx_buf.rb_bc_max_requests;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt->cwnd = credits << RPC_CWNDSHIFT;
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
spin_lock(&xprt->queue_lock);
ret = 0;
* Enqueue the new transport on the accept queue of the listening
* transport
*/
- spin_lock_bh(&listen_xprt->sc_lock);
+ spin_lock(&listen_xprt->sc_lock);
list_add_tail(&newxprt->sc_accept_q, &listen_xprt->sc_accept_q);
- spin_unlock_bh(&listen_xprt->sc_lock);
+ spin_unlock(&listen_xprt->sc_lock);
set_bit(XPT_CONN, &listen_xprt->sc_xprt.xpt_flags);
svc_xprt_enqueue(&listen_xprt->sc_xprt);
listen_rdma = container_of(xprt, struct svcxprt_rdma, sc_xprt);
clear_bit(XPT_CONN, &xprt->xpt_flags);
/* Get the next entry off the accept list */
- spin_lock_bh(&listen_rdma->sc_lock);
+ spin_lock(&listen_rdma->sc_lock);
if (!list_empty(&listen_rdma->sc_accept_q)) {
newxprt = list_entry(listen_rdma->sc_accept_q.next,
struct svcxprt_rdma, sc_accept_q);
}
if (!list_empty(&listen_rdma->sc_accept_q))
set_bit(XPT_CONN, &listen_rdma->sc_xprt.xpt_flags);
- spin_unlock_bh(&listen_rdma->sc_lock);
+ spin_unlock(&listen_rdma->sc_lock);
if (!newxprt)
return NULL;
static unsigned int max_slot_table_size = RPCRDMA_MAX_SLOT_TABLE;
static unsigned int min_inline_size = RPCRDMA_MIN_INLINE;
static unsigned int max_inline_size = RPCRDMA_MAX_INLINE;
-static unsigned int zero;
static unsigned int max_padding = PAGE_SIZE;
static unsigned int min_memreg = RPCRDMA_BOUNCEBUFFERS;
static unsigned int max_memreg = RPCRDMA_LAST - 1;
.maxlen = sizeof(unsigned int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &max_padding,
},
{
module_put(THIS_MODULE);
}
+/* 60 second timeout, no retries */
static const struct rpc_timeout xprt_rdma_default_timeout = {
.to_initval = 60 * HZ,
.to_maxval = 60 * HZ,
if (!xprt)
return ERR_PTR(-ENOMEM);
- /* 60 second timeout, no retries */
xprt->timeout = &xprt_rdma_default_timeout;
+ xprt->connect_timeout = xprt->timeout->to_initval;
+ xprt->max_reconnect_timeout = xprt->timeout->to_maxval;
xprt->bind_timeout = RPCRDMA_BIND_TO;
xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO;
xprt->idle_timeout = RPCRDMA_IDLE_DISC_TO;
}
/**
- * xprt_rdma_connect - try to establish a transport connection
+ * xprt_rdma_set_connect_timeout - set timeouts for establishing a connection
+ * @xprt: controlling transport instance
+ * @connect_timeout: reconnect timeout after client disconnects
+ * @reconnect_timeout: reconnect timeout after server disconnects
+ *
+ */
+static void xprt_rdma_tcp_set_connect_timeout(struct rpc_xprt *xprt,
+ unsigned long connect_timeout,
+ unsigned long reconnect_timeout)
+{
+ struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
+
+ trace_xprtrdma_op_set_cto(r_xprt, connect_timeout, reconnect_timeout);
+
+ spin_lock(&xprt->transport_lock);
+
+ if (connect_timeout < xprt->connect_timeout) {
+ struct rpc_timeout to;
+ unsigned long initval;
+
+ to = *xprt->timeout;
+ initval = connect_timeout;
+ if (initval < RPCRDMA_INIT_REEST_TO << 1)
+ initval = RPCRDMA_INIT_REEST_TO << 1;
+ to.to_initval = initval;
+ to.to_maxval = initval;
+ r_xprt->rx_timeout = to;
+ xprt->timeout = &r_xprt->rx_timeout;
+ xprt->connect_timeout = connect_timeout;
+ }
+
+ if (reconnect_timeout < xprt->max_reconnect_timeout)
+ xprt->max_reconnect_timeout = reconnect_timeout;
+
+ spin_unlock(&xprt->transport_lock);
+}
+
+/**
+ * xprt_rdma_connect - schedule an attempt to reconnect
* @xprt: transport state
- * @task: RPC scheduler context
+ * @task: RPC scheduler context (unused)
*
*/
static void
xprt_rdma_connect(struct rpc_xprt *xprt, struct rpc_task *task)
{
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
+ unsigned long delay;
trace_xprtrdma_op_connect(r_xprt);
+
+ delay = 0;
if (r_xprt->rx_ep.rep_connected != 0) {
- /* Reconnect */
- schedule_delayed_work(&r_xprt->rx_connect_worker,
- xprt->reestablish_timeout);
- xprt->reestablish_timeout <<= 1;
- if (xprt->reestablish_timeout > RPCRDMA_MAX_REEST_TO)
- xprt->reestablish_timeout = RPCRDMA_MAX_REEST_TO;
- else if (xprt->reestablish_timeout < RPCRDMA_INIT_REEST_TO)
- xprt->reestablish_timeout = RPCRDMA_INIT_REEST_TO;
- } else {
- schedule_delayed_work(&r_xprt->rx_connect_worker, 0);
- if (!RPC_IS_ASYNC(task))
- flush_delayed_work(&r_xprt->rx_connect_worker);
+ delay = xprt_reconnect_delay(xprt);
+ xprt_reconnect_backoff(xprt, RPCRDMA_INIT_REEST_TO);
}
+ queue_delayed_work(xprtiod_workqueue, &r_xprt->rx_connect_worker,
+ delay);
}
/**
static void
xprt_rdma_free_slot(struct rpc_xprt *xprt, struct rpc_rqst *rqst)
{
+ struct rpcrdma_xprt *r_xprt =
+ container_of(xprt, struct rpcrdma_xprt, rx_xprt);
+
memset(rqst, 0, sizeof(*rqst));
- rpcrdma_buffer_put(rpcr_to_rdmar(rqst));
+ rpcrdma_buffer_put(&r_xprt->rx_buf, rpcr_to_rdmar(rqst));
rpc_wake_up_next(&xprt->backlog);
}
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(rqst->rq_xprt);
struct rpcrdma_req *req = rpcr_to_rdmar(rqst);
- if (test_bit(RPCRDMA_REQ_F_PENDING, &req->rl_flags))
- rpcrdma_release_rqst(r_xprt, req);
trace_xprtrdma_op_free(task, req);
+
+ if (!list_empty(&req->rl_registered))
+ frwr_unmap_sync(r_xprt, req);
+
+ /* XXX: If the RPC is completing because of a signal and
+ * not because a reply was received, we ought to ensure
+ * that the Send completion has fired, so that memory
+ * involved with the Send is not still visible to the NIC.
+ */
}
/**
goto drop_connection;
rqst->rq_xtime = ktime_get();
- __set_bit(RPCRDMA_REQ_F_PENDING, &req->rl_flags);
if (rpcrdma_ep_post(&r_xprt->rx_ia, &r_xprt->rx_ep, req))
goto drop_connection;
.send_request = xprt_rdma_send_request,
.close = xprt_rdma_close,
.destroy = xprt_rdma_destroy,
+ .set_connect_timeout = xprt_rdma_tcp_set_connect_timeout,
.print_stats = xprt_rdma_print_stats,
.enable_swap = xprt_rdma_enable_swap,
.disable_swap = xprt_rdma_disable_swap,
#if defined(CONFIG_SUNRPC_BACKCHANNEL)
.bc_setup = xprt_rdma_bc_setup,
.bc_maxpayload = xprt_rdma_bc_maxpayload,
+ .bc_num_slots = xprt_rdma_bc_max_slots,
.bc_free_rqst = xprt_rdma_bc_free_rqst,
.bc_destroy = xprt_rdma_bc_destroy,
#endif
*/
static void rpcrdma_xprt_drain(struct rpcrdma_xprt *r_xprt)
{
- struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
struct rpcrdma_ia *ia = &r_xprt->rx_ia;
/* Flush Receives, then wait for deferred Reply work
* to complete.
*/
ib_drain_rq(ia->ri_id->qp);
- drain_workqueue(buf->rb_completion_wq);
/* Deferred Reply processing might have scheduled
* local invalidations.
* completions recently. This is a sign the Send Queue is
* backing up. Cause the caller to pause and try again.
*/
- set_bit(RPCRDMA_BUF_F_EMPTY_SCQ, &buf->rb_flags);
+ xprt_wait_for_buffer_space(&r_xprt->rx_xprt);
r_xprt->rx_stats.empty_sendctx_q++;
return NULL;
}
/* Paired with READ_ONCE */
smp_store_release(&buf->rb_sc_tail, next_tail);
- if (test_and_clear_bit(RPCRDMA_BUF_F_EMPTY_SCQ, &buf->rb_flags)) {
- smp_mb__after_atomic();
- xprt_write_space(&sc->sc_xprt->rx_xprt);
- }
+ xprt_write_space(&sc->sc_xprt->rx_xprt);
}
static void
r_xprt->rx_stats.mrs_allocated += count;
spin_unlock(&buf->rb_mrlock);
trace_xprtrdma_createmrs(r_xprt, count);
-
- xprt_write_space(&r_xprt->rx_xprt);
}
static void
rx_buf);
rpcrdma_mrs_create(r_xprt);
+ xprt_write_space(&r_xprt->rx_xprt);
}
/**
if (!req->rl_recvbuf)
goto out4;
- req->rl_buffer = buffer;
INIT_LIST_HEAD(&req->rl_registered);
spin_lock(&buffer->rb_lock);
list_add(&req->rl_all, &buffer->rb_allreqs);
return NULL;
}
-static bool rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt, bool temp)
+static struct rpcrdma_rep *rpcrdma_rep_create(struct rpcrdma_xprt *r_xprt,
+ bool temp)
{
- struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
struct rpcrdma_rep *rep;
rep = kzalloc(sizeof(*rep), GFP_KERNEL);
DMA_FROM_DEVICE, GFP_KERNEL);
if (!rep->rr_rdmabuf)
goto out_free;
+
xdr_buf_init(&rep->rr_hdrbuf, rdmab_data(rep->rr_rdmabuf),
rdmab_length(rep->rr_rdmabuf));
-
rep->rr_cqe.done = rpcrdma_wc_receive;
rep->rr_rxprt = r_xprt;
- INIT_WORK(&rep->rr_work, rpcrdma_deferred_completion);
rep->rr_recv_wr.next = NULL;
rep->rr_recv_wr.wr_cqe = &rep->rr_cqe;
rep->rr_recv_wr.sg_list = &rep->rr_rdmabuf->rg_iov;
rep->rr_recv_wr.num_sge = 1;
rep->rr_temp = temp;
-
- spin_lock(&buf->rb_lock);
- list_add(&rep->rr_list, &buf->rb_recv_bufs);
- spin_unlock(&buf->rb_lock);
- return true;
+ return rep;
out_free:
kfree(rep);
out:
- return false;
+ return NULL;
}
/**
struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
int i, rc;
- buf->rb_flags = 0;
buf->rb_max_requests = r_xprt->rx_ep.rep_max_requests;
buf->rb_bc_srv_max_requests = 0;
spin_lock_init(&buf->rb_mrlock);
if (rc)
goto out;
- buf->rb_completion_wq = alloc_workqueue("rpcrdma-%s",
- WQ_MEM_RECLAIM | WQ_HIGHPRI,
- 0,
- r_xprt->rx_xprt.address_strings[RPC_DISPLAY_ADDR]);
- if (!buf->rb_completion_wq) {
- rc = -ENOMEM;
- goto out;
- }
-
return 0;
out:
rpcrdma_buffer_destroy(buf);
{
cancel_delayed_work_sync(&buf->rb_refresh_worker);
- if (buf->rb_completion_wq) {
- destroy_workqueue(buf->rb_completion_wq);
- buf->rb_completion_wq = NULL;
- }
-
rpcrdma_sendctxs_destroy(buf);
while (!list_empty(&buf->rb_recv_bufs)) {
/**
* rpcrdma_buffer_put - Put request/reply buffers back into pool
+ * @buffers: buffer pool
* @req: object to return
*
*/
-void
-rpcrdma_buffer_put(struct rpcrdma_req *req)
+void rpcrdma_buffer_put(struct rpcrdma_buffer *buffers, struct rpcrdma_req *req)
{
- struct rpcrdma_buffer *buffers = req->rl_buffer;
struct rpcrdma_rep *rep = req->rl_reply;
req->rl_reply = NULL;
struct ib_send_wr *send_wr = &req->rl_sendctx->sc_wr;
int rc;
- if (!ep->rep_send_count ||
- test_bit(RPCRDMA_REQ_F_TX_RESOURCES, &req->rl_flags)) {
+ if (!ep->rep_send_count || kref_read(&req->rl_kref) > 1) {
send_wr->send_flags |= IB_SEND_SIGNALED;
ep->rep_send_count = ep->rep_send_batch;
} else {
{
struct rpcrdma_buffer *buf = &r_xprt->rx_buf;
struct rpcrdma_ep *ep = &r_xprt->rx_ep;
- struct ib_recv_wr *wr, *bad_wr;
+ struct ib_recv_wr *i, *wr, *bad_wr;
+ struct rpcrdma_rep *rep;
int needed, count, rc;
rc = 0;
count = 0;
+
needed = buf->rb_credits + (buf->rb_bc_srv_max_requests << 1);
if (ep->rep_receive_count > needed)
goto out;
if (!temp)
needed += RPCRDMA_MAX_RECV_BATCH;
- count = 0;
+ /* fast path: all needed reps can be found on the free list */
wr = NULL;
+ spin_lock(&buf->rb_lock);
while (needed) {
- struct rpcrdma_regbuf *rb;
- struct rpcrdma_rep *rep;
-
- spin_lock(&buf->rb_lock);
rep = list_first_entry_or_null(&buf->rb_recv_bufs,
struct rpcrdma_rep, rr_list);
- if (likely(rep))
- list_del(&rep->rr_list);
- spin_unlock(&buf->rb_lock);
- if (!rep) {
- if (!rpcrdma_rep_create(r_xprt, temp))
- break;
- continue;
- }
+ if (!rep)
+ break;
- rb = rep->rr_rdmabuf;
- if (!rpcrdma_regbuf_dma_map(r_xprt, rb)) {
- rpcrdma_recv_buffer_put(rep);
+ list_del(&rep->rr_list);
+ rep->rr_recv_wr.next = wr;
+ wr = &rep->rr_recv_wr;
+ --needed;
+ }
+ spin_unlock(&buf->rb_lock);
+
+ while (needed) {
+ rep = rpcrdma_rep_create(r_xprt, temp);
+ if (!rep)
break;
- }
- trace_xprtrdma_post_recv(rep->rr_recv_wr.wr_cqe);
rep->rr_recv_wr.next = wr;
wr = &rep->rr_recv_wr;
- ++count;
--needed;
}
- if (!count)
+ if (!wr)
goto out;
+ for (i = wr; i; i = i->next) {
+ rep = container_of(i, struct rpcrdma_rep, rr_recv_wr);
+
+ if (!rpcrdma_regbuf_dma_map(r_xprt, rep->rr_rdmabuf))
+ goto release_wrs;
+
+ trace_xprtrdma_post_recv(rep->rr_recv_wr.wr_cqe);
+ ++count;
+ }
+
rc = ib_post_recv(r_xprt->rx_ia.ri_id->qp, wr,
(const struct ib_recv_wr **)&bad_wr);
+out:
+ trace_xprtrdma_post_recvs(r_xprt, count, rc);
if (rc) {
- for (wr = bad_wr; wr; wr = wr->next) {
+ for (wr = bad_wr; wr;) {
struct rpcrdma_rep *rep;
rep = container_of(wr, struct rpcrdma_rep, rr_recv_wr);
+ wr = wr->next;
rpcrdma_recv_buffer_put(rep);
--count;
}
}
ep->rep_receive_count += count;
-out:
- trace_xprtrdma_post_recvs(r_xprt, count, rc);
+ return;
+
+release_wrs:
+ for (i = wr; i;) {
+ rep = container_of(i, struct rpcrdma_rep, rr_recv_wr);
+ i = i->next;
+ rpcrdma_recv_buffer_put(rep);
+ }
}
#include <linux/wait.h> /* wait_queue_head_t, etc */
#include <linux/spinlock.h> /* spinlock_t, etc */
-#include <linux/atomic.h> /* atomic_t, etc */
+#include <linux/atomic.h> /* atomic_t, etc */
+#include <linux/kref.h> /* struct kref */
#include <linux/workqueue.h> /* struct work_struct */
#include <rdma/rdma_cm.h> /* RDMA connection api */
bool rr_temp;
struct rpcrdma_regbuf *rr_rdmabuf;
struct rpcrdma_xprt *rr_rxprt;
- struct work_struct rr_work;
+ struct rpc_rqst *rr_rqst;
struct xdr_buf rr_hdrbuf;
struct xdr_stream rr_stream;
- struct rpc_rqst *rr_rqst;
struct list_head rr_list;
struct ib_recv_wr rr_recv_wr;
};
* An external memory region is any buffer or page that is registered
* on the fly (ie, not pre-registered).
*/
-enum rpcrdma_frwr_state {
- FRWR_IS_INVALID, /* ready to be used */
- FRWR_IS_VALID, /* in use */
- FRWR_FLUSHED_FR, /* flushed FASTREG WR */
- FRWR_FLUSHED_LI, /* flushed LOCALINV WR */
-};
-
+struct rpcrdma_req;
struct rpcrdma_frwr {
struct ib_mr *fr_mr;
struct ib_cqe fr_cqe;
- enum rpcrdma_frwr_state fr_state;
struct completion fr_linv_done;
+ struct rpcrdma_req *fr_req;
union {
struct ib_reg_wr fr_regwr;
struct ib_send_wr fr_invwr;
struct rpcrdma_req {
struct list_head rl_list;
struct rpc_rqst rl_slot;
- struct rpcrdma_buffer *rl_buffer;
struct rpcrdma_rep *rl_reply;
struct xdr_stream rl_stream;
struct xdr_buf rl_hdrbuf;
struct rpcrdma_regbuf *rl_recvbuf; /* rq_rcv_buf */
struct list_head rl_all;
- unsigned long rl_flags;
+ struct kref rl_kref;
struct list_head rl_registered; /* registered segments */
struct rpcrdma_mr_seg rl_segments[RPCRDMA_MAX_SEGS];
};
-/* rl_flags */
-enum {
- RPCRDMA_REQ_F_PENDING = 0,
- RPCRDMA_REQ_F_TX_RESOURCES,
-};
-
static inline struct rpcrdma_req *
rpcr_to_rdmar(const struct rpc_rqst *rqst)
{
struct list_head rb_recv_bufs;
struct list_head rb_allreqs;
- unsigned long rb_flags;
u32 rb_max_requests;
u32 rb_credits; /* most recent credit grant */
u32 rb_bc_srv_max_requests;
u32 rb_bc_max_requests;
- struct workqueue_struct *rb_completion_wq;
struct delayed_work rb_refresh_worker;
};
-/* rb_flags */
-enum {
- RPCRDMA_BUF_F_EMPTY_SCQ = 0,
-};
-
/*
* Statistics for RPCRDMA
*/
struct rpcrdma_ep rx_ep;
struct rpcrdma_buffer rx_buf;
struct delayed_work rx_connect_worker;
+ struct rpc_timeout rx_timeout;
struct rpcrdma_stats rx_stats;
};
}
struct rpcrdma_req *rpcrdma_buffer_get(struct rpcrdma_buffer *);
-void rpcrdma_buffer_put(struct rpcrdma_req *);
+void rpcrdma_buffer_put(struct rpcrdma_buffer *buffers,
+ struct rpcrdma_req *req);
void rpcrdma_recv_buffer_put(struct rpcrdma_rep *);
bool rpcrdma_regbuf_realloc(struct rpcrdma_regbuf *rb, size_t size,
/* Memory registration calls xprtrdma/frwr_ops.c
*/
bool frwr_is_supported(struct ib_device *device);
+void frwr_reset(struct rpcrdma_req *req);
int frwr_open(struct rpcrdma_ia *ia, struct rpcrdma_ep *ep);
int frwr_init_mr(struct rpcrdma_ia *ia, struct rpcrdma_mr *mr);
void frwr_release_mr(struct rpcrdma_mr *mr);
struct rpcrdma_mr **mr);
int frwr_send(struct rpcrdma_ia *ia, struct rpcrdma_req *req);
void frwr_reminv(struct rpcrdma_rep *rep, struct list_head *mrs);
-void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt,
- struct list_head *mrs);
+void frwr_unmap_sync(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req);
+void frwr_unmap_async(struct rpcrdma_xprt *r_xprt, struct rpcrdma_req *req);
/*
* RPC/RDMA protocol calls - xprtrdma/rpc_rdma.c
void rpcrdma_set_max_header_sizes(struct rpcrdma_xprt *);
void rpcrdma_complete_rqst(struct rpcrdma_rep *rep);
void rpcrdma_reply_handler(struct rpcrdma_rep *rep);
-void rpcrdma_release_rqst(struct rpcrdma_xprt *r_xprt,
- struct rpcrdma_req *req);
-void rpcrdma_deferred_completion(struct work_struct *work);
static inline void rpcrdma_set_xdrlen(struct xdr_buf *xdr, size_t len)
{
#if defined(CONFIG_SUNRPC_BACKCHANNEL)
int xprt_rdma_bc_setup(struct rpc_xprt *, unsigned int);
size_t xprt_rdma_bc_maxpayload(struct rpc_xprt *);
+unsigned int xprt_rdma_bc_max_slots(struct rpc_xprt *);
int rpcrdma_bc_post_recv(struct rpcrdma_xprt *, unsigned int);
void rpcrdma_bc_receive_call(struct rpcrdma_xprt *, struct rpcrdma_rep *);
int xprt_rdma_bc_send_reply(struct rpc_rqst *rqst);
req->rq_slen);
/* Protect against races with write_space */
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
/* Don't race with disconnect */
if (xprt_connected(xprt)) {
} else
ret = -ENOTCONN;
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
/* Race breaker in case memory is freed before above code is called */
if (ret == -EAGAIN) {
static void
xs_stream_prepare_request(struct rpc_rqst *req)
{
+ xdr_free_bvec(&req->rq_rcv_buf);
req->rq_task->tk_status = xdr_alloc_bvec(&req->rq_rcv_buf, GFP_KERNEL);
}
struct sock_xprt *transport = container_of(xprt, struct sock_xprt, xprt);
clear_bit(XPRT_SOCK_DATA_READY, &transport->sock_state);
+ clear_bit(XPRT_SOCK_WAKE_ERROR, &transport->sock_state);
+ clear_bit(XPRT_SOCK_WAKE_WRITE, &transport->sock_state);
+ clear_bit(XPRT_SOCK_WAKE_DISCONNECT, &transport->sock_state);
+}
+
+static void xs_run_error_worker(struct sock_xprt *transport, unsigned int nr)
+{
+ set_bit(nr, &transport->sock_state);
+ queue_work(xprtiod_workqueue, &transport->error_worker);
}
static void xs_sock_reset_connection_flags(struct rpc_xprt *xprt)
*/
static void xs_error_report(struct sock *sk)
{
+ struct sock_xprt *transport;
struct rpc_xprt *xprt;
int err;
if (!(xprt = xprt_from_sock(sk)))
goto out;
+ transport = container_of(xprt, struct sock_xprt, xprt);
err = -sk->sk_err;
if (err == 0)
goto out;
dprintk("RPC: xs_error_report client %p, error=%d...\n",
xprt, -err);
trace_rpc_socket_error(xprt, sk->sk_socket, err);
- xprt_wake_pending_tasks(xprt, err);
+ xs_run_error_worker(transport, XPRT_SOCK_WAKE_ERROR);
out:
read_unlock_bh(&sk->sk_callback_lock);
}
cancel_delayed_work_sync(&transport->connect_worker);
xs_close(xprt);
cancel_work_sync(&transport->recv_worker);
+ cancel_work_sync(&transport->error_worker);
xs_xprt_free(xprt);
module_put(THIS_MODULE);
}
}
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt_adjust_cwnd(xprt, task, copied);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
spin_lock(&xprt->queue_lock);
xprt_complete_rqst(task, copied);
__UDPX_INC_STATS(sk, UDP_MIB_INDATAGRAMS);
trace_rpc_socket_state_change(xprt, sk->sk_socket);
switch (sk->sk_state) {
case TCP_ESTABLISHED:
- spin_lock(&xprt->transport_lock);
if (!xprt_test_and_set_connected(xprt)) {
xprt->connect_cookie++;
clear_bit(XPRT_SOCK_CONNECTING, &transport->sock_state);
xprt->stat.connect_count++;
xprt->stat.connect_time += (long)jiffies -
xprt->stat.connect_start;
- xprt_wake_pending_tasks(xprt, -EAGAIN);
+ xs_run_error_worker(transport, XPRT_SOCK_WAKE_PENDING);
}
- spin_unlock(&xprt->transport_lock);
break;
case TCP_FIN_WAIT1:
/* The client initiated a shutdown of the socket */
/* The server initiated a shutdown of the socket */
xprt->connect_cookie++;
clear_bit(XPRT_CONNECTED, &xprt->state);
- xs_tcp_force_close(xprt);
+ xs_run_error_worker(transport, XPRT_SOCK_WAKE_DISCONNECT);
/* fall through */
case TCP_CLOSING:
/*
xprt_clear_connecting(xprt);
clear_bit(XPRT_CLOSING, &xprt->state);
/* Trigger the socket release */
- xs_tcp_force_close(xprt);
+ xs_run_error_worker(transport, XPRT_SOCK_WAKE_DISCONNECT);
}
out:
read_unlock_bh(&sk->sk_callback_lock);
static void xs_write_space(struct sock *sk)
{
struct socket_wq *wq;
+ struct sock_xprt *transport;
struct rpc_xprt *xprt;
if (!sk->sk_socket)
if (unlikely(!(xprt = xprt_from_sock(sk))))
return;
+ transport = container_of(xprt, struct sock_xprt, xprt);
rcu_read_lock();
wq = rcu_dereference(sk->sk_wq);
if (!wq || test_and_clear_bit(SOCKWQ_ASYNC_NOSPACE, &wq->flags) == 0)
goto out;
- if (xprt_write_space(xprt))
- sk->sk_write_pending--;
+ xs_run_error_worker(transport, XPRT_SOCK_WAKE_WRITE);
+ sk->sk_write_pending--;
out:
rcu_read_unlock();
}
*/
static void xs_udp_timer(struct rpc_xprt *xprt, struct rpc_task *task)
{
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
xprt_adjust_cwnd(xprt, task, -ETIMEDOUT);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
static int xs_get_random_port(void)
unsigned int opt_on = 1;
unsigned int timeo;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
keepidle = DIV_ROUND_UP(xprt->timeout->to_initval, HZ);
keepcnt = xprt->timeout->to_retries + 1;
timeo = jiffies_to_msecs(xprt->timeout->to_initval) *
(xprt->timeout->to_retries + 1);
clear_bit(XPRT_SOCK_UPD_TIMEOUT, &transport->sock_state);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
/* TCP Keepalive options */
kernel_setsockopt(sock, SOL_SOCKET, SO_KEEPALIVE,
struct rpc_timeout to;
unsigned long initval;
- spin_lock_bh(&xprt->transport_lock);
+ spin_lock(&xprt->transport_lock);
if (reconnect_timeout < xprt->max_reconnect_timeout)
xprt->max_reconnect_timeout = reconnect_timeout;
if (connect_timeout < xprt->connect_timeout) {
xprt->connect_timeout = connect_timeout;
}
set_bit(XPRT_SOCK_UPD_TIMEOUT, &transport->sock_state);
- spin_unlock_bh(&xprt->transport_lock);
+ spin_unlock(&xprt->transport_lock);
}
static int xs_tcp_finish_connecting(struct rpc_xprt *xprt, struct socket *sock)
xprt_wake_pending_tasks(xprt, status);
}
-static unsigned long xs_reconnect_delay(const struct rpc_xprt *xprt)
-{
- unsigned long start, now = jiffies;
-
- start = xprt->stat.connect_start + xprt->reestablish_timeout;
- if (time_after(start, now))
- return start - now;
- return 0;
-}
-
-static void xs_reconnect_backoff(struct rpc_xprt *xprt)
-{
- xprt->reestablish_timeout <<= 1;
- if (xprt->reestablish_timeout > xprt->max_reconnect_timeout)
- xprt->reestablish_timeout = xprt->max_reconnect_timeout;
- if (xprt->reestablish_timeout < XS_TCP_INIT_REEST_TO)
- xprt->reestablish_timeout = XS_TCP_INIT_REEST_TO;
-}
-
/**
* xs_connect - connect a socket to a remote endpoint
* @xprt: pointer to transport structure
/* Start by resetting any existing state */
xs_reset_transport(transport);
- delay = xs_reconnect_delay(xprt);
- xs_reconnect_backoff(xprt);
+ delay = xprt_reconnect_delay(xprt);
+ xprt_reconnect_backoff(xprt, XS_TCP_INIT_REEST_TO);
} else
dprintk("RPC: xs_connect scheduled xprt %p\n", xprt);
delay);
}
+static void xs_wake_disconnect(struct sock_xprt *transport)
+{
+ if (test_and_clear_bit(XPRT_SOCK_WAKE_DISCONNECT, &transport->sock_state))
+ xs_tcp_force_close(&transport->xprt);
+}
+
+static void xs_wake_write(struct sock_xprt *transport)
+{
+ if (test_and_clear_bit(XPRT_SOCK_WAKE_WRITE, &transport->sock_state))
+ xprt_write_space(&transport->xprt);
+}
+
+static void xs_wake_error(struct sock_xprt *transport)
+{
+ int sockerr;
+ int sockerr_len = sizeof(sockerr);
+
+ if (!test_bit(XPRT_SOCK_WAKE_ERROR, &transport->sock_state))
+ return;
+ mutex_lock(&transport->recv_mutex);
+ if (transport->sock == NULL)
+ goto out;
+ if (!test_and_clear_bit(XPRT_SOCK_WAKE_ERROR, &transport->sock_state))
+ goto out;
+ if (kernel_getsockopt(transport->sock, SOL_SOCKET, SO_ERROR,
+ (char *)&sockerr, &sockerr_len) != 0)
+ goto out;
+ if (sockerr < 0)
+ xprt_wake_pending_tasks(&transport->xprt, sockerr);
+out:
+ mutex_unlock(&transport->recv_mutex);
+}
+
+static void xs_wake_pending(struct sock_xprt *transport)
+{
+ if (test_and_clear_bit(XPRT_SOCK_WAKE_PENDING, &transport->sock_state))
+ xprt_wake_pending_tasks(&transport->xprt, -EAGAIN);
+}
+
+static void xs_error_handle(struct work_struct *work)
+{
+ struct sock_xprt *transport = container_of(work,
+ struct sock_xprt, error_worker);
+
+ xs_wake_disconnect(transport);
+ xs_wake_write(transport);
+ xs_wake_error(transport);
+ xs_wake_pending(transport);
+}
+
/**
* xs_local_print_stats - display AF_LOCAL socket-specifc stats
* @xprt: rpc_xprt struct containing statistics
#ifdef CONFIG_SUNRPC_BACKCHANNEL
.bc_setup = xprt_setup_bc,
.bc_maxpayload = xs_tcp_bc_maxpayload,
+ .bc_num_slots = xprt_bc_max_slots,
.bc_free_rqst = xprt_free_bc_rqst,
.bc_destroy = xprt_destroy_bc,
#endif
xprt->timeout = &xs_local_default_timeout;
INIT_WORK(&transport->recv_worker, xs_stream_data_receive_workfn);
+ INIT_WORK(&transport->error_worker, xs_error_handle);
INIT_DELAYED_WORK(&transport->connect_worker, xs_dummy_setup_socket);
switch (sun->sun_family) {
xprt->timeout = &xs_udp_default_timeout;
INIT_WORK(&transport->recv_worker, xs_udp_data_receive_workfn);
+ INIT_WORK(&transport->error_worker, xs_error_handle);
INIT_DELAYED_WORK(&transport->connect_worker, xs_udp_setup_socket);
switch (addr->sa_family) {
(xprt->timeout->to_retries + 1);
INIT_WORK(&transport->recv_worker, xs_stream_data_receive_workfn);
+ INIT_WORK(&transport->error_worker, xs_error_handle);
INIT_DELAYED_WORK(&transport->connect_worker, xs_tcp_setup_socket);
switch (addr->sa_family) {
__skb_queue_head_init(&xmitq);
/* Ensure message is well-formed before touching the header */
+ TIPC_SKB_CB(skb)->validated = false;
if (unlikely(!tipc_msg_validate(&skb)))
goto discard;
hdr = buf_msg(skb);
#include <linux/sysctl.h>
-static int zero;
-static int one = 1;
static struct ctl_table_header *tipc_ctl_hdr;
static struct ctl_table tipc_table[] = {
.maxlen = sizeof(sysctl_tipc_rmem),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = SYSCTL_ONE,
},
{
.procname = "named_timeout",
.maxlen = sizeof(sysctl_tipc_named_timeout),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
},
{
.procname = "sk_filter",
struct netdev_bpf bpf;
int err = 0;
+ ASSERT_RTNL();
+
force_zc = flags & XDP_ZEROCOPY;
force_copy = flags & XDP_COPY;
if (force_zc && force_copy)
return -EINVAL;
- rtnl_lock();
- if (xdp_get_umem_from_qid(dev, queue_id)) {
- err = -EBUSY;
- goto out_rtnl_unlock;
- }
+ if (xdp_get_umem_from_qid(dev, queue_id))
+ return -EBUSY;
err = xdp_reg_umem_at_qid(dev, umem, queue_id);
if (err)
- goto out_rtnl_unlock;
+ return err;
umem->dev = dev;
umem->queue_id = queue_id;
if (force_copy)
/* For copy-mode, we are done. */
- goto out_rtnl_unlock;
+ return 0;
if (!dev->netdev_ops->ndo_bpf ||
!dev->netdev_ops->ndo_xsk_async_xmit) {
err = dev->netdev_ops->ndo_bpf(dev, &bpf);
if (err)
goto err_unreg_umem;
- rtnl_unlock();
umem->zc = true;
return 0;
err = 0; /* fallback to copy mode */
if (err)
xdp_clear_umem_at_qid(dev, queue_id);
-out_rtnl_unlock:
- rtnl_unlock();
return err;
}
mutex_lock(&xs->mutex);
+ if (xs->queue_id >= xs->dev->real_num_tx_queues)
+ goto out;
+
while (xskq_peek_desc(xs->tx, &desc)) {
char *buffer;
u64 addr;
goto out;
}
- if (xskq_reserve_addr(xs->umem->cq))
- goto out;
-
- if (xs->queue_id >= xs->dev->real_num_tx_queues)
- goto out;
-
len = desc.len;
skb = sock_alloc_send_skb(sk, len, 1, &err);
if (unlikely(!skb)) {
addr = desc.addr;
buffer = xdp_umem_get_data(xs->umem, addr);
err = skb_store_bits(skb, 0, buffer, len);
- if (unlikely(err)) {
+ if (unlikely(err) || xskq_reserve_addr(xs->umem->cq)) {
kfree_skb(skb);
goto out;
}
if (flags & ~(XDP_SHARED_UMEM | XDP_COPY | XDP_ZEROCOPY))
return -EINVAL;
+ rtnl_lock();
mutex_lock(&xs->mutex);
if (xs->state != XSK_READY) {
err = -EBUSY;
xs->state = XSK_BOUND;
out_release:
mutex_unlock(&xs->mutex);
+ rtnl_unlock();
return err;
}
When enabled, this builds both a sample kernel module for
the connector interface and a user space tool to communicate
with it.
- See also Documentation/connector/connector.txt
+ See also Documentation/driver-api/connector.rst
config SAMPLE_HIDRAW
bool "hidraw sample"
$(Q)$(CLANG) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) -I$(obj) \
-I$(srctree)/tools/testing/selftests/bpf/ \
-D__KERNEL__ -D__BPF_TRACING__ -Wno-unused-value -Wno-pointer-sign \
- -D__TARGET_ARCH_$(ARCH) -Wno-compare-distinct-pointer-types \
+ -D__TARGET_ARCH_$(SRCARCH) -Wno-compare-distinct-pointer-types \
-Wno-gnu-variable-sized-type-not-at-end \
-Wno-address-of-packed-member -Wno-tautological-compare \
-Wno-unknown-warning-option $(CLANG_ARCH_ARGS) \
{
int ret = 0;
unsigned long minsz, outsz;
- struct mdev_state *mdev_state;
-
- mdev_state = mdev_get_drvdata(mdev);
switch (cmd) {
case VFIO_DEVICE_GET_INFO:
* Global Structures
*/
-struct mtty_dev {
+static struct mtty_dev {
dev_t vd_devt;
struct class *vd_class;
struct cdev vd_cdev;
};
#if defined(DEBUG_REGS)
-const char *wr_reg[] = {
+static const char *wr_reg[] = {
"TX",
"IER",
"FCR",
"SCR"
};
-const char *rd_reg[] = {
+static const char *rd_reg[] = {
"RX",
"IER",
"IIR",
int nr_ports;
};
-struct mutex mdev_list_lock;
-struct list_head mdev_devices_list;
+static struct mutex mdev_list_lock;
+static struct list_head mdev_devices_list;
static const struct file_operations vd_fops = {
.owner = THIS_MODULE,
return NULL;
}
-void dump_buffer(u8 *buf, uint32_t count)
+static void dump_buffer(u8 *buf, uint32_t count)
{
#if defined(DEBUG)
int i;
return ret;
}
-int mtty_create(struct kobject *kobj, struct mdev_device *mdev)
+static int mtty_create(struct kobject *kobj, struct mdev_device *mdev)
{
struct mdev_state *mdev_state;
char name[MTTY_STRING_LEN];
return 0;
}
-int mtty_remove(struct mdev_device *mdev)
+static int mtty_remove(struct mdev_device *mdev)
{
struct mdev_state *mds, *tmp_mds;
struct mdev_state *mdev_state = mdev_get_drvdata(mdev);
return ret;
}
-int mtty_reset(struct mdev_device *mdev)
+static int mtty_reset(struct mdev_device *mdev)
{
struct mdev_state *mdev_state;
return 0;
}
-ssize_t mtty_read(struct mdev_device *mdev, char __user *buf, size_t count,
- loff_t *ppos)
+static ssize_t mtty_read(struct mdev_device *mdev, char __user *buf,
+ size_t count, loff_t *ppos)
{
unsigned int done = 0;
int ret;
return -EFAULT;
}
-ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
+static ssize_t mtty_write(struct mdev_device *mdev, const char __user *buf,
size_t count, loff_t *ppos)
{
unsigned int done = 0;
return ret;
}
-int mtty_get_region_info(struct mdev_device *mdev,
+static int mtty_get_region_info(struct mdev_device *mdev,
struct vfio_region_info *region_info,
u16 *cap_type_id, void **cap_type)
{
return 0;
}
-int mtty_get_irq_info(struct mdev_device *mdev, struct vfio_irq_info *irq_info)
+static int mtty_get_irq_info(struct mdev_device *mdev,
+ struct vfio_irq_info *irq_info)
{
switch (irq_info->index) {
case VFIO_PCI_INTX_IRQ_INDEX:
return 0;
}
-int mtty_get_device_info(struct mdev_device *mdev,
+static int mtty_get_device_info(struct mdev_device *mdev,
struct vfio_device_info *dev_info)
{
dev_info->flags = VFIO_DEVICE_FLAGS_PCI;
return -ENOTTY;
}
-int mtty_open(struct mdev_device *mdev)
+static int mtty_open(struct mdev_device *mdev)
{
pr_info("%s\n", __func__);
return 0;
}
-void mtty_close(struct mdev_device *mdev)
+static void mtty_close(struct mdev_device *mdev)
{
pr_info("%s\n", __func__);
}
.attrs = mtty_dev_attrs,
};
-const struct attribute_group *mtty_dev_groups[] = {
+static const struct attribute_group *mtty_dev_groups[] = {
&mtty_dev_group,
NULL,
};
.attrs = mdev_dev_attrs,
};
-const struct attribute_group *mdev_dev_groups[] = {
+static const struct attribute_group *mdev_dev_groups[] = {
&mdev_dev_group,
NULL,
};
return -EINVAL;
}
-MDEV_TYPE_ATTR_RO(name);
+static MDEV_TYPE_ATTR_RO(name);
static ssize_t
available_instances_show(struct kobject *kobj, struct device *dev, char *buf)
return sprintf(buf, "%d\n", (MAX_MTTYS - used)/ports);
}
-MDEV_TYPE_ATTR_RO(available_instances);
+static MDEV_TYPE_ATTR_RO(available_instances);
static ssize_t device_api_show(struct kobject *kobj, struct device *dev,
return sprintf(buf, "%s\n", VFIO_DEVICE_API_PCI_STRING);
}
-MDEV_TYPE_ATTR_RO(device_api);
+static MDEV_TYPE_ATTR_RO(device_api);
static struct attribute *mdev_types_attrs[] = {
&mdev_type_attr_name.attr,
.attrs = mdev_types_attrs,
};
-struct attribute_group *mdev_type_groups[] = {
+static struct attribute_group *mdev_type_groups[] = {
&mdev_type_group1,
&mdev_type_group2,
NULL,
"unknown module license " . $extracted_string . "\n" . $herecurr);
}
}
+
+# check for sysctl duplicate constants
+ if ($line =~ /\.extra[12]\s*=\s*&(zero|one|int_max)\b/) {
+ WARN("DUPLICATED_SYSCTL_CONST",
+ "duplicated sysctl range checking value '$1', consider using the shared one in include/linux/sysctl.h\n" . $herecurr);
+ }
}
# If we have no input at all, then there is nothing to report on
/// functions. Values allocated using the devm_functions are freed when
/// the device is detached, and thus the use of the standard freeing
/// function would cause a double free.
-/// See Documentation/driver-model/devres.rst for more information.
+/// See Documentation/driver-api/driver-model/devres.rst for more information.
///
/// A difficulty of detecting this problem is that the standard freeing
/// function might be called from a different function than the one
GCC plugins are loadable modules that provide extra features to the
compiler. They are useful for runtime instrumentation and static analysis.
- See Documentation/gcc-plugins.txt for details.
+ See Documentation/core-api/gcc-plugins.rst for details.
menu "GCC plugins"
depends on GCC_PLUGINS
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) NXP 2019
+
+import gdb
+
+from linux.utils import CachedType
+from linux.utils import container_of
+from linux.lists import list_for_each_entry
+
+
+device_private_type = CachedType('struct device_private')
+device_type = CachedType('struct device')
+
+subsys_private_type = CachedType('struct subsys_private')
+kobject_type = CachedType('struct kobject')
+kset_type = CachedType('struct kset')
+
+bus_type = CachedType('struct bus_type')
+class_type = CachedType('struct class')
+
+
+def dev_name(dev):
+ dev_init_name = dev['init_name']
+ if dev_init_name:
+ return dev_init_name.string()
+ return dev['kobj']['name'].string()
+
+
+def kset_for_each_object(kset):
+ return list_for_each_entry(kset['list'],
+ kobject_type.get_type().pointer(), "entry")
+
+
+def for_each_bus():
+ for kobj in kset_for_each_object(gdb.parse_and_eval('bus_kset')):
+ subsys = container_of(kobj, kset_type.get_type().pointer(), 'kobj')
+ subsys_priv = container_of(subsys, subsys_private_type.get_type().pointer(), 'subsys')
+ yield subsys_priv['bus']
+
+
+def for_each_class():
+ for kobj in kset_for_each_object(gdb.parse_and_eval('class_kset')):
+ subsys = container_of(kobj, kset_type.get_type().pointer(), 'kobj')
+ subsys_priv = container_of(subsys, subsys_private_type.get_type().pointer(), 'subsys')
+ yield subsys_priv['class']
+
+
+def get_bus_by_name(name):
+ for item in for_each_bus():
+ if item['name'].string() == name:
+ return item
+ raise gdb.GdbError("Can't find bus type {!r}".format(name))
+
+
+def get_class_by_name(name):
+ for item in for_each_class():
+ if item['name'].string() == name:
+ return item
+ raise gdb.GdbError("Can't find device class {!r}".format(name))
+
+
+klist_type = CachedType('struct klist')
+klist_node_type = CachedType('struct klist_node')
+
+
+def klist_for_each(klist):
+ return list_for_each_entry(klist['k_list'],
+ klist_node_type.get_type().pointer(), 'n_node')
+
+
+def bus_for_each_device(bus):
+ for kn in klist_for_each(bus['p']['klist_devices']):
+ dp = container_of(kn, device_private_type.get_type().pointer(), 'knode_bus')
+ yield dp['device']
+
+
+def class_for_each_device(cls):
+ for kn in klist_for_each(cls['p']['klist_devices']):
+ dp = container_of(kn, device_private_type.get_type().pointer(), 'knode_class')
+ yield dp['device']
+
+
+def device_for_each_child(dev):
+ for kn in klist_for_each(dev['p']['klist_children']):
+ dp = container_of(kn, device_private_type.get_type().pointer(), 'knode_parent')
+ yield dp['device']
+
+
+def _show_device(dev, level=0, recursive=False):
+ gdb.write('{}dev {}:\t{}\n'.format('\t' * level, dev_name(dev), dev))
+ if recursive:
+ for child in device_for_each_child(dev):
+ _show_device(child, level + 1, recursive)
+
+
+class LxDeviceListBus(gdb.Command):
+ '''Print devices on a bus (or all buses if not specified)'''
+
+ def __init__(self):
+ super(LxDeviceListBus, self).__init__('lx-device-list-bus', gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ if not arg:
+ for bus in for_each_bus():
+ gdb.write('bus {}:\t{}\n'.format(bus['name'].string(), bus))
+ for dev in bus_for_each_device(bus):
+ _show_device(dev, level=1)
+ else:
+ bus = get_bus_by_name(arg)
+ if not bus:
+ raise gdb.GdbError("Can't find bus {!r}".format(arg))
+ for dev in bus_for_each_device(bus):
+ _show_device(dev)
+
+
+class LxDeviceListClass(gdb.Command):
+ '''Print devices in a class (or all classes if not specified)'''
+
+ def __init__(self):
+ super(LxDeviceListClass, self).__init__('lx-device-list-class', gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ if not arg:
+ for cls in for_each_class():
+ gdb.write("class {}:\t{}\n".format(cls['name'].string(), cls))
+ for dev in class_for_each_device(cls):
+ _show_device(dev, level=1)
+ else:
+ cls = get_class_by_name(arg)
+ for dev in class_for_each_device(cls):
+ _show_device(dev)
+
+
+class LxDeviceListTree(gdb.Command):
+ '''Print a device and its children recursively'''
+
+ def __init__(self):
+ super(LxDeviceListTree, self).__init__('lx-device-list-tree', gdb.COMMAND_DATA)
+
+ def invoke(self, arg, from_tty):
+ if not arg:
+ raise gdb.GdbError('Please provide pointer to struct device')
+ dev = gdb.parse_and_eval(arg)
+ if dev.type != device_type.get_type().pointer():
+ raise gdb.GdbError('Please provide pointer to struct device')
+ _show_device(dev, level=0, recursive=True)
+
+
+class LxDeviceFindByBusName(gdb.Function):
+ '''Find struct device by bus and name (both strings)'''
+
+ def __init__(self):
+ super(LxDeviceFindByBusName, self).__init__('lx_device_find_by_bus_name')
+
+ def invoke(self, bus, name):
+ name = name.string()
+ bus = get_bus_by_name(bus.string())
+ for dev in bus_for_each_device(bus):
+ if dev_name(dev) == name:
+ return dev
+
+
+class LxDeviceFindByClassName(gdb.Function):
+ '''Find struct device by class and name (both strings)'''
+
+ def __init__(self):
+ super(LxDeviceFindByClassName, self).__init__('lx_device_find_by_class_name')
+
+ def invoke(self, cls, name):
+ name = name.string()
+ cls = get_class_by_name(cls.string())
+ for dev in class_for_each_device(cls):
+ if dev_name(dev) == name:
+ return dev
+
+
+LxDeviceListBus()
+LxDeviceListClass()
+LxDeviceListTree()
+LxDeviceFindByBusName()
+LxDeviceFindByClassName()
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) NXP 2019
+
+import gdb
+import sys
+
+from linux.utils import CachedType
+from linux.lists import list_for_each_entry
+
+generic_pm_domain_type = CachedType('struct generic_pm_domain')
+pm_domain_data_type = CachedType('struct pm_domain_data')
+device_link_type = CachedType('struct device_link')
+
+
+def kobject_get_path(kobj):
+ path = kobj['name'].string()
+ parent = kobj['parent']
+ if parent:
+ path = kobject_get_path(parent) + '/' + path
+ return path
+
+
+def rtpm_status_str(dev):
+ if dev['power']['runtime_error']:
+ return 'error'
+ if dev['power']['disable_depth']:
+ return 'unsupported'
+ _RPM_STATUS_LOOKUP = [
+ "active",
+ "resuming",
+ "suspended",
+ "suspending"
+ ]
+ return _RPM_STATUS_LOOKUP[dev['power']['runtime_status']]
+
+
+class LxGenPDSummary(gdb.Command):
+ '''Print genpd summary
+
+Output is similar to /sys/kernel/debug/pm_genpd/pm_genpd_summary'''
+
+ def __init__(self):
+ super(LxGenPDSummary, self).__init__('lx-genpd-summary', gdb.COMMAND_DATA)
+
+ def summary_one(self, genpd):
+ if genpd['status'] == 0:
+ status_string = 'on'
+ else:
+ status_string = 'off-{}'.format(genpd['state_idx'])
+
+ slave_names = []
+ for link in list_for_each_entry(
+ genpd['master_links'],
+ device_link_type.get_type().pointer(),
+ 'master_node'):
+ slave_names.apend(link['slave']['name'])
+
+ gdb.write('%-30s %-15s %s\n' % (
+ genpd['name'].string(),
+ status_string,
+ ', '.join(slave_names)))
+
+ # Print devices in domain
+ for pm_data in list_for_each_entry(genpd['dev_list'],
+ pm_domain_data_type.get_type().pointer(),
+ 'list_node'):
+ dev = pm_data['dev']
+ kobj_path = kobject_get_path(dev['kobj'])
+ gdb.write(' %-50s %s\n' % (kobj_path, rtpm_status_str(dev)))
+
+ def invoke(self, arg, from_tty):
+ gdb.write('domain status slaves\n');
+ gdb.write(' /device runtime status\n');
+ gdb.write('----------------------------------------------------------------------\n');
+ for genpd in list_for_each_entry(
+ gdb.parse_and_eval('&gpd_list'),
+ generic_pm_domain_type.get_type().pointer(),
+ 'gpd_list_node'):
+ self.summary_one(genpd)
+
+
+LxGenPDSummary()
import linux.constants
import linux.timerlist
import linux.clk
+ import linux.genpd
+ import linux.device
my $email_maintainer = 1;
my $email_reviewer = 1;
my $email_list = 1;
+my $email_moderated_list = 1;
my $email_subscriber_list = 0;
my $email_git_penguin_chiefs = 0;
my $email_git = 0;
'r!' => \$email_reviewer,
'n!' => \$email_usename,
'l!' => \$email_list,
+ 'moderated!' => \$email_moderated_list,
's!' => \$email_subscriber_list,
'multiline!' => \$output_multiline,
'roles!' => \$output_roles,
--r => include reviewer(s) if any
--n => include name 'Full Name <addr\@domain.tld>'
--l => include list(s) if any
- --s => include subscriber only list(s) if any
+ --moderated => include moderated lists(s) if any (default: true)
+ --s => include subscriber only list(s) if any (default: false)
--remove-duplicates => minimize duplicate email names/addresses
--roles => show roles (status:subsystem, git-signer, list, etc...)
--rolestats => show roles and statistics (commits/total_commits, %)
} else {
if ($email_list) {
if (!$hash_list_to{lc($list_address)}) {
- $hash_list_to{lc($list_address)} = 1;
if ($list_additional =~ m/moderated/) {
- push(@list_to, [$list_address,
- "moderated list${list_role}"]);
+ if ($email_moderated_list) {
+ $hash_list_to{lc($list_address)} = 1;
+ push(@list_to, [$list_address,
+ "moderated list${list_role}"]);
+ }
} else {
+ $hash_list_to{lc($list_address)} = 1;
push(@list_to, [$list_address,
"open list${list_role}"]);
}
See <http://www.intel.com/technology/security/> for more information
about Intel(R) TXT.
See <http://tboot.sourceforge.net> for more information about tboot.
- See Documentation/intel_txt.txt for a description of how to enable
+ See Documentation/x86/intel_txt.rst for a description of how to enable
Intel TXT support in a kernel boot.
If you are unsure as to whether this is required, answer N.
#include <linux/capability.h>
#include <linux/rcupdate.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/poll.h>
#include <uapi/linux/major.h>
#include <uapi/linux/magic.h>
.show_path = aafs_show_path,
};
-static int fill_super(struct super_block *sb, void *data, int silent)
+static int apparmorfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
static struct tree_descr files[] = { {""} };
int error;
return 0;
}
-static struct dentry *aafs_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int apparmorfs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, fill_super);
+ return get_tree_single(fc, apparmorfs_fill_super);
+}
+
+static const struct fs_context_operations apparmorfs_context_ops = {
+ .get_tree = apparmorfs_get_tree,
+};
+
+static int apparmorfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &apparmorfs_context_ops;
+ return 0;
}
static struct file_system_type aafs_ops = {
.owner = THIS_MODULE,
.name = AAFS_NAME,
- .mount = aafs_mount,
+ .init_fs_context = apparmorfs_init_fs_context,
.kill_sb = kill_anon_super,
};
* This is one of the three key functions for hierarchy implementation.
* This function is responsible for re-evaluating all the cgroup's active
* exceptions due to a parent's exception change.
- * Refer to Documentation/cgroup-v1/devices.rst for more details.
+ * Refer to Documentation/admin-guide/cgroup-v1/devices.rst for more details.
*/
static void revalidate_active_exceptions(struct dev_cgroup *devcg)
{
#include <linux/sysfs.h>
#include <linux/kobject.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/mount.h>
#include <linux/pagemap.h>
#include <linux/init.h>
.free_inode = securityfs_free_inode,
};
-static int fill_super(struct super_block *sb, void *data, int silent)
+static int securityfs_fill_super(struct super_block *sb, struct fs_context *fc)
{
static const struct tree_descr files[] = {{""}};
int error;
return 0;
}
-static struct dentry *get_sb(struct file_system_type *fs_type,
- int flags, const char *dev_name,
- void *data)
+static int securityfs_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, fill_super);
+ return get_tree_single(fc, securityfs_fill_super);
+}
+
+static const struct fs_context_operations securityfs_context_ops = {
+ .get_tree = securityfs_get_tree,
+};
+
+static int securityfs_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &securityfs_context_ops;
+ return 0;
}
static struct file_system_type fs_type = {
.owner = THIS_MODULE,
.name = "securityfs",
- .mount = get_sb,
+ .init_fs_context = securityfs_init_fs_context,
.kill_sb = kill_litter_super,
};
#include <linux/sysctl.h>
#include "internal.h"
-static const int zero, one = 1, max = INT_MAX;
-
struct ctl_table key_sysctls[] = {
{
.procname = "maxkeys",
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &one,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ONE,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
{
.procname = "maxbytes",
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &one,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ONE,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
{
.procname = "root_maxkeys",
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &one,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ONE,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
{
.procname = "root_maxbytes",
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &one,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ONE,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
{
.procname = "gc_delay",
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &zero,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ZERO,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
#ifdef CONFIG_PERSISTENT_KEYRINGS
{
.maxlen = sizeof(unsigned),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = (void *) &zero,
- .extra2 = (void *) &max,
+ .extra1 = (void *) SYSCTL_ZERO,
+ .extra2 = (void *) SYSCTL_INT_MAX,
},
#endif
{ }
static DEFINE_SPINLOCK(pinned_root_spinlock);
#ifdef CONFIG_SYSCTL
-static int zero;
-static int one = 1;
static struct ctl_path loadpin_sysctl_path[] = {
{ .procname = "kernel", },
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &zero,
- .extra2 = &one,
+ .extra1 = SYSCTL_ZERO,
+ .extra2 = SYSCTL_ONE,
},
{ }
};
#define pr_fmt(fmt) "SafeSetID: " fmt
-#include <linux/hashtable.h>
#include <linux/lsm_hooks.h>
#include <linux/module.h>
#include <linux/ptrace.h>
#include <linux/sched/task_stack.h>
#include <linux/security.h>
+#include "lsm.h"
/* Flag indicating whether initialization completed */
int safesetid_initialized;
-#define NUM_BITS 8 /* 128 buckets in hash table */
+struct setuid_ruleset __rcu *safesetid_setuid_rules;
-static DEFINE_HASHTABLE(safesetid_whitelist_hashtable, NUM_BITS);
-
-/*
- * Hash table entry to store safesetid policy signifying that 'parent' user
- * can setid to 'child' user.
- */
-struct entry {
- struct hlist_node next;
- struct hlist_node dlist; /* for deletion cleanup */
- uint64_t parent_kuid;
- uint64_t child_kuid;
-};
-
-static DEFINE_SPINLOCK(safesetid_whitelist_hashtable_spinlock);
-
-static bool check_setuid_policy_hashtable_key(kuid_t parent)
+/* Compute a decision for a transition from @src to @dst under @policy. */
+enum sid_policy_type _setuid_policy_lookup(struct setuid_ruleset *policy,
+ kuid_t src, kuid_t dst)
{
- struct entry *entry;
-
- rcu_read_lock();
- hash_for_each_possible_rcu(safesetid_whitelist_hashtable,
- entry, next, __kuid_val(parent)) {
- if (entry->parent_kuid == __kuid_val(parent)) {
- rcu_read_unlock();
- return true;
- }
+ struct setuid_rule *rule;
+ enum sid_policy_type result = SIDPOL_DEFAULT;
+
+ hash_for_each_possible(policy->rules, rule, next, __kuid_val(src)) {
+ if (!uid_eq(rule->src_uid, src))
+ continue;
+ if (uid_eq(rule->dst_uid, dst))
+ return SIDPOL_ALLOWED;
+ result = SIDPOL_CONSTRAINED;
}
- rcu_read_unlock();
-
- return false;
+ return result;
}
-static bool check_setuid_policy_hashtable_key_value(kuid_t parent,
- kuid_t child)
+/*
+ * Compute a decision for a transition from @src to @dst under the active
+ * policy.
+ */
+static enum sid_policy_type setuid_policy_lookup(kuid_t src, kuid_t dst)
{
- struct entry *entry;
+ enum sid_policy_type result = SIDPOL_DEFAULT;
+ struct setuid_ruleset *pol;
rcu_read_lock();
- hash_for_each_possible_rcu(safesetid_whitelist_hashtable,
- entry, next, __kuid_val(parent)) {
- if (entry->parent_kuid == __kuid_val(parent) &&
- entry->child_kuid == __kuid_val(child)) {
- rcu_read_unlock();
- return true;
- }
- }
+ pol = rcu_dereference(safesetid_setuid_rules);
+ if (pol)
+ result = _setuid_policy_lookup(pol, src, dst);
rcu_read_unlock();
-
- return false;
+ return result;
}
static int safesetid_security_capable(const struct cred *cred,
int cap,
unsigned int opts)
{
- if (cap == CAP_SETUID &&
- check_setuid_policy_hashtable_key(cred->uid)) {
- if (!(opts & CAP_OPT_INSETID)) {
- /*
- * Deny if we're not in a set*uid() syscall to avoid
- * giving powers gated by CAP_SETUID that are related
- * to functionality other than calling set*uid() (e.g.
- * allowing user to set up userns uid mappings).
- */
- pr_warn("Operation requires CAP_SETUID, which is not available to UID %u for operations besides approved set*uid transitions",
- __kuid_val(cred->uid));
- return -1;
- }
- }
- return 0;
+ /* We're only interested in CAP_SETUID. */
+ if (cap != CAP_SETUID)
+ return 0;
+
+ /*
+ * If CAP_SETUID is currently used for a set*uid() syscall, we want to
+ * let it go through here; the real security check happens later, in the
+ * task_fix_setuid hook.
+ */
+ if ((opts & CAP_OPT_INSETID) != 0)
+ return 0;
+
+ /*
+ * If no policy applies to this task, allow the use of CAP_SETUID for
+ * other purposes.
+ */
+ if (setuid_policy_lookup(cred->uid, INVALID_UID) == SIDPOL_DEFAULT)
+ return 0;
+
+ /*
+ * Reject use of CAP_SETUID for functionality other than calling
+ * set*uid() (e.g. setting up userns uid mappings).
+ */
+ pr_warn("Operation requires CAP_SETUID, which is not available to UID %u for operations besides approved set*uid transitions\n",
+ __kuid_val(cred->uid));
+ return -EPERM;
}
-static int check_uid_transition(kuid_t parent, kuid_t child)
+/*
+ * Check whether a caller with old credentials @old is allowed to switch to
+ * credentials that contain @new_uid.
+ */
+static bool uid_permitted_for_cred(const struct cred *old, kuid_t new_uid)
{
- if (check_setuid_policy_hashtable_key_value(parent, child))
- return 0;
- pr_warn("UID transition (%d -> %d) blocked",
- __kuid_val(parent),
- __kuid_val(child));
+ bool permitted;
+
+ /* If our old creds already had this UID in it, it's fine. */
+ if (uid_eq(new_uid, old->uid) || uid_eq(new_uid, old->euid) ||
+ uid_eq(new_uid, old->suid))
+ return true;
+
/*
- * Kill this process to avoid potential security vulnerabilities
- * that could arise from a missing whitelist entry preventing a
- * privileged process from dropping to a lesser-privileged one.
+ * Transitions to new UIDs require a check against the policy of the old
+ * RUID.
*/
- force_sig(SIGKILL);
- return -EACCES;
+ permitted =
+ setuid_policy_lookup(old->uid, new_uid) != SIDPOL_CONSTRAINED;
+ if (!permitted) {
+ pr_warn("UID transition ((%d,%d,%d) -> %d) blocked\n",
+ __kuid_val(old->uid), __kuid_val(old->euid),
+ __kuid_val(old->suid), __kuid_val(new_uid));
+ }
+ return permitted;
}
/*
int flags)
{
- /* Do nothing if there are no setuid restrictions for this UID. */
- if (!check_setuid_policy_hashtable_key(old->uid))
+ /* Do nothing if there are no setuid restrictions for our old RUID. */
+ if (setuid_policy_lookup(old->uid, INVALID_UID) == SIDPOL_DEFAULT)
return 0;
- switch (flags) {
- case LSM_SETID_RE:
- /*
- * Users for which setuid restrictions exist can only set the
- * real UID to the real UID or the effective UID, unless an
- * explicit whitelist policy allows the transition.
- */
- if (!uid_eq(old->uid, new->uid) &&
- !uid_eq(old->euid, new->uid)) {
- return check_uid_transition(old->uid, new->uid);
- }
- /*
- * Users for which setuid restrictions exist can only set the
- * effective UID to the real UID, the effective UID, or the
- * saved set-UID, unless an explicit whitelist policy allows
- * the transition.
- */
- if (!uid_eq(old->uid, new->euid) &&
- !uid_eq(old->euid, new->euid) &&
- !uid_eq(old->suid, new->euid)) {
- return check_uid_transition(old->euid, new->euid);
- }
- break;
- case LSM_SETID_ID:
- /*
- * Users for which setuid restrictions exist cannot change the
- * real UID or saved set-UID unless an explicit whitelist
- * policy allows the transition.
- */
- if (!uid_eq(old->uid, new->uid))
- return check_uid_transition(old->uid, new->uid);
- if (!uid_eq(old->suid, new->suid))
- return check_uid_transition(old->suid, new->suid);
- break;
- case LSM_SETID_RES:
- /*
- * Users for which setuid restrictions exist cannot change the
- * real UID, effective UID, or saved set-UID to anything but
- * one of: the current real UID, the current effective UID or
- * the current saved set-user-ID unless an explicit whitelist
- * policy allows the transition.
- */
- if (!uid_eq(new->uid, old->uid) &&
- !uid_eq(new->uid, old->euid) &&
- !uid_eq(new->uid, old->suid)) {
- return check_uid_transition(old->uid, new->uid);
- }
- if (!uid_eq(new->euid, old->uid) &&
- !uid_eq(new->euid, old->euid) &&
- !uid_eq(new->euid, old->suid)) {
- return check_uid_transition(old->euid, new->euid);
- }
- if (!uid_eq(new->suid, old->uid) &&
- !uid_eq(new->suid, old->euid) &&
- !uid_eq(new->suid, old->suid)) {
- return check_uid_transition(old->suid, new->suid);
- }
- break;
- case LSM_SETID_FS:
- /*
- * Users for which setuid restrictions exist cannot change the
- * filesystem UID to anything but one of: the current real UID,
- * the current effective UID or the current saved set-UID
- * unless an explicit whitelist policy allows the transition.
- */
- if (!uid_eq(new->fsuid, old->uid) &&
- !uid_eq(new->fsuid, old->euid) &&
- !uid_eq(new->fsuid, old->suid) &&
- !uid_eq(new->fsuid, old->fsuid)) {
- return check_uid_transition(old->fsuid, new->fsuid);
- }
- break;
- default:
- pr_warn("Unknown setid state %d\n", flags);
- force_sig(SIGKILL);
- return -EINVAL;
- }
- return 0;
-}
-
-int add_safesetid_whitelist_entry(kuid_t parent, kuid_t child)
-{
- struct entry *new;
-
- /* Return if entry already exists */
- if (check_setuid_policy_hashtable_key_value(parent, child))
+ if (uid_permitted_for_cred(old, new->uid) &&
+ uid_permitted_for_cred(old, new->euid) &&
+ uid_permitted_for_cred(old, new->suid) &&
+ uid_permitted_for_cred(old, new->fsuid))
return 0;
- new = kzalloc(sizeof(struct entry), GFP_KERNEL);
- if (!new)
- return -ENOMEM;
- new->parent_kuid = __kuid_val(parent);
- new->child_kuid = __kuid_val(child);
- spin_lock(&safesetid_whitelist_hashtable_spinlock);
- hash_add_rcu(safesetid_whitelist_hashtable,
- &new->next,
- __kuid_val(parent));
- spin_unlock(&safesetid_whitelist_hashtable_spinlock);
- return 0;
-}
-
-void flush_safesetid_whitelist_entries(void)
-{
- struct entry *entry;
- struct hlist_node *hlist_node;
- unsigned int bkt_loop_cursor;
- HLIST_HEAD(free_list);
-
/*
- * Could probably use hash_for_each_rcu here instead, but this should
- * be fine as well.
+ * Kill this process to avoid potential security vulnerabilities
+ * that could arise from a missing whitelist entry preventing a
+ * privileged process from dropping to a lesser-privileged one.
*/
- spin_lock(&safesetid_whitelist_hashtable_spinlock);
- hash_for_each_safe(safesetid_whitelist_hashtable, bkt_loop_cursor,
- hlist_node, entry, next) {
- hash_del_rcu(&entry->next);
- hlist_add_head(&entry->dlist, &free_list);
- }
- spin_unlock(&safesetid_whitelist_hashtable_spinlock);
- synchronize_rcu();
- hlist_for_each_entry_safe(entry, hlist_node, &free_list, dlist) {
- hlist_del(&entry->dlist);
- kfree(entry);
- }
+ force_sig(SIGKILL);
+ return -EACCES;
}
static struct security_hook_list safesetid_security_hooks[] = {
#define _SAFESETID_H
#include <linux/types.h>
+#include <linux/uidgid.h>
+#include <linux/hashtable.h>
/* Flag indicating whether initialization completed */
extern int safesetid_initialized;
-/* Function type. */
-enum safesetid_whitelist_file_write_type {
- SAFESETID_WHITELIST_ADD, /* Add whitelist policy. */
- SAFESETID_WHITELIST_FLUSH, /* Flush whitelist policies. */
+enum sid_policy_type {
+ SIDPOL_DEFAULT, /* source ID is unaffected by policy */
+ SIDPOL_CONSTRAINED, /* source ID is affected by policy */
+ SIDPOL_ALLOWED /* target ID explicitly allowed */
};
-/* Add entry to safesetid whitelist to allow 'parent' to setid to 'child'. */
-int add_safesetid_whitelist_entry(kuid_t parent, kuid_t child);
+/*
+ * Hash table entry to store safesetid policy signifying that 'src_uid'
+ * can setuid to 'dst_uid'.
+ */
+struct setuid_rule {
+ struct hlist_node next;
+ kuid_t src_uid;
+ kuid_t dst_uid;
+};
+
+#define SETID_HASH_BITS 8 /* 256 buckets in hash table */
+
+struct setuid_ruleset {
+ DECLARE_HASHTABLE(rules, SETID_HASH_BITS);
+ char *policy_str;
+ struct rcu_head rcu;
+};
+
+enum sid_policy_type _setuid_policy_lookup(struct setuid_ruleset *policy,
+ kuid_t src, kuid_t dst);
-void flush_safesetid_whitelist_entries(void);
+extern struct setuid_ruleset __rcu *safesetid_setuid_rules;
#endif /* _SAFESETID_H */
* published by the Free Software Foundation.
*
*/
+
+#define pr_fmt(fmt) "SafeSetID: " fmt
+
#include <linux/security.h>
#include <linux/cred.h>
#include "lsm.h"
-static struct dentry *safesetid_policy_dir;
-
-struct safesetid_file_entry {
- const char *name;
- enum safesetid_whitelist_file_write_type type;
- struct dentry *dentry;
-};
-
-static struct safesetid_file_entry safesetid_files[] = {
- {.name = "add_whitelist_policy",
- .type = SAFESETID_WHITELIST_ADD},
- {.name = "flush_whitelist_policies",
- .type = SAFESETID_WHITELIST_FLUSH},
-};
+static DEFINE_MUTEX(policy_update_lock);
/*
* In the case the input buffer contains one or more invalid UIDs, the kuid_t
- * variables pointed to by 'parent' and 'child' will get updated but this
+ * variables pointed to by @parent and @child will get updated but this
* function will return an error.
+ * Contents of @buf may be modified.
*/
-static int parse_safesetid_whitelist_policy(const char __user *buf,
- size_t len,
- kuid_t *parent,
- kuid_t *child)
+static int parse_policy_line(struct file *file, char *buf,
+ struct setuid_rule *rule)
{
- char *kern_buf;
- char *parent_buf;
- char *child_buf;
- const char separator[] = ":";
+ char *child_str;
int ret;
- size_t first_substring_length;
- long parsed_parent;
- long parsed_child;
+ u32 parsed_parent, parsed_child;
- /* Duplicate string from user memory and NULL-terminate */
- kern_buf = memdup_user_nul(buf, len);
- if (IS_ERR(kern_buf))
- return PTR_ERR(kern_buf);
+ /* Format of |buf| string should be <UID>:<UID>. */
+ child_str = strchr(buf, ':');
+ if (child_str == NULL)
+ return -EINVAL;
+ *child_str = '\0';
+ child_str++;
- /*
- * Format of |buf| string should be <UID>:<UID>.
- * Find location of ":" in kern_buf (copied from |buf|).
- */
- first_substring_length = strcspn(kern_buf, separator);
- if (first_substring_length == 0 || first_substring_length == len) {
- ret = -EINVAL;
- goto free_kern;
- }
+ ret = kstrtou32(buf, 0, &parsed_parent);
+ if (ret)
+ return ret;
+
+ ret = kstrtou32(child_str, 0, &parsed_child);
+ if (ret)
+ return ret;
- parent_buf = kmemdup_nul(kern_buf, first_substring_length, GFP_KERNEL);
- if (!parent_buf) {
- ret = -ENOMEM;
- goto free_kern;
+ rule->src_uid = make_kuid(file->f_cred->user_ns, parsed_parent);
+ rule->dst_uid = make_kuid(file->f_cred->user_ns, parsed_child);
+ if (!uid_valid(rule->src_uid) || !uid_valid(rule->dst_uid))
+ return -EINVAL;
+
+ return 0;
+}
+
+static void __release_ruleset(struct rcu_head *rcu)
+{
+ struct setuid_ruleset *pol =
+ container_of(rcu, struct setuid_ruleset, rcu);
+ int bucket;
+ struct setuid_rule *rule;
+ struct hlist_node *tmp;
+
+ hash_for_each_safe(pol->rules, bucket, tmp, rule, next)
+ kfree(rule);
+ kfree(pol->policy_str);
+ kfree(pol);
+}
+
+static void release_ruleset(struct setuid_ruleset *pol)
+{
+ call_rcu(&pol->rcu, __release_ruleset);
+}
+
+static void insert_rule(struct setuid_ruleset *pol, struct setuid_rule *rule)
+{
+ hash_add(pol->rules, &rule->next, __kuid_val(rule->src_uid));
+}
+
+static int verify_ruleset(struct setuid_ruleset *pol)
+{
+ int bucket;
+ struct setuid_rule *rule, *nrule;
+ int res = 0;
+
+ hash_for_each(pol->rules, bucket, rule, next) {
+ if (_setuid_policy_lookup(pol, rule->dst_uid, INVALID_UID) ==
+ SIDPOL_DEFAULT) {
+ pr_warn("insecure policy detected: uid %d is constrained but transitively unconstrained through uid %d\n",
+ __kuid_val(rule->src_uid),
+ __kuid_val(rule->dst_uid));
+ res = -EINVAL;
+
+ /* fix it up */
+ nrule = kmalloc(sizeof(struct setuid_rule), GFP_KERNEL);
+ if (!nrule)
+ return -ENOMEM;
+ nrule->src_uid = rule->dst_uid;
+ nrule->dst_uid = rule->dst_uid;
+ insert_rule(pol, nrule);
+ }
}
+ return res;
+}
- ret = kstrtol(parent_buf, 0, &parsed_parent);
- if (ret)
- goto free_both;
+static ssize_t handle_policy_update(struct file *file,
+ const char __user *ubuf, size_t len)
+{
+ struct setuid_ruleset *pol;
+ char *buf, *p, *end;
+ int err;
- child_buf = kern_buf + first_substring_length + 1;
- ret = kstrtol(child_buf, 0, &parsed_child);
- if (ret)
- goto free_both;
+ pol = kmalloc(sizeof(struct setuid_ruleset), GFP_KERNEL);
+ if (!pol)
+ return -ENOMEM;
+ pol->policy_str = NULL;
+ hash_init(pol->rules);
- *parent = make_kuid(current_user_ns(), parsed_parent);
- if (!uid_valid(*parent)) {
- ret = -EINVAL;
- goto free_both;
+ p = buf = memdup_user_nul(ubuf, len);
+ if (IS_ERR(buf)) {
+ err = PTR_ERR(buf);
+ goto out_free_pol;
}
+ pol->policy_str = kstrdup(buf, GFP_KERNEL);
+ if (pol->policy_str == NULL) {
+ err = -ENOMEM;
+ goto out_free_buf;
+ }
+
+ /* policy lines, including the last one, end with \n */
+ while (*p != '\0') {
+ struct setuid_rule *rule;
+
+ end = strchr(p, '\n');
+ if (end == NULL) {
+ err = -EINVAL;
+ goto out_free_buf;
+ }
+ *end = '\0';
+
+ rule = kmalloc(sizeof(struct setuid_rule), GFP_KERNEL);
+ if (!rule) {
+ err = -ENOMEM;
+ goto out_free_buf;
+ }
- *child = make_kuid(current_user_ns(), parsed_child);
- if (!uid_valid(*child)) {
- ret = -EINVAL;
- goto free_both;
+ err = parse_policy_line(file, p, rule);
+ if (err)
+ goto out_free_rule;
+
+ if (_setuid_policy_lookup(pol, rule->src_uid, rule->dst_uid) ==
+ SIDPOL_ALLOWED) {
+ pr_warn("bad policy: duplicate entry\n");
+ err = -EEXIST;
+ goto out_free_rule;
+ }
+
+ insert_rule(pol, rule);
+ p = end + 1;
+ continue;
+
+out_free_rule:
+ kfree(rule);
+ goto out_free_buf;
}
-free_both:
- kfree(parent_buf);
-free_kern:
- kfree(kern_buf);
- return ret;
+ err = verify_ruleset(pol);
+ /* bogus policy falls through after fixing it up */
+ if (err && err != -EINVAL)
+ goto out_free_buf;
+
+ /*
+ * Everything looks good, apply the policy and release the old one.
+ * What we really want here is an xchg() wrapper for RCU, but since that
+ * doesn't currently exist, just use a spinlock for now.
+ */
+ mutex_lock(&policy_update_lock);
+ rcu_swap_protected(safesetid_setuid_rules, pol,
+ lockdep_is_held(&policy_update_lock));
+ mutex_unlock(&policy_update_lock);
+ err = len;
+
+out_free_buf:
+ kfree(buf);
+out_free_pol:
+ release_ruleset(pol);
+ return err;
}
static ssize_t safesetid_file_write(struct file *file,
size_t len,
loff_t *ppos)
{
- struct safesetid_file_entry *file_entry =
- file->f_inode->i_private;
- kuid_t parent;
- kuid_t child;
- int ret;
-
- if (!ns_capable(current_user_ns(), CAP_MAC_ADMIN))
+ if (!file_ns_capable(file, &init_user_ns, CAP_MAC_ADMIN))
return -EPERM;
if (*ppos != 0)
return -EINVAL;
- switch (file_entry->type) {
- case SAFESETID_WHITELIST_FLUSH:
- flush_safesetid_whitelist_entries();
- break;
- case SAFESETID_WHITELIST_ADD:
- ret = parse_safesetid_whitelist_policy(buf, len, &parent,
- &child);
- if (ret)
- return ret;
-
- ret = add_safesetid_whitelist_entry(parent, child);
- if (ret)
- return ret;
- break;
- default:
- pr_warn("Unknown securityfs file %d\n", file_entry->type);
- break;
- }
-
- /* Return len on success so caller won't keep trying to write */
- return len;
+ return handle_policy_update(file, buf, len);
}
-static const struct file_operations safesetid_file_fops = {
- .write = safesetid_file_write,
-};
-
-static void safesetid_shutdown_securityfs(void)
+static ssize_t safesetid_file_read(struct file *file, char __user *buf,
+ size_t len, loff_t *ppos)
{
- int i;
+ ssize_t res = 0;
+ struct setuid_ruleset *pol;
+ const char *kbuf;
- for (i = 0; i < ARRAY_SIZE(safesetid_files); ++i) {
- struct safesetid_file_entry *entry =
- &safesetid_files[i];
- securityfs_remove(entry->dentry);
- entry->dentry = NULL;
+ mutex_lock(&policy_update_lock);
+ pol = rcu_dereference_protected(safesetid_setuid_rules,
+ lockdep_is_held(&policy_update_lock));
+ if (pol) {
+ kbuf = pol->policy_str;
+ res = simple_read_from_buffer(buf, len, ppos,
+ kbuf, strlen(kbuf));
}
-
- securityfs_remove(safesetid_policy_dir);
- safesetid_policy_dir = NULL;
+ mutex_unlock(&policy_update_lock);
+ return res;
}
+static const struct file_operations safesetid_file_fops = {
+ .read = safesetid_file_read,
+ .write = safesetid_file_write,
+};
+
static int __init safesetid_init_securityfs(void)
{
- int i;
int ret;
+ struct dentry *policy_dir;
+ struct dentry *policy_file;
if (!safesetid_initialized)
return 0;
- safesetid_policy_dir = securityfs_create_dir("safesetid", NULL);
- if (IS_ERR(safesetid_policy_dir)) {
- ret = PTR_ERR(safesetid_policy_dir);
+ policy_dir = securityfs_create_dir("safesetid", NULL);
+ if (IS_ERR(policy_dir)) {
+ ret = PTR_ERR(policy_dir);
goto error;
}
- for (i = 0; i < ARRAY_SIZE(safesetid_files); ++i) {
- struct safesetid_file_entry *entry =
- &safesetid_files[i];
- entry->dentry = securityfs_create_file(
- entry->name, 0200, safesetid_policy_dir,
- entry, &safesetid_file_fops);
- if (IS_ERR(entry->dentry)) {
- ret = PTR_ERR(entry->dentry);
- goto error;
- }
+ policy_file = securityfs_create_file("whitelist_policy", 0600,
+ policy_dir, NULL, &safesetid_file_fops);
+ if (IS_ERR(policy_file)) {
+ ret = PTR_ERR(policy_file);
+ goto error;
}
return 0;
error:
- safesetid_shutdown_securityfs();
+ securityfs_remove(policy_dir);
return ret;
}
fs_initcall(safesetid_init_securityfs);
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
+#include <linux/fs_context.h>
#include <linux/mount.h>
#include <linux/mutex.h>
#include <linux/init.h>
#define NULL_FILE_NAME "null"
-static int sel_fill_super(struct super_block *sb, void *data, int silent)
+static int sel_fill_super(struct super_block *sb, struct fs_context *fc)
{
struct selinux_fs_info *fsi;
int ret;
return ret;
}
-static struct dentry *sel_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int sel_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, sel_fill_super);
+ return get_tree_single(fc, sel_fill_super);
+}
+
+static const struct fs_context_operations sel_context_ops = {
+ .get_tree = sel_get_tree,
+};
+
+static int sel_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &sel_context_ops;
+ return 0;
}
static void sel_kill_sb(struct super_block *sb)
static struct file_system_type sel_fs_type = {
.name = "selinuxfs",
- .mount = sel_mount,
+ .init_fs_context = sel_init_fs_context,
.kill_sb = sel_kill_sb,
};
#include <linux/ctype.h>
#include <linux/audit.h>
#include <linux/magic.h>
+#include <linux/fs_context.h>
#include "smack.h"
#define BEBITS (sizeof(__be32) * 8)
/**
* smk_fill_super - fill the smackfs superblock
* @sb: the empty superblock
- * @data: unused
- * @silent: unused
+ * @fc: unused
*
* Fill in the well known entries for the smack filesystem
*
* Returns 0 on success, an error code on failure
*/
-static int smk_fill_super(struct super_block *sb, void *data, int silent)
+static int smk_fill_super(struct super_block *sb, struct fs_context *fc)
{
int rc;
}
/**
- * smk_mount - get the smackfs superblock
- * @fs_type: passed along without comment
- * @flags: passed along without comment
- * @dev_name: passed along without comment
- * @data: passed along without comment
+ * smk_get_tree - get the smackfs superblock
+ * @fc: The mount context, including any options
*
* Just passes everything along.
*
* Returns what the lower level code does.
*/
-static struct dentry *smk_mount(struct file_system_type *fs_type,
- int flags, const char *dev_name, void *data)
+static int smk_get_tree(struct fs_context *fc)
{
- return mount_single(fs_type, flags, data, smk_fill_super);
+ return get_tree_single(fc, smk_fill_super);
+}
+
+static const struct fs_context_operations smk_context_ops = {
+ .get_tree = smk_get_tree,
+};
+
+/**
+ * smk_init_fs_context - Initialise a filesystem context for smackfs
+ * @fc: The blank mount context
+ */
+static int smk_init_fs_context(struct fs_context *fc)
+{
+ fc->ops = &smk_context_ops;
+ return 0;
}
static struct file_system_type smk_fs_type = {
.name = "smackfs",
- .mount = smk_mount,
+ .init_fs_context = smk_init_fs_context,
.kill_sb = kill_litter_super,
};
return proc_dointvec_minmax(&table_copy, write, buffer, lenp, ppos);
}
-static int zero;
static int max_scope = YAMA_SCOPE_NO_ATTACH;
static struct ctl_path yama_sysctl_path[] = {
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = yama_dointvec_minmax,
- .extra1 = &zero,
+ .extra1 = SYSCTL_ZERO,
.extra2 = &max_scope,
},
{ }
{
struct snd_seq_client *client = file->private_data;
int written = 0, len;
- int err;
+ int err, handled;
struct snd_seq_event event;
if (!(snd_seq_file_flags(file) & SNDRV_SEQ_LFLG_OUTPUT))
if (!client->accept_output || client->pool == NULL)
return -ENXIO;
+ repeat:
+ handled = 0;
/* allocate the pool now if the pool is not allocated yet */
mutex_lock(&client->ioctl_mutex);
if (client->pool->size > 0 && !snd_seq_write_pool_allocated(client)) {
0, 0, &client->ioctl_mutex);
if (err < 0)
break;
+ handled++;
__skip_event:
/* Update pointers and counts */
count -= len;
buf += len;
written += len;
+
+ /* let's have a coffee break if too many events are queued */
+ if (++handled >= 200) {
+ mutex_unlock(&client->ioctl_mutex);
+ goto repeat;
+ }
}
out:
struct snd_ctl_elem_value *ucontrol)
{
a3dsrc_t *a = kcontrol->private_data;
- int changed = 1, i;
+ int i;
int coord[6];
for (i = 0; i < 6; i++)
coord[i] = ucontrol->value.integer.value[i];
vortex_a3d_coord2hrtf(a->hrtf[1], coord);
a3dsrc_SetHrtfTarget(a, a->hrtf[0], a->hrtf[1]);
a3dsrc_SetHrtfCurrent(a, a->hrtf[0], a->hrtf[1]);
- return changed;
+ return 1;
}
static int
{
a3dsrc_t *a = kcontrol->private_data;
int coord[6];
- int i, changed = 1;
+ int i;
for (i = 0; i < 6; i++)
coord[i] = ucontrol->value.integer.value[i];
/* Translate orientation coordinates to a3d params. */
a3dsrc_SetItdTarget(a, a->itd[0], a->itd[1]);
a3dsrc_SetItdCurrent(a, a->itd[0], a->itd[1]);
a3dsrc_SetItdDline(a, a->dline);
- return changed;
+ return 1;
}
static int
struct snd_ctl_elem_value *ucontrol)
{
a3dsrc_t *a = kcontrol->private_data;
- int changed = 1;
int l, r;
/* There may be some scale tranlation needed here. */
l = ucontrol->value.integer.value[0];
/* Left Right panning. */
a3dsrc_SetGainTarget(a, l, r);
a3dsrc_SetGainCurrent(a, l, r);
- return changed;
+ return 1;
}
static int
struct snd_ctl_elem_value *ucontrol)
{
a3dsrc_t *a = kcontrol->private_data;
- int i, changed = 1;
+ int i;
int params[6];
for (i = 0; i < 6; i++)
params[i] = ucontrol->value.integer.value[i];
a3dsrc_SetAtmosCurrent(a, a->filter[0],
a->filter[1], a->filter[2],
a->filter[3], a->filter[4]);
- return changed;
+ return 1;
}
static const struct snd_kcontrol_new vortex_a3d_kcontrol = {
{
struct emu10k1x *emu = snd_kcontrol_chip(kcontrol);
unsigned int val;
- int change = 0;
val = ucontrol->value.integer.value[0] ;
snd_emu10k1x_ptr_write(emu, ROUTING, 0, 0x1003F);
snd_emu10k1x_gpio_write(emu, 0x1080);
}
- return change;
+ return 0;
}
static const struct snd_kcontrol_new snd_emu10k1x_shared_spdif =
#ifdef CONFIG_PM_SLEEP
static int hda_codec_force_resume(struct device *dev)
{
+ struct hda_codec *codec = dev_to_hda_codec(dev);
+ bool forced_resume = !codec->relaxed_resume;
int ret;
/* The get/put pair below enforces the runtime resume even if the
* device hasn't been used at suspend time. This trick is needed to
* update the jack state change during the sleep.
*/
- pm_runtime_get_noresume(dev);
+ if (forced_resume)
+ pm_runtime_get_noresume(dev);
ret = pm_runtime_force_resume(dev);
- pm_runtime_put(dev);
+ if (forced_resume)
+ pm_runtime_put(dev);
return ret;
}
struct hdmi_spec *spec = codec->spec;
int pin_idx, pcm_idx;
- if (codec_has_acomp(codec))
+ if (codec_has_acomp(codec)) {
snd_hdac_acomp_register_notifier(&codec->bus->core, NULL);
+ codec->relaxed_resume = 0;
+ }
for (pin_idx = 0; pin_idx < spec->num_pins; pin_idx++) {
struct hdmi_spec_per_pin *per_pin = get_pin(spec, pin_idx);
}
#define INTEL_GET_VENDOR_VERB 0xf81
-#define INTEL_GET_VENDOR_VERB 0xf81
#define INTEL_SET_VENDOR_VERB 0x781
#define INTEL_EN_DP12 0x02 /* enable DP 1.2 features */
#define INTEL_EN_ALL_PIN_CVTS 0x01 /* enable 2nd & 3rd pins and convertors */
return -1;
}
+static int intel_port2pin(struct hda_codec *codec, int port)
+{
+ struct hdmi_spec *spec = codec->spec;
+
+ if (!spec->port_num) {
+ /* we assume only from port-B to port-D */
+ if (port < 1 || port > 3)
+ return 0;
+ /* intel port is 1-based */
+ return port + intel_base_nid(codec) - 1;
+ }
+
+ if (port < 1 || port > spec->port_num)
+ return 0;
+ return spec->port_map[port - 1];
+}
+
static void intel_pin_eld_notify(void *audio_ptr, int port, int pipe)
{
struct hda_codec *codec = audio_ptr;
int pin_nid;
int dev_id = pipe;
- /* we assume only from port-B to port-D */
- if (port < 1 || port > 3)
+ pin_nid = intel_port2pin(codec, port);
+ if (!pin_nid)
return;
-
- pin_nid = port + intel_base_nid(codec) - 1; /* intel port is 1-based */
-
/* skip notification during system suspend (but not in runtime PM);
* the state will be updated at resume
*/
spec->drm_audio_ops.pin_eld_notify = intel_pin_eld_notify;
snd_hdac_acomp_register_notifier(&codec->bus->core,
&spec->drm_audio_ops);
+ /* no need for forcible resume for jack check thanks to notifier */
+ codec->relaxed_resume = 1;
}
/* setup_stream ops override for HSW+ */
{0x12, 0x90a60130},
{0x17, 0x90170110},
{0x21, 0x03211020}),
- SND_HDA_PIN_QUIRK(0x10ec0295, 0x1028, "Dell", ALC269_FIXUP_DELL1_MIC_NO_PRESENCE,
+ SND_HDA_PIN_QUIRK(0x10ec0295, 0x1028, "Dell", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE,
{0x14, 0x90170110},
{0x21, 0x04211020}),
+ SND_HDA_PIN_QUIRK(0x10ec0295, 0x1028, "Dell", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE,
+ {0x14, 0x90170110},
+ {0x21, 0x04211030}),
SND_HDA_PIN_QUIRK(0x10ec0295, 0x1028, "Dell", ALC269_FIXUP_DELL1_MIC_NO_PRESENCE,
ALC295_STANDARD_PINS,
{0x17, 0x21014020},
{0x18, 0x01a19030},
{0x1a, 0x01813040},
{0x21, 0x01014020}),
+ SND_HDA_PIN_QUIRK(0x10ec0867, 0x1028, "Dell", ALC891_FIXUP_DELL_MIC_NO_PRESENCE,
+ {0x16, 0x01813030},
+ {0x17, 0x02211010},
+ {0x18, 0x01a19040},
+ {0x21, 0x01014020}),
SND_HDA_PIN_QUIRK(0x10ec0662, 0x1028, "Dell", ALC662_FIXUP_DELL_MIC_NO_PRESENCE,
{0x14, 0x01014010},
{0x18, 0x01a19020},
static int lx_pcm_close(struct snd_pcm_substream *substream)
{
- int err = 0;
dev_dbg(substream->pcm->card->dev, "->lx_pcm_close\n");
- return err;
+ return 0;
}
static snd_pcm_uframes_t lx_pcm_stream_pointer(struct snd_pcm_substream
{
struct snd_rme9652 *rme9652 = snd_pcm_substream_chip(substream);
unsigned long flags;
- int result = 0;
spin_lock_irqsave(&rme9652->lock, flags);
if (!rme9652->running)
rme9652_reset_hw_pointer(rme9652);
spin_unlock_irqrestore(&rme9652->lock, flags);
- return result;
+ return 0;
}
static const struct snd_pcm_hardware snd_rme9652_playback_subinfo =
int cmd)
{
struct snd_ps3_card_info *card = snd_pcm_substream_chip(substream);
- int ret = 0;
switch (cmd) {
case SNDRV_PCM_TRIGGER_START:
}
- return ret;
+ return 0;
};
/*
* q6asm_write_async() - non blocking write
*
* @ac: audio client pointer
- * @len: lenght in bytes
+ * @len: length in bytes
* @msw_ts: timestamp msw
* @lsw_ts: timestamp lsw
* @wflags: flags associated with write
[BPF_PROG_TYPE_SK_REUSEPORT] = "sk_reuseport",
[BPF_PROG_TYPE_FLOW_DISSECTOR] = "flow_dissector",
[BPF_PROG_TYPE_CGROUP_SYSCTL] = "cgroup_sysctl",
+ [BPF_PROG_TYPE_RAW_TRACEPOINT_WRITABLE] = "raw_tracepoint_writable",
[BPF_PROG_TYPE_CGROUP_SOCKOPT] = "cgroup_sockopt",
};
* based on a user-provided identifier for all traffic coming from
* the tasks belonging to the related cgroup. See also the related
* kernel documentation, available from the Linux sources in file
- * *Documentation/cgroup-v1/net_cls.rst*.
+ * *Documentation/admin-guide/cgroup-v1/net_cls.rst*.
*
* The Linux kernel has two versions for cgroups: there are
* cgroups v1 and cgroups v2. Both are available to users, who can
__u32 user_ip4; /* Allows 1,2,4-byte read and 4-byte write.
* Stored in network byte order.
*/
- __u32 user_ip6[4]; /* Allows 1,2,4-byte read and 4,8-byte write.
+ __u32 user_ip6[4]; /* Allows 1,2,4,8-byte read and 4,8-byte write.
* Stored in network byte order.
*/
__u32 user_port; /* Allows 4-byte read and write.
__u32 msg_src_ip4; /* Allows 1,2,4-byte read and 4-byte write.
* Stored in network byte order.
*/
- __u32 msg_src_ip6[4]; /* Allows 1,2,4-byte read and 4,8-byte write.
+ __u32 msg_src_ip6[4]; /* Allows 1,2,4,8-byte read and 4,8-byte write.
* Stored in network byte order.
*/
__bpf_md_ptr(struct bpf_sock *, sk);
}
attr.size = sizeof(attr);
attr.type = type;
- attr.config1 = (uint64_t)(void *)name; /* kprobe_func or uprobe_path */
- attr.config2 = offset; /* kprobe_addr or probe_offset */
+ attr.config1 = ptr_to_u64(name); /* kprobe_func or uprobe_path */
+ attr.config2 = offset; /* kprobe_addr or probe_offset */
/* pid filter is meaningful only for uprobes */
pfd = syscall(__NR_perf_event_open, &attr,
err = -errno;
goto out_socket;
}
- strncpy(xsk->ifname, ifname, IFNAMSIZ);
+ strncpy(xsk->ifname, ifname, IFNAMSIZ - 1);
+ xsk->ifname[IFNAMSIZ - 1] = '\0';
err = xsk_set_xdp_socket_config(&xsk->config, usr_config);
if (err)
--------------
Each probe argument follows below syntax.
- [NAME=]LOCALVAR|$retval|%REG|@SYMBOL[:TYPE]
+ [NAME=]LOCALVAR|$retval|%REG|@SYMBOL[:TYPE][@user]
'NAME' specifies the name of this argument (optional). You can use the name of local variable, local data structure member (e.g. var->field, var.field2), local array with fixed index (e.g. array[1], var->array[0], var->pointer[2]), or kprobe-tracer argument format (e.g. $retval, %ax, etc). Note that the name of this argument will be set as the last member name if you specify a local data structure member (e.g. field2 for 'var->field1.field2'.)
'$vars' and '$params' special arguments are also available for NAME, '$vars' is expanded to the local variables (including function parameters) which can access at given probe point. '$params' is expanded to only the function parameters.
'TYPE' casts the type of this argument (optional). If omitted, perf probe automatically set the type based on debuginfo (*). Currently, basic types (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal integers (x/x8/x16/x32/x64), signedness casting (u/s), "string" and bitfield are supported. (see TYPES for detail)
On x86 systems %REG is always the short form of the register: for example %AX. %RAX or %EAX is not valid.
+"@user" is a special attribute which means the LOCALVAR will be treated as a user-space memory. This is only valid for kprobe event.
TYPES
-----
str = tmp + 1;
}
+ tmp = strchr(str, '@');
+ if (tmp && tmp != str && strcmp(tmp + 1, "user")) { /* user attr */
+ if (!user_access_is_supported()) {
+ semantic_error("ftrace does not support user access\n");
+ return -EINVAL;
+ }
+ *tmp = '\0';
+ arg->user_access = true;
+ pr_debug("user_access ");
+ }
+
tmp = strchr(str, ':');
if (tmp) { /* Type setting */
*tmp = '\0';
struct probe_trace_arg_ref {
struct probe_trace_arg_ref *next; /* Next reference */
long offset; /* Offset value */
+ bool user_access; /* User-memory access */
};
/* kprobe-tracer and uprobe-tracer tracing argument */
char *var; /* Variable name */
char *type; /* Type name */
struct perf_probe_arg_field *field; /* Structure fields */
+ bool user_access; /* User-memory access */
};
/* Perf probe probing event (point + arg) */
FTRACE_README_PROBE_TYPE_X = 0,
FTRACE_README_KRETPROBE_OFFSET,
FTRACE_README_UPROBE_REF_CTR,
+ FTRACE_README_USER_ACCESS,
FTRACE_README_END,
};
DEFINE_TYPE(FTRACE_README_PROBE_TYPE_X, "*type: * x8/16/32/64,*"),
DEFINE_TYPE(FTRACE_README_KRETPROBE_OFFSET, "*place (kretprobe): *"),
DEFINE_TYPE(FTRACE_README_UPROBE_REF_CTR, "*ref_ctr_offset*"),
+ DEFINE_TYPE(FTRACE_README_USER_ACCESS, "*[u]<offset>*"),
};
static bool scan_ftrace_readme(enum ftrace_readme type)
{
return scan_ftrace_readme(FTRACE_README_UPROBE_REF_CTR);
}
+
+bool user_access_is_supported(void)
+{
+ return scan_ftrace_readme(FTRACE_README_USER_ACCESS);
+}
bool probe_type_is_available(enum probe_type type);
bool kretprobe_offset_is_supported(void);
bool uprobe_ref_ctr_is_supported(void);
+bool user_access_is_supported(void);
#else /* ! HAVE_LIBELF_SUPPORT */
static inline struct probe_cache *probe_cache__new(const char *tgt __maybe_unused, struct nsinfo *nsi __maybe_unused)
{
static int convert_variable_type(Dwarf_Die *vr_die,
struct probe_trace_arg *tvar,
- const char *cast)
+ const char *cast, bool user_access)
{
struct probe_trace_arg_ref **ref_ptr = &tvar->ref;
Dwarf_Die type;
pr_debug("%s type is %s.\n",
dwarf_diename(vr_die), dwarf_diename(&type));
- if (cast && strcmp(cast, "string") == 0) { /* String type */
+ if (cast && (!strcmp(cast, "string") || !strcmp(cast, "ustring"))) {
+ /* String type */
ret = dwarf_tag(&type);
if (ret != DW_TAG_pointer_type &&
ret != DW_TAG_array_type) {
pr_warning("Out of memory error\n");
return -ENOMEM;
}
+ (*ref_ptr)->user_access = user_access;
}
if (!die_compare_name(&type, "char") &&
!die_compare_name(&type, "unsigned char")) {
static int convert_variable_fields(Dwarf_Die *vr_die, const char *varname,
struct perf_probe_arg_field *field,
struct probe_trace_arg_ref **ref_ptr,
- Dwarf_Die *die_mem)
+ Dwarf_Die *die_mem, bool user_access)
{
struct probe_trace_arg_ref *ref = *ref_ptr;
Dwarf_Die type;
*ref_ptr = ref;
}
ref->offset += dwarf_bytesize(&type) * field->index;
+ ref->user_access = user_access;
goto next;
} else if (tag == DW_TAG_pointer_type) {
/* Check the pointer and dereference */
}
}
ref->offset += (long)offs;
+ ref->user_access = user_access;
/* If this member is unnamed, we need to reuse this field */
if (!dwarf_diename(die_mem))
return convert_variable_fields(die_mem, varname, field,
- &ref, die_mem);
+ &ref, die_mem, user_access);
next:
/* Converting next field */
if (field->next)
return convert_variable_fields(die_mem, field->name,
- field->next, &ref, die_mem);
+ field->next, &ref, die_mem, user_access);
else
return 0;
}
else if (ret == 0 && pf->pvar->field) {
ret = convert_variable_fields(vr_die, pf->pvar->var,
pf->pvar->field, &pf->tvar->ref,
- &die_mem);
+ &die_mem, pf->pvar->user_access);
vr_die = &die_mem;
}
if (ret == 0)
- ret = convert_variable_type(vr_die, pf->tvar, pf->pvar->type);
+ ret = convert_variable_type(vr_die, pf->tvar, pf->pvar->type,
+ pf->pvar->user_access);
/* *expr will be cached in libdw. Don't free it. */
return ret;
}
# SPDX-License-Identifier: GPL-2.0
+include ../../../../scripts/Kbuild.include
+include ../../../scripts/Makefile.arch
LIBDIR := ../../../lib
BPFDIR := $(LIBDIR)/bpf
$(OUTPUT)/urandom_read: $(OUTPUT)/%: %.c
$(CC) -o $@ $< -Wl,--build-id
-$(OUTPUT)/test_maps: map_tests/*.c
+$(OUTPUT)/test_stub.o: test_stub.c
+ $(CC) $(TEST_PROGS_CFLAGS) $(CFLAGS) -c -o $@ $<
BPFOBJ := $(OUTPUT)/libbpf.a
-$(TEST_GEN_PROGS): test_stub.o $(BPFOBJ)
+$(TEST_GEN_PROGS): $(OUTPUT)/test_stub.o $(BPFOBJ)
-$(TEST_GEN_PROGS_EXTENDED): test_stub.o $(OUTPUT)/libbpf.a
+$(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/test_stub.o $(OUTPUT)/libbpf.a
$(OUTPUT)/test_dev_cgroup: cgroup_helpers.c
$(OUTPUT)/test_skb_cgroup_id_user: cgroup_helpers.c
CLANG_FLAGS = -I. -I./include/uapi -I../../../include/uapi \
$(CLANG_SYS_INCLUDES) \
- -Wno-compare-distinct-pointer-types
+ -Wno-compare-distinct-pointer-types \
+ -D__TARGET_ARCH_$(SRCARCH)
$(OUTPUT)/test_l4lb_noinline.o: CLANG_FLAGS += -fno-inline
$(OUTPUT)/test_xdp_noinline.o: CLANG_FLAGS += -fno-inline
endif
TEST_PROGS_CFLAGS := -I. -I$(OUTPUT)
+TEST_MAPS_CFLAGS := -I. -I$(OUTPUT)
TEST_VERIFIER_CFLAGS := -I. -I$(OUTPUT) -Iverifier
ifneq ($(SUBREG_CODEGEN),)
$(ALU32_BUILD_DIR):
mkdir -p $@
-$(ALU32_BUILD_DIR)/urandom_read: $(OUTPUT)/urandom_read
+$(ALU32_BUILD_DIR)/urandom_read: $(OUTPUT)/urandom_read | $(ALU32_BUILD_DIR)
cp $< $@
$(ALU32_BUILD_DIR)/test_progs_32: test_progs.c $(OUTPUT)/libbpf.a\
- $(ALU32_BUILD_DIR) \
- $(ALU32_BUILD_DIR)/urandom_read
+ $(ALU32_BUILD_DIR)/urandom_read \
+ | $(ALU32_BUILD_DIR)
$(CC) $(TEST_PROGS_CFLAGS) $(CFLAGS) \
-o $(ALU32_BUILD_DIR)/test_progs_32 \
test_progs.c test_stub.c trace_helpers.c prog_tests/*.c \
$(ALU32_BUILD_DIR)/test_progs_32: $(PROG_TESTS_H)
$(ALU32_BUILD_DIR)/test_progs_32: prog_tests/*.c
-$(ALU32_BUILD_DIR)/%.o: progs/%.c $(ALU32_BUILD_DIR) \
- $(ALU32_BUILD_DIR)/test_progs_32
- $(CLANG) $(CLANG_FLAGS) \
- -O2 -target bpf -emit-llvm -c $< -o - | \
+$(ALU32_BUILD_DIR)/%.o: progs/%.c $(ALU32_BUILD_DIR)/test_progs_32 \
+ | $(ALU32_BUILD_DIR)
+ ($(CLANG) $(CLANG_FLAGS) -O2 -target bpf -emit-llvm -c $< -o - || \
+ echo "clang failed") | \
$(LLC) -march=bpf -mattr=+alu32 -mcpu=$(CPU) $(LLC_FLAGS) \
-filetype=obj -o $@
ifeq ($(DWARF2BTF),y)
# Have one program compiled without "-target bpf" to test whether libbpf loads
# it successfully
$(OUTPUT)/test_xdp.o: progs/test_xdp.c
- $(CLANG) $(CLANG_FLAGS) \
- -O2 -emit-llvm -c $< -o - | \
+ ($(CLANG) $(CLANG_FLAGS) -O2 -emit-llvm -c $< -o - || \
+ echo "clang failed") | \
$(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@
ifeq ($(DWARF2BTF),y)
$(BTF_PAHOLE) -J $@
endif
$(OUTPUT)/%.o: progs/%.c
- $(CLANG) $(CLANG_FLAGS) \
- -O2 -target bpf -emit-llvm -c $< -o - | \
+ ($(CLANG) $(CLANG_FLAGS) -O2 -target bpf -emit-llvm -c $< -o - || \
+ echo "clang failed") | \
$(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@
ifeq ($(DWARF2BTF),y)
$(BTF_PAHOLE) -J $@
endif
-PROG_TESTS_H := $(OUTPUT)/prog_tests/tests.h
-test_progs.c: $(PROG_TESTS_H)
-$(OUTPUT)/test_progs: CFLAGS += $(TEST_PROGS_CFLAGS)
-$(OUTPUT)/test_progs: prog_tests/*.c
-
PROG_TESTS_DIR = $(OUTPUT)/prog_tests
$(PROG_TESTS_DIR):
mkdir -p $@
-
+PROG_TESTS_H := $(PROG_TESTS_DIR)/tests.h
PROG_TESTS_FILES := $(wildcard prog_tests/*.c)
-$(PROG_TESTS_H): $(PROG_TESTS_DIR) $(PROG_TESTS_FILES)
+test_progs.c: $(PROG_TESTS_H)
+$(OUTPUT)/test_progs: CFLAGS += $(TEST_PROGS_CFLAGS)
+$(OUTPUT)/test_progs: test_progs.c $(PROG_TESTS_H) $(PROG_TESTS_FILES)
+$(PROG_TESTS_H): $(PROG_TESTS_FILES) | $(PROG_TESTS_DIR)
$(shell ( cd prog_tests/; \
echo '/* Generated header, do not edit */'; \
echo '#ifdef DECLARE'; \
echo '#endif' \
) > $(PROG_TESTS_H))
-TEST_MAPS_CFLAGS := -I. -I$(OUTPUT)
MAP_TESTS_DIR = $(OUTPUT)/map_tests
$(MAP_TESTS_DIR):
mkdir -p $@
MAP_TESTS_H := $(MAP_TESTS_DIR)/tests.h
+MAP_TESTS_FILES := $(wildcard map_tests/*.c)
test_maps.c: $(MAP_TESTS_H)
$(OUTPUT)/test_maps: CFLAGS += $(TEST_MAPS_CFLAGS)
-MAP_TESTS_FILES := $(wildcard map_tests/*.c)
-$(MAP_TESTS_H): $(MAP_TESTS_DIR) $(MAP_TESTS_FILES)
+$(OUTPUT)/test_maps: test_maps.c $(MAP_TESTS_H) $(MAP_TESTS_FILES)
+$(MAP_TESTS_H): $(MAP_TESTS_FILES) | $(MAP_TESTS_DIR)
$(shell ( cd map_tests/; \
echo '/* Generated header, do not edit */'; \
echo '#ifdef DECLARE'; \
echo '#endif' \
) > $(MAP_TESTS_H))
-VERIFIER_TESTS_H := $(OUTPUT)/verifier/tests.h
-test_verifier.c: $(VERIFIER_TESTS_H)
-$(OUTPUT)/test_verifier: CFLAGS += $(TEST_VERIFIER_CFLAGS)
-
VERIFIER_TESTS_DIR = $(OUTPUT)/verifier
$(VERIFIER_TESTS_DIR):
mkdir -p $@
-
+VERIFIER_TESTS_H := $(VERIFIER_TESTS_DIR)/tests.h
VERIFIER_TEST_FILES := $(wildcard verifier/*.c)
-$(OUTPUT)/verifier/tests.h: $(VERIFIER_TESTS_DIR) $(VERIFIER_TEST_FILES)
+test_verifier.c: $(VERIFIER_TESTS_H)
+$(OUTPUT)/test_verifier: CFLAGS += $(TEST_VERIFIER_CFLAGS)
+$(OUTPUT)/test_verifier: test_verifier.c $(VERIFIER_TESTS_H)
+$(VERIFIER_TESTS_H): $(VERIFIER_TEST_FILES) | $(VERIFIER_TESTS_DIR)
$(shell ( cd verifier/; \
echo '/* Generated header, do not edit */'; \
echo '#ifdef FILL_ARRAY'; \
#if defined(__TARGET_ARCH_x86)
#define bpf_target_x86
#define bpf_target_defined
-#elif defined(__TARGET_ARCH_s930x)
- #define bpf_target_s930x
+#elif defined(__TARGET_ARCH_s390)
+ #define bpf_target_s390
#define bpf_target_defined
#elif defined(__TARGET_ARCH_arm)
#define bpf_target_arm
#ifndef bpf_target_defined
#if defined(__x86_64__)
#define bpf_target_x86
-#elif defined(__s390x__)
- #define bpf_target_s930x
+#elif defined(__s390__)
+ #define bpf_target_s390
#elif defined(__arm__)
#define bpf_target_arm
#elif defined(__aarch64__)
#if defined(bpf_target_x86)
+#ifdef __KERNEL__
#define PT_REGS_PARM1(x) ((x)->di)
#define PT_REGS_PARM2(x) ((x)->si)
#define PT_REGS_PARM3(x) ((x)->dx)
#define PT_REGS_RC(x) ((x)->ax)
#define PT_REGS_SP(x) ((x)->sp)
#define PT_REGS_IP(x) ((x)->ip)
+#else
+#ifdef __i386__
+/* i386 kernel is built with -mregparm=3 */
+#define PT_REGS_PARM1(x) ((x)->eax)
+#define PT_REGS_PARM2(x) ((x)->edx)
+#define PT_REGS_PARM3(x) ((x)->ecx)
+#define PT_REGS_PARM4(x) 0
+#define PT_REGS_PARM5(x) 0
+#define PT_REGS_RET(x) ((x)->esp)
+#define PT_REGS_FP(x) ((x)->ebp)
+#define PT_REGS_RC(x) ((x)->eax)
+#define PT_REGS_SP(x) ((x)->esp)
+#define PT_REGS_IP(x) ((x)->eip)
+#else
+#define PT_REGS_PARM1(x) ((x)->rdi)
+#define PT_REGS_PARM2(x) ((x)->rsi)
+#define PT_REGS_PARM3(x) ((x)->rdx)
+#define PT_REGS_PARM4(x) ((x)->rcx)
+#define PT_REGS_PARM5(x) ((x)->r8)
+#define PT_REGS_RET(x) ((x)->rsp)
+#define PT_REGS_FP(x) ((x)->rbp)
+#define PT_REGS_RC(x) ((x)->rax)
+#define PT_REGS_SP(x) ((x)->rsp)
+#define PT_REGS_IP(x) ((x)->rip)
+#endif
+#endif
-#elif defined(bpf_target_s390x)
+#elif defined(bpf_target_s390)
-#define PT_REGS_PARM1(x) ((x)->gprs[2])
-#define PT_REGS_PARM2(x) ((x)->gprs[3])
-#define PT_REGS_PARM3(x) ((x)->gprs[4])
-#define PT_REGS_PARM4(x) ((x)->gprs[5])
-#define PT_REGS_PARM5(x) ((x)->gprs[6])
-#define PT_REGS_RET(x) ((x)->gprs[14])
-#define PT_REGS_FP(x) ((x)->gprs[11]) /* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_RC(x) ((x)->gprs[2])
-#define PT_REGS_SP(x) ((x)->gprs[15])
-#define PT_REGS_IP(x) ((x)->psw.addr)
+/* s390 provides user_pt_regs instead of struct pt_regs to userspace */
+struct pt_regs;
+#define PT_REGS_S390 const volatile user_pt_regs
+#define PT_REGS_PARM1(x) (((PT_REGS_S390 *)(x))->gprs[2])
+#define PT_REGS_PARM2(x) (((PT_REGS_S390 *)(x))->gprs[3])
+#define PT_REGS_PARM3(x) (((PT_REGS_S390 *)(x))->gprs[4])
+#define PT_REGS_PARM4(x) (((PT_REGS_S390 *)(x))->gprs[5])
+#define PT_REGS_PARM5(x) (((PT_REGS_S390 *)(x))->gprs[6])
+#define PT_REGS_RET(x) (((PT_REGS_S390 *)(x))->gprs[14])
+/* Works only with CONFIG_FRAME_POINTER */
+#define PT_REGS_FP(x) (((PT_REGS_S390 *)(x))->gprs[11])
+#define PT_REGS_RC(x) (((PT_REGS_S390 *)(x))->gprs[2])
+#define PT_REGS_SP(x) (((PT_REGS_S390 *)(x))->gprs[15])
+#define PT_REGS_IP(x) (((PT_REGS_S390 *)(x))->psw.addr)
#elif defined(bpf_target_arm)
#elif defined(bpf_target_arm64)
-#define PT_REGS_PARM1(x) ((x)->regs[0])
-#define PT_REGS_PARM2(x) ((x)->regs[1])
-#define PT_REGS_PARM3(x) ((x)->regs[2])
-#define PT_REGS_PARM4(x) ((x)->regs[3])
-#define PT_REGS_PARM5(x) ((x)->regs[4])
-#define PT_REGS_RET(x) ((x)->regs[30])
-#define PT_REGS_FP(x) ((x)->regs[29]) /* Works only with CONFIG_FRAME_POINTER */
-#define PT_REGS_RC(x) ((x)->regs[0])
-#define PT_REGS_SP(x) ((x)->sp)
-#define PT_REGS_IP(x) ((x)->pc)
+/* arm64 provides struct user_pt_regs instead of struct pt_regs to userspace */
+struct pt_regs;
+#define PT_REGS_ARM64 const volatile struct user_pt_regs
+#define PT_REGS_PARM1(x) (((PT_REGS_ARM64 *)(x))->regs[0])
+#define PT_REGS_PARM2(x) (((PT_REGS_ARM64 *)(x))->regs[1])
+#define PT_REGS_PARM3(x) (((PT_REGS_ARM64 *)(x))->regs[2])
+#define PT_REGS_PARM4(x) (((PT_REGS_ARM64 *)(x))->regs[3])
+#define PT_REGS_PARM5(x) (((PT_REGS_ARM64 *)(x))->regs[4])
+#define PT_REGS_RET(x) (((PT_REGS_ARM64 *)(x))->regs[30])
+/* Works only with CONFIG_FRAME_POINTER */
+#define PT_REGS_FP(x) (((PT_REGS_ARM64 *)(x))->regs[29])
+#define PT_REGS_RC(x) (((PT_REGS_ARM64 *)(x))->regs[0])
+#define PT_REGS_SP(x) (((PT_REGS_ARM64 *)(x))->sp)
+#define PT_REGS_IP(x) (((PT_REGS_ARM64 *)(x))->pc)
#elif defined(bpf_target_mips)
#endif
-#ifdef bpf_target_powerpc
+#if defined(bpf_target_powerpc)
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ (ip) = (ctx)->link; })
#define BPF_KRETPROBE_READ_RET_IP BPF_KPROBE_READ_RET_IP
-#elif bpf_target_sparc
+#elif defined(bpf_target_sparc)
#define BPF_KPROBE_READ_RET_IP(ip, ctx) ({ (ip) = PT_REGS_RET(ctx); })
#define BPF_KRETPROBE_READ_RET_IP BPF_KPROBE_READ_RET_IP
#else
return -EINVAL;
}
-#ifdef __x86_64__
-#define SYS_KPROBE_NAME "__x64_sys_nanosleep"
-#else
-#define SYS_KPROBE_NAME "sys_nanosleep"
-#endif
-
void test_attach_probe(void)
{
const char *kprobe_name = "kprobe/sys_nanosleep";
kprobe_link = bpf_program__attach_kprobe(kprobe_prog,
false /* retprobe */,
- SYS_KPROBE_NAME);
+ SYS_NANOSLEEP_KPROBE_NAME);
if (CHECK(IS_ERR(kprobe_link), "attach_kprobe",
"err %ld\n", PTR_ERR(kprobe_link))) {
kprobe_link = NULL;
}
kretprobe_link = bpf_program__attach_kprobe(kretprobe_prog,
true /* retprobe */,
- SYS_KPROBE_NAME);
+ SYS_NANOSLEEP_KPROBE_NAME);
if (CHECK(IS_ERR(kretprobe_link), "attach_kretprobe",
"err %ld\n", PTR_ERR(kretprobe_link))) {
kretprobe_link = NULL;
#include <sys/socket.h>
#include <test_progs.h>
-#ifdef __x86_64__
-#define SYS_KPROBE_NAME "__x64_sys_nanosleep"
-#else
-#define SYS_KPROBE_NAME "sys_nanosleep"
-#endif
-
static void on_sample(void *ctx, int cpu, void *data, __u32 size)
{
int cpu_data = *(int *)data, duration = 0;
/* attach kprobe */
link = bpf_program__attach_kprobe(prog, false /* retprobe */,
- SYS_KPROBE_NAME);
+ SYS_NANOSLEEP_KPROBE_NAME);
if (CHECK(IS_ERR(link), "attach_kprobe", "err %ld\n", PTR_ERR(link)))
goto out_close;
return test_send_signal_common(&attr, BPF_PROG_TYPE_TRACEPOINT, "tracepoint");
}
+static int test_send_signal_perf(void)
+{
+ struct perf_event_attr attr = {
+ .sample_period = 1,
+ .type = PERF_TYPE_SOFTWARE,
+ .config = PERF_COUNT_SW_CPU_CLOCK,
+ };
+
+ return test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
+ "perf_sw_event");
+}
+
static int test_send_signal_nmi(void)
{
struct perf_event_attr attr = {
.type = PERF_TYPE_HARDWARE,
.config = PERF_COUNT_HW_CPU_CYCLES,
};
+ int pmu_fd;
+
+ /* Some setups (e.g. virtual machines) might run with hardware
+ * perf events disabled. If this is the case, skip this test.
+ */
+ pmu_fd = syscall(__NR_perf_event_open, &attr, 0 /* pid */,
+ -1 /* cpu */, -1 /* group_fd */, 0 /* flags */);
+ if (pmu_fd == -1) {
+ if (errno == ENOENT) {
+ printf("%s:SKIP:no PERF_COUNT_HW_CPU_CYCLES\n",
+ __func__);
+ return 0;
+ }
+ /* Let the test fail with a more informative message */
+ } else {
+ close(pmu_fd);
+ }
- return test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT, "perf_event");
+ return test_send_signal_common(&attr, BPF_PROG_TYPE_PERF_EVENT,
+ "perf_hw_event");
}
void test_send_signal(void)
int ret = 0;
ret |= test_send_signal_tracepoint();
+ ret |= test_send_signal_perf();
ret |= test_send_signal_nmi();
if (!ret)
printf("test_send_signal:OK\n");
for (j = 0; j < 300; j++)
for (i = 0; i < j; i++) {
if (j & 1)
- m = ctx->rax;
+ m = PT_REGS_RC(ctx);
else
m = j;
sum += i * m;
int i = 0;
while (true) {
- if (ctx->rax & 1)
+ if (PT_REGS_RC(ctx) & 1)
i += 3;
else
i += 7;
__u64 i = 0, sum = 0;
do {
i++;
- sum += ctx->rax;
+ sum += PT_REGS_RC(ctx);
} while (i < 0x100000000ULL);
return sum;
}
* issue and avoid complicated C programming massaging.
* This is an acceptable workaround since there is one entry here.
*/
+typedef __u64 raw_stack_trace_t[2 * MAX_STACK_RAWTP];
struct {
__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
- __u64 (*value)[2 * MAX_STACK_RAWTP];
+ __type(value, raw_stack_trace_t);
} rawdata_map SEC(".maps");
SEC("tracepoint/raw_syscalls/sys_enter")
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 128);
__type(key, __u32);
- /* there seems to be a bug in kernel not handling typedef properly */
- struct bpf_stack_build_id (*value)[PERF_MAX_STACK_DEPTH];
+ __type(value, stack_trace_t);
} stack_amap SEC(".maps");
/* taken from /sys/kernel/debug/tracing/events/random/urandom_read/format */
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 16384);
__type(key, __u32);
- __u64 (*value)[PERF_MAX_STACK_DEPTH];
+ __type(value, stack_trace_t);
} stack_amap SEC(".maps");
/* taken from /sys/kernel/debug/tracing/events/sched/sched_switch/format */
#include <linux/tcp.h>
#include <linux/udp.h>
#include "bpf_helpers.h"
+#include "bpf_endian.h"
static __u32 rol32(__u32 word, unsigned int shift)
{
ip6h->nexthdr = IPPROTO_IPV6;
ip_suffix = pckt->flow.srcv6[3] ^ pckt->flow.port16[0];
ip6h->payload_len =
- __builtin_bswap16(pkt_bytes + sizeof(struct ipv6hdr));
+ bpf_htons(pkt_bytes + sizeof(struct ipv6hdr));
ip6h->hop_limit = 4;
ip6h->saddr.in6_u.u6_addr32[0] = 1;
struct real_definition *dst, __u32 pkt_bytes)
{
- __u32 ip_suffix = __builtin_bswap16(pckt->flow.port16[0]);
+ __u32 ip_suffix = bpf_ntohs(pckt->flow.port16[0]);
struct eth_hdr *new_eth;
struct eth_hdr *old_eth;
__u16 *next_iph_u16;
iph->protocol = IPPROTO_IPIP;
iph->check = 0;
iph->tos = 1;
- iph->tot_len = __builtin_bswap16(pkt_bytes + sizeof(struct iphdr));
+ iph->tot_len = bpf_htons(pkt_bytes + sizeof(struct iphdr));
/* don't update iph->daddr, since it will overwrite old eth_proto
* and multiple iterations of bpf_prog_run() will fail
*/
iph_len = sizeof(struct ipv6hdr);
*protocol = ip6h->nexthdr;
pckt->flow.proto = *protocol;
- *pkt_bytes = __builtin_bswap16(ip6h->payload_len);
+ *pkt_bytes = bpf_ntohs(ip6h->payload_len);
off += iph_len;
if (*protocol == 45) {
return XDP_DROP;
return XDP_DROP;
*protocol = iph->protocol;
pckt->flow.proto = *protocol;
- *pkt_bytes = __builtin_bswap16(iph->tot_len);
+ *pkt_bytes = bpf_ntohs(iph->tot_len);
off += 20;
if (iph->frag_off & 65343)
return XDP_DROP;
nh_off = sizeof(struct eth_hdr);
if (data + nh_off > data_end)
return XDP_DROP;
- eth_proto = eth->eth_proto;
- if (eth_proto == 8)
+ eth_proto = bpf_ntohs(eth->eth_proto);
+ if (eth_proto == ETH_P_IP)
return process_packet(data, nh_off, data_end, 0, ctx);
- else if (eth_proto == 56710)
+ else if (eth_proto == ETH_P_IPV6)
return process_packet(data, nh_off, data_end, 1, ctx);
else
return XDP_DROP;
.value_type_id = 1,
.max_entries = 4,
},
+/*
+ * typedef int arr_t[16];
+ * struct s {
+ * arr_t *a;
+ * };
+ */
+{
+ .descr = "struct->ptr->typedef->array->int size resolution",
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_TBD, 1, 8), /* [1] */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 0),
+ BTF_PTR_ENC(3), /* [2] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4), /* [3] */
+ BTF_TYPE_ARRAY_ENC(5, 5, 16), /* [4] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0s\0a\0arr_t"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "ptr_mod_chain_size_resolve_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int) * 16,
+ .key_type_id = 5 /* int */,
+ .value_type_id = 3 /* arr_t */,
+ .max_entries = 4,
+},
+/*
+ * typedef int arr_t[16][8][4];
+ * struct s {
+ * arr_t *a;
+ * };
+ */
+{
+ .descr = "struct->ptr->typedef->multi-array->int size resolution",
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_TBD, 1, 8), /* [1] */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 0),
+ BTF_PTR_ENC(3), /* [2] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4), /* [3] */
+ BTF_TYPE_ARRAY_ENC(5, 7, 16), /* [4] */
+ BTF_TYPE_ARRAY_ENC(6, 7, 8), /* [5] */
+ BTF_TYPE_ARRAY_ENC(7, 7, 4), /* [6] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [7] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0s\0a\0arr_t"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "multi_arr_size_resolve_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int) * 16 * 8 * 4,
+ .key_type_id = 7 /* int */,
+ .value_type_id = 3 /* arr_t */,
+ .max_entries = 4,
+},
+/*
+ * typedef int int_t;
+ * typedef int_t arr3_t[4];
+ * typedef arr3_t arr2_t[8];
+ * typedef arr2_t arr1_t[16];
+ * struct s {
+ * arr1_t *a;
+ * };
+ */
+{
+ .descr = "typedef/multi-arr mix size resolution",
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_TBD, 1, 8), /* [1] */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 0),
+ BTF_PTR_ENC(3), /* [2] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4), /* [3] */
+ BTF_TYPE_ARRAY_ENC(5, 10, 16), /* [4] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 6), /* [5] */
+ BTF_TYPE_ARRAY_ENC(7, 10, 8), /* [6] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 8), /* [7] */
+ BTF_TYPE_ARRAY_ENC(9, 10, 4), /* [8] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 10), /* [9] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [10] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0s\0a\0arr1_t\0arr2_t\0arr3_t\0int_t"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "typedef_arra_mix_size_resolve_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int) * 16 * 8 * 4,
+ .key_type_id = 10 /* int */,
+ .value_type_id = 3 /* arr_t */,
+ .max_entries = 4,
+},
}; /* struct btf_raw_test raw_tests[] */
int compare_stack_ips(int smap_fd, int amap_fd, int stack_trace_len);
int extract_build_id(char *build_id, size_t size);
void *spin_lock_thread(void *arg);
+
+#ifdef __x86_64__
+#define SYS_NANOSLEEP_KPROBE_NAME "__x64_sys_nanosleep"
+#elif defined(__s390x__)
+#define SYS_NANOSLEEP_KPROBE_NAME "__s390x_sys_nanosleep"
+#else
+#define SYS_NANOSLEEP_KPROBE_NAME "sys_nanosleep"
+#endif
int fixup_sk_storage_map[MAX_FIXUPS];
const char *errstr;
const char *errstr_unpriv;
- uint32_t retval, retval_unpriv, insn_processed;
+ uint32_t insn_processed;
int prog_len;
enum {
UNDEF,
} result, result_unpriv;
enum bpf_prog_type prog_type;
uint8_t flags;
- __u8 data[TEST_DATA_LEN];
void (*fill_helper)(struct bpf_test *self);
uint8_t runs;
- struct {
- uint32_t retval, retval_unpriv;
- union {
- __u8 data[TEST_DATA_LEN];
- __u64 data64[TEST_DATA_LEN / 8];
- };
- } retvals[MAX_TEST_RUNS];
+#define bpf_testdata_struct_t \
+ struct { \
+ uint32_t retval, retval_unpriv; \
+ union { \
+ __u8 data[TEST_DATA_LEN]; \
+ __u64 data64[TEST_DATA_LEN / 8]; \
+ }; \
+ }
+ union {
+ bpf_testdata_struct_t;
+ bpf_testdata_struct_t retvals[MAX_TEST_RUNS];
+ };
enum bpf_attach_type expected_attach_type;
};
uint32_t expected_val;
int i;
- if (!test->runs) {
- expected_val = unpriv && test->retval_unpriv ?
- test->retval_unpriv : test->retval;
-
- err = do_prog_test_run(fd_prog, unpriv, expected_val,
- test->data, sizeof(test->data));
- if (err)
- run_errs++;
- else
- run_successes++;
- }
+ if (!test->runs)
+ test->runs = 1;
for (i = 0; i < test->runs; i++) {
if (unpriv && test->retvals[i].retval_unpriv)
BPF_LD_MAP_FD(BPF_REG_1, 0),
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 1),
- BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0),
BPF_EXIT_INSN(),
},
.fixup_map_array_ro = { 3 },
BPF_EMIT_CALL(BPF_FUNC_map_lookup_elem),
BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1),
BPF_EXIT_INSN(),
- BPF_LDX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, 0),
+ BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, 0),
BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 3),
BPF_MOV64_IMM(BPF_REG_2, 0),
BPF_MOV64_IMM(BPF_REG_3, 0x100000),
--- /dev/null
+#define BPF_SOCK_ADDR_STORE(field, off, res, err) \
+{ \
+ "wide store to bpf_sock_addr." #field "[" #off "]", \
+ .insns = { \
+ BPF_MOV64_IMM(BPF_REG_0, 1), \
+ BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, \
+ offsetof(struct bpf_sock_addr, field[off])), \
+ BPF_EXIT_INSN(), \
+ }, \
+ .result = res, \
+ .prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR, \
+ .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, \
+ .errstr = err, \
+}
+
+/* user_ip6[0] is u64 aligned */
+BPF_SOCK_ADDR_STORE(user_ip6, 0, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_STORE(user_ip6, 1, REJECT,
+ "invalid bpf_context access off=12 size=8"),
+BPF_SOCK_ADDR_STORE(user_ip6, 2, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_STORE(user_ip6, 3, REJECT,
+ "invalid bpf_context access off=20 size=8"),
+
+/* msg_src_ip6[0] is _not_ u64 aligned */
+BPF_SOCK_ADDR_STORE(msg_src_ip6, 0, REJECT,
+ "invalid bpf_context access off=44 size=8"),
+BPF_SOCK_ADDR_STORE(msg_src_ip6, 1, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_STORE(msg_src_ip6, 2, REJECT,
+ "invalid bpf_context access off=52 size=8"),
+BPF_SOCK_ADDR_STORE(msg_src_ip6, 3, REJECT,
+ "invalid bpf_context access off=56 size=8"),
+
+#undef BPF_SOCK_ADDR_STORE
+
+#define BPF_SOCK_ADDR_LOAD(field, off, res, err) \
+{ \
+ "wide load from bpf_sock_addr." #field "[" #off "]", \
+ .insns = { \
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, \
+ offsetof(struct bpf_sock_addr, field[off])), \
+ BPF_MOV64_IMM(BPF_REG_0, 1), \
+ BPF_EXIT_INSN(), \
+ }, \
+ .result = res, \
+ .prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR, \
+ .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, \
+ .errstr = err, \
+}
+
+/* user_ip6[0] is u64 aligned */
+BPF_SOCK_ADDR_LOAD(user_ip6, 0, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_LOAD(user_ip6, 1, REJECT,
+ "invalid bpf_context access off=12 size=8"),
+BPF_SOCK_ADDR_LOAD(user_ip6, 2, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_LOAD(user_ip6, 3, REJECT,
+ "invalid bpf_context access off=20 size=8"),
+
+/* msg_src_ip6[0] is _not_ u64 aligned */
+BPF_SOCK_ADDR_LOAD(msg_src_ip6, 0, REJECT,
+ "invalid bpf_context access off=44 size=8"),
+BPF_SOCK_ADDR_LOAD(msg_src_ip6, 1, ACCEPT,
+ NULL),
+BPF_SOCK_ADDR_LOAD(msg_src_ip6, 2, REJECT,
+ "invalid bpf_context access off=52 size=8"),
+BPF_SOCK_ADDR_LOAD(msg_src_ip6, 3, REJECT,
+ "invalid bpf_context access off=56 size=8"),
+
+#undef BPF_SOCK_ADDR_LOAD
+++ /dev/null
-#define BPF_SOCK_ADDR(field, off, res, err) \
-{ \
- "wide store to bpf_sock_addr." #field "[" #off "]", \
- .insns = { \
- BPF_MOV64_IMM(BPF_REG_0, 1), \
- BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, \
- offsetof(struct bpf_sock_addr, field[off])), \
- BPF_EXIT_INSN(), \
- }, \
- .result = res, \
- .prog_type = BPF_PROG_TYPE_CGROUP_SOCK_ADDR, \
- .expected_attach_type = BPF_CGROUP_UDP6_SENDMSG, \
- .errstr = err, \
-}
-
-/* user_ip6[0] is u64 aligned */
-BPF_SOCK_ADDR(user_ip6, 0, ACCEPT,
- NULL),
-BPF_SOCK_ADDR(user_ip6, 1, REJECT,
- "invalid bpf_context access off=12 size=8"),
-BPF_SOCK_ADDR(user_ip6, 2, ACCEPT,
- NULL),
-BPF_SOCK_ADDR(user_ip6, 3, REJECT,
- "invalid bpf_context access off=20 size=8"),
-
-/* msg_src_ip6[0] is _not_ u64 aligned */
-BPF_SOCK_ADDR(msg_src_ip6, 0, REJECT,
- "invalid bpf_context access off=44 size=8"),
-BPF_SOCK_ADDR(msg_src_ip6, 1, ACCEPT,
- NULL),
-BPF_SOCK_ADDR(msg_src_ip6, 2, REJECT,
- "invalid bpf_context access off=52 size=8"),
-BPF_SOCK_ADDR(msg_src_ip6, 3, REJECT,
- "invalid bpf_context access off=56 size=8"),
-
-#undef BPF_SOCK_ADDR
exit $1
}
+# default error
+err_ret=1
+
+# kselftest skip code is 4
+err_skip=4
+
errexit() { # message
echo "Error: $1" 1>&2
- exit 1
+ exit $err_ret
}
# Ensuring user privilege
}
# Parameters
-DEBUGFS_DIR=`grep debugfs /proc/mounts | cut -f2 -d' ' | head -1`
-if [ -z "$DEBUGFS_DIR" ]; then
- TRACING_DIR=`grep tracefs /proc/mounts | cut -f2 -d' ' | head -1`
-else
- TRACING_DIR=$DEBUGFS_DIR/tracing
+TRACING_DIR=`grep tracefs /proc/mounts | cut -f2 -d' ' | head -1`
+if [ -z "$TRACING_DIR" ]; then
+ DEBUGFS_DIR=`grep debugfs /proc/mounts | cut -f2 -d' ' | head -1`
+ if [ -z "$DEBUGFS_DIR" ]; then
+ # If tracefs exists, then so does /sys/kernel/tracing
+ if [ -d "/sys/kernel/tracing" ]; then
+ mount -t tracefs nodev /sys/kernel/tracing ||
+ errexit "Failed to mount /sys/kernel/tracing"
+ TRACING_DIR="/sys/kernel/tracing"
+ # If debugfs exists, then so does /sys/kernel/debug
+ elif [ -d "/sys/kernel/debug" ]; then
+ mount -t debugfs nodev /sys/kernel/debug ||
+ errexit "Failed to mount /sys/kernel/debug"
+ TRACING_DIR="/sys/kernel/debug/tracing"
+ else
+ err_ret=$err_skip
+ errexit "debugfs and tracefs are not configured in this kernel"
+ fi
+ else
+ TRACING_DIR="$DEBUGFS_DIR/tracing"
+ fi
+fi
+if [ ! -d "$TRACING_DIR" ]; then
+ err_ret=$err_skip
+ errexit "ftrace is not configured in this kernel"
fi
TOP_DIR=`absdir $0`
reset_events_filter
reset_ftrace_filter
disable_events
- echo > set_event_pid # event tracer is always on
- echo > set_ftrace_pid
+ [ -f set_event_pid ] && echo > set_event_pid
+ [ -f set_ftrace_pid ] && echo > set_ftrace_pid
[ -f set_ftrace_filter ] && echo | tee set_ftrace_*
[ -f set_graph_function ] && echo | tee set_graph_*
[ -f stack_trace_filter ] && echo > stack_trace_filter
--- /dev/null
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Kprobe event user-memory access
+
+[ -f kprobe_events ] || exit_unsupported # this is configurable
+
+grep -q '\$arg<N>' README || exit_unresolved # depends on arch
+grep -A10 "fetcharg:" README | grep -q 'ustring' || exit_unsupported
+grep -A10 "fetcharg:" README | grep -q '\[u\]<offset>' || exit_unsupported
+
+:;: "user-memory access syntax and ustring working on user memory";:
+echo 'p:myevent do_sys_open path=+0($arg2):ustring path2=+u0($arg2):string' \
+ > kprobe_events
+
+grep myevent kprobe_events | \
+ grep -q 'path=+0($arg2):ustring path2=+u0($arg2):string'
+echo 1 > events/kprobes/myevent/enable
+echo > /dev/null
+echo 0 > events/kprobes/myevent/enable
+
+grep myevent trace | grep -q 'path="/dev/null" path2="/dev/null"'
+
+:;: "user-memory access syntax and ustring not working with kernel memory";:
+echo 'p:myevent vfs_symlink path=+0($arg3):ustring path2=+u0($arg3):string' \
+ > kprobe_events
+echo 1 > events/kprobes/myevent/enable
+ln -s foo $TMPDIR/bar
+echo 0 > events/kprobes/myevent/enable
+
+grep myevent trace | grep -q 'path=(fault) path2=(fault)'
+
+exit 0
ksft_skip=4
# all tests in this script. Can be overridden with -t option
-TESTS="unregister down carrier nexthop ipv6_rt ipv4_rt ipv6_addr_metric ipv4_addr_metric ipv6_route_metrics ipv4_route_metrics ipv4_route_v6_gw"
+TESTS="unregister down carrier nexthop ipv6_rt ipv4_rt ipv6_addr_metric ipv4_addr_metric ipv6_route_metrics ipv4_route_metrics ipv4_route_v6_gw rp_filter"
VERBOSE=0
PAUSE_ON_FAIL=no
PAUSE=no
IP="ip -netns ns1"
+NS_EXEC="ip netns exec ns1"
log_test()
{
fib_carrier_unicast_test
}
+fib_rp_filter_test()
+{
+ echo
+ echo "IPv4 rp_filter tests"
+
+ setup
+
+ set -e
+ $IP link set dev lo address 52:54:00:6a:c7:5e
+ $IP link set dummy0 address 52:54:00:6a:c7:5e
+ $IP link add dummy1 type dummy
+ $IP link set dummy1 address 52:54:00:6a:c7:5e
+ $IP link set dev dummy1 up
+ $NS_EXEC sysctl -qw net.ipv4.conf.all.rp_filter=1
+ $NS_EXEC sysctl -qw net.ipv4.conf.all.accept_local=1
+ $NS_EXEC sysctl -qw net.ipv4.conf.all.route_localnet=1
+
+ $NS_EXEC tc qd add dev dummy1 parent root handle 1: fq_codel
+ $NS_EXEC tc filter add dev dummy1 parent 1: protocol arp basic action mirred egress redirect dev lo
+ $NS_EXEC tc filter add dev dummy1 parent 1: protocol ip basic action mirred egress redirect dev lo
+ set +e
+
+ run_cmd "ip netns exec ns1 ping -I dummy1 -w1 -c1 198.51.100.1"
+ log_test $? 0 "rp_filter passes local packets"
+
+ run_cmd "ip netns exec ns1 ping -I dummy1 -w1 -c1 127.0.0.1"
+ log_test $? 0 "rp_filter passes loopback packets"
+
+ cleanup
+}
+
################################################################################
# Tests on nexthop spec
fib_unreg_test|unregister) fib_unreg_test;;
fib_down_test|down) fib_down_test;;
fib_carrier_test|carrier) fib_carrier_test;;
+ fib_rp_filter_test|rp_filter) fib_rp_filter_test;;
fib_nexthop_test|nexthop) fib_nexthop_test;;
ipv6_route_test|ipv6_rt) ipv6_route_test;;
ipv4_route_test|ipv4_rt) ipv4_route_test;;
/read
/self
/setns-dcache
+/setns-sysvipc
/thread-self
TEST_GEN_PROGS += read
TEST_GEN_PROGS += self
TEST_GEN_PROGS += setns-dcache
+TEST_GEN_PROGS += setns-sysvipc
TEST_GEN_PROGS += thread-self
include ../lib.mk
"ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]\n";
#ifdef __x86_64__
+static void sigaction_SIGSEGV(int _, siginfo_t *__, void *___)
+{
+ _exit(1);
+}
+
/*
* vsyscall page can't be unmapped, probe it with memory load.
*/
if (pid == 0) {
struct rlimit rlim = {0, 0};
(void)setrlimit(RLIMIT_CORE, &rlim);
+
+ /* Hide "segfault at ffffffffff600000" messages. */
+ struct sigaction act;
+ memset(&act, 0, sizeof(struct sigaction));
+ act.sa_flags = SA_SIGINFO;
+ act.sa_sigaction = sigaction_SIGSEGV;
+ (void)sigaction(SIGSEGV, &act, NULL);
+
*(volatile int *)0xffffffffff600000UL;
exit(0);
}
- wait(&wstatus);
- if (WIFEXITED(wstatus)) {
+ waitpid(pid, &wstatus, 0);
+ if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) == 0) {
g_vsyscall = true;
}
}
--- /dev/null
+/*
+ * Copyright © 2019 Alexey Dobriyan <adobriyan@gmail.com>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+/*
+ * Test that setns(CLONE_NEWIPC) points to new /proc/sysvipc content even
+ * if old one is in dcache.
+ */
+#undef NDEBUG
+#include <assert.h>
+#include <errno.h>
+#include <stdio.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <sys/ipc.h>
+#include <sys/shm.h>
+
+static pid_t pid = -1;
+
+static void f(void)
+{
+ if (pid > 0) {
+ kill(pid, SIGTERM);
+ }
+}
+
+int main(void)
+{
+ int fd[2];
+ char _ = 0;
+ int nsfd;
+
+ atexit(f);
+
+ /* Check for priviledges and syscall availability straight away. */
+ if (unshare(CLONE_NEWIPC) == -1) {
+ if (errno == ENOSYS || errno == EPERM) {
+ return 4;
+ }
+ return 1;
+ }
+ /* Distinguisher between two otherwise empty IPC namespaces. */
+ if (shmget(IPC_PRIVATE, 1, IPC_CREAT) == -1) {
+ return 1;
+ }
+
+ if (pipe(fd) == -1) {
+ return 1;
+ }
+
+ pid = fork();
+ if (pid == -1) {
+ return 1;
+ }
+
+ if (pid == 0) {
+ if (unshare(CLONE_NEWIPC) == -1) {
+ return 1;
+ }
+
+ if (write(fd[1], &_, 1) != 1) {
+ return 1;
+ }
+
+ pause();
+
+ return 0;
+ }
+
+ if (read(fd[0], &_, 1) != 1) {
+ return 1;
+ }
+
+ {
+ char buf[64];
+ snprintf(buf, sizeof(buf), "/proc/%u/ns/ipc", pid);
+ nsfd = open(buf, O_RDONLY);
+ if (nsfd == -1) {
+ return 1;
+ }
+ }
+
+ /* Reliably pin dentry into dcache. */
+ (void)open("/proc/sysvipc/shm", O_RDONLY);
+
+ if (setns(nsfd, CLONE_NEWIPC) == -1) {
+ return 1;
+ }
+
+ kill(pid, SIGTERM);
+ pid = 0;
+
+ {
+ char buf[4096];
+ ssize_t rv;
+ int fd;
+
+ fd = open("/proc/sysvipc/shm", O_RDONLY);
+ if (fd == -1) {
+ return 1;
+ }
+
+#define S32 " key shmid perms size cpid lpid nattch uid gid cuid cgid atime dtime ctime rss swap\n"
+#define S64 " key shmid perms size cpid lpid nattch uid gid cuid cgid atime dtime ctime rss swap\n"
+ rv = read(fd, buf, sizeof(buf));
+ if (rv == strlen(S32)) {
+ assert(memcmp(buf, S32, strlen(S32)) == 0);
+ } else if (rv == strlen(S64)) {
+ assert(memcmp(buf, S64, strlen(S64)) == 0);
+ } else {
+ assert(0);
+ }
+ }
+
+ return 0;
+}
+get_syscall_info
peeksiginfo
# SPDX-License-Identifier: GPL-2.0-only
CFLAGS += -iquote../../../../include/uapi -Wall
-TEST_GEN_PROGS := peeksiginfo
+TEST_GEN_PROGS := get_syscall_info peeksiginfo
include ../lib.mk
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (c) 2018 Dmitry V. Levin <ldv@altlinux.org>
+ * All rights reserved.
+ *
+ * Check whether PTRACE_GET_SYSCALL_INFO semantics implemented in the kernel
+ * matches userspace expectations.
+ */
+
+#include "../kselftest_harness.h"
+#include <err.h>
+#include <signal.h>
+#include <asm/unistd.h>
+#include "linux/ptrace.h"
+
+static int
+kill_tracee(pid_t pid)
+{
+ if (!pid)
+ return 0;
+
+ int saved_errno = errno;
+
+ int rc = kill(pid, SIGKILL);
+
+ errno = saved_errno;
+ return rc;
+}
+
+static long
+sys_ptrace(int request, pid_t pid, unsigned long addr, unsigned long data)
+{
+ return syscall(__NR_ptrace, request, pid, addr, data);
+}
+
+#define LOG_KILL_TRACEE(fmt, ...) \
+ do { \
+ kill_tracee(pid); \
+ TH_LOG("wait #%d: " fmt, \
+ ptrace_stop, ##__VA_ARGS__); \
+ } while (0)
+
+TEST(get_syscall_info)
+{
+ static const unsigned long args[][7] = {
+ /* a sequence of architecture-agnostic syscalls */
+ {
+ __NR_chdir,
+ (unsigned long) "",
+ 0xbad1fed1,
+ 0xbad2fed2,
+ 0xbad3fed3,
+ 0xbad4fed4,
+ 0xbad5fed5
+ },
+ {
+ __NR_gettid,
+ 0xcaf0bea0,
+ 0xcaf1bea1,
+ 0xcaf2bea2,
+ 0xcaf3bea3,
+ 0xcaf4bea4,
+ 0xcaf5bea5
+ },
+ {
+ __NR_exit_group,
+ 0,
+ 0xfac1c0d1,
+ 0xfac2c0d2,
+ 0xfac3c0d3,
+ 0xfac4c0d4,
+ 0xfac5c0d5
+ }
+ };
+ const unsigned long *exp_args;
+
+ pid_t pid = fork();
+
+ ASSERT_LE(0, pid) {
+ TH_LOG("fork: %m");
+ }
+
+ if (pid == 0) {
+ /* get the pid before PTRACE_TRACEME */
+ pid = getpid();
+ ASSERT_EQ(0, sys_ptrace(PTRACE_TRACEME, 0, 0, 0)) {
+ TH_LOG("PTRACE_TRACEME: %m");
+ }
+ ASSERT_EQ(0, kill(pid, SIGSTOP)) {
+ /* cannot happen */
+ TH_LOG("kill SIGSTOP: %m");
+ }
+ for (unsigned int i = 0; i < ARRAY_SIZE(args); ++i) {
+ syscall(args[i][0],
+ args[i][1], args[i][2], args[i][3],
+ args[i][4], args[i][5], args[i][6]);
+ }
+ /* unreachable */
+ _exit(1);
+ }
+
+ const struct {
+ unsigned int is_error;
+ int rval;
+ } *exp_param, exit_param[] = {
+ { 1, -ENOENT }, /* chdir */
+ { 0, pid } /* gettid */
+ };
+
+ unsigned int ptrace_stop;
+
+ for (ptrace_stop = 0; ; ++ptrace_stop) {
+ struct ptrace_syscall_info info = {
+ .op = 0xff /* invalid PTRACE_SYSCALL_INFO_* op */
+ };
+ const size_t size = sizeof(info);
+ const int expected_none_size =
+ (void *) &info.entry - (void *) &info;
+ const int expected_entry_size =
+ (void *) &info.entry.args[6] - (void *) &info;
+ const int expected_exit_size =
+ (void *) (&info.exit.is_error + 1) -
+ (void *) &info;
+ int status;
+ long rc;
+
+ ASSERT_EQ(pid, wait(&status)) {
+ /* cannot happen */
+ LOG_KILL_TRACEE("wait: %m");
+ }
+ if (WIFEXITED(status)) {
+ pid = 0; /* the tracee is no more */
+ ASSERT_EQ(0, WEXITSTATUS(status));
+ break;
+ }
+ ASSERT_FALSE(WIFSIGNALED(status)) {
+ pid = 0; /* the tracee is no more */
+ LOG_KILL_TRACEE("unexpected signal %u",
+ WTERMSIG(status));
+ }
+ ASSERT_TRUE(WIFSTOPPED(status)) {
+ /* cannot happen */
+ LOG_KILL_TRACEE("unexpected wait status %#x", status);
+ }
+
+ switch (WSTOPSIG(status)) {
+ case SIGSTOP:
+ ASSERT_EQ(0, ptrace_stop) {
+ LOG_KILL_TRACEE("unexpected signal stop");
+ }
+ ASSERT_EQ(0, sys_ptrace(PTRACE_SETOPTIONS, pid, 0,
+ PTRACE_O_TRACESYSGOOD)) {
+ LOG_KILL_TRACEE("PTRACE_SETOPTIONS: %m");
+ }
+ ASSERT_LT(0, (rc = sys_ptrace(PTRACE_GET_SYSCALL_INFO,
+ pid, size,
+ (unsigned long) &info))) {
+ LOG_KILL_TRACEE("PTRACE_GET_SYSCALL_INFO: %m");
+ }
+ ASSERT_EQ(expected_none_size, rc) {
+ LOG_KILL_TRACEE("signal stop mismatch");
+ }
+ ASSERT_EQ(PTRACE_SYSCALL_INFO_NONE, info.op) {
+ LOG_KILL_TRACEE("signal stop mismatch");
+ }
+ ASSERT_TRUE(info.arch) {
+ LOG_KILL_TRACEE("signal stop mismatch");
+ }
+ ASSERT_TRUE(info.instruction_pointer) {
+ LOG_KILL_TRACEE("signal stop mismatch");
+ }
+ ASSERT_TRUE(info.stack_pointer) {
+ LOG_KILL_TRACEE("signal stop mismatch");
+ }
+ break;
+
+ case SIGTRAP | 0x80:
+ ASSERT_LT(0, (rc = sys_ptrace(PTRACE_GET_SYSCALL_INFO,
+ pid, size,
+ (unsigned long) &info))) {
+ LOG_KILL_TRACEE("PTRACE_GET_SYSCALL_INFO: %m");
+ }
+ switch (ptrace_stop) {
+ case 1: /* entering chdir */
+ case 3: /* entering gettid */
+ case 5: /* entering exit_group */
+ exp_args = args[ptrace_stop / 2];
+ ASSERT_EQ(expected_entry_size, rc) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(PTRACE_SYSCALL_INFO_ENTRY, info.op) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_TRUE(info.arch) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_TRUE(info.instruction_pointer) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_TRUE(info.stack_pointer) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[0], info.entry.nr) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[1], info.entry.args[0]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[2], info.entry.args[1]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[3], info.entry.args[2]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[4], info.entry.args[3]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[5], info.entry.args[4]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ ASSERT_EQ(exp_args[6], info.entry.args[5]) {
+ LOG_KILL_TRACEE("entry stop mismatch");
+ }
+ break;
+ case 2: /* exiting chdir */
+ case 4: /* exiting gettid */
+ exp_param = &exit_param[ptrace_stop / 2 - 1];
+ ASSERT_EQ(expected_exit_size, rc) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_EQ(PTRACE_SYSCALL_INFO_EXIT, info.op) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_TRUE(info.arch) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_TRUE(info.instruction_pointer) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_TRUE(info.stack_pointer) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_EQ(exp_param->is_error,
+ info.exit.is_error) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ ASSERT_EQ(exp_param->rval, info.exit.rval) {
+ LOG_KILL_TRACEE("exit stop mismatch");
+ }
+ break;
+ default:
+ LOG_KILL_TRACEE("unexpected syscall stop");
+ abort();
+ }
+ break;
+
+ default:
+ LOG_KILL_TRACEE("unexpected stop signal %#x",
+ WSTOPSIG(status));
+ abort();
+ }
+
+ ASSERT_EQ(0, sys_ptrace(PTRACE_SYSCALL, pid, 0, 0)) {
+ LOG_KILL_TRACEE("PTRACE_SYSCALL: %m");
+ }
+ }
+
+ ASSERT_EQ(ARRAY_SIZE(args) * 2, ptrace_stop);
+}
+
+TEST_HARNESS_MAIN
static void write_policies(void)
{
+ static char *policy_str =
+ "1:2\n"
+ "1:3\n"
+ "2:2\n"
+ "3:3\n";
ssize_t written;
int fd;
fd = open(add_whitelist_policy_file, O_WRONLY);
if (fd < 0)
die("cant open add_whitelist_policy file\n");
- written = write(fd, "1:2", strlen("1:2"));
- if (written != strlen("1:2")) {
- if (written >= 0) {
- die("short write to %s\n", add_whitelist_policy_file);
- } else {
- die("write to %s failed: %s\n",
- add_whitelist_policy_file, strerror(errno));
- }
- }
- written = write(fd, "1:3", strlen("1:3"));
- if (written != strlen("1:3")) {
+ written = write(fd, policy_str, strlen(policy_str));
+ if (written != strlen(policy_str)) {
if (written >= 0) {
die("short write to %s\n", add_whitelist_policy_file);
} else {
unsigned long msg;
static bool entry;
- /* Make sure we got an empty message. */
+ /*
+ * The traditional way to tell PTRACE_SYSCALL entry/exit
+ * is by counting.
+ */
+ entry = !entry;
+
+ /* Make sure we got an appropriate message. */
ret = ptrace(PTRACE_GETEVENTMSG, tracee, NULL, &msg);
EXPECT_EQ(0, ret);
- EXPECT_EQ(0, msg);
+ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
+ : PTRACE_EVENTMSG_SYSCALL_EXIT, msg);
- /* The only way to tell PTRACE_SYSCALL entry/exit is by counting. */
- entry = !entry;
if (!entry)
return;
- mkfs/ mkfs.ext4
For more information please refer:
-kernel-source-tree/Documentation/blockdev/zram.txt
+kernel-source-tree/Documentation/admin-guide/blockdev/zram.rst
When multiple directories and files are specified then the
initramfs image will be the aggregate of all of them.
- See <file:Documentation/early-userspace/README> for more details.
+ See <file:Documentation/driver-api/early-userspace/early_userspace_support.rst> for more details.
If you are not sure, leave it blank.
projects / platform / kernel / linux-starfive.git / commitdiff
